The Machine: A computer science education podcast

Building a Local Large Language Model (LLM)

Listen Later

Another #ComputingWeek talk turned into a podcast! Two Red Hat software engineers, both recent graduates of SETU, returned to discuss the issues surrounding running your own LLM on a local machine, how models and datasets are built and reduced (quantised) so as to run on a laptop rather than an array of servers. Mark Campbell and Dimitri Saradkis provided excellent insight on the technical issues surround this topic, before getting into some of the ethical and moral issues with host Rob O'Connor at the end.

You can connect with all the people on this podcast on LinkedIn at:

  • Mark Campbell https://www.linkedin.com/in/mark-campbell-76846b194/
  • Dimitri Saradakis https://www.linkedin.com/in/dimitri-saridakis-32a087139/
  • Rob O'Connor https://www.linkedin.com/in/robertoconnorirl/

    • Here are links to some of the tools referenced in the podcast:

    • Red Hat OpenShift AI https://www.redhat.com/en/technologies/cloud-computing/openshift/openshift-ai
      • LMStudio https://lmstudio.ai/
      • Ollama https://ollama.ai/
      • HuggingFace https://huggingface.co/
        • ...more
        View all episodesView all episodes
        Download on the App Store
        The Machine: A computer science education podcastBy Rob O'Connor
        • 4.8
        • 4.8
        • 4.8
        • 4.8
        • 4.8

        4.8

        8 ratings

        More shows like The Machine: A computer science education podcast

        View all
        Crime Junkie by Audiochuck

        Crime Junkie

        368,943 Listeners

        Computer Science/Software Engineering College Courses Review by Andres Arriaga

        Computer Science/Software Engineering College Courses Review

        11 Listeners

        Hard Fork by The New York Times

        Hard Fork

        5,525 Listeners

        The Ezra Klein Show by New York Times Opinion

        The Ezra Klein Show

        15,955 Listeners