The Machine: A computer science education podcast

Building a Local Large Language Model (LLM)


Listen Later

Another #ComputingWeek talk turned into a podcast! Two Red Hat software engineers, both recent graduates of SETU, returned to discuss the issues surrounding running your own LLM on a local machine, how models and datasets are built and reduced (quantised) so as to run on a laptop rather than an array of servers. Mark Campbell and Dimitri Saradkis provided excellent insight on the technical issues surround this topic, before getting into some of the ethical and moral issues with host Rob O'Connor at the end.

You can connect with all the people on this podcast on LinkedIn at:

  • Mark Campbell https://www.linkedin.com/in/mark-campbell-76846b194/
  • Dimitri Saradakis https://www.linkedin.com/in/dimitri-saridakis-32a087139/
  • Rob O'Connor https://www.linkedin.com/in/robertoconnorirl/
  • Here are links to some of the tools referenced in the podcast:

    • Red Hat OpenShift AI https://www.redhat.com/en/technologies/cloud-computing/openshift/openshift-ai
      • LMStudio https://lmstudio.ai/
      • Ollama https://ollama.ai/
      • HuggingFace https://huggingface.co/
      • ...more
        View all episodesView all episodes
        Download on the App Store

        The Machine: A computer science education podcastBy Rob O'Connor

        • 4.8
        • 4.8
        • 4.8
        • 4.8
        • 4.8

        4.8

        8 ratings


        More shows like The Machine: A computer science education podcast

        View all
        Global News Podcast by BBC World Service

        Global News Podcast

        7,901 Listeners

        The Sunday Magazine by CBC

        The Sunday Magazine

        66 Listeners

        The Joe Rogan Experience by Joe Rogan

        The Joe Rogan Experience

        226,206 Listeners

        CodeNewbie by CodeNewbie

        CodeNewbie

        574 Listeners

        The Ben Shapiro Show by The Daily Wire

        The Ben Shapiro Show

        153,960 Listeners

        The Daily by The New York Times

        The Daily

        111,864 Listeners

        Up First from NPR by NPR

        Up First from NPR

        56,221 Listeners

        Dwarkesh Podcast by Dwarkesh Patel

        Dwarkesh Podcast

        417 Listeners

        Hard Fork by The New York Times

        Hard Fork

        5,448 Listeners

        The Ezra Klein Show by New York Times Opinion

        The Ezra Klein Show

        15,237 Listeners

        Hello World by Hello World

        Hello World

        0 Listeners

        The Rest Is Politics by Goalhanger

        The Rest Is Politics

        3,276 Listeners

        The Rest Is Politics: Leading by Goalhanger

        The Rest Is Politics: Leading

        988 Listeners

        The Daily AI Show by The Daily AI Show Crew - Brian, Beth, Jyunmi, Andy, Karl, and Eran

        The Daily AI Show

        4 Listeners

        The Rest Is Politics: US by Goalhanger

        The Rest Is Politics: US

        2,295 Listeners