Wes Roth

Wes Roth

Download on the App Store

Wes Roth episodes

  • LoRA - Low-rank Adaption of AI Large Language Models: LoRA and QLoRA Explained Simply

    What is LoRA in AI?

    You may have heard of a concept called LoRA or QLoRA referring to AI and Large Language Models.

    Imagine you have a giant box full of Legos.

    You can build all kinds of things with this giant box - houses, cars, spaceships.

    But it's so big and heavy that it's hard to carry around.

    And most of the time, you don't need all these Legos to build what you want to build.

    So instead, you pick out a smaller box of your favorite, most useful Legos.

    This smaller box is easier to carry around, and you can still build most of the things you want.

    In this analogy, the giant box of Legos is like a large language model, like GPT-4.

    It's powerful and can do lots of things, but it's also big and heavy (it requires a lot of computational resources to use).

    The smaller box of Legos is like a "low-rank adaptation" of the large language model.

    It's a smaller, lighter version of the model that's been adapted for a specific task.

    It's not as powerful as the full model - there might be some things it can't do - but it's more efficient and easier to use.

    “Low-Rank Adaptation”

    LoRA stands for

    “Low-Rank Adaptation”

    "Low-rank" in this context refers to a mathematical technique used to create this smaller, lighter model.

    You can also think of “Low-Rank” as just reading all the highlighted parts in a book.

    “Full Rank” would be reading the entire book and “Low-Rank” would be reading just the important, highlighted parts.

    Why is LoRA important?

    Let’s say you have a large and advanced AI model trained on recognizing all sorts of images.

    You can "fine-tune" it to do a related task (like recognizing images of cats, specifically) by making small adjustments to that large model.

    You can also “fine-tune” it to add behaviors you want or remove behaviors you don’t.

    But this can be very expensive in terms of what computers you would need and how long it would take.

    Lora solves this problem by making it cheap and fast to fine tune smaller models




    LoRA is important because:

    1. Efficiency

    Using LoRA can greatly reduce the amount of resources used to train AI models to perform these tasks.

    2. Speed

    These lower-rank models are faster to train,but they also can provide faster outputs.

    This can be crucial in applications where results need to happen in real-time.

    3. Limited Resources

    In many real world applications the devices that are available to run AI models may have limited computational power or memory.

    Your smartphone may not be able to run a large AI model, but a Low-Rank Adaptation can be used for specific tasks you may need.

    4. Stacking and Transfering

    Low-rank adaptations can be helpful for transfer learning where a model trained on one task can be adapted to a different, but related task.

    This is much more efficient than training the large model to do something from scratch.

    The updates and new skills learned by these low rank adaptations can also stack with other such adaptations, so multiple models can benefit each other as well as the original larger model.

    QLoRa

    QLoRA is a similar concept.

    The Q is for Quantized, so QLoRA is Quantized Low Rank Adaptation.

    Quantized refers to data compression.

    Quantization is converting a continuous range of values into a finite set of possible values.

    Image if you’re an artist mixing paint.

    You have an almost infinite range of colors you can create by mixing different amounts of colors together.

    This is like a continuous signal in the real world.

    But if you are working with a computer graphics program, it can’t handle an infinite range of colors.

    It might only allow each color component (red, green, and blue) to have one of several levels of intensity.

    This limited set of possible colors is like a quantized signal.

    Here it can apply to reducing the number of decimal places we need to express a number.
    For example Pi is an infinitely long number, bu

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    9 min
  • BREAKING: Grok 4.20 might be *too* good...

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRothMoney
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    14 min
  • Zapier Central Unleashed ADVANCED Automation

    AI News by Wes Roth. We cover AGI, open source and AI Agents. From Sam Altman, ChatGPT and OpenAI to Elon Musk, XAI and Grok.

    My Links 🔗
    ➡️ Subscribe: https://www.youtube.com/@WesRoth?sub_confirmation=1
    ➡️ Twitter: https://x.com/WesRothMoney
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    #ainews #ai #agi



    00:00 what is Zapier?
    09:16 Zapier Presents Zapier Central


    #ai #openai #llm

    BUSINESS, MEDIA & SPONSORSHIPS:
    Wes Roth Business @ Gmail . com
    [email protected]

    Just shoot me an email to the above address.

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    28 min
  • we JUST figured out how AI thinks

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRoth
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    28 min
  • OpenAI Just Released New Models and Developer Tools | GPT-4 no longer "lazy" 😍

    POLL:
    Will you use OpenAI's new embedding models?
    https://www.youtube.com/post/UgkxtEP9eVamBpx6dMyw7dggpimiOJgSJqzJ




    Get on my daily AI newsletter 🔥
    https://natural20.beehiiv.com/subscribe
    [News, Research and Tutorials on AI]



    My AI Playlist:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XROkUAwkYhcHb7OY9yoGGZH

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    11 min
  • Meta just did the thing

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    Alexandr Wang:
    https://x.com/alexandr_wang/status/2005766469771223106

    Remote Labor Index:
    Measuring AI Automation of Remote Work
    https://www.remotelabor.ai/

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRothMoney
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    16 min
  • "Almost UNIMAGINABLE Power" - Anthropic Founder

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    Dario's Blog Post:
    https://www.darioamodei.com/essay/the-adolescence-of-technology

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRoth
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    32 min
  • "Code Red" Over, OpenAI is about to blow...

    Big AI news today! We dive into the upcoming release of Grok 4.20, which is dominating the profit-based Alpha Arena and is expected to launch before the end of the year. Meanwhile, the battle for the top spot on the LM Arena heats up as Google’s Gemini 3.0 Pro holds the lead, forcing OpenAI into "Code Red" with mysterious new test models like Emperor and Rockhopper appearing in the wild.

    We also break down a massive shift in hardware strategy: reports suggest Google is now selling TPUs directly to companies like Anthropic, challenging Nvidia's dominance. Plus, we explore Google’s new "Titans" and "Mira" papers on long-term memory and "surprise" metrics in AI, and discuss Elon Musk’s plan to launch solar-powered AI data centers into space via SpaceX.
    ______________________________________________
    Keywords: Grok 4.20, Gemini 3.0, OpenAI Code Red, Google TPUs, Artificial Intelligence News, SpaceX Data Centers, Project SunCatcher, AI Memory Architecture, Wes Roth, LM Arena, Nvidia Short, Michael Burry, Alpha Arena, AI Investing.
    ______________________________________________
    VIDEO LINKS:

    GROK 4.20 keeps making money
    https://nof1.ai/leaderboard

    Simulators
    Janus aka Repligate
    https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

    Our interview with Karan 4D, co‑founder & Head of Behavior at Nous Research
    (I just unlocked SHOGGOTH MODE)
    https://www.youtube.com/watch?v=7ZEHdaABIJU

    Titans + MIRAS: Helping AI have long-term memory
    https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/

    Ali Behrouz - Titans
    https://x.com/behrouz_ali/status/1878859776299343999

    GROK 4.20 keeps making money
    https://nof1.ai/leaderboard

    ELON MUSK ROCKETS
    https://x.com/elonmusk/status/1997706687155720229

    ARK’s Expected Value For SpaceX In 2030: ~$2.5 Trillion Enterprise Value
    https://www.ark-invest.com/articles/valuation-models/ark-expected-value-spacex-2030

    Andrej Karpathy
    @karpathy
    https://x.com/karpathy/status/1997731268969304070

    ______________________________________________
    TIMELINE
    00:00 Grok 4.20 Release & Alpha Arena Profits
    02:08 Gemini 3.0 Dominance & Polymarket Betting
    03:59 Mystery OpenAI Models (Emperor, Rockhopper)
    04:40 Google Research: Titans, Mira & Long-Term Memory
    06:00 The "Surprise" Metric: How AI Learns Like Humans
    08:26 Update: Google Selling TPUs to Anthropic
    11:05 Michael Burry Shorts Nvidia
    12:00 Andre Karpathy on LLM Psychology
    14:10 SpaceX Valuation & Space-Based Data Centers
    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRothMoney
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe
    ______________________________________________
    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]
    ______________________________________________
    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    25 min
  • How To Use Midjourney | The ULTIMATE Beginners Guide to Midjourney 2025

    FULL GUIDE: https://natural20.com/how-to-use-midjourney/

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRothMoney
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/@Wes-Dylan



    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    25 min
  • GPT-5.6 about to DROP

    The latest AI News. Learn about LLMs, Gen AI and get ready for the rollout of AGI. Wes Roth covers the latest happenings in the world of OpenAI, Google, Anthropic, NVIDIA and Open Source AI.

    ______________________________________________
    My Links 🔗
    ➡️ Twitter: https://x.com/WesRoth
    ➡️ AI Newsletter: https://natural20.beehiiv.com/subscribe

    Want to work with me?
    Brand, sponsorship & business inquiries: [email protected]

    Check out my AI Podcast where me and Dylan interview AI experts:
    https://www.youtube.com/playlist?list=PLb1th0f6y4XSKLYenSVDUXFjSHsZTTfhk
    ______________________________________________



    #ai #openai #llm

    Learn more about your ad choices. Visit megaphone.fm/adchoices

    23 min

About Wes Roth

From the publisher's feed

Artificial Intelligence, AI, Machine Learning, Google DeepMind, OpenAI, ChatGPT, Anthropic, Tech, Coding, AI News, Large Language Models and... (most importantly)... optimism. AGI rolls around only…