In today's thrilling episode, we dissect “LongNet”, a groundbreaking paper that scales transformers to a whopping 1 billion tokens. Next, we discuss Uncertainty Alignment and its implications for robotics. Finally, we cover "Motion Retargeting", a method of creating 3D avatars from minimal user input data, primarily headset and controller information.

Key Points

1️⃣ LongNet

* A method called "LongNet" scales transformer models to handle a billion tokens, using dilated attention to avoid quadratic complexity, achieving linear scaling.

* While this method technically handles a billion tokens, it's different as it looks at pieces, not the entire attention, compromising performance beyond context window.

* It's viewed as a clever innovation in computational scaling, despite trade-offs, and other methods like 'alibi' are suggested for better performance.

2️⃣ Uncertainty Alignment

* The paper introduces "uncertainty alignment," a method for robots to handle ambiguous tasks by seeking minimum user help and providing statistical guarantees before executing a task.

* This approach reduces fine-tuning and prompt tuning, aligns with how people think, and improves user experience by asking follow-up questions when uncertain.

* While not groundbreaking, it simplifies complex tasks using probability and statistics, potentially becoming a standard practice for various chatbots and robotics applications.

3️⃣ Motion Retargeting

* “Motion retargeting" is a method of creating 3D avatars from minimal user input data, primarily headset and controller information.

* This technology transfers human movements to various virtual characters, demonstrating realistic movements despite the difference in character structure, like a dinosaur or a mouse.

* Though promising, the technique depends heavily on the user's movements, and edge cases like extreme physical behavior can disrupt the avatar's realistic representation.

🔗 Episode Links

* LongNet

* Uncertainty Alignment

* Motion Retargeting

* AI-Laser Pesticide & Herbicide

Connect With Us:

Follow us on Threads

Subscribe to our Substack

Follow us on Twitter:

* AI Daily

* Farb

* Ethan

* Conner

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aidailypod.com