Links

• Notes and resources at ocdevel.com/mlg/mla-24
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools

Tool Use in AI Code Agents

• File Operations: Agents can read, edit, and search files using sophisticated regular expressions.
• Executable Commands: They can recommend and perform installations like pip or npm installs, with user approval.
• Browser Integration: Allows agents to perform actions and verify outcomes through browser interactions.

Model Context Protocol (MCP)

• Standardization: MCP was created by Anthropic to standardize how AI tools and agents communicate with each other and with external tools.
• Implementation:
  • MCP Client: Converts AI agent requests into structured commands.
  • MCP Server: Executes commands and sends structured responses back to the client.
• Local and Cloud Frameworks:
  • Local (S-T-D-I-O MCP): Examples include utilizing Playwright for local browser automation and connecting to local databases like Postgres.
  • Cloud (SSE MCP): SaaS providers offer cloud-hosted MCPs to enhance external integrations.

Expanding AI Capabilities with MCP Servers

• Directories: Various directories exist listing MCP servers for diverse functions beyond programming. modelcontextprotocol/servers
• Use Cases:
  • Automation Beyond Coding: Implementing MCPs that extend automation into non-programming tasks like sales, marketing, or personal project management.
  • Creative Solutions: Encourages innovation in automating routine tasks by integrating diverse MCP functionalities.

AI Tools in Machine Learning

• Automating ML Process:
  • Auto ML and Feature Engineering: AI tools assist in transforming raw data, optimizing hyperparameters, and inventing new ML solutions.
  • Pipeline Construction and Deployment: Facilitates the use of infrastructure as code for deploying ML models efficiently.
• Active Experimentation:
  • Jupyter Integration Challenges: While integrations are possible, they often lag and may not support the latest models.
  • Practical Strategies: Suggests alternating between Jupyter and traditional Python files to maximize tool efficiency.
• Action Plan for ML Engineers:
  • Setup structured folders and documentation to leverage AI tools effectively.
  • Encourage systematic exploration of MCPs to enhance both direct programming tasks and associated workflows.

Links

• Notes and resources at  ocdevel.com/mlg/mla-23
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools

Model Current Leaders

According to the Aider Leaderboard (as of April 12, 2025), leading models include for vibe-coding:

• Gemini 2.5 Pro Preview 03-25: most accurate and cost-effective option currently.
• Claude 3.7 Sonnet: Performs well in both architect and code modes with enabled reasoning flags.
• DeepSeek R1 with Claude 3.5 Sonnet: A popular combination for its balance of cost and performance between reasoning and non-reasoning tasks.

Local Models

• Tools for Local Models: Ollama is the standard tool to manage local models, enabling usage without internet connectivity.
• Privacy and Security: Utilizing local models enhances data security, suitable for sensitive projects or corporate environments that require data to remain onsite.
• Performance Trade-offs: Local models, due to distillation and size constraints, often perform slightly worse than cloud-hosted models but offer privacy benefits.

Fine-Tuning Models

• Customization: Developers can fine-tune pre-trained models to specialize them for their specific codebase, enhancing relevance and accuracy.
• Advanced Usage: Suitable for long-term projects, fine-tuning helps models understand unique aspects of a project, resulting in consistent code quality improvements.

Tips and Best Practices

• Judicious Use of the @ Key: Improves model efficiency by specifying the context of commands, reducing the necessity for AI-initiated searches.
  • Examples include specifying file paths, URLs, or git commits to inform AI actions more precisely.
• Concurrent Feature Implementation: Leverage tools like Boomerang mode to manage multiple features simultaneously, acting more as a manager overseeing several tasks at once, enhancing productivity.
• Continued Learning: Staying updated with documentation, particularly Roo Code's, due to its comprehensive feature set and versatility among AI coding tools.

Links

• Notes and resources at  ocdevel.com/mlg/mla-22
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools

I currently favor Roo Code. Plus either gemini-2.5-pro-exp-03-25 for Architect, Boomerang, or Code with large contexts. And Claude 3.7 for code with small contexts, eg Boomerang subtasks. Many others favor Cursor, Aider, or Cline. Copilot and Windsurf are less vogue lately. I found Copilot to struggle more; and their pricing - previously their winning point - is less compelling now.

Why I favor Roo. The default settings have it as stable and effective as Cline, Cursor. But you can tinker more with these settings - eg, for Gemini 2.5 I disable partial file reads (since it has a huge context window). Their modes are elegantly just custom system prompts (an oversimplification), making custom workflows very powerful. A potent example is their Boomerang Mode, which is an orchestrator that delegates planning and edit subtasks, to keep context windows tight. Boomerang mode specifically is a plugin-seller, it's incredibly powerful. Aider is still a darn decent exacto-knife, but as Roo has grown, I haven't found much need for Aider.

Tools discussed:

1. Roo Code
2. Aider
3. Cursor
4. Cline
5. Copilot
6. Windsurf

Other:

1. Leaderboards
2. Video of speed-demon
3. Reddit

"Vibe coding" using AI agents in software development. It uses LLMs for code generation and project management. Developers are increasingly relying on agentic tools and IDE plugins to improve productivity.

Use of AI in Code Generation

• AI tools facilitate the generation and editing of code.
• Integration typically occurs within IDEs or as plugins.
• These tools offer features like inline editing, bug fixing, and project scaffolding.

Evolution and Adoption

• The concept is gaining popularity due to its efficiency and competitive edge in development.

Popular AI Tools for Vibe Coding Cursor

• Characteristics: Most popular, stable, with advanced agentic capabilities.
• Pricing: $20 per month, additional charges for power-use.
• Strengths: Reliable, focuses on integrating new models effectively.

Windsurf

• Characteristics: Cost-effective, a VS Code fork.
• Pricing: Starts at $15, with higher usage at $60.
• Strengths: Similar to Cursor, with a competitive pricing model.

GitHub Copilot

• Characteristics: Operates within GitHub code spaces, developed by Microsoft.
• Pricing: $10 to $40 monthly.
• Strengths: Deep integration with cloud-based development environments.

Cline

• Characteristics: Open-source, known for customizable features.
• Pricing: BYOM (Bring Your Own Model), costs based on individual API usage.
• Strengths: Community-driven, rapid development cycles.

Roo Code

• Characteristics: Fast-moving, offers the latest technological advancements.
• Pricing: Uses BYOM model, similar to Cline.
• Strengths: Frequent updates, for users wanting cutting-edge features.

Aider

• Characteristics: CLI-based, focuses on precision and minimal token usage.
• Pricing: BYOM, efficient token usage strategies.
• Strengths: High accuracy for small adjustments, good for backup use.

Choosing the Right Tool

• Beginner Recommendation: Start with Cursor for reliability.
• Experimentation: Try Copilot and Windsurf for comparisons.
• Advanced Configuration: Use Kline or Roo Code for sophisticated tasks and ER for precise adjustments.

Cost Management

• Open Router: Centralize API billing to manage interactions across multiple models, preventing fragmented payments.

Links:

• Notes and resources at ocdevel.com/mlg/33
• 3Blue1Brown videos: https://3blue1brown.com/
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools

Background & Motivation

• RNN Limitations: Sequential processing prevents full parallelization—even with attention tweaks—making them inefficient on modern hardware.
• Breakthrough: “Attention Is All You Need” replaced recurrence with self-attention, unlocking massive parallelism and scalability.

Core Architecture

• Layer Stack: Consists of alternating self-attention and feed-forward (MLP) layers, each wrapped in residual connections and layer normalization.
• Positional Encodings: Since self-attention is permutation invariant, add sinusoidal or learned positional embeddings to inject sequence order.

Self-Attention Mechanism

• Q, K, V Explained:
  • Query (Q): The representation of the token seeking contextual info.
  • Key (K): The representation of tokens being compared against.
  • Value (V): The information to be aggregated based on the attention scores.
• Multi-Head Attention: Splits Q, K, V into multiple “heads” to capture diverse relationships and nuances across different subspaces.
• Dot-Product & Scaling: Computes similarity between Q and K (scaled to avoid large gradients), then applies softmax to weigh V accordingly.

Masking

• Causal Masking: In autoregressive models, prevents a token from “seeing” future tokens, ensuring proper generation.
• Padding Masks: Ignore padded (non-informative) parts of sequences to maintain meaningful attention distributions.

Feed-Forward Networks (MLPs)

• Transformation & Storage: Post-attention MLPs apply non-linear transformations; many argue they’re where the “facts” or learned knowledge really get stored.
• Depth & Expressivity: Their layered nature deepens the model’s capacity to represent complex patterns.

Residual Connections & Normalization

• Residual Links: Crucial for gradient flow in deep architectures, preventing vanishing/exploding gradients.
• Layer Normalization: Stabilizes training by normalizing across features, enhancing convergence.

Scalability & Efficiency Considerations

• Parallelization Advantage: Entire architecture is designed to exploit modern parallel hardware, a huge win over RNNs.
• Complexity Trade-offs: Self-attention’s quadratic complexity with sequence length remains a challenge; spurred innovations like sparse or linearized attention.

Training Paradigms & Emergent Properties

• Pretraining & Fine-Tuning: Massive self-supervised pretraining on diverse data, followed by task-specific fine-tuning, is the norm.
• Emergent Behavior: With scale comes abilities like in-context learning and few-shot adaptation, aspects that are still being unpacked.

Interpretability & Knowledge Distribution

• Distributed Representation: “Facts” aren’t stored in a single layer but are embedded throughout both attention heads and MLP layers.
• Debate on Attention: While some see attention weights as interpretable, a growing view is that real “knowledge” is diffused across the network’s parameters.

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/mla-21

• Raybeam and Databricks: Ming Chang from Raybeam discusses Raybeam's focus on data science and analytics, and how their recent acquisition by Dept Agency has expanded their scope into ML Ops and AI. Raybeam often utilizes Databricks due to its comprehensive nature.

• Understanding Databricks: Contrary to initial assumptions, Databricks is not just an analytics platform like Tableau but an ML Ops platform competing with tools like SageMaker and Kubeflow. It offers functionalities for creating notebooks, executing Python code, and using a hosted Spark cluster and Delta Lake for data storage.

• Choosing the Right MLOps Tool: Depending on client requirements, Raybeam might recommend different tools. Decision factors include client's existing expertise, infrastructure needs, and scaling challenges. Databricks is often recommended for its ease of use and features.

• Databricks Features: Offers a hosted solution for Spark clusters on AWS, Azure, or GCP; integrates with IDEs like VSCode through Databricks Connect; provides a unique Git integration for version control of notebooks; and utilizes Delta Lake for version control of Parquet files, enhancing operations like edit and delete.

• Parquet and Delta Lake: Parquet files are optimized for big data, and Delta Lake provides transaction-like operations over Parquet by maintaining version history.

• Pricing and Usage: Databricks adds a nominal fee on top of cloud provider charges. It's accessible for single developers and startups, making it suitable for various scales of operations.

• Ming Chang's Picks: Discusses interests in automated stock trading projects and building drones with Raspberry Pi, highlighting the intersection of programming and physical computing.

Additional Resources

• Databricks Homepage
• Delta Lake on Databricks
• Parquet Format
• Raybeam Overview
• MLFlow Documentation

For a hands-on look at Ming Chang's drone project, follow his developments or connect for insights on building a Raspberry Pi-powered drone.

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/mla-20

Conversation with Dirk-Jan Kubeflow (vs cloud native solutions like SageMaker)

Dirk-Jan Verdoorn - Data Scientist at Dept Agency

Kubeflow. (From the website:) The Machine Learning Toolkit for Kubernetes. The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be able to run Kubeflow.

TensorFlow Extended (TFX). If using TensorFlow with Kubeflow, combine with TFX for maximum power. (From the website:) TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines. When you're ready to move your models from research to production, use TFX to create and manage a production pipeline.

Alternatives:

• Airflow
• MLflow

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/mla-19

Chatting with co-workers about the role of DevOps in a machine learning engineer's life

Expert coworkers at Dept

• Matt Merrill - Principal Software Developer
• Jirawat Uttayaya - DevOps Lead
• The Ship It Podcast (where Matt features often)

Devops tools

• Terraform
• Ansible

Pictures (funny and serious)

• Which AWS container service should I use?
• A visual guide on troubleshooting Kubernetes deployments
• Public Cloud Services Comparison
• Killed by Google
• aCloudGuru AWS curriculum

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/mla-18

(Optional episode) just showcasing a cool application using machine learning

Dept uses Descript for some of their podcasting. I'm using it like a maniac, I think they're surprised at how into it I am. Check out the transcript & see how it performed.

• Descript
• The Ship It Podcast How to ship software, from the front lines. We talk with software developers about their craft, developer tools, developer productivity and what makes software development awesome. Hosted by your friends at Rocket Insights. AKA shipit.io
• Brandbeats Podcast by BASIC An agency podcast with views on design, technology, art, and culture. Explore the new microsite at www.brandbeats.basicagency.com

Try a walking desk to stay healthy while you study or work!

Show notes: ocdevel.com/mlg/mla-17

Developing on AWS first (SageMaker or other)

Consider developing against AWS as your local development environment, rather than only your cloud deployment environment. Solutions:

1. Stick to AWS Cloud IDEs (Lambda, SageMaker Studio, Cloud9

2. Connect to deployed infrastructure via Client VPN

   • Terraform example
   • YouTube tutorial
   • Creating the keys
3. LocalStack

Infrastructure as Code

• Terraform
• CDK
• Serverless

Try a walking desk to stay healthy while you study or work!

Full note at ocdevel.com/mlg/mla-16

Part 2 of deploying your ML models to the cloud with SageMaker (MLOps)

MLOps is deploying your ML models to the cloud. See MadeWithML for an overview of tooling (also generally a great ML educational run-down.)

• SageMaker
• Jumpstart
• Deploy
• Pipelines
• Monitor
• Kubernetes
• Neo