In this episode, we explore how Udemy tackled the tricky challenge of understanding learner intent in their AI Assistant — from a simple similarity-based embedding model, through experiments with larger models and fine-tuning, to a hybrid system that intelligently leverages both embeddings and large language model classification. This evolution demonstrates how real-world ML systems often require balancing accuracy, cost, latency, and user experience — especially in AI features that directly interact with users.
For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/udemy-engineering/evolution-of-the-udemy-ai-assistant-intent-understanding-system-ec3ee0039364