Why does every naive data scientist who tries to predict stock prices end up depressed? Finance systematically breaks standard AI. You'll discover the four methodological pitfalls: data scarcity (10 years of daily data = only 2,500 observations—laughably insufficient), look-ahead bias (accidentally using future data), the unconditional trap (models validate but can't predict what matters), and heavy tails (the rare crashes that define risk). The analogy that sticks: "It's like having an umbrella that doesn't work when it rains." But there's a solution. Task-driven training matches the P&L of benchmark strategies instead of learning impossible 10,000-dimensional distributions. You'll hear about dynamic portfolios that spontaneously switched hedging instruments during COVID, lasso regression for cost-effective hedging, and the "Persona Ledger" method—LLM-generated synthetic data with accounting rules as constraints. Finance breaks AI, but sophisticated methodologies are fixing it.
Topics Covered
- The "naive data scientist depression": why finance breaks standard AI
- Four methodological pitfalls: data scarcity, look-ahead bias, unconditional trap, heavy tails
- Task-driven training: matching strategy P&L instead of price prediction
- Dynamic vs. static portfolios (encoding timing and regime changes)
- Lasso regression for sparse hedging (minimizing transaction costs)
- Agentic pipelines: GPU-accelerated end-to-end workflows
- LLM challenges: time travel problem, implicit investment biases, stubbornness
- Persona Ledger: LLM-generated synthetic data with stateful verification