Frontier AI Architect

Legal research tools built on retrieval and on some of the best document collections in any industry still hallucinated on 17% to 33% of test queries. This episode explains where retrieval breaks and what a production agent needs on top of it.

In this episode:

- What the Stanford and Yale study of AI legal research tools found

- Why similarity search fails on multi-document and corpus-wide questions

- Three layers: better retrieval, graph structure and agent memory, with the cost of each

- What the GraphRAG evidence shows, and what an independent benchmark added

- The case that long context windows make most of this unnecessary

Read the full article with all sources: https://substack.com/@frontieraiarchitect/note/p-221333964?r=978zej

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

A fake Postmark package copied every email its users sent through an AI assistant. This episode explains why the protocol that connects agents to tools cannot enforce trust, and what a real boundary looks like.

In this episode:

- The postmark-mcp backdoor and what the npm record shows

- Three ways a server turns on you: poisoned descriptions, the rug pull, shadowing

- What independent benchmarks measured, and how far to trust them

- Shadow AI as the way these servers get in

- Five controls and their costs, and what to build at each company stage

Read the full article with all sources: https://substack.com/@frontieraiarchitect/note/p-220957777?r=978zej

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

Stolen OAuth tokens from an AI chatbot opened hundreds of Salesforce tenants, and every call looked legitimate. This episode explains how to give agents an identity of their own and credentials that are short-lived, narrow and attributable.

In this episode:

- What happened in the Salesloft Drift breach, from the public record

- Why a shared service token and user impersonation are both the wrong answer

- Three pillars: workload identity, delegation through token exchange, enforcement at the resource

- What to build at seed, at Series A and B, and at enterprise scale

- The honest caveat: short-lived tokens would not have prevented the intrusion

Read the full article with all sources: https://substack.com/@frontieraiarchitect/note/p-220524477?r=978zej

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

One instruction in a system prompt measurably degraded a coding product at the company that built the model. This episode explains how to build the test suite that catches that kind of change, and why that suite is the asset competitors cannot copy.

In this episode:

- What three public postmortems from Anthropic and OpenAI have in common

- The four layers: code checks, a calibrated judge, a regression suite, a gate on every change

- Why a model judge has to be measured before it is trusted

- How a suite turns a forced model migration from weeks into days

- The honest caveat: every one of those incidents passed the evals that existed

Read the full article with all sources: https://frontieraiarchitect.substack.com/p/evals-driven-development-your-test?utm_source=share&utm_medium=android&r=978zej

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

A single email made Microsoft 365 Copilot leak data, past three layers of filtering. This episode argues that agent security is about what authority the model's output carries, not about catching bad prompts.

In this episode:

- How EchoLeak worked, and why every defense it passed was a filter

- What happened when researchers attacked 12 published defenses adaptively

- The confused deputy problem from 1988 and why it describes today's agents

- Five controls that hold when the model is fooled, and what each one costs

- The one diligence question investors can ask about any agent

Read the full article with all sources: [LINK]

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

Salesforce priced its AI agents three different ways in about a year and ended up close to a seat again. This episode explains what that teaches anyone who has to put a price on an AI product.

In this episode:

- Why AI breaks the link between headcount and revenue

- Consumption, workflow and outcome pricing, and where each one breaks

- What Intercom Fin's billing rules reveal about the real cost of outcome pricing

- The hybrid model most contracts end up with, and how to compute a margin floor

- The evidence that the seat is winning again

Read the full article with all sources:

https://substack.com/@frontieraiarchitect/note/p-219471997?r=978zej

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

Klarna claimed its AI assistant did the work of 700 agents, then went back to hiring people, then claimed even bigger savings. This episode uses that story to explain the shift from selling software tools to selling finished work.

In this episode:

- Why a copilot is priced as a tool and an autopilot is priced against payroll

- What Sequoia's $10 trillion figure does and does not measure

- Why consistency, not average accuracy, decides whether work can be delegated

- The exception architecture: a deterministic gate, risk tiers and a human queue

- The strongest case for staying a tool company

Read the full article with all sources: https://frontieraiarchitect.substack.com/p/services-as-software-the-10-trillion?r=978zej&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Frontier AI Architect is written by David Amar.

LinkedIn: https://www.linkedin.com/in/david-amar-575192441

Newsletter: https://frontieraiarchitect.substack.com

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.

In October 2022 Jasper raised $125M at a $1.5B valuation. About six weeks later its own supplier released ChatGPT for free. This episode is a teardown of the wrapper trap: why a product whose value is the model is replaceable, and what makes an AI product hard to remove.

We cover:

- Why falling model prices help your customer as much as they help you

- The four pillars of structural defensibility, and where each one breaks

- The honest case for starting as a wrapper

- A Monday morning checklist for founders and CTOs

Read the full article with all sources: https://substack.com/@frontieraiarchitect/p-218787354

Follow us on LinkedIn:

https://www.linkedin.com/in/david-amar-575192441

Frontier AI Architect publishes three architectural teardowns a week for founders, engineering leaders and investors.

Contact: [email protected]

This episode's audio is AI-generated, based on an article written by David Amar.