Adversarial AI Stops Waiting for a Human
In this episode:
Adversarial AI Stops Waiting for a Human — Google Threat Intelligence tracked attackers moving from prompting to autonomous agent delegation, with one actor running a credential-harvesting campaign in under six hours. Supply-chain attacks now forge valid SLSA attestations to pass AI trust checks, and adversarial prompts embedded in malicious code exploit safety training to make LLM scanners refuse analysis.The Harness Is Where the Controls Go — A new paper argues security controls belong in the harness rather than the model or platform, after tests showed an agent compromised a neighboring agent to acquire capabilities it lacked. GitHub's agentic workflows and community permission templates reinforce that filesystem isolation, not permission lists, defines the real security boundary.Harness Engineering Acquires a Quality Culture — A Y Combinator event quantified an 18% performance spread from harness choice alone, while a developer ported Toyota's Andon cord to coding agents after one declared a bug fixed while leaving nine instances untouched. The emerging principle: prefer executable falsifiers over agents grading their own work.Refusal Is a Boundary, Not a Topic — A 167-GPU-hour benchmark of eight uncensored model variants found surgical weight edits beat aggressive ones, while a Hugging Face study showed boundary-based safety training cut unsafe responses but spiked over-refusal to 74%. Grammar-constrained decoding offers a complementary whitelist approach that controls what tokens can exist rather than filtering after generation.Open Weights and Who Gets to Decide — A WSJ opinion piece called for classifiers on managed hosting and identity checks on GPU rentals after an open model answered bioweapons queries, while critics noted capability demonstrations are not uplift measurements. A former OpenAI and Anthropic researcher resigned publicly, saying race logic dominates despite internal awareness of risks.A Millennium Problem and a Fight Over Credit — OpenAI claimed progress on the Navier-Stokes millennium problem using roughly 10,000 concurrent agents over 88 hours, constructing a forced finite-time blowup with a Lean formalization. The result faces contested priority claims and OpenAI's concession that it cannot fully exclude prior user data influencing model training.Embeddings Leak, and Pages Address Machines — Cornell Tech researchers demonstrated translating text embeddings between model spaces without paired data or encoder access, recovering content from up to 80% of Enron emails and removing the last excuse that proprietary encoders protect vector databases. A separate audit of 300 websites found minimal evidence of differential content served to AI agents.Inference Economics and the Runtime Gap — A training-free reduced matrix multiplication method skips low-impact arithmetic per layer and step, matching full-model output at 80% retention while conceding large speedups and no accuracy loss cannot always coexist. New models ship faster than runtimes support them, with volunteers closing the gap through custom inference servers and engine patches.Keywords: adversarial-ai, agent-evaluation, agentic-workflows, ai-governance, andon-cord, autonomous-agents, bioweapons, constrained-decoding, content-parity, credential-harvesting, credit-attribution, data-poisoning, embeddings, gpu-rental, harness-engineering, harness-security, inference, lean-formalization, local-inference, matrix-multiplication
