Scalpels in the Lab

The first Scalpels in the Lab episode, from CI Week. Ed Grosvenor is reworking continuous integration for the way we build software now, starting with a seven-year-old client Laravel app with tens of thousands of tests and a suite that takes 25 to 30 minutes, even in parallel.

Instead of handing the suite to an agent and asking it to cut 30%, Ed built a deterministic classifier that sorts every test into three buckets. The gate runs on every pull request and blocks merging: money, security, data loss, tenant isolation, critical journeys. Nightly holds the brittle, implementation-bound tests, run on a schedule, with failures opened as issues. Delete is for tests that can never fail or just assert a line of copy. On this repo, about 60% of the tests moved out of the gate, and only 1% were marked for deletion, which says the delete rules need loosening.

Next up: tombstoning. Static analysis finds dead code, which gets marked to report rather than crash if it ever runs in production, while its tests are skipped. Hone and the logs catch anything that turns out to be alive. Plus what's coming tomorrow for GitHub workflows, and an open question about self-hosted runners and services like Blacksmith.

Chapters
00:00 CI Week begins
01:18 A seven-year-old suite to test on
02:18 Three buckets: gate, nightly, delete
05:14 Deterministic rules, and your own rules
07:34 First results: 60% out of the gate
09:27 Why not let an agent prune the suite?
10:35 Dead code and tombstoning
15:19 Packaging it up
17:16 Tomorrow: the GitHub workflows
19:10 Self-hosted runners and Blacksmith
22:43 Need help with CI?

Links
Scalpels: https://scalpels.app
Scalpels in the Lab: https://scalpels.app/lab
Hone: https://scalpels.app/products/hone
Wake, central logs for every Laravel app: https://scalpels.app/products/wake
Freelancers and agencies in the Scalpels directory: https://scalpels.app/partners
Artisan Build: https://artisan.build

Mentioned
Blacksmith: https://www.blacksmith.sh
Aaron Francis: https://aaronfrancis.com

Have you run CI on self-hosted runners or a service like Blacksmith? Ed would love to talk about it on a future episode.

#Laravel #CI #Testing #GitHubActions #AI #Podcast