SourceShift
Engineering notes from the SourceShift team. Post-mortems, LLM gateway scars, and the occasional working theory — drafted from real production fires by the engineers running them. No newsletter, no popups, no tracking.
- 2026aug 20"Nothing Got Checked" Is Not "Pass": The Vacuous Verdict
The cheapest, highest-yield honesty rule in an agent orchestrator: a run that executes cleanly but proves nothing gets stamped amber, never green. 23 real runs in a live database show why exit code zero is not evidence.
- 2026jul 14The Self-Report Gap: I Audited 4,785 Agent Claims Against What Actually Ran
An agent claimed a verification it never ran — 90% of the time. I measured it against the tool-call record, and the culprit turned out to be the trigger, not the model.
- 2026jun 23The orchestrator that learns which model to trust
You're running five AI models behind one workflow and overpaying for the wrong one on every job, because nothing measures which is actually best. mini-ork is a CLI orchestrator that makes models compete, grades the results, and re-routes the next job to whoever won — so your routing gets cheaper and better on its own instead of staying frozen at whatever you guessed on day one.
- 2026jun 17Coordinating a fleet of LLM agents on one codebase
Point three coding agents at the same repo and they overwrite each other within minutes. The fix is not to make them talk; it is to give them an external referee. We built one using six ideas that are 30+ years old.
- 2026jun 15You can't align what you haven't measured
A Vapnik decision boundary, an RL exploration result, and a Hubinger sleeper-agent finding all say the same thing: training must include the failure mode. The agentic-era addition: train an agent to elicit it, because the human red-team budget runs out before the action space does.
- 2026jun 11Fable may be smarter. Show me the token bill.
A critique of Anthropic-style intelligence claims: if the gain comes from spending more test-time compute, publish the cost frontier.
