3 posts tagged with "AI Engineering"

Releasing model weights is a one-way door, and the model behind it is a room no one can read. Those two facts — irreversibility and inscrutability — sit underneath the most-quoted thing Dario Amodei has ever said about open models, that they are heading down a "dangerous path." A 2023 clip of Anthropic's CEO warning the U.S. Senate resurfaced on Hacker News this month, and the top comment wrote the dunk for everyone: these tools will become dangerously powerful, which is why nobody should be allowed to have them except by buying them from me. It is an easy laugh. The actual argument is more careful than the clip, the evidence behind it is thinner than Anthropic implies, and the way 2026 has judged it is sharper than either side expected — because the regulatory lever Amodei spent years asking for came home in June 2026 as an export-control order that switched off Anthropic's own flagship models.

This is the long version. It runs through what "open" earned the right to mean across forty years of software; what Amodei actually argues, in his own essays rather than the meme; what the biosecurity studies actually found; and why the closed, "safe" path turned out to be the one with a government-sized switch on it.

Claude 3.5 Sonnet rates its own outputs approximately 25% higher than a human panel would. GPT-4 gives itself a 10% boost. Swap the order of two candidate responses in a pairwise comparison, and the verdict flips in 10--30% of cases -- not because the quality changed, but because the judge has a position preference it cannot override.

These are not edge cases. They are the default behavior of every LLM-as-judge pipeline that ships without explicit mitigation. And most ship without it.

LLM-as-judge -- the practice of using a capable large language model to score or compare outputs from another LLM -- has become the dominant evaluation method for production AI systems. 53.3% of teams with deployed AI agents now use it, according to LangChain's 2025 State of AI Agents survey. The economics are compelling: 80% agreement with human preferences at 500x--5,000x lower cost. But agreement rates and cost savings obscure a deeper problem. Most teams adopt the method, measure the savings, and never measure the biases. The result is evaluation infrastructure that looks automated but is quietly wrong in systematic, reproducible ways.

This article covers the mechanism, the research, and the biases that break LLM judges in production.

What is LLM as a judge? LLM-as-a-Judge is an evaluation methodology where a capable large language model scores or compares outputs from another LLM application against defined criteria -- such as helpfulness, factual accuracy, and relevance -- using structured prompts that request chain-of-thought reasoning before a final score. The method achieves approximately 80% agreement with human evaluators, matching human-to-human consistency, at 500x--5,000x lower cost than manual review.

We built an Architect agent — a fully autonomous code reviewer powered by Claude Opus 4.6 — that explores a repository, runs audits, and produces a comprehensive architecture report. One command, zero human intervention, a professional-grade review in under 10 minutes.

This article covers how the agent is structured, how it leverages Anthropic's agentic tool-use loop, and what we learned shipping it.