A year of AI-assisted development: what actually changed
Twelve months of using AI tooling on client work, and an honest accounting of where it helped, where it did not, and what we changed about how we work.
We have been using AI tooling as a standard part of delivery for a year now. The hype and the backlash are both unhelpful, so here is what the year actually looked like.
Where it clearly helped
**Understanding code we did not write.** The biggest single win. Discovery on modernization work is meaningfully shorter, and that phase used to be where estimates went wrong.
**Test coverage.** The tests everyone agrees should exist - the edge cases, the error paths - now get written, because the cost of writing them dropped below the threshold where they get cut. This showed up as fewer regressions about six months in.
**Boilerplate.** Migrations, fixtures, scaffolding, the fourth CRUD resource that looks like the first three. Real hours, and hours clients were never happy paying for.
**Review throughput.** An automated pass over every diff catches the dull mistakes, so human review attention goes to design.
Where it did not help
**Architecture.** Every time we have asked for a design decision, we have got a competent-sounding average of what is on the internet. Averages are exactly wrong for the decisions that matter, which are specific to constraints a model cannot see.
**Anything needing system context.** Why a check exists, which of two paths is live, what a workaround was protecting against. It answers confidently and is sometimes wrong, which is worse than not answering.
**Debugging genuinely novel problems.** If the problem has been seen before, it helps. If it is specific to your system, you are back to measurement and thinking.
What we changed
- Generated code goes through identical review to handwritten code
- If the author cannot explain it, it does not merge
- Nothing load-bearing is accepted without verification against the running system
- Clients who need source kept out of third-party tools get an engagement that works that way
The honest summary
It is a meaningful productivity improvement concentrated in specific phases, not a step change across the board. The parts of software that were hard - deciding what to build, understanding the domain, operating it in production - are exactly as hard as they were.
Anyone selling you more than that has not shipped enough with it yet.