Kevlin Henney (independent consultant, keynote speaker, editor of the “97 Things” series, among other things) joined Ralf D. Müller for a wide-ranging conversation on where generative AI is actually taking software development.

Below are some bullet point notes taken during that event. These are not intended to be way to skip checking the event at all. I fully recommended watching the while thing. It was an awesome conversation. It’s available here.

The error screen collection

Kevlin is known for collecting photos of public software failures such as crash screens, stack traces, exposed endpoints, etc.
Today he framed those as “accidental installation art”, which was a rather interesting way to describe it.
When software crashes, it reveals how it was built: tech stack, what was tested, what wasn’t. And from a security angle failures are often indistinguishable from attacks.

Testing in software development

Testing has always mattered but the industry has just been slow to accept it, especially in late 90’s, early 2000’s.
His prediction years ago was that AI-generated code would need more testing, not less.
Current increase on AI adoption has been showing an increase in bugs too. Testing is even more critical now.
There was also time to discuss the 80% test coverage and how that’s a joke. It’s easy to trick the metrics by testing getters and setters to meet the target.

Failure demand vs Value demand

Most developer work is failure demand (fixing bugs and poor quality code), not value demand (what customers really need).
If good code would take 30 minutes but bad code takes a week to understand, that extra time is all failure demand.
Ticket-driven development was already a dysfunction before AI. And AI is a multiplier so it just amplifies whatever culture and competence already exist.
Therefore, good teams get better and dysfunctional teams get more dysfunctional.
AI is being used to create more “legacy code” instead of reducing it. So failure demand is being multiplied, not eliminated.
Java given as an example of a language that had refactoring tools for decades but legacy code still exists. It was never a lack of tooling problem.
Instead of producing more and faster, we should be using AI to do things that we couldn’t do before.
Adressing legacy code, improving coverage and documentation quality, etc. is what ultimately shifts from feature demand to value demand.

Why AI adoption might not be working out as expected

Some companies have been adopting AI not because of identified needs but because of the hype around it.
Most developers are doing worse than before AI adoption. They just don’t realized it yet.
A lot of people using AI are not developers. “meat proxying” (taking the LLM output and passing it into the world without adding anything) is now widespread.
And again the multiplier framing: AI doesn’t make you the person doing it well, rather it just multiplies what you already are.
Sprint metaphor in software development is wrong. It’s a long-distance race, not a 100-meter sprint.
Organizations heard “sprint” and imeddiately went for “go faster” ignoring everything else.
Sudden speed makes us hit a wall where AI stops helping. Teams drop to normal speed if not slower.
Many companies have a “foot on the brake and on the accelerator at the same time”. Taking the foot of the brake is what really would help.
Identifying the brakes in the organization is crucial. Then ask if AI can help with that, or not.

Using AI for software architecture and AI limitations

Architecture is the sum of decisions, but decisions require options; a decision without options is just a constraint.
Most people anchor on the first solution generated instead of using AI to explore options.
- Quote: “There is nothing more dangerous than an idea when you have only one idea” (Émile-Auguste Chartier)
LLMs are like humans. They work best with small, loosely coupled, well-named, clearly structured code.

The Next Generation

The next generation does not need to code the way previous generations did.
They must be able to read the code otherwise they won’t be able to assess what’s being generated on their behalf.
Understanding the code enables: knowing what you have, knowing how to change it, knowing what to do when things go wrong

See you in the next post.