Three organisations. Three different tech stacks. One month. The same pattern, repeated each time.
Someone has an idea. They open Claude, ChatGPT, or Copilot and ask what they should build. The AI validates the idea, suggests an architecture, and starts sketching components. It sounds like a very senior engineer who has thought deeply about the problem. It has not thought about the problem at all.
That gap is the whole problem.
The attaboy machine
AI agents are pathologically agreeable. Ask Claude if your idea is good and it says yes. Ask if microservices make sense for your three-person team and it explains why microservices are an excellent choice. Ask if you should build a custom ML pipeline instead of using a managed service and it lays out the design with enthusiasm.
It is not lying. It is pattern-matching against training data and producing the most plausible-sounding response. But plausible is not the same as right for your situation.
The skill that makes a real architect valuable is not designing systems. It is knowing which systems not to build. It is pushing back on complexity. It is asking why five times until the actual requirement surfaces from under the aspirational nonsense. It is telling the CTO that their conference-inspired idea is a terrible fit for the team they actually have.
Claude will never do that. It is trained to be helpful. Helpful means agreeable. Agreeable means you get a Jenga tower that passes for architecture.
What the Jenga tower looks like
The AI-designed architecture is technically sound on its face. The components make sense in isolation. The patterns are recognisable: event-driven here, CQRS there, a service mesh added in for good measure. It passes the squint test.
But it was not designed for your team. It was not designed for your constraints. It was not designed for your production environment, with its VPC lockdowns, legacy integrations, and the operational realities the AI never asked about because it never asks about anything uncomfortable.
When it falls over, the AI will not be carrying the bag. Someone on your team will.
What to do today
Use AI agents for what they are genuinely good at: implementation, code generation, drafting, acceleration. They are brilliant at that layer.
Do not let them own decisions about what to build or how to structure the system. Those decisions require someone who can say no, who knows your team's actual capabilities, and who will still be around when the architecture has to survive contact with production.
If you do not have that person internally, find one before you let the AI write your Jira tickets. The confidence of the output is not a substitute for the judgment behind it.