How We’re Learning to Build Maintainable Software with AI
The ability to generate software and the ability to maintain control of it are very different things. Lessons from ForgeOS — and the engineering process Epistudious now uses to keep AI-built software understandable, verifiable and owned by the business.
Every few weeks, a new model demonstrates that it can build something in one attempt that would have taken a software team days or weeks not long ago.
Entire applications appear from prompts. Playable games are generated in a single session. Coding agents work autonomously for hours, adding features, fixing tests and refactoring their own work.
It is remarkable progress.
But if you are responsible for a business, the most important question often starts the morning after the demo.
Who understands what it built?
Can someone explain the architecture? Do you know which decisions were deliberate and which were simply choices the model made along the way? If a business process changes six months from now, can you determine what else will be affected? Does your delivery team understand it? Can your support team operate it? Do you know what technical, legal or financial risks were introduced?
And if the AI that built it suddenly became unavailable, prohibitively expensive or legally constrained, would you still understand your own software?
These are the questions we have become increasingly interested in at Epistudious.
We use AI extensively to build software. We want models to become more capable, autonomous and productive. We are actively designing our own development processes around the assumption that they will.
But we have also learned that the ability to generate software and the ability to maintain control of that software are very different things.
When speed outran understanding
We learned this most clearly through an internal experiment called ForgeOS.
We were using autonomous AI agents to develop a substantial software system. The productivity was extraordinary. We could leave agents working overnight and wake up to tens of thousands of new lines of code.
Seen purely as a demonstration of AI capability, this was exciting.
As a software engineering process, it quickly became a problem.
There was no sufficiently developed mechanism for the agents to synchronise with the human directing the project. They were building faster than we could understand what they were building.
We attempted to direct the work, but the implementation increasingly outran our mental model of the system. Eventually ForgeOS grew beyond 200,000 lines of code.
We abandoned that iteration.
- Overnight agent work
- Tens of thousands of lines
- Rapid accumulation
- 200,000+ lines
- Human understanding falls behind
- Iteration abandoned
The lesson was not that the AI was incapable of writing software. Quite the opposite. Its ability to write software was part of the problem.
The speed cost us understanding.
That experience changed how we think about AI-assisted development. We no longer consider the amount of code an AI can produce to be a particularly interesting measure of engineering success.
What matters is whether that capability remains connected to human understanding and business intent.
The speed cost us understanding.
The better AI gets, the more important that becomes
There is an understandable fascination with “one-shotting” software.
It is a useful way to demonstrate model capability. Give a model a prompt, watch it create an application, run it, and see what happens.
There is also an economy built around those demonstrations. A technically impressive result makes a good video, post or benchmark.
The incentives for a software business are different.
The creator of a demonstration needs the application to work long enough to demonstrate what the model can do.
A business may have to operate that application for ten years.
Those are very different success criteria.
As models improve, we expect them to one-shot increasingly large pieces of software. They will make more implementation decisions independently. They will test more of their own work. They will solve problems that previously required several engineers.
That makes the human role more important in a different way.
If an AI can produce ten times as much work before it needs you, then it can also travel ten times further in a direction you did not intend.
Sometimes that direction is technically poor.
More interestingly, sometimes it is technically excellent and commercially pointless.
We have repeatedly seen capable models attempt to harden systems against extraordinarily unlikely edge cases. From the model’s local perspective, this can look like diligent engineering. From the perspective of a business trying to deliver a useful product, several days of work protecting against a scenario with a 0.01% chance of occurring may be an irrational use of resources.
We generally prefer a Pareto approach. We want the majority of the available value without automatically spending the majority of the project hardening the final few percentage points.
We see the same problem in product design. An agent can produce a technically sophisticated solution for a user who never wanted that complexity in the first place.
More capability does not automatically mean better judgement about where effort should be spent.
That judgement still has to come from somewhere.
Engineers are moving up a level
We increasingly think about AI engineering teams in a way that resembles software organisations from twenty years ago.
A programme manager did not review every line written by every developer.
They needed a different kind of understanding.
They needed to know what the engineering team was doing, whether the work aligned with the customer’s needs, what problems were emerging, what decisions needed to be made and whether the organisation was still going to deliver what it had promised.
When something went wrong, they had to be sufficiently plugged into the engineering work to represent both the team and the company.
That is increasingly how we see the role of experienced engineers working with AI.
The agents are becoming the engineering team.
The human moves up a level.
This does not mean disengaging from engineering. It means concentrating human attention where judgement has the highest leverage.
Traditional
- Business / client
- Programme / engineering leadership
- Software engineering team
- Implementation
AI-native
- Business / client
- Experienced human engineer / technical lead
- AI engineering agents
- Implementation + tests
We are comfortable allowing AI to make a large number of non-creative technical decisions, provided those decisions remain understandable, reviewable and adjustable.
What we do not want is an opaque engineering organisation operating beneath us.
Few business owners would hire a team of twenty software engineers, put nobody in charge of them, provide a broad objective and then disappear for six months.
Yet that is surprisingly close to how some people imagine autonomous AI development should work.
“Look what my engineers built.”
How does it work?
“I don’t know.”
Why was it built this way?
“I don’t know.”
Can we safely change this process?
“I suppose we can ask them.”
What happens if those engineers disappear?
That last question is not hypothetical in traditional software businesses.
We have seen what happens when the knowledge leaves
Our experience building software stretches back more than thirty years, and one recurring failure mode has been remarkably consistent.
Sometimes an entire engineering team leaves a company.
Sometimes it only takes one person.
A technical lead, architect or senior engineer moves on, and suddenly the business discovers how much of the product existed inside that person’s head.
The company may legally own the intellectual property. It may possess every line of source code.
But it does not possess the understanding.
Why was this subsystem designed this way?
Which customer requirement caused that constraint?
Which ugly-looking workaround is actually protecting an important process?
Which dependencies are safe to replace?
Which assumptions are still valid?
In extreme cases, products have been abandoned or substantially restarted because nobody remaining could confidently take ownership of them.
AI creates a new version of the same problem, and it can happen much faster.
A capable model reasons through a problem, considers alternatives, makes architectural and implementation choices, writes the code and moves on.
Unless the development process deliberately captures that reasoning, much of the intellectual work that produced the implementation disappears.
The code remains.
The thinking often does not.
And as agents become capable of doing more in a single session, the amount of knowledge that can disappear with them increases.
The code remains. The thinking often does not.
A repository is not organisational memory
This is why we do not think maintainability can be reduced to documentation after the fact.
Nor do we think the answer is simply to produce more documents and feed them into a retrieval system.
We are increasingly interested in preserving project understanding rather than just project text.
That means recording discussions and architectural decisions, but also understanding their relationships.
Where did this requirement originate?
What evidence supports this statement?
Is it an observation, a constraint, a decision or an assumption?
Does something else conflict with it?
Has a later decision superseded it?
Which version is authoritative now?
What remains unresolved?
What downstream parts of the system depend on it?
This thinking has heavily influenced the Project Memory system we are developing internally.
The goal is not merely to make old information searchable. It is to preserve enough structure, provenance and authority that a future human or AI can reconstruct what the project currently believes and why.
Document retrieval
- Documents
- Chunks
- Search / retrieval
Access to everything the project has written — without what it means.
Project Memory
Connected into a coherent, authority-aware understanding of the project as it stands now.
That distinction matters.
Having access to every document a project has ever created does not mean you understand the project.
Human review also has to change
There is another consequence of increasing agent capability: traditional code review cannot remain the primary human control mechanism.
Agents are becoming much better at writing tests, executing them, diagnosing failures and correcting their own implementations.
If an agent can produce thousands of lines of code and validate substantial portions of its own work, it makes little sense to respond by requiring a human to manually retrace every implementation decision line by line.
The human review surface has to move upward too.
We increasingly care about validating outcomes and validating where effort goes next.
Our development process therefore includes direct reporting from agents that bubbles upward to the human regularly. We use daily check-ins to understand what happened, examine proposals, make decisions and observe how those decisions alter the overall project.
Those meetings are not a wall of agent-generated text followed by a human saying “OK”.
They are control points.
What did we learn?
What is being proposed?
What decisions require human judgement?
What has changed?
What does that do to the rest of the project?
Where should effort go next?
We use a Delivery Map to make those consequences visible so that decisions can be evaluated against the wider deliverable rather than in isolation.
- AI agents implement
- AI agents test / diagnose / iterate
- Results + proposals surface
- Human evaluates outcomes and consequences
- Decision
- Delivery Map updates
- Next work
A continuous loop — next work returns to the agents with updated human direction.
This is one of the most direct changes we made after ForgeOS.
The answer to agents becoming more productive was not to try to read faster.
It was to formalise synchronisation.
Maintainability is a business property
There is a tendency to discuss maintainable software primarily in terms of code quality.
Those things matter. Good architecture matters. Tests matter. Sensible dependencies matter.
But from a business owner’s perspective, maintainability is broader.
A business should understand every important stage between the initial proposal and the final deliverable.
It should understand what was built, why it was built, what assumptions it relies upon and what risks were introduced along the way.
Those risks may be technical.
They may involve security or data handling.
They may involve software licences, intellectual property, regulatory obligations or dependencies on particular providers.
They may be financial. A technically excellent architecture that costs more to operate than the value it produces is still a bad business outcome.
And they may be operational.
If you change an important business process, do you understand what that change will affect downstream?
This becomes especially important when the natural response is simply to ask an AI to make the change.
Suppose you provide the model with all of the context you personally know about the system.
Is that enough?
If you do not understand the full picture yourself, how can you know that the AI received the full picture?
A frontier model may be perfectly capable of implementing exactly the change you requested while being unaware of a constraint, dependency or historical decision that nobody thought to provide.
The problem is no longer whether the AI is smart enough to modify the code.
The problem is whether the organisation is capable of giving that intelligence a sufficiently complete understanding of its own system.
Owning the code is not enough
This is also why we think businesses should consider what happens if access to frontier AI changes.
Today it is easy to assume that tomorrow’s models will simply be better, cheaper and more available.
Perhaps they will be.
But businesses generally do not build serious operational systems on the assumption that an important external dependency can never change.
Frontier AI could become substantially more expensive. Providers could change. Regulations could constrain particular uses. A major geopolitical or regulatory event could result in governments restricting access to advanced systems.
The exact scenario matters less than the principle.
If the intelligence that built your software becomes unavailable, does your organisation still understand its own system?
Owning a Git repository does not answer that question.
We think genuine ownership means possessing enough of the architecture, decisions, provenance, risks, dependencies and operating knowledge to continue making informed decisions about the system.
The AI can help create and maintain that knowledge.
But the organisation has to deliberately keep it.
How much useful complexity can we allow AI to create while the organisation remains able to understand, verify, operate and change it?
The limiting factor may be the person directing the AI
If current trends continue, models will almost certainly build more software with less direct human implementation.
We do not think that makes engineering knowledge less valuable.
It may make it considerably more leveraged.
The quality of the outcome increasingly depends on the intellect and knowledge of the person directing the AI.
Someone who understands engineering, delivery, customers, economics and risk can use powerful agents to produce an extraordinary amount of valuable work.
Someone who does not understand those things can also produce an extraordinary amount of work.
The second outcome can be expensive.
Tokens are burned. Software is produced that a delivery team cannot reliably deliver. Support teams inherit systems they do not understand. Operating economics do not make sense. Customers receive technically impressive capabilities that do not solve problems they actually have.
Or everything appears successful initially and the company quietly accumulates risk that only becomes visible later.
This is why we are much less interested in asking how much software the latest model can one-shot.
We already expect that number to keep increasing.
The benchmark we care about is different:
How much useful complexity can we allow AI to create while the organisation remains able to understand, verify, operate and change it?
For us, building maintainable software with AI means designing the engineering process around that question.
The agents can write the code.
They can increasingly test it, diagnose it and maintain it too.
Our job is increasingly less about making AI capable of building software and more about making sure the business understands what its AI has built — and remains capable of directing what it builds next.
If your organisation is starting to build software faster than it can understand it, contact us.