Case study 02 · Recovery and governed delivery
The reconstruction
A production platform at a higher-education institution, offline. The public face of the institution and its primary channel to prospective students.
- 13 days to recovery
- 0 lines of original source
- 39 routes recovered
- 34 collections mapped
- 17 engineering documents
- Repository in client custody
The state it was found in
The immediate cause was a change to the SSL configuration made outside any change process. The deeper problem surfaced within a day of looking properly: the platform existed only as a compiled runtime. No original source code existed anywhere — not in a repository, not in a handover archive, not with anyone. The running system was the only copy of itself.
A system in that state cannot be safely repaired. Every future change carries the same risk as the one that broke it, because nobody can read what they're changing.
The recovery doctrine
Step 1
Stabilise
Get the institution back to a defensible position without touching the evidence.
Step 2
Preserve
Full artefact extraction from production — front end, back end, database, infrastructure configuration — read-only, catalogued. Nearly fifty thousand recovery files verified.
Step 3
Document
Decode the compiled bundles into a written specification: thirty-nine routes, thirty-four database collections, the deployment shape.
Step 4
Rebuild
Isolated environment, every version pinned, rebuilt to correct design rather than to match the defects.
That order is not procedural politeness. You cannot diagnose a system you have already started fixing, and the moment you touch production you have destroyed the only record of what actually failed.
The decision
Patch the binary, or reconstruct.
Patching would have been faster and cheaper that month, and wrong. I recommended reconstruction and said plainly it would cost more in the short term. The argument was straightforward: an institution that cannot read its own platform does not own it.
Reverse engineering, method by method
Reconstruction from a compiled runtime is archaeology with a deadline.
The compiled bundles were the specification, so they were read as one — every route recovered from the compiled chunks, thirty-nine of them, decoded rather than guessed at. The database, thirty-four collections of it, gave the content model. The infrastructure archives gave the deployment shape. Between them, the system's actual behaviour could be reconstructed as a written specification, which is the thing that had never existed.
An isolated rebuild environment.
Reconstruction never happens where production lives.
Every version pinned. "Latest" is banned.
The original platform's exact framework and CMS versions were identified and locked. Every version number is load-bearing — this is the discipline whose absence caused the outage in the first place.
Rebuilt to correct design, not to match the defects.
Reproducing production faithfully would have reproduced its faults. The reconstruction fixed what was wrong while preserving what the institution actually depended on.
AI agents with defined roles and human gates.
Agents with scoped responsibilities, a human approval gate on every shared component, page-by-page verification against a design source, and regression on every change. The agents produced their own documentation as they worked, which is the only reliable way documentation gets written.
Where it stands
Full-stack reconstruction, front end and back end, delivered and in production.
Seventeen engineering documents covering deployment, recovery, operations and architecture. A private repository under the institution's control with a tagged baseline and independent backups. Containerised deployment with documented rollback and recovery procedures. A published incident and recovery record.
The institution now holds something it did not hold before the outage: its own platform, in a form it can read.
What it taught
Source code is an asset. Control it from day one.
If someone else holds the only copy, you don't own your platform. You rent it, and you'll find out on the worst possible day.
Change management isn't bureaucracy.
The process that would have prevented this costs ten minutes. Its absence cost thirteen days.
Preservation before repair.
The instinct to fix immediately is the instinct that destroys the evidence.
AI-augmented delivery is real — under governance.
Defined roles, human gates, pinned versions, continuous verification. The same tools without that discipline produce fast, confident garbage.
If your platform exists only in production, we should talk.