Rebuilding Lost Places: What a Leonardo Sketch and Ground-Penetrating Radar Teach About AI Reconstruction

In 1460, Francesco Sforza seized the ruins of a medieval fortress in northern Milan and rebuilt it into a ducal residence. A legend followed the building for centuries: a secret network of passages beneath its foundations. The legend had a strange feature. One of the few pieces of evidence for it was a sketch by Leonardo da Vinci, who worked at the castle for Ludovico Sforza and recorded the passages in the Codex Forster I in the late 15th century.

Between 2021 and 2023, a team from the Politecnico di Milano, working with the castle itself and the geophysics firm Codevintec, mapped the subsurface with ground-penetrating radar and laser scanners. Announced in January 2025, the results confirmed voids and walkways at depths of a few decimeters, in places matching Leonardo's drawings, including a plausible route toward the Basilica of Santa Maria delle Grazie, where Ludovico's wife Beatrice d'Este is buried. Research has continued into 2026 with scientific publications on the castle's digital model.

For heritage people, this is a great archaeology story. For anyone who builds visual work with generative AI, it is something more useful: a clean, public example of the hardest category of reconstruction problem, reconstructing a place from partial evidence, and the project's stated next step is the part worth studying.

The digital twin problem

Franco Guzzetti, the geomatics professor on the project, described the goal plainly: a digital model that does not just represent the castle's current appearance, but lets you explore the past, "recovering historical elements that are no longer visible." The team plans to pair the model with augmented reality so visitors can walk the tunnels physically nobody can enter.

Here is what the instruments actually deliver. Radar says "there is a void, this deep, this wide, running this direction." Laser scanning says "the surviving walls look like this." Leonardo's sketch says "in the 1490s, someone drew passages in roughly this arrangement."

None of it says what the tunnel floors were paved with, where the torch sconces sat, how the brick vaults met the foundation stone, which sections collapsed and when. The measured data ends and the inferred data begins, and a walkable AR experience needs all of it, measured and inferred, rendered at the same visual resolution. The viewer cannot tell the layers apart unless the pipeline keeps them apart on purpose.

That gap between evidence and experience is where generative models now live. Image and 3D generation can fill the large unmeasured remainder convincingly, which is exactly why the discipline around them matters.

Rules the heritage project implies

The Sforza researchers did not publish a methodology for generative reconstruction, but the structure of their project points at the rules any serious reconstruction pipeline needs.

Keep measured and inferred geometry on separate layers. Radar-derived voids and scanned walls are evidence; a generated brick vault is a hypothesis. They can be rendered together, but they must be authored and versioned separately, so a later scan can replace a hypothesis without touching measurements.

Let references constrain the fills. A credible 15th-century Milanese service tunnel draws from surviving vault construction of the same period and region, not from a generic "old stone corridor" prior. Period-correct references are the difference between reconstruction and set design.

Show the seams. A heritage twin that presents inferred passages with the same confidence as scanned ones is not a research tool, it is propaganda for its own guesses. The honest version marks what is known, what is inferred, and what is invented, even when the client would prefer it didn't.

Watch the sketch problem. Leonardo's drawing is evidence, but it is also a 500-year-old copy of unknown fidelity, describing tunnels that may have been planned rather than built. Partial sources need weights, not blind trust. This is the same judgment call anyone makes when a client hands over three reference images and calls it a brand.

Why this market matters now

The heritage case is not a one-off. The same pipeline, instruments and archives in, explorable visual world out, applies to museums that cannot exhibit everything they own, to game studios building historically grounded environments, to film previsualization of period locations, and to cities selling virtual tourism.

Each of these buyers has the same constraint the Sforza project has: the source material is fragmentary, and the deliverable has to feel complete. Generative tooling collapsed the cost of the fill. What did not get cheaper is the judgment about where evidence ends, and that judgment is now the sellable skill.

If you already run reference-driven image workflows, you have most of the technical stack. Upload the scanned or photographed evidence, lock the composition to it, generate only into the documented gaps, and keep every generated layer separable so new evidence can invalidate it cleanly. That last habit, separable layers, is what separates a reconstructable project from a pretty picture when the next radar scan comes back.

The people who worked on the castle solved the sensing problem in a focused 2021-2023 survey campaign. The rendering problem they solved last. The evidentiary problem, what to trust and how to show uncertainty, is the one they left open for whoever builds these experiences next.