Hidden Florence 3D: San Pier Maggiore AR

From the National Gallery in London, the streets of Florence or your local park; step back in time and witness the former glory of the San Pier Maggiore church.

We were challenged to produce the 3D and AR content for a heritage app that virtually restores a long-destroyed Florentine church. Working directly with the academic research team, we brought the 3D model to life with accurate details, textures and lighting. At the same time, we carefully designed an augmented reality experience to fit around the surviving altarpiece artwork at the National Gallery. The end result is a powerful experience that allows anyone to walk around the interior of the church as it would have been. The app was officially unveiled at the National Gallery in November 2019.


Zubr are proud to have prepared the 3D model and developed the AR functionality and visual effects for this exciting new augmented reality heritage project. San Pier Maggiore was one of the most important churches in Florence – sadly demolished in the eighteenth century to make way for a covered market, and all that remains today is a street of the same name. Its artworks are now dispersed in collections around the world, including Jacopo di Cione’s altarpiece painted for the church’s high altar in 1371, now at the National Gallery in London. 

Use the altarpiece as your portal

For visitors to the National Gallery, Jacopo di Cione’s beautiful altarpiece artwork now invites its’ admirers to witness the grandeur of the church in which it once stood. Using the app on your iPhone or iPad, stepping up to the altarpiece will trigger a breathtaking transition, immersing you in a 3D render of the church interior. At actual scale, San Pier Maggiore is simply far too big to fit within the National Gallery’s Room 60. In fact, from the altarpiece, you’ll have to walk across four rooms – the entire length of the Sainsbury Wing – in order to reach the front door of the church!

Rebuilding the Church

Working alongside our friends at Calvium, Zubr was tasked with overhauling an existing 3D model produced by the academic research team. This included updating the structure to match the most recent research, creating authentic textures and lighting conditions, and sculpting detailed 3D assets, including the altarpiece frame, columns and capitals.

Optimisation for mobile devices was also an important part of the 3D development. Hundreds of thousands of polygons had to be stripped from unnecessarily high-detail parts of the research model, and new objects we created were optimised by baking intricate detail into normal maps. This ensures the model looks as good as it possibly can, even on devices that are three or four years old.  

Augmenting the National Gallery

Alongside the 3D model process, we worked to ensure the full-sized representation of the church would come to life perfectly in augmented reality at the National Gallery. This involved matching the precise measurements and positioning of the altarpiece, the model, and the floorplan of the gallery itself. We used Apple ARKit to power the AR experience – including image recognition to anchor the 3D scene to the artwork, and plane tracking to ensure it remains calibrated even at the other side of the gallery. 

To raise the sense of magic even higher, we created a bespoke transition effect which animates the church into view through a depth-based camera shader. This has proved to be one of the most breathtaking elements of the experience, and is fundamental for users to comprehend the shape and scale of the architecture as it is revealed.

Available as Open Source

As part of the wider collaborative research project, all of the source files from our development of Hidden Florence 3D have been made Open Source, and are available on GitHub. This includes our reworked 3D model and its textures, the Unity scene, AR implementation and transition effects. Researchers, artists and other 3rd parties are welcome to fork or download the project for any purpose. 

Credits

Hidden Florence 3D: San Pier Maggiore is the result of a collaborative research project led by Fabrizio Nevola (University of Exeter) and Donal Cooper (University of Cambridge). The app and project have been made possible with support from the Getty Foundation, through its Digital Art History initiative; additional support was provided from the University of Exeter and the Arts and Humanities Research Council, UK. The app was produced by Calvium, and 3D/AR development work was carried out by Zubr.

  • Content type: Public release iOS AR app
  • Platform: iOS
  • Launch date: November 2019
  • Additional Technologies: ARKit