UC San Diego Researchers Acquire Data on Renaissance Landmark in Search for da Vinci Mural

By Doug Ramsey, 858-822-5825, dramsey@ucsd.edu

San Diego, CA, January 2, 2008 -- When most tourists visit Florence's famed Palazzo Vecchio, they are ushered off the premises in early evening. But for UCSD graduate students Michael Olsen and Daniel Knoblauch, the museum's closing time was when their real work began. Together with professor Falko Kuester, they spent nearly three weeks in November and December scanning the interior of the historic building's stately Hall of the 500, mostly in the dark. "It was a great opportunity to work in an area where important Renaissance artists worked and so much history happened," recalls Olsen.

Palazzo Vecchio
Interior of the Palazzo Vecchio's Hall of the 500

"We spent entire nights there in the darkness using our laser scanner," says Kuester, Calit2 professor of visualization and virtual reality, who also holds appointments in the structural engineering and computer science and engineering departments of UCSD's Jacobs School of Engineering. "In doing so, you develop a completely different attachment and understanding of the space."

For Kuester's grad students, the research was anything but business as usual. Olsen, a structural engineer, had previously used his laser-scanning skills to detect erosion of coastal bluffs in San Diego County. As a computer scientist, Knoblauch had never ventured into the field. "I've never worked before in the field as a computer scientist," says Knoblauch. "You normally sit in front of a computer in a lab."

Their research in Florence was part of a wider 18-month project to re-discover a long-lost Leonardo da Vinci mural, the "Battle of Anghiari." The project is the largest to date for Calit2's year-old Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3), a joint venture with the Jacobs School and UCSD's Division of Arts & Humanities. Last summer, Italian authorities designated CISA3 director Maurizio Seracini to lead the scientific search for the da Vinci mural, not seen in nearly 500 years.

Daniel in Hall of 500
Grad student Daniel Knoblauch operates the laser scanner in the Hall of the 500.
The primary goal of Kuester's expedition was to acquire all the data necessary to build a massive, interactive, 3D computer model of the Hall of the 500 -- the most extensive laser scan ever attempted in the Italian landmark. The scans and subsequent analysis will be critical to future conservation and renovation efforts. "It's a digital blueprint in three dimensions," says Kuester. "When it's complete, the computer model will be used as a baseline for future work, and everything else can be referenced against it."

"We are interested in building a digital clinical chart of the structure," explains Kuester. "We want to understand the building, its performance, and its history. Hopefully one outcome will also be the re-discovery of the 'Battle of Anghiari'."

Painted in 1505, the "Battle of Anghiari" disappeared from view some 50 years later, when the hall was completely renovated by artist Giorgio Vasari. Seracini and other experts believe the missing mural was preserved by Vasari, who may have built a brick wall in front of the da Vinci mural before painting his own fresco on the new wall.

To acquire the 3D data, Kuester and his grad students used a state-of-the-art laser scanner. Commonly known as LIDAR (light detection and ranging), the device emits light, which bounces off line-of-sight surfaces, allowing researchers to "map" the location of each "point" in 3D space. The team acquired some 500 million points in space that constitute the nearest walls, ceilings, etc.

They began by sampling points every centimeter (compared to the 5-centimeter resolution-about two inches-of previous scans). By deploying the latest scanning technology, CISA3 was able to execute ever-more-detailed scans, determining points barely one millimeter apart.

The LIDAR scans yielded 500 million x-y-z point locations, as well as color and intensity information for each point, providing some insights about the properties of the material.

Falko Kuester
Professor Falko Kuester with scanned interior of the Palazzo Vecchio's Hall of the 500 on the 220-megapixel HIPerSpace tiled display.
"When Leonardo da Vinci was standing there in 1505 on a sunny day, were the rays of sunlight illuminating the wall where he chose to paint his mural? And how could Michelangelo have agreed to work on the shaded side of the building rather than working where the lighting conditions were better?" asks Kuester. "By scanning the structure and reconstructing how it has changed over the past 700 years, we can create a 3D computer model that should be able to answer those and many other questions that art historians have struggled to answer until now."

According to grad student Daniel Knoblauch, whose expertise is in rendering 3D graphics, the Hall of the 500 presented its own special difficulties. "We had to figure out how to get points from every corner of the room and it was hard because there were a lot of occlusions," he says, referring to areas where structural details may be largely hidden to the naked eye. "We had to use multiple scanning positions, so in the end we amassed over 500 million points."

Occlusions
To scan the Hall required hundreds of scans to take into account occlusions -- areas hidden from view when scanning most of the room's surfaces.
In order to create a photorealistic model though, the researchers could not rely only on the laser scans. The team used a high-resolution camera to record the color value at each point sampled by the laser.

In the first quarter of 2008, the team will be processing the 25 gigabytes of data they collected in Florence, fusing the points and color information. "We now have a lot of data and a lot of it has noise in it, so now we have to cut out points that we don't want," says Knoblauch. "We will clean up all the point clouds and get rid of any bad areas and fill from other scans the points that we do want. Then we will just have to figure out how to deal with so many points."

"I don't know of another project that renders this much data in real time," says Iman Sadeghi, a Ph.D. student in computer science who is expected to use his computer-graphics expertise to put the icing on the cake. "I call it the Virtual Vecchio, because we want viewers to feel as if they are walking into the real structure, while navigating and viewing the Hall of the 500 from any angle, and even seeing inside those walls" (after X-ray and other multispectral data are integrated into the model).

"When I started on the project a few months ago we were trying to create photorealistic renderings of the main hall of the Palazzo Vecchio, based on old data models showing 20 million points," notes Sadeghi, who won the top prize last spring in UCSD's computer graphics competition for photorealistic rendering. "Now with the new data, we have over 500 million points to work with, so the realism of this computer model will be intense."

Ceiling of Hall of the 500
The ceiling of the Hall of the 500
"Once we have this complete, volumetric model, our research challenge is to develop new visualization techniques that can handle these massive, multi-dimensional data amounts in real time," says Kuester. "We need to be able to manipulate the model interactively, and that means we only have about one-sixteenth of a second to generate the new result by processing each of those points."

Putting his structural engineering hat on, Kuester says that by learning what has happened to the Palazzo Vecchio over the past half millennium will provide new insights on the way structures change over time. "I am intrigued about the impact this research can have on our built environment today, including structural and nonstructural components," he says. "We can learn more about how to predict the performance of that environment in order to keep it safe."

Kuester lauds Calit2 and CISA3 for offering truly interdisciplinary research experiences. "To find structural engineers and computer scientists talking with artists and art historians was a far-fetched idea until recently," he notes. "But on this project, that's how it happens, and the extent of the exchange of ideas is fascinating. You gain insights from the different perspectives, and the methodologies we are developing will help us improve on the way that historic structures are safeguarded."

"We were able to walk with Leonardo da Vinci and Michelangelo, and to apply information technology to a space that was one of the landmarks of the Renaissance," adds Kuester.

The research team is also building algorithms and software tools to bring the 3D model's real-time interactivity to Calit2's 220-million-pixel HIPerSpace tiled display, the highest-resolution multi-tile  display in the world, installed in Kuester's second-floor lab in Atkinson Hall.

Related Links

CISA3 

Falko Kuester Website  

Daniel Knoblauch Website  

Michael Olsen Website 

Iman Sadeghi Website