> This scan is made possible by recent advances in Gaussian Splatting. This is an emerging technology that lets us quickly create very detailed models just from photographs. For this model (or splat, as we call them), my friend Daylen and I flew our drones around Sutro Tower at a respectful distance for an afternoon until we had collected a few thousand photographs.
> I then aligned these pictures in free software called RealityCapture. Alignment is the process that teaches the computer that a bunch of points in different images all actually correspond with the same point in real life. Then I used another piece of free software called gsplat to produce the 3D model itself.
Yes, I also added a bit of context on twitter (https://xcancel.com/fulligin/status/2079296985811873796) and will probably blog about the process a bit more. Grace was a lot more work than Sutro. I wanted to be ambitious in combining interior/exterior scenes in a new way, and I also wanted it to be less of a "toy". Grace actually plans to use this to show prospective renters the space, and we spent a lot of time making sure it could run well on a wide variety of devices.
In case it's useful, given your last sentence, it runs really well on Chrome on Android (on a Pixel 10) but it's a bit glitchy (although not impossible) on Firefox on Android. The loading indicator on Chrome got stuck briefly whereas the Firefox one was smoother, but that's about the only case there Firefox is better.
The cathedral is the star of the show, but I found myself more fascinated by all the incidental little details captured in this. Like the trees and the buildings nearby, the basketball court, the little solar panel on the side, the apartment towers.
It's like seeing the promise of Google Street View realized.
The technology is in such an early stage and it already slaps this hard.
Wonder what the next steps are for this tech? "Un-splatting" optimizations to convert things like flat surfaces into classic rigid textured geometry? Factoring the lighting away from the geometry? Filling the pipeline with AI tooling - segmentation to remove the "variables" like humans automatically, generative AI to fill in the missing non-semantic details based on the surroundings, more generative AI to "port" the data from things like public high resolution images of plaques or other undercaptured details into the scene?
Seeing this kind of reconstruction now makes me very excited for what the 3D environments of the future are going to be like.
One other little easter egg you may not have noticed. There's an American flag gently fluttering in the breeze on top of a nearby building. We were discussing whether to adds some birds as well, but that never made the final cut. You ask about factoring lighting away from the scene - actually, PlayCanvas can already do relighting and shadow-casting for gaussian splat-based scenes:
Some kind of day/night toggle would have been another great feature...but again, it didn't make the final cut of the experience. Maybe a HD Remix is needed. :o)
Reminded me of a VRML demonstrator inside Notre Dame from decades ago. Different approach, similar outcome. (it used surface texture mapping inside a game engine from my memory)
A lot more architecture would benefit from being available like this splat model. The Pharonic tombs which you can visit in google/other virtual tourism are good, but this is also good!
I noticed that complex patterns - marble, some paintings, some stained glass - are rendered in what seems like stretched, hazy ellipses. Is that the fundamental building block of a splat?
Also, I was interested if any specularity would be captured by the splat. Or any other aspect of the building which shifts from viewpoint - like lights reflected on the marble floor, or where a metal element outdoors is reflecting the sun but only from one vantage point. I didn’t see that. As I understand it, splats are a kind of solution to the “problem” posed by the series of photos so they are capable of capturing that sort of thing. Is that present here, and I missed it? Or is that something that you need a lot of processing power for, to generate a more complex solution for the light fields?
Splats can indeed come with view dependent colors to represent lighting, reflections… Those are captured in values called spherical harmonics of different levels 0, 1, 2, 3 representing different aspects like diffuse, directional lightning, highlights… Not all splat reconstructions come with spherical harmonics though. And they add rendering overhead.
Besides spherical harmonics splats are also good at representing high frequency detail that is harder with traditional meshes: Hair, clouds, vegetation… notice how trees in these reconstructions already look much better than in google maps / earth 3D views.
I spent a lot of time hand-masking out inconsistencies between shoots. I waited too long between sessions and they actually built out an entirely new section of ceiling. PPISP guidance in splatting also helps a lot in solving sensor inconsistencies between the many different cameras and lighting conditions that went into the source dataset.
Hmm, today it works all of a sudden. No changes, no reboots, no new drivers, not restarted Firefox either. Just repeated what I did yesterday: open the link in a private window.
Yesterday when it didn't work I did try multiple times, and as mentioned that other site as well.
Guess Firefox on Windows has some flaky WebGPU support?
edit: must be, worked fine a couple of times, then it stopped with same error as before even after multiple attempts.
Did you animate the cars separately? That really did a lot to add to the immersion (and the sound effects too). If nothing moved I think the effect would be significantly diminished.
Yes, the cars are separated out and procedurally animated. They're splats, like the rest of the scene. Easily done in PlayCanvas - simple but effective!
Extraordinary. Thank you for sharing. If there would ever be the possibility of seeing European Gothic cathedrals in this manner, that would also be really cool.
I think it's probably more like an "incorrect" focal length for what you'd expect. And it's kind of hard to know what you'd expect because you don't typically have arbitrary spatial control over your point-of-view, so some POVs you're used to only seeing through cameras (or even certain types of cameras), while others you're mostly used to seeing through your own eyes.
Hi Vincent, the Sutro Tower 3dgs demo is one I enjoyed a lot. I'd whip it out on my phone and show people saying, 'hey look this is whats possible with 3D Gaussian Splatting'. And this Grace Catherdral splat is stunning. It showcases one of my favorite parts of 3dgs which is reflections and glass. The color and essence of the stained glass comes through so well.
The cutaway Peek feature is like a sci fi version of an Stephen Biesty's Incredible Cross-Sections book. Incredible work!
The Keith Haring altarpiece is a really cool surprise!
It's unfortunate that this tour makes it very difficult to see. Really not a fan of this "on rails" presentation style. I understand the angles matter when viewing a splat, but come on.
From OP's prior work scanning Sutro Tower in SF: https://vincentwoo.com/2025/02/18/sutro-tower-in-3d/.
> This scan is made possible by recent advances in Gaussian Splatting. This is an emerging technology that lets us quickly create very detailed models just from photographs. For this model (or splat, as we call them), my friend Daylen and I flew our drones around Sutro Tower at a respectful distance for an afternoon until we had collected a few thousand photographs.
> I then aligned these pictures in free software called RealityCapture. Alignment is the process that teaches the computer that a bunch of points in different images all actually correspond with the same point in real life. Then I used another piece of free software called gsplat to produce the 3D model itself.
I'm assuming this was done similarly. Very cool.
Yes, I also added a bit of context on twitter (https://xcancel.com/fulligin/status/2079296985811873796) and will probably blog about the process a bit more. Grace was a lot more work than Sutro. I wanted to be ambitious in combining interior/exterior scenes in a new way, and I also wanted it to be less of a "toy". Grace actually plans to use this to show prospective renters the space, and we spent a lot of time making sure it could run well on a wide variety of devices.
Looking forward to it. I'm also a big fan of https://vincentwoo.com/2023/07/20/tunnel-vision-an-unauthori..., thanks for keeping the internet interesting.
Beautiful work, thank you for sharing it!
In case it's useful, given your last sentence, it runs really well on Chrome on Android (on a Pixel 10) but it's a bit glitchy (although not impossible) on Firefox on Android. The loading indicator on Chrome got stuck briefly whereas the Firefox one was smoother, but that's about the only case there Firefox is better.
FWIW, it works great in Firefox focus on an iPhone 16 Pro.
So beautifully done, its mindblowing. Please blog about your process a bit more.
absolutely beautiful. I'd love to learn this process and attempt to capture a local rock climbing route.
anything you write about the process, or how you learned are very welcome.
The cathedral is the star of the show, but I found myself more fascinated by all the incidental little details captured in this. Like the trees and the buildings nearby, the basketball court, the little solar panel on the side, the apartment towers.
It's like seeing the promise of Google Street View realized.
The technology is in such an early stage and it already slaps this hard.
Wonder what the next steps are for this tech? "Un-splatting" optimizations to convert things like flat surfaces into classic rigid textured geometry? Factoring the lighting away from the geometry? Filling the pipeline with AI tooling - segmentation to remove the "variables" like humans automatically, generative AI to fill in the missing non-semantic details based on the surroundings, more generative AI to "port" the data from things like public high resolution images of plaques or other undercaptured details into the scene?
Seeing this kind of reconstruction now makes me very excited for what the 3D environments of the future are going to be like.
One other little easter egg you may not have noticed. There's an American flag gently fluttering in the breeze on top of a nearby building. We were discussing whether to adds some birds as well, but that never made the final cut. You ask about factoring lighting away from the scene - actually, PlayCanvas can already do relighting and shadow-casting for gaussian splat-based scenes:
https://playcanvas.com/examples/#/gaussian-splatting/relight...
Some kind of day/night toggle would have been another great feature...but again, it didn't make the final cut of the experience. Maybe a HD Remix is needed. :o)
Apple (“streets”) Maps uses that tech in the latest beta. (Apple Maps is generally better than google maps apart for the data, tbh)
I had a bit of an amusing time trying to see if I could find my apartment. I could though it was quite blurry :)
Reminded me of a VRML demonstrator inside Notre Dame from decades ago. Different approach, similar outcome. (it used surface texture mapping inside a game engine from my memory)
A lot more architecture would benefit from being available like this splat model. The Pharonic tombs which you can visit in google/other virtual tourism are good, but this is also good!
It’s beautifully done. A few questions
I noticed that complex patterns - marble, some paintings, some stained glass - are rendered in what seems like stretched, hazy ellipses. Is that the fundamental building block of a splat?
Also, I was interested if any specularity would be captured by the splat. Or any other aspect of the building which shifts from viewpoint - like lights reflected on the marble floor, or where a metal element outdoors is reflecting the sun but only from one vantage point. I didn’t see that. As I understand it, splats are a kind of solution to the “problem” posed by the series of photos so they are capable of capturing that sort of thing. Is that present here, and I missed it? Or is that something that you need a lot of processing power for, to generate a more complex solution for the light fields?
Splats can indeed come with view dependent colors to represent lighting, reflections… Those are captured in values called spherical harmonics of different levels 0, 1, 2, 3 representing different aspects like diffuse, directional lightning, highlights… Not all splat reconstructions come with spherical harmonics though. And they add rendering overhead.
Besides spherical harmonics splats are also good at representing high frequency detail that is harder with traditional meshes: Hair, clouds, vegetation… notice how trees in these reconstructions already look much better than in google maps / earth 3D views.
The ellipsoids are what a splat renderer uses as primitives, usually.
Very impressive. How are you managing to produce such a clean model without artifacts?
I spent a lot of time hand-masking out inconsistencies between shoots. I waited too long between sessions and they actually built out an entirely new section of ceiling. PPISP guidance in splatting also helps a lot in solving sensor inconsistencies between the many different cameras and lighting conditions that went into the source dataset.
Do you see potential for automation based on what you learned from your hand tuning?
Gsplat is indeed the future for graphics. Handling much more detail in mobile hardware spec. Check this work too. (Not mine) https://demo.immersivetech.ch/chateau-la-sarraz/
Gsplat + dlss
The Peek mode is insanely good, it reminds me of those exploded views of buildings and "how things work" children books.
The model seems to be having a hard time with the marble floor in the quire (low resolution), is this because of its reflection?
Looks good the few seconds it works here. Then stops working and I have to close the tab and reopen for it to work again for another few seconds.
Firefox, Windows 11, Nvidia
Get the following in the console when it stops working:
It looks like a webgpu driver issue - can you tell me what model of graphics chip you're using?
RTX 5090 with 610.62 drivers, Windows 11 25H2, Firefox 153.0b9.
FWIW I tried the other one linked here[1] which also throws the same error and stops. Using the same engine perhaps?
[1]: https://news.ycombinator.com/item?id=48986175
Hmm, today it works all of a sudden. No changes, no reboots, no new drivers, not restarted Firefox either. Just repeated what I did yesterday: open the link in a private window.
Yesterday when it didn't work I did try multiple times, and as mentioned that other site as well.
Guess Firefox on Windows has some flaky WebGPU support?
edit: must be, worked fine a couple of times, then it stopped with same error as before even after multiple attempts.
thanks for double checking, that's very good to know!
Totally hung my Windows 11 machine with Firefox and 32GB of RAM. Ryzen 8745HS (Radeon 780M iGPU).
Did you animate the cars separately? That really did a lot to add to the immersion (and the sound effects too). If nothing moved I think the effect would be significantly diminished.
Yes, the cars are separated out and procedurally animated. They're splats, like the rest of the scene. Easily done in PlayCanvas - simple but effective!
It's the next day and I'm still thinking about how much of a difference that small touch made.
Extraordinary. Thank you for sharing. If there would ever be the possibility of seeing European Gothic cathedrals in this manner, that would also be really cool.
Don't miss this one: https://superspl.at/scene/c67edb74
I'm not sure why, but my brain always gives a strong "this is not real" alert with Gaussian splats. Something about the lighting is really off for me.
And whenever I see videos explaining the Gaussian splats, they always claim they are supposed to fix lighting issues.
Is this a function of "not enough detail" or.. Something else?
I think it's probably more like an "incorrect" focal length for what you'd expect. And it's kind of hard to know what you'd expect because you don't typically have arbitrary spatial control over your point-of-view, so some POVs you're used to only seeing through cameras (or even certain types of cameras), while others you're mostly used to seeing through your own eyes.
I imagine all of this will get better though.
Interesting, thanks for the reply.
[dead]
Hi Vincent, the Sutro Tower 3dgs demo is one I enjoyed a lot. I'd whip it out on my phone and show people saying, 'hey look this is whats possible with 3D Gaussian Splatting'. And this Grace Catherdral splat is stunning. It showcases one of my favorite parts of 3dgs which is reflections and glass. The color and essence of the stained glass comes through so well.
The cutaway Peek feature is like a sci fi version of an Stephen Biesty's Incredible Cross-Sections book. Incredible work!
Thank you!
This is fabulous. It captures the feeling of being in the cathedral quite well - it fully transported me back to that spot. Great work!
- top tier work
- let us say i wanted to bring a city to life like this
- what does the procedure look like on my end
- what inputs, tools, libraries would i need to go about doing this?
I imagine it is based off World Labs
[dead]
Combining the interior and exterior captures sounds like the hard part. How did you get the two sets registered to each other?
Anchor points on the windows!
Wow that's crazy that they built an apartment building where it would shadow all that stained glass.
I bet there's a story there.
Wow Did you have any apps in Apple Vision Pro?
I like the moving cars. They're copies of the stopped ones right? Adds a good effect.
Yes. They're Gaussian splats like the rest of the scene. PlayCanvas has a global sort so they render correctly regardless of camera angle.
This is such a cool immersive experience on mobile that works so well. Great job on this!
Woaah, zooming all the way out is really cool
The cars moving in the beginning is a nice touch.
absolutely crazy level of detail... in awe
That's really cool
what kind of drones did you use for this.
The Keith Haring altarpiece is a really cool surprise!
It's unfortunate that this tour makes it very difficult to see. Really not a fan of this "on rails" presentation style. I understand the angles matter when viewing a splat, but come on.
You can also walk freely FPS style with WASD and mouse-dragging to look around.
I only ever look at HN on a phone, and I'm pretty sure I'm in the majority.
Bad mobile support is a sore spot for any "Show HN" announcement.
Hello, I'm wondering whatever you mean? You can absolutely walk around the church indoors on mobile. Just tap to walk.
(And if that doesn't work for you, please let me know!)
Unreal. It’s been years since I’ve been here. It’s crazy the fidelity you can capture. Stunning work.
Unrelated silly question. Are you the same Vincent Woo who went over the whole unit combination story on Medium of a supervisor?
yep
[dead]
Wow, I didn't expect to see transsexual steps in front of a church.