Rune Spaans has released a new script for 3DS Max that sets up the camera data required for generating VR images using Google’s VR Seurat system. According to Rune, the script “lets you setup rendered input data (a bunch of animated camera frames) from any 3ds Max renderer, with an accompanying JSON data file, which are then ready to be processed using the Seurat Pipeline binary to a real-time 6DoF VR file.”
If you’re not familiar with Seurat, it’s a scene simplification tool for converting complex 3D scenes for VR display on mobile. Best of all, it is now free and open-source. Seurat works based on the assumption that VR is viewed from within a limited region. It takes RGBD images (images with depth information) from within that region and then reconstructs a textured 3D version of the scene with options for the poly-count, texture size, and fill rate.
You can read more about Seurat on Google’s blog and download Rune’s script from GitHub.
I would assume this is cheaper, bandwidth wise, than light fields…it seems a camera like the Detu MAX 8K 3D 360 is capable of capturing data from reality that Seurat could potentially process. So now the question is, who will be the first 360 camera manufacturer to offer it as a feature? Is the market even savvy enough that such a feature would entice more than a handful of purchases?
Actually, the Seurat format is a simpler version of Light Fields dubbed “Surface Light Fields”. Right now, the process of capturing 360 surface light fields from real locations required one heck of a camera setup: https://www.blog.google/products/google-vr/experimenting-light-fields/ But the results are amazing! If you want to see it in action on your Oculus or Vive, check out the “Welcome to Light Fields” app on Steam.
How does this approach deal with specular effects?
Rune mentions on the Github page that specular effects don’t seem to work.
The reflections should change according to where you stand when you view the images in VR. I have only checked the images in the Google preview app (which is just on the desktop and not in VR), so I have not been able to confirm this yet. I hope I can find someone with some Unity or Unreal skills to test this for me. Will keep you updated!
Maybe I’m not understanding exactly what Seurat is doing, but isn’t it baking the images to geometry that it creates, so specular and reflections are stuck on the geometry and don’t change with viewing angle? The Bladerunner sample on their blog appears that way as well.
I haven’t tried it yet myself but what you are saying would be true for stereoscopic capture. I believe because of the camera setup and resulting images and provided information should allow you to get some reflection changes as you tilt your head. Could be wrong, but I plan to find out.
After some more investigation, I’ve found that the Seurat converter simply bakes the speculars into the texture, and does not have a way of changing it according to where you position your head. I was confused since they announced “Welcome to Light Fields” just after they released Seurat, and some news articles also mentioned changing speculars. Crossing fingers Google VR (or someone else) gets to incorporate changing speculars into the format at some point, since that’s one of the things making the “Welcome to Light Fields” app so amazingly real.
Thanks for sharing Rune!
Thanks Tobbe!