Interact with your holographic characters and worlds with Unity 3D

Hello and welcome, all you Looking Glass Portrait-curious readers. My name is Nolan and I’m a long time developer of weird cool future tech, from AR to VR to Immersive Displays like the Looking Glass. Today I hope to expand your ideas of what’s possible with the new Looking Glass Portrait display.

After you’ve modeled your characters, or captured real world content, or created a digital dreamscape, you have two options. First, you could convert it into a photo or video, and use the HoloPlay Studio to view on your Looking Glass Portrait at your leisure. Second, you could pull it into a Game Engine like Unity to add dynamic behaviors that let people interact with your creation using a computer!

Unity, like many game engines, already supports data, transformations, and calculations in 3D. It uses these to create a natural-seeming view into the world, even though you see it on a 2D screen - for example, it needs to know if your character is in front of or behind a crate so it can draw one in front of the other. This may seem simple, but it’s incredibly important to state: people are accustomed to working in 3D, even though they’re using 2D monitors.

The Looking Glass HoloPlay Unity Plugin uses this already-existing 3D information and shows it to the user. All you have to do is drag the Holoplay Camera prefab into your scene and position the framing box around your imported model to have it appear in a Looking Glass instantaneously. There’s no need for additional preprocessing or alterations of any kind. Any existing 3D content (that Unity supports) can be viewed in a Looking Glass. It’s true that rendering a model in 3D is more complicated than rendering it in 2D, but the model itself does not need to change. And HoloPlay Unity Plugin takes care of the 3D rendering, leaving you to focus on the most exciting part: interaction!

If you only want to examine your model from different angles and distances, HoloPlay Unity Plugin contains an example scene with prebuilt controls for simple interaction (and every concept up to there!) See Holoplay > Examples > “5 - 3D Cursor” for more information. Just drop your model into the scene and you’ll be up and running with a museum-like experience to show off your work.

Let’s keep it going! Now that you’re leveling up our visual output, why not also level up your user input? All the same controls and commands that worked in 2D will also work in 3D, but a Looking Glass display offers the unique opportunity to view and interact with 3D models like never before. Why not connect up an Azure Kinect or Leap Motion (both of which have officially supported Unity Plugins you can download)? You interact with the real world in three dimensions, you can bring that same nuance to the way you manipulate your 3D models.

From here, you can dive into the wild world of scripting to animate your creations. Even if you don’t think of yourself as a developer, there are easy ways to make a living scene. Unity has a bunch of great tutorials with cookie-cutter components you can drop in, as well as the Asset Store full of expertly made packages that are ready for import. Simple behaviors go a long way - with a little bit of custom scripting, you can, say, make your model blush when you wave at it! Hey there good-looking!

And that’s just the start! There’s a whole community of 3D creators who can share and give feedback on your interactive 3D experience. Join our Discord server to find new friends and discuss new ways of adding fun behaviors to your existing worlds and characters. When you get something working, make sure you let us know on Twitter - we’re always excited to see what people create in the Looking Glass.

To pre-order Looking Glass Portrait, head on over to our Kickstarter page which will be live until January 14, 2021.

Fauxlan model was made by Nolan Filter, with the use of the Hair Shader 1.0 pack by RR Freelance