This post was guest-written by Kevin Watters & Fernando Ramallo, creator duo behind Content Creator Simulator. This post assumes some basic Unity knowledge and familiarity with the Looking Glass displays and the HoloPlay Unity Plugin. For more information, visit this page.

To skip straight to downloading the Looking Glass GUI Examples for Unity, click here.

Hey there! We are the developers of Content Creator Simulator (CCS) for the Looking Glass -- a quirky tool available on MadeWith for creating fun and weird 3D dioramas using internet GIFs. This post serves as an overview of how we created the holographic user interface for CCS, some best practices, and a run through of what some of the tools and techniques are that we included in the downloadable Unity package.

Background

Because CCS relied heavily on the player using its User Interface, we had some special requirements for this projects’ UI:

• It had to be simple, easy to use, and rely on icons as much as possible.
• It had to showcase and utilize the depth aspect of the Looking Glass displays.
• The text had to be easy to read on the Looking Glass displays.

Just like in the early days of developing for VR headsets, there wasn’t a whole lot of existing research or work created on holographic interfaces that we could replicate or start from, so we had to come up with our own techniques to achieve our goals of readability and usability.

Defining an exclusive depth area for the UI

Early in the project, we prototyped the main interactions and found a camera setup that worked best. We stuck to this camera angle and based our UI layouts on the assumption that the camera wouldn’t move. It’s important to always be testing on a Looking Glass display— it’s difficult to really get a feel for how things look in 3D on a normal two-dimensional display!

It quickly became clear that we needed to define different areas on the scene where our UI would sit, and how deep the UI elements are allowed to go as to not overlap with the geometry of the scene, and vice versa.

To make sure the UI elements would still be readable even if the geometry of the scene came too close or went past their place in the scene, we rendered some UI elements to always be in front of the geometry.

Instead of having custom shaders on the UI, one way to “overlay” the UI on top of everything else is just to add your UI elements to their own layer, and then have a second Holoplay Capture/Camera pair, and set the second camera’s Clear Flags to “Depth only” and its Culling Mask to include your UI layers.

Making icons smooth using Signed Distance Fields (SDF)

We found that UI icons look better on a Looking Glass display when given a bit of softness to smooth out their edges and compensate for the “jagged edges effect” produced by the way the Looking Glass renders all the multiple views.

It is possible to blur the edges of your icons on an image editor, but the process might be cumbersome. You could also apply an anti-aliasing post processing effect to smooth out all the edges on your scene, but that can be very expensive to render and results might not be good for smaller icons.

A third option is to use Signed Distance Fields (SDF), where, instead of sampling the icon’s texture directly you can use a pre-computed image of how close any given pixel is to the contour of the shape. This results in much smoother edges and clearly defined shapes, regardless of texture size. TextMeshPro uses this technique to render its text with sharpness.

While developing CSS we came across Catlike Coding’s SDF Texture Generator, which makes the process of generating the SDF textures much easier, so we integrated that into our pipeline.

The end result of this technique is a few knobs that allowed us to tweak exactly how smooth each icon is, on an icon-by-icon basis, with much better results on the display.

Included in our package is the SDFIcon.cs class that we use on Image components to use and tweak the SDF parameters of the icons.

To generate an SDF image from your icon, click the “SDF Generator” button, load your icon in the “Input Texture” field, and press “Generate” to see the SDF image. You may adjust the Inside and Outside Distance fields to add or remove softness, depending on your image and resolution. Then use the Save SDF Texture button to save the SDF image to your textures folder. For more information about SDF and how to create useful SDF textures, see this Catlike Coding’s article.

Finally, play with the Threshold and Smoothness value to tweak the way the icon looks. Make sure to do this while the icon is visible on your Looking Glass display for best results. To use the icon in other Images in your scene, you don’t need to regenerate the SDF — just select it in the Source Image field of your Image component.

Keeping text readable

There were four main techniques we used to make text more readable in the Looking Glass:

1. Font Size

We kept font sizes larger than we would in a 2D game and stuck to bold, thick typefaces.

2. Focus plane

There’s a “sweet spot” in the Looking Glass depth where things look their sharpest. Any closer or further from this focal plane and things start looking less defined. This isn’t a problem for regular geometry, but it becomes very important when you’re displaying text or icons that need to be readable.

It’s important to play with a Depth-of-Field post processing effect to make the near and far parts of your scene blurrier and hide some of the artifacts inherent in the display (and to also make the objects in the main area “pop”), but this doesn’t apply to text. We learned that text needs to stay very close to that sweet spot to be at its most readable.

Other non-text based UI elements can go a bit further back, and there are ways to keep them readable by blurring the edges a bit (more on that below).

3. Smoothness

TextMeshPro’s materials include a “Smoothness” slider that blurs the edges of fonts. This helps get rid of some of the “jagged pixels effect” on the display. This is the same SDF technique described above we use in our icons.

4. Volumetric “Drop Shadow”

Wherever our Text objects didn’t have a clear solid background behind them, we would add some volume and depth by just cloning the text object a few times and making darker the text in the back. This would help separate that text from the background elements with the added bonus of creating a subtle effect giving the letters depth.

Making sprites volumetric

In addition to text, one simple way we added depth to 2D sprites is to layer multiple instances of them underneath each other in a “stack”. This creates a simple but neat effect on the display where an image “pops out” and feels more placed in the world by its sides having depth as you look around the scene.

You can use the included component RepeatRenderer.cs for an easy-to-use script that will duplicate your sprite automatically and give it depth.

GUI Tools and Examples Package

Ah! You’ve made it to the end!

We’ve put most of what we built for Content Creator Simulator into a neat Unity package that you can use in your own Looking Glass projects. It includes prefabs for buttons and text that render well in the display, as well as a whole template with scripting for moving across different screens.We hope it’s a great starting point if your Looking Glass Unity project relies on the user interacting with a lot of UI elements.

What’s included:

• Easy to use Prefabs for buttons, text and menus
• Fully working template with working buttons, different screens, tutorial text and other UI ready to use in the Looking Glass
• Component for creating quick volumetric Sprites
• Components for creating and tweaking smooth icons using SDF

Download

You can download the Unity GUI Tools for Looking Glass package here.

Thank you so much for tuning in and I hope that this was helpful! If you do end up making something with our HoloGraphic UI package, don’t forget to tag us. Follow along on Looking Glass Factory’s Twitter, Facebook, LinkedIn and YouTube.