Platforms Tools

A New Dimension for UI: Using Unity for Virtual Reality

virtual reality unity ui

The advent of virtual reality solutions, ranging from gaming to trainings and simulations, is raising new questions about previously standard industry practices. User interfaces (UI), in particular, require a complete re-thinking of function, layout, and implementation. Traditionally, user interfaces have been divided into diegetic (part of the game world), non-diegetic (separate from the game world), spatial and meta components. Most successful games use a combination of them to provide a balanced experience. In this, we break down each category, its advantages/disadvantages for virtual reality, and how to implement them in Unity. Meta UI components are rare in general and largely disregarded in VR programming. For that reason, they are not considered in this analysis.

Non-Diegetic UI

Historically, non-diegetic user interfaces have been the most common in the gaming industry. The key defining feature of them is that the components of the UI exist on a completely different plane than the actual 3D game space. Imagine here a heads-up display (HUD) as they are likely the most ubiquitous examples of non-diegetic user interfaces. A health bar, for example, does not exist within the 3D space that the game supposes nor can characters in-game interact with it. It is outside both the game’s narrative and space.


This modality offers the user a very clear display of relevant information and allows for quick navigation. The fear, however, is that the distinct separation of the game world from the structures that manipulate it results in a lack of immersion.

Use in Virtual Reality With Unity

For virtual reality, non-diegetic user interfaces can be very difficult to successfully implement. The largest obstacle is the fact that a HUD a la traditional gaming can be too close to the user’s face, resulting in highly uncomfortable eye strain. In Unity, the typical way to design a non-diegetic HUD is through the Screen Space – Overlay or Screen Space – Camera functions. It is unsupported, however, in Unity VR due to discomfort-related concerns. A developer can, however, fix a model to the user’s vector of vision. This, in effect, serves the purpose of a HUD. Once again, though, it can prove awkward. It would be like walking all day with a phone directly in front of you. In order to focus on it, you would need to re-focus your view from the rest of the world. Additionally, its presence when focusing on other tasks would be distracting. In short, stay away from strictly non-diegetic UIs when developing solutions for virtual reality.

Diegetic UI

This model of user interface holistically embeds all of the information typically represented in a HUD into the game’s 3D space. An example of this in a game would be if instead of a mini-map in the corner of the screen, the avatar/user would pull out and look at a map that exists within in the game world. Thus, the user interface is part of the game’s narrative and exists within the game space. From a player perspective, the Deadspace video game franchise is generally regarded as having implemented one of the best diegetic UIs to date.


The advantage of this style is the belief that it increases the realism of the gaming experience and thereby results in deeper immersion. The drawback, however, is it requires developers to seek ingenious ways of representing typical information, such as health, items in inventory, etc. These, in turn, must be intuitive and effective, otherwise, they will frustrate the user and result in a loss of immersion.

Use in Virtual Reality With Unity

In many ways, the goal of virtual reality is to provide a level of engagement and immersion that mimics real-life. With this in mind, diegesis seems like the logical, and even necessary, method of crafting user interfaces. The logic seems to go, if real-life is without menus and speech bubbles shouldn’t virtual real-life be so too? In lieu of this, there are several ways to create more diegetic experiences using Unity in new innovative ways. One way is to use the Raycast function to initiate interaction. Let’s imagine, for example, that in an RPG the user wishes to interact with an NPC. Instead of clicking and using a menu, the user could simply stare at them for an appropriate amount of time, which mirrors how we use eye contact in real-life to initiate conversation.

Spatial UI

A spatial UI lies half between traditional diegetic and non-diegetic models by offering elements that exist within the 3D game space but are not part of the game’s narrative. Perhaps the simplest iteration of this would be if you were to select a unit in a real-time strategy. Around the unit would appear some sort of circle or symbol to represent that the unit has been selected. In a first-person shooter, a way-marker for an objective is another example of spatial UI. The way-marker exists in the game space but if you were to live inside your character’s head, you wouldn’t see it.


In many ways, the advantages and disadvantages of spatial UIs mimic those of diegetic models. The key upside is it provides a lot of clarity to the user; all the relevant information for a user can be tagged to the relevant models. This, however, is offset by the fear that the presence of meta-information could break the immersive dimension of the game.

Use in Virtual Reality With Unity

When it comes virtual reality, spatial UI is the simplest and most effective option. When programming with Unity this means selecting World Space as the render mode for the Canvas. This allows components of the UI to be placed anywhere in the game space. In order for the best results and most comfortable experience for the user, set the text at a comfortable distance (3-5 meters) away and make sure it is clear, large, and readable.

In order to reduce clutter on the screen and keep immersion-levels high, it is often advisable not to permanently tag UI information to a model. It can appear unrealistic and unnecessary. Instead, allow notifications and status updates to flow in and out of the game as organically as possible. For example, don’t always have a health bar floating above a character’s head but instead have an aura appear around the character or have a health bar flash in the game space near the character. Unity also allows the implementation of arrows to help direct users if they’re looking in the wrong the direction. The easiest way to add this to a game is GUIArrows and customising which vector should be prioritized can be done with the Show Angle function.

An effective use of spatial user interfaces that is subtle but clear is overwhelmingly the simplest and most effective model. It provides the necessary instruction without — if done tastefully — shattering the user’s level of immersion.


The key consideration, whether choosing to pursue non-diegetic, diegetic or spatial components, is to strike a balance between immersion and usability. The greatest strength of virtual reality is that it’s 360° of 3D space naturally induces a degree of engagement that far surpasses even the most advanced screen-based solutions. The fear for some developers is that immersion could be broken by clunky interfaces that divorce the user from the actual experience. With this in mind, it’s important to remember that many games featuring non-diegetic/spatial features still boast impressive levels of immersion. MMOs that allow highly customizable HUDS immediately come to mind. They may clutter the screen but they also allow the user to feel at home in the experience, which in turn induces immersion.

In short, according to our experience at Program-Ace when designing an interface for virtual reality, pay careful attention to making sure the experience remains intuitive and comfortable while also trying at every moment to submerge components into the game space and game narrative.


Top Game Development Tools: Pros and Cons

According to our survey, a surprisingly high [tweetable]29% of games developers are primarily building their apps without a third party engine[/tweetable]*. They have either written their own engines, or are building everything from scratch. Large games studios very often build their own engines and tools, or customise open source solutions to suit their own internal processes and workflow. However, two of the most popular developer segments going for this option are Hobbyists and Explorers. It doesn’t make much sense for part-time game developers, or even small studios, to spend a lot of time working on their own tools rather than building games. In this post I’ll take a look at some of the most popular tools they could be using instead.


Which are your favourite tools for games development? Let us know in our new Developer Economics Survey and you might win an Occulus Rift Pro and other hot prizes.

Unity – the people’s champion!


As developer tools go, Unity is incredibly successful. A massive [tweetable]47% of developers in our survey use Unity for some of their projects and 29% use it as their primary development tool[/tweetable]. This is not just Hobbyists taking advantage of the free licensing options, Unity is more popular with professionals in general and most popular with the Hunters (53% of them) who are trying to earn their living from the app stores.


Unity supports both 2D & 3D game development, which is quite unusual for a game engine. That said, Unity was really designed for 3D games with 2D support bolted on afterwards; the 2D features were initially just for building menus and other 2D screens needed in a 3D game, to avoid the need for an external tool. The features were quite generic and developers started building games with them; probably due to the broad cross-platform support. To their credit, Unity have supported this and continue to invest in the area.

Supported languages

Three development languages are officially supported: C#, UnityScript (basically JavaScript with type annotations) and Boo. The last of these, Boo is not widely used and probably best avoided. Given its name, you’d be forgiven for thinking that UnityScript is the main development language, it’s not. The Unity community has widely adopted C# and you’ll find the majority of plugins and examples use it. If you prefer JavaScript and only have a very simple project in mind then UnityScript is a good option. Once you start using plugins written in C# that potentially need to call back into your UnityScript you’ll find issues with compilation order and wish you’d gone with C# from the beginning.


Unity has a lot of great features:

  • Unity has a very strong community of asset and plugin creators – there’s lots of free and reasonable priced content available.
  • Unity’s visual editing tools are excellent and the editor can be extended with plugins.
  • It supports a wide range of asset formats and converts automatically to optimal formats for the target platform.
  • It supports a very wide range of platforms, mobile, desktop, web and console.
  • Deployment to multiple platforms is very easy to manage.
  • The 3D engine produces high quality results without any complex configuration (I’ve personally written a licensed game with Unity that Apple has featured in lots of countries).
  • There is a free license that covers the majority of features.
  • Paid licenses are very affordable for most professional developers, available on subscription for $75 per platform currently (some platforms are free).


There are a few issues which are worth considering before choosing to go with Unity:

  • Collaboration is difficult. Unity has an expensive asset server product to help teams collaborate. If you don’t use it, sharing code and assets between team members can be painful. The best option is to enable and use external source control but there are several binary files (which don’t need to be) that can’t be merged and updating assets often causes them to break things in scenes, losing connections to scripts and other objects.
  • Performance is not great – until very recently Unity ran almost entirely in a single thread and made almost no use of the extra cores in most mobile devices – this is improving in Unity 5. The compilers are not at all well optimised for the ARM processors in almost all mobile devices – Unity have decided to transpile to C++ and use LLVM to get a more optimised build rather than solve this problem directly in future releases.
  • The engine source code is not available. Even paying users don’t get to see the Unity source code, which means if you come across a bug in the engine you have to wait for them to fix it or work around it. It’s always going to be more critical for you than it is for them. This also limits the ways in which you can extend or customise the engine.

Overall, Unity is a great choice, particularly for solo developers who aren’t trying to push the limits of what the platforms can do.

Cocos2d – perfect for casual games


Cocos2d is, as the name suggests, a 2D games engine. It originated around the same time as the iPhone SDK and quickly switched to Objective-C, growing in popularity as the best free and open source option for mobile games. However, Apple released their own highly performance optimised 2D engine for Objective-C developers called SpriteKit. That, along with the rise of Android, has caused the focus of Cocos2d development to shift towards the cross-platform Cocos2d-x branch written in C++. The Cocos2d family of engines is the most popular open source option in the world, used by 19% of game developers in our survey and by 8% as their primary tool.

Supported languages

There are different versions of Cocos2d available in Objective-C, C++, C#, Java, JavaScript and Ruby. As mentioned above, the C++ version is the most actively maintained, it also has the widest range of supported platforms. There are also scripting language bindings to the C++ version in both Lua and JavaScript, enabling developers to write in their preferred scripting language but get the full native performance of the underlying engine.


As with most thriving open source products, there’s a lot to like about Cocos2d:

  • Broad range of supported platforms, particularly mobile ones.
  • Free and open source (MIT license).
  • Wide range of extensions, tools and open source code available.
  • Lots of community created examples and learning resources.
  • Large peer support community.
  • Hardware accelerated graphics and good performance.
  • Audio support (in most versions).


Nothing’s perfect, here are a few issues with Cocos2d:

  • There’s no large commercial entity providing support and bug fixes. It’s great that you can fix it yourself, or hire someone who knows how. The community might even fix your issue for free but sometimes when a big project hits a bug or performance issue close to a deadline you just want to be able to pay someone to make it go away.
  • The APIs are somewhat unorthodox. The history of the project is such that it started in Python and moved to Objective-C very early. The Objective-C wasn’t exactly following standard practices and then that got ported to C++, retaining the Objective-C idioms.
  • It doesn’t do much to encourage good structure. Some developers like frameworks that don’t impose a style on their apps but Cocos2d goes a bit far. It’s possible to write code that’s hard to maintain in any system but it’s easy to find examples of Cocos2d games with really long functions and a lot of global state.

OK, the cons are nit-picking and more warnings that really negative points. After all, poorly structured code and unusual APIs are not exactly barriers to success. I’ve ported a game from iOS that was written with Cocos2d (the Objective-C version, before the C++ variant existed) and almost one giant method with tens of global variables. At one time it was the number 1 paid download on iOS in several countries. Cocos2d-x is an excellent choice for a 2D game.

Adobe AIR – what remains of Flash

Adobe AIR

In 2007 Adobe seemed to be winning the casual games runtime battle, with Flash having become the defacto standard for games on the web and Flash Lite almost ubiquitous on more advanced mobile devices. Then the iPhone came along and Steve Jobs said it wasn’t going to support Flash. This knife wound wasn’t immediately fatal but Flash has been slowly bleeding to death ever since. By 2011 Adobe eventually produced a version of AIR that compiled Flash to native iOS apps but by then the damage was done. Android initially supported Flash, poorly, in the browser but Adobe eventually gave in and stopped developing the browser plugin to focus on AIR. There are still a lot of Flash developers in the world, 15% of mobile game developers use it and 6% of them as their primary tool. It’s also still, just about, the only way to target rich gaming experiences to the majority of the world’s desktop web browsers. Adobe is now focussing on tools for HTML5 developers and Flash/AIR has not really evolved in a long time. Given this background, I won’t focus on detailed technical pros and cons as with the other tools.

Supported languages

Adobe AIR applications are developed in Flash, coded using ActionScript. There’s an integrated web view which can be targeted with HTML, CSS and JavaScript. It’s also possible to build native extensions for AIR apps, individually for each targeted platform.


Flash is still a capable environment for simple 2D games. If you already know Flash it’s one of the fastest ways to build a mobile game.


The platform is a dead end. I couldn’t recommend anyone who doesn’t already know Flash to learn it. Those who are fans of ActionScript but don’t like HTML5 should probably look at Haxe.

Unreal Engine 4 – the AAA king goes mass market


The Unreal Engine has a long history as one of the top 3D games engines for PC and console platforms. The 3rd generation of the engine supported mobile platforms but it was really only for hobbyist developers tinkering with their limited UDK or the multi-million dollar licensees of the engine for console games porting their titles to mobile devices. In March this year, Epic Games released the Unreal Engine 4 to anyone for $19/month plus 5% revenue share. This offering includes full access to the engine source code and their suite of tools. This change was not long enough before our survey was launched to see significant adoption by developers but 13% were using it with only 3% as their primary tool.

Supported languages

The Unreal Engine is written in C++ and that’s the only supported development language. However, it’s possible to do a lot of development without writing any code using Blueprints – a visual programming environment where nodes are connected with lines.


The Unreal Engine is AAA game quality:

  • Incredible performance. The Unreal Engine was demoed using Apple’s new Metal graphics interface at WWDC. It can produce the most realistic graphics ever seen on an iOS device. The same will be true for (high end) Android devices.
  • They have state of the art tools for all aspects of game development.
  • Full source access enables extension, customisation and engine bug fixing.
  • The pricing model is an excellent match for the high risks of failure on the App Store.


The Unreal Engine is designed for professionals:

  • Development is in C++, not a beginner friendly language.
  • The learning curve for the tools and engine is significant, greater than Unity.
  • The engine has limited support for older devices.
  • The pricing model is very expensive for a successful title, unless you expect significant success and use the engine under a different licensing model.

The Unreal Engine is an excellent option for high quality 3D games on high end mobile devices but it won’t be for everyone. I expect to see increasing adoption across the next couple of years. If you’d like a free 3D engine with scripting language support it might be worth checking out Project Anarchy by Havok. It’s effectively subsidised by Intel (owners of Havok) for mobile devices. There’s a co-marketing option in the license and you have to build an x86 variant if you build for Android (or Tizen, if that ever happens), otherwise it’s completely free, only the PC version is paid.

* Apple’s SpriteKit on iOS is actually a fully functional 2D game engine but as it’s part of the platform, developers may legitimately have said they were only using native code. If this is popular then it may significantly reduce the 29% figure.

Which are the best and worst aspects of each tool? Here’s your chance to have your say. Take the survey.