Coroutines in Unity Part 3

Coroutines in Unity – Encapsulating with Promises [Part 3]

 

In the last part of the series we’re going to build a real example of a REST API Interface using Unity’s Coroutines as an internal web requests tool and Promises as an encapsulation layer. We’re going to use the fake REST API service available for everyone to test their services on. It’s a simple API that implements the classic user to-do lists, posts & comments section as well as the album & photos scenario. Very useful if you’re building your own front-end but don’t have your own running server just yet.

NOTE: This tutorial is a bit more advanced and it won’t teach you REST backend theory or JSON serialization. It also assumes you’re already familiar with the RSG Promises we’ve covered in Part 2.

The project

Our project is going to be based around the user to-do lists. The main feature is simple: it’ll take a username as input and provide a list of tasks associated with that user. The application is going to get a list of all the users, find the searched username in it and if that user exists it will then grab all the tasks. Ideally you’d want the user searching to be done server-side, but for the sake of this example let’s say someone hadn’t thought it through and left you to do your job.

For JSON deserialization we’re using the popular JSON .NET framework. If your project is going to be cross-platform you should take a look at JSON.NET for Unity, which uses the same namespace and structure so it can easily be used as a drop-in replacement.

We’re going to work with Unity 5.4.0f3. You can download the .unitypackage here with a complete project and all the necessary plugins. Let’s dive into it.

The project features a Plugins directory, Scenes directory with the single example scene as well as Scripts directory, where the whole codebase sits. The code is structured as follows:

devenv_2016-08-24_11-09-12

Let’s start from the top.

The Models

The Models directory is where the data models’ classes are. They’re essentially the classes with properties mapped to JSON object keys. For example our JSON of a single Task object looks like this:

The associated Model class is then being implemented in the following way:

As you can see, JSON .NET allows for an extremely easy mapping using the JsonProperty attribute. Actually, you can skip these entirely if the property name matches the JSON key. Personally I prefer camelCase in my JSONs and PascalCase in my properties. Keep in mind that for AOT platforms you should use JSON .NET for Unity or use regular fields. Refer to the documentation for more information.

The User model is a stripped model as the jsonplaceholder returns a much bigger JSON, but for the purpose of this example we will not implement all the properties.

Promises as a service interface

Let’s say you’re working on a REST API for a month only to find out that management of your company decided to move to Websocket. Or perhaps you’re ahead of your backend department and want to test new features on your own without the need of using a real server. To remedy this it’s a good idea to implement the Factory pattern which let you choose the exact implementation of your service (APIServiceFactory) by encapsulating a common interface for all of them (IAPIService). The interface is using Promises as the abstraction layer so it’s very easy to use.

To search users and list their tasks we only need two functions:

If you’re going to need another API implementation in the future all you have to do is create a new class that will implement these two methods. The instantiation is done via the factory and the provided config (IClientConfig and ClientConfig):

RestAPIService

The REST API implementation uses the Unity’s Coroutines and the UnityWebRequest class under the hood. Because of that, the factory creates a GameObject and attaches the RestAPIService class, which also extends the MonoBehaviour. This lets us encapsulate the coroutines even further – you’ll be able to use the service in all of the classes, because the interface deals with Promises only. For example, getting the user tasks is done like so:

Notice how easily we control the output with promise.Resolve() and promise.Reject().

TestAPIService

The test implementation is just an example of what you can do. It returns objects without any external calls, but you could also use it as a room to test your JSON deserialization without launching a real server. The bottom line is that this should be the space to mess around with no worries that you’re hard-coding some test scenarios which need to be commented out later. All you have to do is change the config to the real service and you’re done.

So for example, if you’d like to test how your tasks` UI look, but don’t have the tasks functionality done server-side just yet, simply implement the test service and the GetUserTasks() method to return a bunch of test objects:

Notice that you can resolve promises instantly when needed.

The result

The payoff is the example test scene and the MainScreenController class which utilizes the interface. First, it initializes the service using the Factory:

As mentioned earlier, changing to Test implementation is as simple as swapping that APIType property in the config. The UI consists of a single input field where you type in the username and a button to get all the user tasks. Using the common interface it couldn’t be any simpler:

2016-08-24_10-46-14

Summary

And at last we come to an end of the series. Again, the final project can be downloaded as the .unitypackage here. In conclusion Promises prove to be a great way to abstract your code from Unity specific Coroutines. They can also be used in many different cases and are an elegant way to create clean interfaces. We hope you’ll enjoy them as much as we do. If you have any questions about the series or the example project, feel free to leave a comment in the section below.

Should You Install Unity Cache Server on localhost

Should you install Unity Cache Server on localhost?

Do you already know how to use Unity Cache Server? If you’re one of the maniacs trying to optimize every aspect of their development environment, then you most probably do! It’s a great solution for the teams that are working on large projects, especially for mobile devices. But does it make sense to install it for only one person… locally?

How Unity imports assets

The first thing you need to understand is the way Unity imports assets. When you put a file into your Assets folder, Unity executes an asset processor. The textures will be converted to optimal texture format for your target platform: sounds to mp3/ogg (or any other) conversion, models to internal Unity mesh format and so on. Some conversions are done so fast, that you won’t even notice it, but for some it may take enormous amount of time. For instance, processing Android and iOS textures may even take several hours!

This is not an issue if you’re working on a game for a single platform, but usually you are not. For instance, if you’re working on an Android game, most probably you’d like to build an iOS version too. Unfortunately when you switch your target platform, Unity removes previously processed data as you’d never intend to use it again. As a result, switching between multiple platforms may take more time than actual development of your game!

Let’s install Cache Server

Cache Server solves that issue by storing processed assets in the persistent database for later use. If you’re working alone or if your internet connection is not that great, it may be a wise choice to install it locally.

Cache Server is written on the top of NodeJS. If you’re not familiar with this technology, don’t worry. It’s distributed along with cache server zip file, so you don’t even need to install it! (well, I don’t know now why I even mentioned it…)

Now it’s time to get the Cache Server files. These can be found on Team License web page (Team License is now available for everyone). Just click on the download button.

a280306e-a6e8-4ebe-806a-9ff04bd439a3

Unpack the downloaded zip contents in the location where you want to keep your processed asset database. Cache Server by default creates its database in-place, so be prepared to have at least 50 gigabytes of free space on your drive. Now you’re ready to launch it. Go into the CacheServer directory and double-click on RunWin.cmd file. There are appropriate script files for Mac and Linux too.

d4a57c8c-7df2-41ad-824d-881f87cdbfd2

When you see the output like this one, you know that your Cache Server is operational:

f7548b51-d00c-46d0-8afe-be95a1b26bd5

Now, don’t close this command line window! Your cache server works only when it’s open. When closed, you can re-launch it again and your data will be still there, but if it’s not running Unity is not storing anything in it (obviously).

Configuring Unity

Let’s now open Unity preferences. Go into the Cache Server tab, and use localhost as the IP Address.

3fc6f3ce-5cd7-43a8-8a67-6c7a6ea9b3bf

When you see Connection successful message, this means that your Cache Server is fully operational! Now you have to reimport your assets at least once to have it uploaded to the cache server. Later on instead of usual asset importing progress you will be seeing something like this:

using cache server

How long it will take depends only on the speed of your hard drive.

If you’re interested in more information about the Cache Server, you can find it in the official Unity manual. Please also note that Cache Server may require a license different than free (it got a little confusing after latest changes in licensing, so I cannot tell for sure).

7 Ways to Keep Your Unity Project Organized

I saw a person on Quora the other day, asking how programmers are able to write projects that consist of over 10,000 lines of code. When software gets bigger, it is more difficult to maintain and that’s a fact. So here’s the thing – if you don’t keep your project organized, you’re going to have a hard time to keep the pace. Later on, you will find yourself wasting time with a messy project instead of adding new features. This is also true regarding any Unity Project. Here are (in my opinion) the most important tips that will help you with keeping your project organized.

1. Directory Structure

We cannot talk about organization without mentioning organizing project directory structure. Unity gives you a total freedom in that matter, but because of that, it can frequently get really messy. This is the directory structure I personally use:

  • 3rd-Party
  • Animations
  • Audio
    • Music
    • SFX
  • Materials
  • Models
  • Plugins
  • Prefabs
  • Resources
  • Textures
  • Sandbox
  • Scenes
    • Levels
    • Other
  • Scripts
    • Editor
  • Shaders
  1. Do not store any asset files in the root directory. Use subdirectories whenever possible.
  2. Do not create any additional directories in the root directory, unless you really need to.
  3. Be consistent with naming. If you decide to use camel case for directory names and low letters for assets, stick to that convention.
  4. Don’t try to move context-specific assets to the general directories. For instance, if there are materials generated from the model, don’t move them to Materials directory because later you won’t know where these come from.
  5. Use 3rd-Party to store assets imported from the Asset Store. They usually have their own structure that shouldn’t be altered.
  6. Use Sandbox directory for any experiments you’re not entirely sure about. While working on this kind of things, the last thing that you want to care about is a proper organization. Do what you want, then remove it or organize when you’re certain that you want to include it in your project. When you’re working on a project with other people, create your personal Sandbox subdirectory like: Sandbox/JohnyC.

2. Scene hierarchy structure

Next to the project’s hierarchy there’s also scene hierarchy. As before, I will present you a template. You can adjust it to your needs.

  • Management
  • GUI
  • Cameras
  • Lights
  • World
    • Terrain
    • Props
  • _Dynamic

There are several rules you should follow:

  1. All empty objects should be located at 0,0,0 with default rotation and scale.
  2. When you’re instantiating an object in runtime, make sure to put it in _Dynamic – do not pollute the root of your hierarchy or you will find it difficult to navigate through it.
  3. For empty objects that are only containers for scripts, use “@” as prefix – e.g. @Cheats

3. Use prefabs for everything

Prefabs in Unity are not perfect, but they are the best thing you will find to share pre-configured hierarchies of objects. Generally speaking, try to prefab everything that you put on your scenes. You should be able to create a new level from an empty scene just by adding one or more prefabs to it.

The reason why you should use prefabs is that when a prefab changes, all the instances change too. Have 100 levels and want to add a camera effect on all of them? Not a problem! If your camera is a prefab, just add a camera effect to the camera prefab!

Be aware that you cannot have a prefab in another prefab. Use links instead – have a field that requires a prefab to be assigned and make sure to assign it when instance is created. Consider auto-connecting prefab instances in Awake() or OnEnable() when it makes sense.

4. Learn how to use version control system (VCS)

git logoYou may already know something about GIT, Subversion or any other VCS out there. As a matter of fact, “knowing something” is only a small piece of what you may learn. You should focus on learning about important but infrequently used features of VCS of your choice. Why? Mostly because VCS systems are much more powerful that you think, and unfortunately many users are using these as nothing more than a backup and synchronized solutions. For example, did you know that GIT allows you to stash your changes, so you can work on them later without committing anything to your master branch?

Programmers tend to comment out blocks of code in case it’s needed later. Don’t do that! If you’re using VCS learn how to quickly browse previous versions of a file. When you are familiar with it, your code looks a lot nicer without unnecessary block of commented code.

Here’s a nice resource of tips for GIT users: http://gitready.com/

5. Learn to write editor scripts

Unity is a great game engine in the matter of extensibility (see Asset Store). Learn how to write editor scripts and utilize this knowledge. You don’t necessary need to create fancy GUI for your scripts, it can be something simple, as menu entries that are doing something useful. Here are some examples of editor scripts that I have created not so long ago:

  • Google Sheets .csv download – I had a translation spreadsheet saved on Google Drive. It automatically downloaded the newest version as .csv file, so I never had to do it manually.
  • Randomize the position, rotation and size of trees – I had a lot of trees and wanted it to look more like a forest than a grid.
  • Create distribution – Built for specified target, zips all the files and copy to the right place.
  • String replace in the sources – I had several files that contained the application version.

You can learn how to create editor scripts from the official documentation.

6. Learn to program defensively

Have you heard about defensive programming? Wikipedia defines it as follows:

Defensive programming is a form of defensive design intended to ensure the continuing function of a piece of software under unforeseen circumstances. Defensive programming techniques are used especially when a piece of software could be misused.

Generally when you’re writing MonoBehaviours, you should make sure that:

  • All needed references are set
  • All required components are present
  • If you’re using singletons, make sure that they exists
  • If you’re searching for objects and expect to find something, do it as fast as possible
  • Mix-in editor code (ExecuteInEditMode and #if UNITY_EDITOR) to do as many checks as possible before you run the scene

For many of these checks you can use asserts. You should also read A Story of NullPointerException Part 1 and 2.

7. Implement in-editor and/or in-game cheats

After you learn how to write editor script, you should be able to write a set of in-editor cheats. It can work as menu entry that unlocks something (all levels for instance). It’s really easy to create:

Generally you should write cheats that will allow you to:

  • Unlock all levels, characters, items etc
  • Give you immortality
  • Add/subtract values like time, money, coins etc
  • Allow you to see things not meant to be seen by players
  • Anything else that will help you with testing your game

Of course more practical (but harder to write) are in-game cheats. These type of cheats can be executed outside Unity editor, but you have to think how you would like to execute it. See our other article about implementing cheats subsystem controlled by mouse.

Choosing Between Forward or Deferred Rendering Paths in Unity

One of the most important Unity features is the ability to choose a rendering path.  For those who are not very familiar with Unity, choosing (usually) between forward and deferred rendering paths may be something comparable to choosing between “normal”  and “strange looking and something’s broken” rendering methods. To understand better why there is more than one rendering path, first you will need to understand the motivation behind it.

It’s all about lighting

Lights are expensive, mostly because a lot calculations has to be done to find out the valid color of a pixel when there’s a light in range. In Unity lights can be evaluated per-vertex, per-pixel or as Spherical Harmonics (SH). In this article we will talk only about the former two.

pixel vs vertex lighting

In per-pixel lighting, each pixel color is computed individually (as on the left.) You can see that even when I use low-poly sphere for this example, the lighting still makes it look round. If it wasn’t for the edges, it’d be really hard to spot where all the vertices are. Then, there’s per-vertex lighting. It makes one light calculation per vertex. All the other pixels between vertices evaluate the color using regular color blending algorithm (without further light calculations.) This is the cheapest method of lighting and yeah… it looks cheap (if you’re wondering where’s the pixel and vertex lighting switch, it’s hidden in the Light component under Render Mode option. Important option is forces the light to be pixel light, Not Important is vertex light, Auto makes the strongest light a pixel light.)

It’s not a secret that game developers love per-pixel lighting much more than per-vertex lighting. Yet it has a significant downside. Each light causes the additional rendering pass of each object in the range. There’s a limit of four lights that can affect the object. What’s more, there’s also a limit of shadows – based on Unity documentation only one light can have shadows (for some reason I’ve managed to get two shadows in Unity 5.3.4, so I’m not really sure about this one.)

Deferred rendering to the rescue!

There’s a technique that allows you to use as much lights as you want on your scene with keeping the performance at reasonable level. It does not limit the number of shadows and it does not cause additional draw passes if scene objects are within light range (objects casting shadows are exceptions.) It’s called Deferred Shading Rendering Path.

4 lights deferred lighting

Why is it so different? Mostly because most of the models are rendered without lighting calculations and when the scene rendering is nearly done lights are applied to rendered 2D image. Making changes on this stage is usually called doing something in screen-space. Knowing that, we can say lighting in deferred rendering is screen-space. To understand it better, let’s look at the Frame Debugger.

Scene rendering starts with rendering all geometries:

deferred opaque

This is a flat image, so how graphics card will know how to apply lights and shadows? Thanks to the depth buffer! You can think of depth buffer as of another image that is hidden from you and that stores the information about how far from the camera each pixel is located. When represented as image, it may look like this:

cf538839-afaf-4afb-911b-52db5c062af7

Depth information alone isn’t enough to figure out how light should be applied on the surface. Still, we need at least one more thing – the orientation. Orientation in 3D space is usually represented by normals. The unusual thing is that along with color buffer and depth buffer, there is a buffer with normals!

346210b1-c144-4a66-b1bc-bd7488e3b06b

How can you tell that these are normals? It’s pretty easy! Just look at the Scene Gizmo.

scene gizmo

Do you see the color resemblance? Red cone (x) points to the left, so do left faces on previous image. Green (y) to the top and blue (z) bottom-right (from this perspective). It all matches the colors of faces from before.

Basing on that information, lights and shadows can be rendered. It really doesn’t matter how many objects there are on your scene. Everything gets done only on the final image.

After lighting pass

After lighting pass

The image above is the result is an inverted version (1 – color) of lighting pass. At the end it is blended with the first opaque image to get the final result.

Which one should I choose?

After reading all of this you may be full of enthusiasm to use the new rendering path, but hold your horses! Deferred rendering is not a remedy for all of the world problems. It has some…

Limitations

It would be too great to be true, wouldn’t it? There are some limitations.

First of all, deferred rendering does not allow us to render semi-transparent objects. That’s because if something semi-transparent exists on the scene, there’s no way to write down depth and normals for objects that is visible through semi-transparent objects and for current object itself. Unity handles this limitation rendering semi-transparent objects using forward rendering path at the end of the whole process. It works quite well, these objects can cast a shadow, but unfortunately they are unable to receive shadows from other objects. They can also cause some unexpected issues, not known when using forward rendering.

The second limitation is the lack of anti-aliasing support. The reason is similar to the issue with semi-transparent objects, but Unity does not try to workaround it in any way. Instead you can use screen-space AA algorithms (image effects), but the visual effect may be less good-looking.

Another limitation is that you can use up to four culling masks. In the documentation you can read:

that is, your culling layer mask must at least contain all layers minus four arbitrary layers, so 28 of the 32 layers must be set. Otherwise you will get graphical artifacts.

And finally there’s no support for the Mesh Renderer’s Receive Shadows flag.

Requirements

If that’s not enough, deferred rendering works only on a limited set of graphics cards. When it comes to PCs, you can safely assume that all graphics cards not older than 10 years will support it. When it comes to mobile devices, you should assume nothing. But that’s not a big issue, because…

Performance

The most important thing is that deferred rendering in most cases will get a worse performance on mobile devices than forward rendering. It’s because of additional passes that need to be done on each frame. If you’re using only one light, then it may not be worth it.

On the other hand, adding extra lights is quite cheap. In the worst case scenario performance will drop linearly and compared to forward lighting, it’s independent of number of objects on the scene.

Cities: Skylines (done in Unity) decided to use deferred rendering path. There's a lot of small lights in this game and it still performs really well.

Cities: Skylines (made with Unity) decided to use deferred rendering path. There’s a lot of small lights in this game and it still performs really well.

Resources

I hope that this article will cast some light on what rendering path you should choose for your game. Anyway, you may also be interested in these resources:

How to Integrate Steamworks with Unity Games

Integrating Unity Games with Steamworks

I believe that many of you have thought of publishing a game on Steam. It wouldn’t be surprising, as Steam is a great distribution platform for PC and now also for Mac and Linux games. But Steam is not only about distribution. When you get approved by Valve, you gain the access to something that may help you a lot with your game development. This little thing is called Steamworks.

Steamworks features

Here’s a list of some most known Steamworks features:

  • Achievements – provide free grass roots marketing for your application. As players unlock achievements it exposes your product to their friends.
  • Error Reporting – provides dead simple error collection so that you can quickly find and fix your most common bugs. With a few simple api calls Steam will automatically collect the most common crash reports for the game or software. You can then review error reports on the error reporting page, which you can find from your application landing page in Steamworks.
  • Cloud Saves – is free storage that gives players ability to play where they choose as well as the peace of mind tha they won’t lose all the work they’ve put into your game. Cloud can also be used for software applications to store work-in-progress or special configuration settings.
  • Steam Workshop –  is a system of storing, organizing, and downloading user-created content uploaded through your application. This makes sharing custom levels, skins, or complete mods easy and user-friendly.
  • Other features to consider are stats, leaderboards, and multi-player matchmaking.
As you will accept Steamworks SDK terms & conditions you will get the access to official Steamworks SDK documentation.

As you will accept Steamworks SDK terms & conditions you will get the access to the official Steamworks SDK documentation.

Integrating Unity game with Steamworks

Steamworks SDK is distributed as a native DLL file (*.so when talking about Mac and Linux). In order to make it work with Unity you have to create a binding. Fortunately such binding already exists and it is distributed also as an easy to install, unitypackage file!

I am of course talking about Steamworks.NET. It’s an open source wrapper distributed under MIT license (you’re free to use it even in commercial projects!). The good thing about Steamworks.NET is that the authors value API compatibility over simplicity. That means that you only need a quick look over how it should be used and when you’re familiar with the concept, all you need is the official Steamworks documentation. The downside of this approach is that callback setup need one extra step, but it’s not a hassle.

Installation

To make Steamworks.NET  work you have to be a Steamworks developer and you need an AppID (this is just a number in Steam database). At the time of writing of this article you can get one after passing Steam Greenlight or by making a custom deal with Valve.

When you have acquired an AppID all you have to do is import Steamworks.NET unitypackage file to your Unity project. At the time of writing of this article the current stable version is 7.0.0, but please use installation page links instead to always get the latest version.

steamworks unitypackage installation

Steamworks.NET package includes libraries for Windows, Mac and Linux in x86 and x86_64 architectures. After importing it you don’t need to add anything else to your project. Even official Steam dll/so is included, so there are just two more steps to go.

After importing the package, a new file called steam_appid.txt will be created in your project root directory (this is the one that contains the Assets and Library folders). Open it in the text editor and replace 480 with your Steam AppID.

Finally, the last step – create a new empty game object on your scene and add SteamManager script to it. There! Now you’re good to go!

Checking to make sure it works

Make sure that Steam is running. Then create a script like this:

Add this script to a new game object on your scene and hit the Play button. If everything is OK, you will see your Steam name in your Unity Editor console!

When something went wrong you will end with an error message that may not tell you what exactly has gone wrong. If you’re working on Windows then you may want to get DebugView application. Just run it before running your Unity game and after the error is printed out, alt-tab to DebugView window and see if there’s something more in there.

 

More information and getting help

You can learn more about how to get started (and how callbacks should be handled) on the Getting Started page of Steamworks.NET documentation. It you ever feel lost, you can use SteamworksDev discussion group. It’s invite-only so you should contact Steam about getting access to this one. It is worth it!

steamworks discussion group

If you ever feel lost, please leave a comment here or reach the Knights using our Facebook page.