Improving on my guide for Screen-Space Halftones.

Tutorial / 11 February 2024

Why am I writing this?

4 Years ago while I was studying art for games I wrote a guide outlining how to make screen-space halftones using post processing in Unreal engine 4. Between then and now I have learnt so much about games, optimisation, workflows and tech art, so I am here today rewriting my 4 year old guide with a better understanding of halftones and post processing materials in unreal engine. Not a whole lot has changed in the way the code was laid out, however it is now significantly easier to put together, visually cleaner (both in code and game), and I have removed a lot of the redundancies.

Project Setup
Material Setup
Outlines (Simplified)
Halftones
Bokeh's (Stencils/Shapes)

Project Setup

There's a couple dependencies we need to setup before we get started on the materials. I noticed while working on the halftones that I would get a lot of blur as the camera moved around, by setting the project settings anti-aliasing method to FXAA the problem disappeared entirely. I know however that this doesn't work for every project so I will cover an alternative in Material Setup that may suit your needs better. With the project settings complete we also need to setup the scene;

create a post processing volume (or use one already in your scene).
With the volume selected search for Infinite Extents (Unbound) and enable it, this will make the volume influence the camera no matter where it is in the scene.
The last step, with the volume still selected search for the dropdown Render Features, inside this drop down there will be Post Process Materials here we will add an Element --> Asset Reference. This is where we will assign our halftones, outlines or other post processing materials.

Material Setup

Depending on if you can or cannot use FXAA, I have outlined two options for you in the image below. The main points to note are which material domain are we using and what step in the render process is it injected. both of these can be found on the master node of the material.

Outlines (Simplified)

Between my old post and this one, outlines have not changed. We are simply sampling a pixel, checking each adjacent pixel and seeing if the depth has changed, if the answer is yes then we are drawing black (0), otherwise we are drawing PostProcessInput0. I spent the time making a material function in the image after this to reduce the clutter.

Below is the material function for MF_Outline.

Halftones

Compared to my last post, these halftones are a walk in the park. There is two parts behind halftones, the shape and how light masks the output. In unreal engine it is quite simple to get the light value in a post process material. Starting with 2 SceneTexture nodes, we will set one of them to PostProcessInput0 and the other to SceneTexture:BaseColor (for lighting). By dividing our Input0 by the base colour we have now extracted the lighting information in our scene. We then convert this to greyscale and clip/step our shape. This gives us a mask that we then blend between Input0 and black with.

As games are made for all dimensions of monitors it is important that the shape we are sampling is the correct aspect ratio, by dividing the screen width by the screen height we get a ratio that we can use to square our screen UVs. This will eliminate any stretching we would otherwise see on non-square monitors.

Bokeh's (Stencils / Shapes)

With the way this shader is setup we can input whatever shape we like! I find that circles are still the nicest but that comes down to personal opinion and project art style. Below are a few examples of textures to input and their results. It is important that whichever image we use has a gradient from black to white. These were made using Affinity Photo and the "Outer Glow" layer effect.

Interior mapping and building tile-able rooms - UE4

Tutorial / 13 July 2020

Creating a cube map
Creating a basic material for interior mapping
Tiling Rooms by object scale and randomised light values
Pixel depth offset
Adding windows and curtains
Applying the material and uses
Shader complexity
Limitations
Articles and resources

Creating a cube map

To make interior mapping is a simple process in unreal engine that revolves around distorting a texture; in particular a cube-map. To make our cube map we can place a scene capture cube inside 6 textured planes. We can then detail our little room with other objects and furniture, but be careful with prop placement, angle and distance from the walls as some shapes can distort incorrectly.

Creating a material for interior mapping

For a very simple room we can simply have a texture distorted by the interior cube map node. If we have many rooms we may want to randomise the rotation which can be done by adding a static boolean into the randomise rotation input.

Tiling rooms by object scale and randomised light values

A super handy trick to quickly make an apartment building is to tile our texture by the scale of the object. To do this we first need to convert our 3-dimensional coordinates into 2 dimensional.

From here we can plug our vertex interpolator into the UV's of our cube-map. This on its own is enough to make an apartment building but its not fully believable as not all lights will be on or off at the same time. To fix this we can take our same UV co-ordinates and plug them into a Sobol noise node. This creates a grid of cells with values between 0 and 1 which we can then multiply and combine together to give rooms of varying light levels.

Pixel depth offset

Pixel depth offset is the act of changing the render order on a per pixel level by "pushing" selected pixels a certain distance "behind" other objects. In our case we can use it to place objects into our rooms. To achieve this we can take our UV's from our albedo cube-map and plug them into a black-to-white map. From here we are multiplying our pixel depth value by black (0) to white (1), thereby creating a mask that hides objects close to the window edges but fully encapsulates the objects in the centre.

Adding windows and curtains

There are many ways of adding window frames to our rooms. For a very detailed window we could have a mesh placed over the top, another route which i chose was to layer a few textures on top of each other. By using linear interpolators (Lerps) we can have our cube-map as our base layer and then layer curtains and window frames over the top. This uses black where we want our cubemap and white where we don't. Anywhere we have grey is a blend of both which is nice for achieving things like transparent curtains. For my curtains and windows I wanted a few variations, on one texture i had 4 curtains and 4 window frames that i could shift the UV's on to swap out which pairing I had.

Applying the material and uses

Using the material is quite simple, we can use any cube or plane and scale it to whatever size we choose. To place objects further back into the room we simply adjust the pixel depth offset and to place them off to the side we can change our mask.

Shader complexity

Overall the shader isn't too complicated, we could have many of these in scene before seeing a huge performance impact and for the amount of textures and materials it could save using and the time it saves making huge buildings is well worth having in large city projects.

Limitations

This shader is great but comes with a couple limitations. Rooms can become quite repetitive on large scale projects and creating rooms is limited to time and reducing distorted objects.

Articles and resources

https://forums.unrealengine.com/unreal-engine/events/1670169-unreal-engine-livestream-tech-art-with-chris-murphy-october-3-live-from-hq

https://www.youtube.com/watch?v=dUjNoIxQXAA

https://80.lv/articles/interior-mapping-rendering-real-rooms-without-geometry/

https://www.gamasutra.com/view/news/332409/Game_Tech_Deep_Dive_A_window_into_Playground_Games_latest_shader_development.php

A guide to Screen-Space Halftones, Cel Shading and Toon Shading for post processing in Unreal Engine 4.20.3

Tutorial / 20 February 2020

!! Important !!

This tutorial has been revised in a new post here. If you are after Toon or Cell shading continue reading here.

Post processing materials for UE4
Introduction to Cel shading
Cel Shading in UE4
Introduction to Toon shading
Toon Shading in UE4
Introduction to Halftones
Halftones in UE4
Dependencies for Halftones
Known issues
Links and resources used

Post Processing Material Setup for UE4

In Unreal Engine we have access to post processing, it allows us to adjust the look of our game through certain elements like bloom, vignette and more. We will be using the post processing volume to recalculate shading. To create a new post processing volume, under the modes window find the visual effects tab and drag in the post processing volume into your scene. The settings I have used for the volume are as follows:

Exposure min AND max brightness: 1.0

Infinite Extent (unbound): True (ticked), this setting is not necessary for the shader to work as the post process volume only influences cameras within it. With this setting ticked it acts as if you scaled the volume up.

Introduction to Cel Shading

So what is Cel Shading? Its a type of rendering used to imitate the looks of comic books or cartoons. Unlike conventional lighting with smooth gradients, Cel shading has hard edges and a step gradient for displaying light. To make Cel shading you can use two methods: Look up tables (Luts) or you can use maths to iterate the steps in gradient. For this tutorial i will cover only how the second method works.

Cel Shading for UE4

Cel shading for Unreal Engine is quite simple. The method I have chose involves using a value instead of a look up table for a little more modularity. This works by taking the scene colour including lighting (post process input 0) and desaturates the scene. We then multiply the output by the amount of Cels we want in our lighting, reduce it to a whole number, and then divide back to the 0-1 range. Finally we can take this output and add it back to out base colour to reintroduce lighting for our scene. I have included a 3 vector which has a default of white, with a purpose of colour correction if the scene happens to need it at any point. The last step to the graph is to bring back ambient occlusion by multiplying it by our previous results.

Introduction to Toon Shading

Toon shading is very similar to Cel shading with the only difference being added outlines that help to define the shapes of your objects. For our Toon shader we will use what we made in Cel Shading, then add detection for both outlines and normals.

Toon Shading for UE4

In Unreal Engine the process of calculating the outlines is to find the scene depth of the object and compare it to the background depth, at this particular point it will find the edge of the objects but nothing will be drawn on the screen. We can further determine the thickness of what we want drawn and finally our bias of what is drawn through this method:

The node network to create outlines is great for lines on the outer sides of objects, however it fails to draw any lines inside the object. To compensate for this we can duplicate our network for the outlines and convert all the scene textures from scene depth to world normal. For normals it is important to add an "addComponents" node and connect the output of the outlines into the float 3 input, then output from the float 3. From here we can take the maximum value of both graphs and use this as an alpha for a linear interpolation (lerp). For our lerp our alpha is our outlines, our B is the line colour and A will use what we made from Cel Shading.

Of course the outlines are now rendering all objects in scene, including our sky box. This is undesirable as the entire sky box becomes a grid of lines. To fix this we can make a culling mask to stop the lines from rendering beyond a certain threshold.

Introduction to Halftones

What are Halftones? Halftones are a method of printing that creates a gradient by the use of dots with various size or spacing. From a certain distance the dots that make up images become blurred with each other and create a pleasing image to look at, while being efficient at saving ink.

We can replicate this in Unreal Engine with a few methods, the first is a per-texture instance, this is not a dynamic solution and shading will be static across the objects. The second method uses post processing to overlay a pattern across the screen. This allows the camera to move and the dots to follow, but can be effected by the objects, their materials and the lighting on them, making for a fully dynamic solution.

This is a small scene made by Dris Hunt ( https://twitter.com/DrisHunt ) using assets from the Unreal store: Soul: Cave, Medieval Dungeon, Paragon: Revenant and Infinity blade. To find out more there are links at the end. Special thanks to Dris for helping find bugs, bug fix and create this shader.

Halftones for UE4

Using the components from both Toon shading and Cel shading we can develop a halftone shader. To achieve this I have used the values of light as steps in radius size for a spherical mask component. The input A is the position on screen for where the dots are placed and finally input B is the centre of where that dot is. The last input hardness does not need to be changed and can be left at 100, this simply changes the falloff or gradient for the dots. A small adjustment i made to Cel shading means that the shadows will be fully black, this was done by taking the Floor node and multiplying it to the base colour, then adding back in the cel shading afterwards.

From here we can use the output of the sphere mask as an alpha and reintroduce it into our graph. If we want the dot colour to be changed we can use a Lerp with input A as our output of cel shading. Input B will be a colour parameter which allows us to change the colour to whatever we like. As an optional step we can introduce the scene light and multiply it by our new dot colour parameter to bring in some light colours in the scene. This optional step can be a little problematic in some versions of unreal engine where the scene being divided outputs undesirable results.

The final step is to connect out last Lerp node into our outline lerp. Depending on where you would like halftones to influence you can connect either the line colour Lerp, the dot Lerp or the output of the Cel Shader into the A>B, A==B and A<B Statements of the render mask.

Dependencies for Halftones

There are a few dependencies for this shader, the first being materials. They are allowed to be as complicated as you like with only one exception, for the halftones to work correctly the RGB Channels must contain a minimum of 0.05 if you are to have any particular colour. For example if i wanted Red i would do R: 1, G: 0.05 and B: 0.05. However doing blacks and anything grey can be done regularly. The cause of this is due to taking out the scene colour through the cel shader, then multiplying the red and green channels of the screen UVs by the scene colour, thereby influencing the halftones.

Can i do metal and roughness? Metallic and Roughness are interesting, yes you may do both but be weary with metal as it reacts incorrectly.

Another interesting effect to play with is emissive. It reacts to the halftones on the emissive object, but not the areas surrounding it, furthermore the emissive colour is based on the diffuse of the surface rather than the emissive colour.

The final dependency for the shader is translucency. Any material like glass can be rendered, but is dependent on static or movable lights. With baked lighting the shading behind glass becomes completely black when in shadow. Otherwise in the same scenario with movable lighting the shading is as expected.

Known issues

There have been a few issues noticed with the shader. The first one is how shader complexity increases due to transulcency. This reduces the amount of glass able to be used within the scene.

The second issue is due to how light is extracted for the dot colours, this can cause shaded areas to appear the wrong colour and does similarly to some materials and colours.

The third issue is how the halftones are calculated from the cel shading. The cel shading comes in RGB and the red and green channels influence both spacing and size of the dots, not fully desirable but it can be worked around.

The final known issue is with alphas and complexity. As always when overlapping many alphas the shader has more to compute, increasing the overall performance impact.