LEARN PHOTOGRAMMETRY
Arthur Gatineau shares his advice for using photos of objects to create video game character assets
How to use photos of objects to make stunning video game assets
P hotogrammetry is fast-becoming a staple of many video game asset creation workflows. So for this tutorial I will cover the process of making a game character, focusing on the creation of some parts of my latest project, a survival character I imagined, which was inspired by the universes of games such as The Division and The Last of Us.
I will go through the steps of using Agisoft Photoscan to speed up making a pair of shoes and a rucksack. I will also explain how I created a shirt in Marvelous Designer, Zbrush and Substance Painter. While we won’t be covering the modelling process here, it might be useful to know that I used Zbrush and 3ds Max to create my character.
I used Marmoset Toolbag 2 to create a working real-time render, but ultimately I’m aiming to build a full scene in Unreal Engine to add more background to this character, in order to explore her personality in more detail.
If there’s anything you want to ask me about, feel free to email me at arthur.gatineau@gmail.com.
01 GET STARTED IN PHOTOSCAN
I don’t have the best set-up to take photos – just one Canon 550D – but the goal here is not to have a supersharp model right away. Instead, I’m looking to get the proportions and basic forms photographed.
The workflow is pretty straightforward. I start by taking 33 photos of a trainer from different angles, then in Agisoft Photoscan I go to Import photos>create Masks to get rid of the parts I don’t want Agisoft Photoscan to focus on – everything apart from the shoe. Next, I go to Align Photos and choose Build Dense Cloud from the Workflow menu. I set my parameters and click OK. Then in Build Mesh I set more parameters, click OK and export as a .obj file.
Import the raw scan into 3ds Max to remove the ‘floaty bits’ and close any holes. This step can be done in Zbrush, but I find I have more control in 3ds Max.
02 TIDY UP THE SCAN
Once in Zbrush, I tidy up the model using smooth strokes with a low intensity; hold Shift and adjust the Z Intensity slider to access an alternate Smooth brush. I continue sculpting missing parts where the photos didn’t capture anything.
Now, because I sculpted and smoothed some parts of the model, I need to redo the UVS, so I use the Uvmaster plug-in to avoid stretching in the texture. Doing this will also lead to better texture quality – we don’t care a lot about this here since it’s just a shoe, but it’s good to get into this habit. Finally, I import the model into Photoscan and use Workflow>build Texture to generate a new texture.
03 CONVERT TO POLYPAINT
As you can see on this before and after comparison (image 3b), it’s a great deal easier to work with this new UV layout. I take this texture back into Zbrush and convert it to a Polypainting texture by going to Tool>polypaint>polypaint From Texture. When doing this, don’t forget to flip V in the Texture tab, otherwise the texture won’t fit your model. Be careful: the fewer polygons your model has, the less detail you’ll get on your final texture, so make sure to divide quite a lot. I now export this mesh as my final high-poly model.
04 CREATE A LOW-RES MESH
Now I have to make a lowresolution mesh for my asset. I personally use 3ds Max to make the retopology, but the package doesn’t really matter. What’s important is to maintain the volumes of the high-poly asset, for instance the shoelaces. To bake the maps, I use xnormal. When doing this, don’t forget to untick ‘Ignore per vertex color’ in the properties of the corresponding high-poly slot.
I then bake another Polypaint with vivid colours to make an ID map, doing this will mean it’s easier to assign different materials later in Substance Painter.
05 LIGHTING TWEAKS
I notice there’s some lighting information in my texture. This is because the photos were not taken in an evenly lit environment. Here’s a simple method to extract the shadows and highlights in Photoshop: 1. Open texture in Photoshop. 2. Duplicate its layer by pressing Cmd/ctrl+j, and remove all saturation using Hue and Saturation.
3. Invert this layer with Ctrl+i and change the blending mode to Soft Light.
4. Add adjustment layers, such as Levels, and use Brightness/contrast to tweak the effect.
06 ADDING MORE DETAIL
Since the scan didn’t catch all the fabric details I need, I convert my diffuse map into a normal map and overlay it onto my original scan to add an extra layer of detail. Now I have all my maps ready, the
fun begins in Substance Painter! I learned Substance Painter by watching Christophe Desse’s Youtube channel (www.youtube.
com/user/xtrm3d), where he has plenty of videos showing texturing tricks, I really recommend it! I end up modifying the colours of the shoes to create a nice contrast with the trousers.
07 CREATING THE RUCKSACK
That’s it for the shoes. The second asset I need to create with the help of photogrammetry is the rucksack, which I want to be modern, and something that we see all the time so we can better identify with the character – so I choose to make an Eastpack bag.
It’s night-time by this stage and I don’t want to wait until the next day to take photos so the lighting is very bad, but I manage to capture images with the correct proportions and the main folds visible.
The scans of the rucksack are pretty rough and have many
gaps, which will need to be fixed. This is very common when working with scans.
08 FILL IN THE GAPS
I fill the largest gap behind the main side part of the rucksack and rear of the bag with a very simple mesh in 3ds Max. To do this I Dynamesh everything in Zbrush, use Zremesher to achieve a fairly low-poly first subdivision level, then I project the details back.
At this point, I sculpt all the elements that are missing from the scan, such as folds and stitches. I add a zip using the Zbrush IMM brush and then I open 3ds Max and model a zip puller with the polygon modelling tools found in the Polygon Modeling panel – it’s nothing fancy.
09 USING SUBSTANCE PAINTER
I use Substance Painter to bake the maps for the rucksack asset. What I love about Substance Painter is that I don’t have to explode my mesh any more to bake multiple pieces. All you have to do is to name your low-poly meshes with ‘_low’ at the end and your highpoly with the ‘_high’ suffix.
I assign different materials in 3ds Max to generate an ID map in Substance (don’t forget to switch from Vertex Color to Material Color in the ID baker parameters).
With the help of my ID map, I start to assign basic materials to the different parts of the asset. Texturing is an iterative process, layer after layer I develop the look of the asset; I am adding smaller and smaller details such as dust and tiny imperfections to make it look believable. You also have to think about the story. When a viewer sees the asset, he or she has to imagine the background of the model – whether it’s old, whether it was left outside for a long time.
10 MIX MARVELOUS DESIGNER AND ZBRUSH
My attention now turns to the character’s shirt, where I use Marvelous Designer. For me, Designer is a tool to facilitate and accelerate the process of creating clothes. It’s a very good base for sculpting, but I wouldn’t really call my Designer mesh the final high-poly one.
There are a lot of ways of working with Marvelous Designer and Zbrush, and you may already have your own. What I’m showing here is my workflow, inspired by the amazing Yuri Alexander (who has extra steps and explains this technique in more depth on his website www.yuri3d.com).
I start by decreasing the particle distance of my clothes in the Property Editor. To increase the density of the mesh, wait a few seconds for Marvelous Designer to make the new folds and export as an .obj file. Here are my settings for the exporting, under the Export OBJ menu:
1. Set Object to Single Object and turn on Thick.
Story development When texturing, consider the asset’s story. For example, my character could be put in a post-apocalyptic environment and the details would be believable.
2. Under Scale, turn on cm (DAZ Studio). 3. Keep all Invert options under Axis Conversion unticked.
11 EXPORTING THE SHIRT
I export the shirt in different sections, for example I separate the sleeves and pocket from the body. It’s important to do this in such a way that they don’t share borders, so the combination of parts doesn’t matter. I then import all the meshes into Zbrush using the same tool. From here, each of the following steps will have to be repeated on each of the Subtools: Under the Polygroups tab select Auto Groups, under the Modify Topology tab select Weldpoints and under the Zremesher tab make sure Keepgroups is selected to maintain the shape according to the groups we set earlier.
12 MERGE SUBTOOLS
Once finished, I can either continue like this with separate pieces and start dividing to add more details, or I can merge all of the Subtools using the Mergedown option under the Subtool tab – and that’s what I do here.
13 SCULPTING FOLDS
At this point, it’s time to look at my references again and start sculpting folds, memory folds, seams and some damage, such as wear and tear, to give a backstory to the shirt. I don’t use anything fancy here, just the Standard brush, Clay brush and Inflate. You can find some great, free brushes to download at Zbrush Central too, for example the ones by Michael Dunnam at
www.bit.ly/free-brushes. To add small folds, I recommend using the Drag Rectangle tool with a cloth texture such as alpha and a very low intensity on your brush.
14 TEXTURE AND FINESSING
After the retopology and bake (in Substance Painter) is done, it’s time to texture. The images below (14a) show the main evolutions of the look in Substance Painter. As before, it’s a layering workflow. When I feel that I’m getting closer, I like to go back and forth between the game engine and Photoshop to fine-tune the textures before creating a real-time render in Marmoset Toolbag 2.