your smartphone is a material scanner
Anthony Salvi teaches you how to create scanned 3D materials using a DIY Lightbox, a smartphone and Substance Designer 6
Create 3D scanned materials with a DIY lightbox
t he power of material scanning cannot be overstated. When done properly, it is the most accurate way to digitally reproduce a physical material available to us today. For a long time, however, the only way to access it was with a high-end scanner, which many people in the 3D community do not own.
Allegorithmic couldn’t buy everyone a scanner, but we could open up the technology to more people. And the exciting thing is, the research has paid off. In February we launched Substance Designer 6, which included a complete scan processing workflow that finally enables anyone with a smartphone to scan physical materials.
In this tutorial, I will describe each step of the scanning process, from the photoshoot to the final Substance, incorporating insights that we’ve learned over the last few years.
We will start with the concept of material scanning. Next, we will move on and learn how to build a complete cardboard lighting setup. From there, I’ll describe how to capture the material with a smartphone and how to postprocess the result.
For our next step, I’ll go into the node-based scanning process inside Substance Designer.
To finish, I will demonstrate how to convert this scanned material into a hybrid material with the help of Substance.
I really hope you enjoy following this tutorial and have fun with your first smartphonepowered material scan!
01 the concept of Scanning
Think of your material at the scale of a mountain. If the sun turns around the mountain, we can see the shape of the shadows cast in black. These shapes represent the indirect information about the relief of the mountain itself. If we combine enough information, the algorithm can calculate the relief. We need to turn the light eight times at the same distance around our material.
To keep our experiment cost-effective, let’s use some cardboard, a stack of tracing paper sheets and a LED light for our lighting setup. My preferred LED light is made by Manfrotto. For the image capture, it’s important to attain the best process possible, so for our tutorial we used an iphone 7 and the Adobe Lightroom mobile app. This app enables you to capture in the RAW (uncompressed) format on IOS and Android. Other apps like Procam or Camera FV-5 will work as well. Finally, we need a colour chart from X-rite.
03 Diy A SCANBOX
Our Scanbox, meaning the interior area where the scan will occur, is designed to capture a large amount of details from a material that measures up to 10cm x 10cm with opacity. We’ll also need a box to go around the interior, which in our case will be a 27cm x 22cm x 15.5cm cardboard box. It’s a good idea to cover the interior faces in white.
To retain the ability to capture the opacity of a material, we will cut a square hole (10cm x 10cm) at the centre of the box and place a stack of six sheets of tracing paper on top of it to diffuse the light through the material. On top of the paper, we’ll add a Scanning Chart to help us during the capture and in post. This chart is black to reduce the lighting bounces from our LED light onto our material sample. We’ll also add some shapes (like a square, triangle, moon and star) at the eight angles. These shapes are used by the photomerge process in Photoshop to produce a fast and accurate merging of the scan components.
04 make A lightbox
The Lightbox is designed specifically for our Scanbox. All faces inside the Lightbox should be white, while the outside faces should be black. To build this DIY Softbox, use four white foam boards (50cm x 50cm x 3mm), three black foam boards (50cm x 50cm x 5mm), two black paper sheets (42cm x 59.4cm) and a Scotch tape roll. Cut the Scanbox on six foam board elements, plus one paper tracer sheet. On the back, cut a door to put the LED light inside. The diffuse part is created by a tracing paper sheet.
05 create A STAND
To keep this project cost-effective, use a simple cardboard tube with a foam core plate to create a stand for your smartphone. To maximize the final frame, calibrate the stand size to the size of the box. Next add a black paper sheet to remove all potential colour bounces coming from the cylinder. Finally, attach your smartphone with four pieces of regular Scotch tape.
06 Final Setup
Here is our final setup with our smartphone, stand, Scanbox and Lightbox. Note: this setup is a recipe – you can create your own and enhance it however you want. You can switch the smartphone to a DSLR, or switch the LED light by using a wireless flash. It’s up to you and your budget.
In this tutorial, I’m going to focus on the capture process for a complex sportswear fabric from Tex-ray, a material manufacturer. To do this, we need to make sure that the only light on our material is coming from the Lightbox. If the shadow cast is wrong, you’ll have to deal with an erroneous computation during post-processing. So, just close the windows or curtains to darken the room and eliminate as much ambient light as possible. Then turn on your LED light and place it inside the Lightbox. Next, place the fabric on the Scanbox. Be sure the material is not covering the little shapes in the Scanning Chart. The fabric needs to fit perfectly inside the 10cm x 10cm square.
Next, clean your sample, as dust or hair will often fall onto the material. The most important thing is to not move the material sample during the shoot. If the material is moved, the photomerge cannot be done correctly. Once everything is set, it’s time to capture the material. Set the LED lighting power to maximum. Make sure you have enough battery power (smartphone and LED light) and that the LED light doesn’t become too hot. During the shoot, it’s important to keep the same framing as much as possible. However, don’t panic if your pictures are not perfectly aligned. It’s more important to keep all of the shapes (square, triangle, moon, star) present on the Scanning Chart visible in your images.
08 mobile capture
With Adobe Lightroom mobile, you have two options. You can use the PRO mode and manually set up your shot, or use the HDR mode. Both deliver good results. In the PRO mode, turn the flash off, activate the DNG format, set the ISO as low as possible (between 25 to 100) and set the White Balance to Daylight.
To avoid motion blur, keep a speed around 1/50 sec and adjust your ISO to get a good exposure. It can be helpful to add the grid and the level on screen for framing the shot. You can also add a timer for five seconds to act as a remote trigger for capturing the image. This will provide a more stable result as manually touching the shutter button on the camera can inadvertently add a shake, which may produce a blurrier image.
If you have an Adobe Creative Cloud account you can import your images into your Adobe Lightroom desktop library using your Wi-fi or 4G connection.
09 Shoot A Grey card
During the capture, it’s always a good idea to neutralise the colour shift coming from the lighting. By nature, all lights have a colour tint with some tints stronger than others. For example, a candle is red, a tungsten bulb is orange, the sky is blue. To neutralise this colour and keep colour consistency, a Colorchecker is required. Basically, it’s a reference, with a greyscale printed and calibrated. In post-production we’ll adjust the captured grey colour with Lightroom’s White Balance tool. In this example we have only one light source, so one picture with the grey card is required. You can find a grey card easily in a photo supply store, or on the web. X-rite, Datacolor, QP Card or Whibal all produce great White Balance targets.
10 capture A multi-angle light
The next step is to take eight pictures, with the Lightbox around the Scanbox (anticlockwise). We start with the Softbox in the number 1 side of the Scanning Chart. The smartphone is in front of the number 7.
We then take a picture and check the exposure and sharpness. If everything is okay, we move the Lightbox to the number 2 of the Scanning Chart. We repeat this process up to the fifth picture. Now we invert our position with the smartphone in front of the number 3. Now that we’ve completed this process, this step is finished.
11 capture the opacity
For opacity, open the curtains or turn the room lights on, and set up the LED light inside the Scanbox. With the LED light at a lower power, you’ll have enough backlight to capture the opacity. For a better light diffusion, you can also add white paper on the inside, with the light centred in the middle of the Scanbox. A foam core with a hole works well.
12 post-process All images
Next, we import our images into the desktop version of Lightroom. Once loaded, select the Colorchecker image for the LED light and use the White Balance tool. Just click on the middle grey example. Next, set the Whites, Blacks and Clarity parameters at 0. Then copy and paste the parameter value to the eight angle images. For the opacity image, use the gradient tool in Lightroom to reduce the vignette effect. For the last step, export all of the images as TIFF 16-bit using the Adobe 1998 ICC profile.
13 Do A photomerge
Using the photomerge feature in Adobe Photoshop, it’s possible to merge and align the scan images. Just load your nine pictures in photomerge and uncheck the Blend Images Together option.
After that, add a solid black layer at the bottom of the layer stack and crop your image on a square, resizing it to 4,096 pixels. Finally, export your layer stack in a new individual TIFF 16-bit file with the Remove Layers option. If you set up a stronger stand with additional accessories, this photomerge is not necessary, just crop your images in Lightroom in 4K.
14 create An Authoring material
It’s time to convert our images into a PBR material. Open Substance Designer 6 (a 30-day trial of the latest version is available at www.allegorithmic.com) and start by creating a new Substance file using the Physically Based (Metallic/roughness) template set to 4,096 x 4,096. Right-click on the Substance name, then select Link menu>bitmap to link the nine pictures to our project.
15 Select A crop
After a drag and drop from our resource images in the graph node window, hit the spacebar in the graph view and type ‘Multi Crop’ to add the scanning node. With the procedural approach, you can easily test different crops for a better tiling. In our case, just target the centre of the fabric and make sure you have enough space to cover at least a complete pattern. Set the Input Count to 8 and the Input Size to 4,096 on X and Y. To view and edit the crop, double-click on the last node output named Area on a flyover mouse.
16 create the normal
In the same way, add the Multi-angle to Normal node. Set the Normal Format to Directx (if needed), then set the value of Sample Amount to 8, Intensity to 1, First Sample Light Angle to 0.5/180 and the Next Sample Light Angle to Counterclockwise. Next connect the first eight outputs from the Multi Crop to the Multi-angle to Normal node. To remove the slightly normal variation, add a Color Equalizer node after your Multi-angle to Normal node.
17 make the colour map
Once the Normal is set, we can work on the colour map. For the Base Color, we have a node named Multiangle to Albedo. Add it in the graph from the Library or with the spacebar shortcut. There are only two samples – the angle number 1 and 5. If you want to use less than eight samples, just always select the opposite angles.
18 create the opacity map
We have a specific image for this opacity map. After a drag and drop from our resources, copy-paste the Multi Crop node and reduce the Input Count to 1. In doing this, you’ll be synced with the other maps.
After this node, add a Grayscale Conversion node, then a Histogram Scan node to push the contrast and produce a clean mask.
Next, put a Blur HQ Grayscale node to add a little softness on the mask. To control the amount of opacity, use this mask in a Blend node with two Uniform Color nodes as the inputs. All of these nodes are quickly available through the spacebar shortcut.
19 make it tile
The Smart Auto Tile is a new node that can be found via Library>material Filters>scan Processing. You can plug the Color map, Normal map and Opacity map into the three available inputs. One trick is to use the Height input for the Opacity map. As with the Multi Crop node, the last output is the pattern viewer/editor.
20 ADD A height map
One map that’s missing is the height or displacement map. By adding a Normal to Height HQ node, we can convert our Normal map to a Height map. Connect the Normal output from the Smart Auto Tile node to this Normal to Height HQ node. Since we set up the Multi-angle to Normal on Directx format, do the same for the Normal to Height HQ node. Play with the Relief Balance and Height Intensity to find a good balance. Turn the Quality to High.
21 Set the metallic map
In the Disney PBR shader, we have to set parameters for our materials, choosing which will be dielectric and which will be metallic. It’s really useful when only one texture set covers multiple materials, like you would find in a character. With this information the shader can apply a different behaviour.
22 ADD the roughness map
The fabric is composed by the same base material. This textile has no aging either. In this case, we can suppose the Roughness is uniform. As with the metallic, you will also use a uniform colour set to Grayscale, but here, the value depends on what you observe when you manipulate the fabric under the light. I set the value at 70.
23 ADD more outputs
Like before, press spacebar in the graph view and write ‘Output’. Then set one with the Opacity Usage. The second will be used for the Specular Level. With the Metallic/roughness definition, when the Metallic is set to 0, the material is understood to be a dielectric, and the reflectance value at the Fresnel zero angle or f0 is set to 4% reflective. The Specular Level can be used to override the default 4% value used in the metallic/roughness definition. For this material, a Specular Level at 80 was perfect.
24 create A runtime material
Our material is big, encapsulating nine 4K, 16-bit images. This is a huge amount of data if you want to export your Substance, use it in another 3D package, or even share it with your friends or team members as a downloadable file.
One solution is simply to export the Base Color, Normal and Opacity maps.
25 Dive into the hybrid world
It might be nice to go further and change the colour of your material, or add a pattern on top of it. This is where hybrid materials come in. In the next steps, we’ll learn how to customise our scanned material using the hybrid technology within Substance Designer.
26 match the colour
One of the best nodes for the hybridisation process is the Color Match node. You can find this node in the Library, under Filters>adjustments. This node enables us to replace a source colour with a target colour. With a lot of controls for the colour variation and the mask creation, it’s a very efficient node.
As you can see in this example, you can replace the yellow with a nice blue colour. This operation is clean without the loss of any fine details.
27 create A procedural pattern With the help of the Color Match node, the fabric now has two colours. Let’s combine them with a procedural pattern.
In Library>generators>patterns you have a lot of basic components. In this example, we used the Polygon 1 node. Just drag and drop the node from the Library to the graph view to start. This node is interesting due to its special Explode parameter. You can play around for hours and create hundreds of different variations with an artistic look. For this tutorial, we have set up this node to create a double triangle pattern around a circle.
28 ADD realism
A good material is often one that’s close to reality. In our case, that means close to the industrial process. We can imagine our pattern like a printed colour on top of a fabric.
To create that effect, use the Normal map to create some variations on the pattern. Under the Library>filters>effects we find the Vector Warp node for Grayscale input. Then plug in the node, the pattern and the Normal map. Set the intensity and put the Vector Format in Directx. Next, add a Blur HQ Grayscale for a soft touch.
A procedural approach fits well with the blending concept, as it enables us to create masks from nodes, and then keep a blend consistency throughout our work. Any pattern changes made during iterations will be reflected in the mask. To simulate a painted pattern on top of our fabric, we have to use our pattern as a mask.
30 ADD more control
Substance is incredible for controlling each aspect of a material. One of these aspects is the Normal Intensity. We have the ability to fine-tune our Normal and Height intensity and adjust their force by using a Normal Blend node and a Normal Color node. The Normal Blend node is set with an Opacity at 1 and Use Mask at True. The Normal Color node is set at 0. To control the blending, a Uniform Color node set in Grayscale is required.
31 publish our Substance
To finish our work and create a Substance with a hybrid material, we have to expose some of the parameters. In doing this, you keep the ability to modify the material without any access to the node graph. You can expose the Target Color in the Color Match node. To expose a node’s parameter, click on the sine graph icon at the right of the parameter name and choose Expose.
32 play with your material
It’s now time to use your material in a project! Here is an example of the material applied to a running shoe. Rendered in Substance Designer with Iray, this .SBSAR file can be loaded by 3ds Max, Maya, Modo, Cinema 4D, Adobe Fuse, iclone, Houdini, Unreal Engine 4, Unity and Lumberyard.
If your application is not in this list, you can always export your texture set (Basecolor, Normal, Height, Roughness, Metallic, Opacity, Specular Level and so on) from any Substance tool and use it in your shader.
Scan AND ADAPT materials here is our final result: a hybrid material based on our scans. the render is achieved in Substance Designer with iray.