Cre­ate a fun face de­tec­tion photo app

52 Cre­ate a fun photo app

Web Designer - - Con­tents -

Com­bine the power of fa­cial recog­ni­tion and a we­b­cam to build an app us­ing clm­trackr and the p5.js li­brary

With the rise of so­cial ap­pli­ca­tions, a big part of their pop­u­lar­ity has been their built-in photo ma­nip­u­la­tion tools. In­sta­gram has be­come syn­ony­mous with fil­ters and more re­cently so­cial apps have cap­i­talised on the rise of Aug­mented Re­al­ity to recog­nise faces in images and to over­lay props on var­i­ous fa­cial fea­tures. This has been hugely pop­u­lar among users and a con­tribut­ing fac­tor in the mo­bile suc­cess of cer­tain apps. In this tu­to­rial, an app for cre­at­ing photos with face de­tec­tion will be cre­ated run­ning in the web browser. The fa­cial recog­ni­tion is added us­ing the clm­tracker (con­strained lo­cal mod­els tracker) li­brary and then props will be added to the face us­ing P5js to ma­nip­u­late the HTML5 can­vas.

The clm­tracker recog­nises 70 unique points on a face giv­ing ac­cess to the x and y val­ues of those points. Us­ing these points it’s pos­si­ble to add glasses to the eyes, hats to the top of the head, false mous­taches and just about any­thing that you can think of as a photo prop. The ob­jects are con­trolled us­ing code so that they ori­ent to the head if it tilts from side to side by mea­sur­ing the an­gle be­tween the eyes and ap­ply­ing that to the prop. The scal­ing is worked out from the dis­tance be­tween the eyes so that the props scale to the face no mat­ter the dis­tance. A photo app needs to al­low the user to save their im­age once happy, so that will also be the fi­nal fea­ture.

1. The be­gin­ning

Open the ‘start’ folder in your code IDE and then open ‘in­dex.html’ to edit. In the body sec­tion of the page add the logo and the in­struc­tion div tags as shown be­low. These will be styled up to sit over the top of other con­tent with CSS.

<div id=”logo”><img src=”img/logo.png” alt=”photo Boom Web Stu­dio”></div>

<div id=”inst”>use left and right ar­row key to change prop

<br> Press S to save the screen


2. Defin­ing the styles

In the head sec­tion add style tags to add this CSS to, or al­ter­na­tively you could put this in its own CSS file. This code styles the body to fill the screen, re­mov­ing the de­fault browser mar­gins and chang­ing the type­face that will be used for the in­struc­tions. html, body { width: 100%; height: 100%; over­flow: hid­den; pad­ding: 0; mar­gin: 0; font-fam­ily: “Hel­veti­ca­neue-light”, “Hel­vetica Neue Light”, “Hel­vetica Neue”, Hel­vetica, Arial, “Lu­cida Grande”, sansserif; font-weight: 300;

-we­bkit-font-smooth­ing: an­tialiased; }

3. Scal­ing the app

The app will run in a can­vas ele­ment with a video ele­ment in the back­ground. Here these tags are styled to take the full width and height of the screen, so that they are al­ways viewed in the same place one over the other. video { dis­play: block; width: 100% !im­por­tant; height: auto !im­por­tant; ver­ti­cal-align: top; } can­vas { dis­play: block; width: 100% !im­por­tant; height: auto !im­por­tant; ver­ti­cal-align: top; }

4. Po­si­tion the logo

The logo div tag will be placed in the top-left cor­ner of the screen at a higher z-in­dex to other con­tent. The im­age in­side this is then just re­sized to make this fit bet­ter on the dis­play and not take up too much room within the de­sign. #logo { po­si­tion: ab­so­lute; z-in­dex: 1000; }

#logo img {

width: 310px; }

5. Plac­ing the in­struc­tions

The in­struc­tions are po­si­tioned in the bot­tom left of the screen, also above all other con­tent. There is a slight pad­ding added and the back­ground is set to a semi-trans­par­ent black with white text to stand out over the top.

#inst { po­si­tion: ab­so­lute; bot­tom: 0; left: 0; z-in­dex: 1000; pad­ding: 30px; back­ground: rgba(0, 0, 0, 0.6); color: #fff; }

6. Sketch­ing it up

Now save the HTML and switch over to the ‘sketch.js’ file. This will hold all of the code. At the top of the page add in some vari­ables for the tracker, can­vas and video, then other will hold var­i­ous props that will be dis­played on the screen. var ctracker, cnv, videoin­put; var glasses, mous­tache, whiskers, pipe, ears, hat, nose; var mode, sav­ing;

7. Preload­ing the images

Be­fore the images can be dis­played in the app they first have to be loaded in. Thank­fully p5.js has a ‘preload’ func­tion that will load these el­e­ments be­fore the rest of the code runs. Here the images are loaded and stored in vari­ables. func­tion preload() { glasses = load­im­age(“img/glasses.png”); mous­tache = load­im­age(“img/mous­tache. png”); whiskers = load­im­age(“img/whiskers.png”); pipe = load­im­age(“img/pipe.png”); ears = load­im­age(“img/ears.png”); hat = load­im­age(“img/hat.png”); nose = load­im­age(“img/nose.png”); }

8. Set­ting up the app

The ‘setup’ func­tion runs right af­ter the preload has fin­ished. This cre­ates the video from a we­b­cam and sets the size of the video, which is also the same size as the can­vas. The pixel den­sity of the dis­play is set to 1, which means it ig­nores high den­sity dis­plays and this is nec­es­sary to make the im­age sav­ing work. func­tion setup() { videoin­put = cre­ate­cap­ture(video); videoin­put.size(800, 600); videoin­put.po­si­tion(0, 0); cnv = cre­ate­can­vas(800, 600); cnv.po­si­tion(0, 0); pix­elden­sity(1);

9. Add the tracker

The tracker is added and ini­tialised, and it is set to work on the video. The ‘mode’ vari­able is used to change the prop that is be­ing dis­played on the im­age. The one here refers to the first im­age, which is the glasses. The ‘sav­ing’ vari­able is used to save the frame as an im­age, so the user can take a snap­shot. ctracker = new clm.tracker(); ctracker.init(pmodel); ctracker.start(videoin­put.elt); nos­troke(); mode = 1; sav­ing = false; }

10. Up­dat­ing the screen

The ‘draw’ func­tion is called ev­ery frame to up­date the screen. Here the screen is cleared be­tween frames. The ‘po­si­tions’ vari­able is up­dated with the track­ing po­si­tion of the head. The ‘if’ state­ment checks the ‘sav­ing’ vari­able, if it’s set to true the video is writ­ten in as an im­age, oth­er­wise the can­vas has a trans­par­ent back­ground. func­tion draw() { clear(); var po­si­tions = ctracker. getcur­rent­po­si­tion(); if (sav­ing) { im­age(videoin­put, 0, 0, width, height); }

11. De­tect­ing a face

If a face is de­tected, then 70 po­si­tions of the face are stored in the ‘po­si­tions’ ar­ray vari­able. If the ‘po­si­tions’ is greater than one, then a face is de­tected. The an­gle is cal­cu­lated us­ing the po­si­tion of the eyes and also the

dis­tance be­tween the eyes. This will en­able the props to be ro­tated and scaled ac­cu­rately. if (po­si­tions.length > 1) {

//an­gle be­tween eyes gives an­gle of head var ang = atan2(po­si­tions[27][1] po­si­tions[32][1], po­si­tions[27][0] po­si­tions[32][0]);

//dis­tance be­tween eyes so can work out scale

var d = int(dist(po­si­tions[27][0], po­si­tions[27][1], po­si­tions[32][0], po­si­tions[32][1]));

var scl = map(d, 0, 290, 0, 1.8);

12. Po­si­tion the sun­glasses

The sun­glasses are go­ing to be po­si­tioned at point 33, which is be­tween the eyes. The glasses are then ro­tated to the cor­rect an­gle and scaled. Half the width and height is taken off the glasses so that its an­chor point is the cen­tre of the im­age as it’s dis­played in the cen­tre of the eyes. if (mode == 1) { push();

//33 po­si­tion be­tween eyes trans­late(po­si­tions[33][0], po­si­tions[33]

[1]); ro­tate(ang + PI); scale(scl); im­age(glasses, 0 - glasses.width / 2, 0 - glasses.height / 2); pop(); }

13. Chang­ing the mode

The ‘mode’ vari­able will be changed with the ar­row keys so when the mode is set to 2 then the mous­tache will be po­si­tioned at point 37, the base of the nose. You will no­tice that the po­si­tion is set us­ing ‘po­si­tions[37][0]’, which stores the x value and [1] stores the y value. else if (mode == 2) { push();

//37 po­si­tion base of nose trans­late(po­si­tions[37][0], po­si­tions[37]

[1]); ro­tate(ang + PI); scale(scl); im­age(mous­tache, 0 - mous­tache.width / 2,


pop(); }

14. Plac­ing the whiskers

The next prop for mode 3 is the whiskers im­age, which will be placed at the point on the mid nose. The scal­ing of this im­age is in­creased as well so that the whiskers stick out a lit­tle more from the side of the face and so that they start out a lit­tle from the nose on each side. else if (mode == 3) { push();

//41 po­si­tion mid nose trans­late(po­si­tions[41][0], po­si­tions[41] [1]); ro­tate(ang + PI); scl *= 1.8; scale(scl); im­age(whiskers, 0 - whiskers.width / 2,


pop(); }

15. Pipe dreams

Mode 4 will be the pipe im­age and this is go­ing to be placed at the left-hand side of the user’s lip. If the light­ing is good, it tracks pretty well to the mouth open­ing and clos­ing, which adds that re­al­ism for the prop work­ing with the user. else if (mode == 4) { push();

//56 po­si­tion left mouth trans­late(po­si­tions[56][0], po­si­tions[56]

[1]); ro­tate(ang + PI); scale(scl); im­age(pipe, 0 - pipe.width, 0); pop(); }

16. An­i­mal ears

The next prop is an­i­mal ears and a new dis­tance is cal­cu­lated for the length of the nose. The ears are go­ing to be po­si­tioned be­tween the length of the nose above the mid­dle of the user’s eyes, which is just above the hair­line on most people. else if (mode == 5) {

//33 to 62 length of nose place this dis­tance above 33 for ears

var d2 = int(dist(po­si­tions[33][0], po­si­tions[33][1], po­si­tions[37][0], po­si­tions[37][1])); push();

//33 mid eyes trans­late(po­si­tions[33][0], po­si­tions[33]

[1]); ro­tate(ang + PI); scale(scl); im­age(ears, 0 - ears.width / 2, 0 (ears.height + d2));

pop(); }

17. I tip my hat to you

The hat prop is slightly tilted to one side, so this will be placed above the left eye­brow, and once again the length of nose is cal­cu­lated to po­si­tion this above the eye­brow. We have to use a dy­namic po­si­tion as there is no other way to know how close the user is to the cam­era. else if (mode == 6) {

//33 to 62 length of nose place this dis­tance above 33 for ears

var d2 = int(dist(po­si­tions[33][0], po­si­tions[33][1], po­si­tions[37][0], po­si­tions[37][1])); push();

//20 po­si­tion left eye­brow trans­late(po­si­tions[20][0], po­si­tions[20] [1]); ro­tate(ang + PI); scale(scl); im­age(hat, 0 - hat.width / 2, 0 - (hat. height + d2));

pop(); }

18. Last prop

The last prop to be dis­played is a clown’s nose. This is easy to po­si­tion as we have a point that is the nose. All of the props are now ready to go, but the user still needs a way to con­trol them shift­ing from one prop to an­other with the key­board. else if (mode == 7) { push();

//20 po­si­tion left eye­brow trans­late(po­si­tions[62][0], po­si­tions[62]

[1]); ro­tate(ang + PI); scale(scl); im­age(nose, 0 - nose.width / 2, 0 - nose. height / 2);

pop(); } }

19. Fin­ish­ing the draw

The end of the draw func­tion is now com­pleted and the im­age is saved out us­ing the file name ‘me.png’ and the ‘sav­ing’ vari­able is set to false so that hun­dreds of images aren’t cre­ated! The user will con­trol the sav­ing by press­ing ‘s’ on the key­board. if (sav­ing) { save­can­vas(cnv, ‘me’, ‘png’); sav­ing = false; } }

20. De­tect­ing key presses

The last part of the func­tion­al­ity re­quires key presses so this code de­tects if the right cur­sor key is pressed and cy­cles up through the modes. This in turn changes the props on dis­play for the photo app on the screen. func­tion key­pressed() { if (key­code === RIGHT_AR­ROW) { if (mode == 7) { mode = 1;

} else { mode++; } }

21. Fi­nal step

The fi­nal part of the code does the same as the pre­vi­ous step for the left ar­row key to cy­cle down. The ‘s’ key is mapped to save the im­age. Save all your files and test them on a server to see this work­ing. Make sure you have rea­son­able light­ing on your face for best re­sults. if (key­code === LEFT­_AR­ROW) { if (mode == 1) { mode = 7;

} else { mode--; } } if (key­code === 83) { sav­ing = true; } }

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.