Text me a face

Back in the ancient times of 2018 (by AI standards), NVIDIA published an AI model called StyleGAN which could generate very realistic images, particularly images of faces. This was an impressive achievement which probably led to a lot of fake profiles on Facebook. Still, all it can do is generate random faces and it isn’t possible to control what the face should look like with a text description, or prompt, like DALL-E can.

Unfortunately, DALL-E is not open source like StyleGAN and so can’t be used freely. However, it is possible to use prompts with StyleGAN by using the following recipe.

DALL-E and ChatGPT are not the only interesting things OpenAI developed. They also developed a program called CLIP (Contrastive Language-Image Pre-training) which can be used to quantify how much an image matches a text description. This program is open source and can be used to quantify how much a randomly generated

StyleGAN face matches a description. The face is random because StyleGAN, being a GAN (Generative Adversarial Network), converts a bunch of random numbers into an image. Any set of random numbers will give a proper face and slightly changing the numbers will slightly change the image. By optimising these numbers in such a way that CLIP increases the image’s score for a given description, eventually, you will arrive at a face that matches the description.

The University of Malta, together with Threls Ltd, have implemented this process in a mobile app that allows you to generate and edit face images via text prompts. These faces are free to use, even commercially, since AI-generated photos cannot be copyrighted and belong in the public domain.

The project face:LIFT (faces: Lifelike Images From Text) is funded by the Malta Council for Science and Technology FUSION programme, and a demonstrator version will be available both as a mobile app and as a website. For more information, visit www.um.edu.mt/projects/facelift/.

Marc Tanti is a lecturer with the Institute of Linguistics and Language Technologies and Adrian Muscat is a professor with the Department of Communications and Computer Engineering within the Faculty of ICT, University of Malta.

•

•

•

•

NASA’s James Webb Space Telescope’s NIRCam (near-infrared camera) instrument has captured new detail of the Horsehead Nebula (right image), providing the sharpest infrared images to date and portraying the region’s complexity with unprecedented spatial resolution. The ethereal clouds that appear blue at the bottom of the image are dominated by cold, molecular hydrogen. Red-coloured wisps extending above the main nebula represent mainly atomic hydrogen gas. In this area, known as a photodissociation region, ultraviolet light from nearby young, massive stars creates a mostly neutral, warm area of gas and dust between the fully ionised gas above and the colder nebula below. PHOTO: NASA, ESA, CSA, K. MISSELT (UNIVERSITY OF ARIZONA) AND A. ABERGEL (IAS/UNIVERSITY PARIS-SACLAY, CNRS)

These people don’t exist. They are images that were generated by our app using the description ‘An attractive Mediterranean woman looking happy’ for the left and the same description but with ‘man’ instead of ‘woman’ for the right.