Is “Generative Synesthesia” Underway?

In more ways than one, the advent of generative AI suites has stirred heated arguments and debates in creative communities. Some perceive the tech as potential augmenters of creativity, while others are convinced that it inhibits the human capacity to craft and create.

In the midst of these many discussions, the concept of “generative synesthesia” has been brought up – a concept which is largely defined as the amalgam of human creativity and the capabilities of generative AI engines to come up with something novel.

A study published in the PNAS Nexus journal recently roused the attention of creatives in finding that generative AI has the tendency to “enhance” creative productivity – not hinder its progression – and that the concept of “generative synesthesia” may be underway.

The Study

Titled “Generative Artificial Intelligence, Human Creativity, and Art”, the study looked into how the advent of generative AI tools like DALL-E, Stable Diffusion and Midjourney has had an effect on artists.

These tools are largely described as text-to-image generators, and can come up with different types of visuals and imagery based on a set of keywords or prompts.

The study mainly looked into how the adoption of the tools has had an effect on creative expression – assessing if the use of the tools is unlocking heightened levels of creative expression or not.

For the study, the proponents looked at the dataset of one of the biggest artsharing online platforms which included the works of “AI adopters”. The dataset covered more than 4.3 million digital artworks made by more than 53,000 creatives.

Of that number, artworks made by more than 5,800 AI adopters were included, and the researchers analyzed their creative workflows and the reception of their works from January 2022 to June 2023.

They utilized the “Conceptual Spaces” method for the study, the method being one that uses geometric representations in vector (scalable) spaces to identify or determine how much a new or current work aligns or deviates from older works.

In other words, they used the method to compare how AI-assisted works faired when compared to older digital pieces.

The method was complemented with implementations of analysis models and algorithms like BLIP-2 and DINOv2 as a means to qualify a body of work’s content and visual elements.

Its Findings

The study’s proponents found that AI adopters went through a productivity burst during the study’s covered time period, citing a 50 percent rise in the number of artworks that they produced at month-on-month intervals.

They also found that evaluations from their peers and followers on the quality of their works also improved over time, which hints that the perceived value of their works is improving and even resonating with audiences.

But as generally favorable as their findings were, they found a prevailing trend in a general decline in the variety of visual and content elements. In other words, they found a conforming quality in AI-assisted works – a common creative take on styles, so to speak.

However, they noted that a peak in content novelty increased, which implies that though there’s a trend in creative homogeneity, AI adopters are actively pushing for artistic innovation with AI generative tools.

Moving On

Eric Zhou, one of the study’s proponents, describes the study as an “initial exploration among a series of research where we seek to explore the unintended consequences of text-to-image generative algorithms on creative pursuits, both in terms of implications on the natural expression of ideas and the actual labor market dynamics that result from such technologies.”

“Our long-term goal is to provide guidance on how text-to-image generative AI can be leveraged for human enrichment while also exploring ways for traditional artists to continue to succeed,” he further writes.

As its general finding, the study alludes to the possibility that “generative synesthesia” is underway – evinced in finding that though content variety from AI-assisted art is leveling, indicators of content novelty is on the rise.