How human senses are connected
It’s not just about your tongue. New studies on taste find that sight, hearing, smell and touch intertwine in a surprising hack of the senses.
IF YOU HAPPEN TO BE TASTING SOMETHING, THE MUSIC COULD ALTER THE WAY IT TASTES. — JANICE WANG, PSYCHOLOGIST
At its worst, wine criticism can be over the top, pompous, nonsensical, trying so hard to be evocative that it ends up just embarrassing. Claiming, for example, that a Syrah is chewy, or a Chardonnay is like pebbles in the mouth, can end up sounding, as the wine writer Paul White put it, “trite or pretentious or simply slink off into esoteric nonsense.”
The Canadian psychologist Qian Janice Wang knows this better than most. She is on the Oxford University wine tasting team known for its long-standing rivalry with Cambridge, and trained in patisserie at Le Cordon Bleu while working as a Microsoft software engineer. Today, with colleague and fellow “gastrophysicist” Charles Spence, she studies the human experience of taste, and how the other senses can skew it, for good or ill.
Through a series of controlled experiments that aim to change the experience of tasting wine by, for example, playing specially chosen classical music for the drinking subjects, she is learning that some of the most outrageous metaphors in wine criticism are not so far fetched or fanciful as they first seem.
Smells and tastes really can be colourful. They can have textures and shapes. Colours can be sweet or sour. So can sounds. Changing the shape or texture of a food can change its taste, make it seem sweeter or more bitter. So can changing the dinner music.
In this burgeoning field of psychology, these are called crossmodal interactions, and they add up to a novel realization about how humans experience the world. The five traditional human senses — sight, hearing, smell, taste and touch — are not distinct pathways into consciousness. They are not separate ways of perceiving. They are intertwined, in some cases physiologically, in others psychologically.
This, in turn, has raised questions about what senses are in the first place.
YOU CAN ACTUALLY PLAY WITH FOOD AT A VERY HIGH LEVEL.
“There’s no wide agreement on what a sense is,” Wang said in an interview. “It’s not always very clear cut how to separate the senses.”
Crucial ly, these crossmodal interactions seem to be constant in the population. Everybody has them. This is what sets them apart from true synesthesia, the rare cases in which some people seem to literally “hear” tastes, or “taste” sounds or “see” sounds. Synesthesia is also idiosyncratic — for example, one person might hear a middle C as blue, and another as green. But there tends to be broad agreement on crossmodal interactions.
Asked to pick a musical note to match various tastes, for example, a group of subjects in the Oxford taste laboratory gave answers that, when plotted on a graph, showed a clear and consistent spectrum from the low pitches of smoke, musk and chocolate, upward through the intermediate musical pitches with pepper, mushroom, caramel and violet, all the way to the highest pitches for apricot, lemon and apple.
Other new research in Canada shows similar effects on the sense of hearing, and how the appreciation of music can be dramatically affected by a person’s sense of movement.
And when psychologists like Wang try to exploit these crossmodal interactions, for both art and science, they are discovering combinations of, for example, taste and sound, or sight and smell, that complement each other beautifully, and others that clash unpleasantly.
Wang calls it “sonic seasoning,” and she traces her fascination with it to a revelatory dinner at the Chicago restaurant Alinea, famous for its adventurous cooking and dishes that have a dramatic twist, almost like the big reveal in a mystery novel.
She was especially struck by an edible helium balloon made of green apple toffee, which she was invited to in- hale from, talk in that funny high- pitched helium voice, then eat the deflated balloon.
“I was really inspired, and I felt like I could think of other ways to make that experience even more crazy, more interesting,” Wang said. “It showed me that you can actually play with food at a very high level.”
The experience led her to co- found an artistic movement, Crossmodalism, to put on public events that demonstrate this new science, such as a “perfume concert” or “multi- sensory dinner.”
Her PhD thesis was on the possible mechanisms that could explain why crossmodal interaction even happens in the first place.
Mysteries remain, but there are two basic possibilities. One is that this is some kind of neurogastronomical hack, exploiting physiological links between the senses. In mice, for example, there appears to be some neural connection of the auditory nerve and the olfactory bulb, which could be a promising track for explaining why hearing could affect taste.
Or the mechanism might be more like priming, playing mainly with people’s expectations, like a sensory placebo effect. Experiments that bolster this view include one that showed people liked a type of chocolate more if they were told in advance that it was expensive and made in Switzerland, and less if they were told it was cheap and made in China.
As ever with questions about human nature and nurture, the answer is a bit of both.
“We know that your expectations can actually change your sensory experience,” Wang said. And different music might draw your attention to different aspects of food or wine, giving them greater prominence in experience. Likewise, emotion and mood is known to play a big role in how much pleasure people can take from food and drink, and if music can affect mood, that is another possible mechanism.
“Much of the current work doesn’t inform to a high degree what might be causing these interactions, but certainly in terms of practical applications, they’re quite interesting,” said Gary Pickering, professor of biological sciences and psychology, wine science at Brock University in St. Catharines, Ont.
He thinks most crossmodal interaction is psychological, that it takes place in consciousness, rather than peripherally, in the taste buds or the nerves. For example, you can change a person’s perception of sourness in a white wine by simply adjusting how much citrus flavour is in it. And you can make a red wine taste more like berries simply by making it look a deeper, more intense red.
“That’s a psychological association (between citrus flavours and sourness, and the colour red and berry flavour),” he said. “That carries on at an unconscious level when we try complex products like a wine.”
Laurel Trainor, director of the Auditory Development Lab and the Institute for Music and the Mind at McMaster University, takes a different view.
“The short answer is that it happens before consciousness,” she said. Her latest research is on links in the brain between auditory and motor areas, or more broadly, how hearing and movement are linked.
By studying neural oscillations, better known as brain waves, as people listen to music, she has shown that a certain frequency of brain wave reaches a maximum at the very moment the brain expects the next musical beat, almost as if it were predicting the future.
“The punchline is that when you just listen to an auditory rhythm and you don’t move, we not only measure these (brain wave fluctuations) in the auditory cortex, but they also show up in motor cortex,” Trainor said. “So, even though you’re not moving, your motor cortex is acting as if it’s preparing you to move ... It’s showing that the auditory system is using information from the motor system to interpret the rhythm.”
“The basic idea is that there’s this interaction, so whatever you’re hearing, that actually enervates not only auditory areas but motor areas in the brain, and primes them to move along to the beat. And the way you move affects how you hear it, so there’s feedback from motor areas, saying ‘I’m moving at these times, so these must be important beats,’ and that biases your auditory system to say ‘Oh, this must be a waltz, or a march, because this is where the important beats are.’ ”
“This is really true of all of the sensory systems,” Trainor said. The senses are intertwined long before their input shows up in conscious experience. Interaction, on this view, seems inevitable.
Brains evolved to help us survive efficiently, so it makes sense to integrate the experience, Trainor said. Likewise, Wang identified evolutionary reasons for the associations between, for example, large objects and low pitches (like a lion’s growl) and small objects and high pitches (like a bird’s cheeping).
But as to whether these associations are innate or learned, or whether they occur in the mind or in the sensory organs, remains a tantalizing psychological mystery.
“The short answer is we don’t know,” Wang said. “It’s not like if you just hear music you’ll taste things in the mouth. But if you happen to be tasting something, the music could alter the way it tastes.”