Letters to the Editor Understanding the amazing complexity of sign language

MOST people are familiar with sign language, the system that hearing impaired people use to communicate. What fewer may know is that there are many different sign languages around the world, just as there are many different spoken languages.

So how does the grammar of sign language work? Unlike in spoken languages in which grammar is expressed through sound-based signifiers for tense, aspect, mood and syntax (the way we organise individual words), sign languages use hand movements, sign order, as well as body and facial cues to create grammar.

This is called non-manual activity.

To find out whether these cues are comprehensible to signers and non-signers of a country, my team of hearing linguists and translators conducted two studies. The results, which were published in July, demonstrate the incredible complexity of sign language.

What signers and non-signers see

In the first study, which included both signers and non-signers from Austria, we asked participants to watch a set of videos of people using Austrian sign languages.

We instructed them to try to break up the signed text into smaller units — the equivalent of cutting unbroken speech down into prosodic units.

The participants then went through the resulting segments and showed us the cues that had led them to break the videos where they did.

When it came to pauses and signs made with hands, signers and non-signers alike made similar decisions.

All participants identified rest positions, such as crossing one’s arms, as pauses as well as discerned holds — where a signer maintains the same hand position for a longer period of time or repeats the last sign of a segmented unit.

But when it came to cues from other parts of the body — non-manual activity — signers and nonsigners performed very differently.

Almost exclusively, sign language users also listed head and body movements as cues, as well as movements of the eyebrows, gaze direction and blinks.

Non-signers tended to identify only one or two cues from the hands.

Meanings of non-manual elements

The second study involved only hearing impaired Austrian Sign Language users.

Once again, we showed signed videos to the participants. But this time we instructed them to identify the non-manual elements that they thought had relevance to the language. That is, elements that acted as grammar.

Participants had to describe the form, meaning and function of each non-manual element.

The agreement between the signers’ description showed that certain body, head or facial movements have linguistic functions.

They express assertion, negation, conditionality (a phrase using the word if, for instance), hypothetical thoughts and alternatives, as well as time, location, and cause.

While shaking one’s head can be used to simply negate a clause or thought, for instance, other head shakes, performed in a slow, small and tentative way, can express the signer’s negative attitude toward a hypothetical thought.

Gaze can also serve several functions. So far, our data shows that signers consistently looked upward when indicating a hypothetical statement.

The position of the signer’s head, too, conveys different meanings. Positioning the head forward while formulating a hypothetical thought can be used to express a self-addressed, hypothetical question (such as should I go to the movies tonight?).

But moving the head forward can also accompany an “if” clause (If I go to the movies tonight, I might see Wonder Woman).

In other contexts, it can also act as an exclamation or imply possibility. To make our research publicly available, we used an approach that ensured accuracy of interpretation and translation.

Our linguists first discussed the outcomes with hearing impaired native signers in Austrian Sign Language. Then, the native signers described the outcome from their hearing impaired native signers’ perspective. Only then did the translators interpret the phenomena description into written German and English.

In a follow-up project (funded by the Austrian Science Fund), we will investigate the interplay of non-manual cues and clauses in several varieties of Austrian Sign Language, comparing our findings with non-manual activity of other sign languages to determine how the form and function of different sign languages vary across the globe.

Diversity in sign language

Including both non-signers and hearing impaired signers in our study enabled us to learn not only about grammar among people who sign, but about how we all interpret certain gestural behaviour.

The — perhaps unsurprising — fact that a nonsigner’s understanding of visual signals differs tremendously from that of a sign language user has pedagogical implications.

Because some of the same non-manual behaviours may signify different things depending on context, written explanations of sign language, such as textbook lessons, must incorporate the perspectives of native signers to avoid incorrect interpretations.

More broadly, the influence of body movements must be considered when describing any language. Our mental concepts are strongly shaped by visual configurations.

If sign language is any indication, the close relationship between how we gesture and how we think may matter more than previously thought. — The Conversation.