AI models beat humans on Chinese language skills

Artificial intelligence (AI) models from Chinese technology giants Tencent Holdings and Alibaba Group Holding understand the Chinese language better than humans, according to a benchmark test measuring natural language processing (NLP).

The two rival models have achieved record-high scores on the Chinese Language Understanding Evaluation (CLUE) benchmark, which is a set of tasks designed to evaluate how well a machine can understand and respond to Chinese text similar to how humans do.

This marks the first time AI models have scored better than humans on CLUE since the benchmark was established by researchers three years ago.

Tencent’s “Hunyuan AI model” came first with a score of 86.918, followed by Alibaba’s AliceMind with a score of 86.685, according to yesterday’s rankings on the CLUE website.

Both models ranked higher than humans, who have been given a score of 86.678.

AI models from Chinese smartphone maker Oppo and food delivery giant Meituan ranked fourth and fifth, respectively.

“Although the rankings change a lot, the top scorers had never surpassed humans until now,” Alibaba Cloud, the cloud computing arm of the e-commerce giant, said in an article published on Friday.

It added the latest results meant “the Chinese language understanding of the AI model has reached a new level”.

Alibaba owns the Post. China’s tech giants have been working to improve their NLP technology, which is used to support voice-enabled virtual assistants, such as Alibaba’s AliGenie and Tencent’s Xiaowei, as well as other functions like machine translation and spam detection.

Earlier this year, Chinese internet search giant Baidu said its bot Du Xiaoxiao wrote an article that scored higher than most students on China’s notoriously difficult national college entrance examination, although it mistakenly used an internet slang term.

Still, some researchers say most AI models still have a long way to go before they can truly understand the intricacies of languages.

Last year, scientists at Auburn University in the US state of Alabama and the research unit of American software company Adobe found many AIs that performed better than humans on certain comprehension tasks were unable to tell when words in a sentence were randomly shuffled.