Alibaba stands behind open-source model on LLM
Alibaba Group Holding has strengthened its commitment to the open-source development of its large language model (LLM) – the deep-learning technology used to train generative artificial intelligence (AI) services like ChatGPT – months after Tongyi Qianwen was made available to third-party developers.
The e-commerce giant could have been “more aggressive in open-source [development]” over the past year based on the gains made by Tongyi Qianwen, Alibaba’s Lin Junyang, who is in charge of the model’s open-source buildout, said in a WeChat post published on Monday.
“We really feel the power of the open-source community,” Lin said in the post published by Alibaba’s open-source platform, ModelScope. “After contributing to the community, the community has also given us a lot of feedback.”
Open sourcing gives public access to a program’s code, allowing third-party software developers to modify or share its design, fix broken links or scale up its capabilities. Open-source technologies have been a huge contributor to China’s tech industry over the past few decades.
After Tongyi Qianwen’s launch in April last year, Alibaba’s cloud services unit, which is responsible for AI initiatives, open-sourced two simpler forms of its LLM that were trained on 7 billion “parameters” – a machinelearning term for the variables present in the model on which it is trained that can be used to plan new content.
Alibaba Cloud last December opened access to the 72-billion parameter and 1.8-billion parameter versions of its LLM, while also making freely available another model that understood audio. These were made accessible via ModelScope and US opensource platform Hugging Face.
“After the 72B [version] was released, people felt Tongyi Qianwen had risen to a higher level,” Lin said.
He added the next step was to “compare [Alibaba’s progress] with the top international players”, acknowledging French start-up Mistral AI and Meta Platforms currently led the global open-source community in terms of LLM development.
Alibaba’s latest AI push could fuel further debate on whether China can continue relying on the open-source community instead of bolstering its tech ecosystem amid concerns that such collaboration would put the country at a disadvantage, especially if global tensions ratchet up further.
Still, Lin asserted the apparent advantages of continued opensource AI development.
“Open-source [development] has helped promote the commercialisation of our large [language] models,” he said.
He added “the developers [with access to Alibaba’s LLMs] are mainly the core engineers at our potential customers”.
In January, China approved this year’s first batch of LLMs. The latest approvals, for a total of 14 LLMs and AI enterprise applications for commercial use, come after an initial batch of generative AI services was allowed for release to the public last August.
Baidu co-founder, chairman and chief executive Robin Li Yanhong recently said the company’s Ernie LLM would not be open-sourced because this strategy was not cost-effective and reiterated a claim that there were too many open-source AI models on the market.
At the annual X-Lake Forum in Shenzhen last November, Li described the repeated launch of various LLMs in China as a “huge waste of resources”.
The number of governmentapproved LLMs and related AI applications in the country currently totals more than 40. But at present, there are more than 200 China-developed LLMs on the market.
Alibaba’s Lin also expressed global tech industry concerns about the scarcity of advanced graphics processing units, which provide the computing power in data centres for LLMs to train generative AI systems.
That echoed a comment by Alibaba co-founder and chairman Joe Tsai this month that China’s tech companies were “possibly two years behind” the top AI firms in the United States.
Alibaba owns the Post.
Open-source [development] has helped promote the commercialisation of our [LLMs]
LIN JUNYANG, ALIBABA