New research is shedding light on how artificial intelligence language models handle multiple languages, a capability with potential implications for global commerce.
African tech firms have also launched tools supporting multiple local languages, responding to the continent’s linguistic diversity. Researchers are uncovering how these AI systems process different languages while businesses assess the technology’s impact on global markets and customer engagement.
“As businesses expand globally, linguistic diversity in AI tools has become essential,” Eleanor Lightbody, CEO of the AI company Luminance, told PYMNTS. “Contracts are the backbone of business operations, and to scale effectively, companies need tools that transcend language barriers and enable global engagement.”
Multilingual Large Language Models (LLMs) are AI systems trained on texts from multiple languages. This allows them to understand and generate content across various languages. For example, a multilingual LLM might be able to read a question in Spanish, process information from German sources, and provide an answer in English.
These models learn patterns and structures common to many languages and language-specific features. This enables them to perform tasks like translation, cross-lingual information retrieval and multilingual question answering.
One example of these types of LLMs is Google’s multilingual BERT model, which was trained on 104 languages and can reportedly handle tasks in all of them. Another is Meta’s No Language Left Behind project, which created a model capable of translating between 200 languages.
Another initiative is the multilingual LLM XLM-R, developed by Facebook AI. It can supposedly perform tasks like sentiment analysis and named entity recognition across 100 languages. For instance, it can identify positive or negative sentiments in product reviews written in Thai, Russian or Swahili, demonstrating its versatility in understanding context and nuance across diverse languages.
Multilingual LLMs are particularly useful in global contexts. They can help international businesses communicate with customers worldwide or assist researchers in accessing and understanding studies published in different languages.
Multilingual LLMs may become more effective. Researchers have identified a three-step process called “Multilingual Workflow,” offering insights into the inner workings of these sophisticated systems. A study published on the open-access site Arxiv, which has yet to undergo peer review, suggests that large language models (LLMs) employ a nuanced approach to multilingual tasks.
The process involves converting input to English, solving tasks and responding in the original language. The researchers found that they could boost performance across languages by fine-tuning just 0.1% of a model’s neurons using only 400 documents. This method resulted in an average improvement of 3.6% for high-resource languages and 2.3% for low-resource languages across all tasks.
These developments come as African tech companies make strides in creating AI solutions tailored to the continent’s linguistic diversity. Lelapa AI recently launched InkubaLM, Africa’s first multilingual AI language model. This open-source platform supports five African languages: Swahili, Yoruba, IsiXhosa, Hausa and isiZulu.
Botlhale AI Solutions reportedly has also seen increased adoption of its natural language processing suite, Bua. Developed last year for contact centers, Bua has gained clients, including Multichoice and MTN.
As research continues and new products emerge, AI’s ability to handle multiple languages fluently could have significant implications for businesses operating in linguistically diverse markets.
“Businesses can leverage LLMs by training them in multiple languages and editing for cultural and linguistic nuances,” Kyran Schmidt, co-founder of Outverse, an AI b2b customer service platform, told PYMNTS. “This allows for personalized customer support and boosts engagement.”
For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.
We’re always on the lookout for opportunities to partner with innovators and disruptors.
Learn More