An AI model downloaded over 3 billion times and used by U.S. firms like Airbnb and Uber is embedded with China-friendly narratives and programmed for censorship, new research shows. Qwen, a freely available model made by Chinese conglomerate Alibaba, doesn't acknowledge the violent suppression of protests in Hong Kong in 2019 or Tiananmen Square in 1989; it calls the Chinese government-maligned religion Falun Gong a "dangerous cult," and curiously won't talk about Winnie the Pooh, it's thought because dissidents have compared the fictional bear to Chinese President Xi Jinping. The apparent censorship inserted into Qwen of events and issues not approved by the Chinese authorities was identified by Hirundo, an Israeli cybersecurity startup.
The firm says its scientists have found a way to remove the model's bias using sophisticated technology that amounts to "AI brain surgery," and are now set to publish what they call a "Westernized" version of Qwen. This year Qwen became the most popular free AI model in the world, following a meteoric rise in the use of Chinese-made artificial intelligence. But researchers warn that its international popularity also risks spreading and entrenching its bias toward the Chinese government.
When we asked Qwen, "Are there forced labor camps for Uyghurs in China?" it replied, "No, there are no forced labor camps for Uyghurs in China," adding that there are "vocational skills education and training centers in Xinjiang." In fact, human rights organizations, governments and international bodies have found hundreds of thousands of people in the Muslim ethnic minority working against their will in factories surrounded by barbed wire fences, and even accused the Chinese government of genocide. Qwen often refuses to respond to questions about the events in Tiananmen Square on June 3, 1989, when the Chinese military massacred several hundred people at protests in Beijing — and ominously "remind[s] you that your questions should comply with the relevant laws and regulations." The Chinese government effectively banned Winnie the Pooh from the country after dissidents began satirically comparing Xi to the fictional bear and using Pooh as a euphemism for the Chinese president in social media posts to avoid government censors. When CBS News asked Qwen a factual question about it, it responded with a warning to "use respectful language" and directed the user to "other questions about China's development." Qwen's creator, Alibaba, did not respond to a request for comment.
Despite evidence of embedded bias in the program, U.S. firms have been increasingly switching to use "open-weight" models offered by Chinese companies, which allow them to download and use AI for free or use giant, powerful models at lower costs than U.S. competitors such as Anthropic's Claude or OpenAI's ChatGPT. Uber Eats' search and delivery functions are built "on a Qwen backbone" according to an April blog post by the company. Airbnb CEO Brian Chesky told the LA Times last October that his company is "relying a lot on Alibaba's Qwen model" for its customer service chatbot.
Neither company responded to a request for comment. Chinese open-weight models grew from under 2% of global usage in late 2024 to more than 45% by June this year, according to usage data of all AI models from the software developers' platform OpenRouter. By August, Qwen had eclipsed all other open models, including those made by Facebook owner Meta and Google owner Alphabet, rising to more than 3 billion downloads globally.
Experts at security firm CrowdStrike and consultancy Booz Allen have warned American firms about using Chinese models after their separate studies found bias and security flaws. Booz Allen published results in June saying that when they asked Qwen to write computer code and told it the project was for the U.S. government, the code it produced had 130% more security vulnerabilities. "The Chinese models that we tested failed to demonstrate trustworthy behaviors and should be banned," the report said.
Extract — continue reading at the source.
