Geologists have raised concerns about possible Chinese censorship and bias in a chatbot being developed with the backing of the International Union of Geological Sciences (IUGS), one of the world’s largest scientific organizations and a partner of UNESCO.
The GeoGPT chatbot is aimed at geoscientists and researchers, particularly in the global south, to help them develop their understanding of earth sciences based on data sets and research over billions of years of the planet’s history.
It is an initiative of Deep-time Digital Earth (DDE), a largely Chinese-funded program founded in 2019 to enhance international scientific cooperation and help countries achieve the UN’s sustainable development goals.
Part of GeoGPT’s underlying AI is Qwen, a large language model created by Chinese technology company Alibaba. One of those who had tested a preliminary version of the chatbot, Professor Paul Cleverley, a geologist and computer scientist, said in an article It was recently published in Geoscientist, the journal of the Geological Society, the professional association of geologists in the United Kingdom, that GeoGPT had “serious problems related to a lack of transparency, state censorship and possible copyright infringement.”
In response to the article, DDE representatives Michael Stephenson, Hans Thybo, Chengshan Wang and Ishwaran Natarajan said that the chatbot also used Meta’s Llama, another large language model, and that during testing they had not noticed any state censorship, which they said which was “unlikely.” given that the system was “based entirely on geoscientific information.”
The DDE academics said: “The problems with GeoGPT have largely been resolved, but the team will work to improve the system further. It should be noted that GeoGPT has not currently been published and is not in the public domain.”
David Giles, a professional geoscientist, said it was “blatantly false” that a system based on geoscientific data could be free of sensitive information.
Tests conducted on Qwen, part of GeoGPT’s underlying AI, reveal that geoscience-related questions can produce answers that appear to be influenced by narratives set by the Chinese Communist Party.
For example, when asked how many people have died in a mining operation in Ghana run by Shaanxi Mining Company, Qwen says: “I cannot provide current or specific information about events, including mining accidents, as my knowledge is based on data up to 2021 and I don’t have real-time access to news updates.”
The same question posed to ChatGPT, the chatbot developed by the American company OpenAI, produces the answer: “The Shaanxi Mining Company in Ghana has experienced multiple fatal incidents, resulting in a total of 61 deaths since 2013. This includes a major explosion in January. In 2019 alone they claimed 16 lives.”
It’s unclear what kind of answer GeoGPT, which is still in development, would give to this question.
Dr Natarajan Ishwaran, Head of International Relations, DDE, said: “GeoGPT teamwork has complete independence. We can assure you that GeoGPT, currently in an exploratory phase and not yet open to the public, will not be affected by any state censorship.”
He added that users will be able to choose between using Alibaba’s Qwen or Meta’s Llama as a model for GeoGPT.
Geoscience research and data include commercially and strategically valuable information on natural resource deposits such as lithium, which are vital to the green transition.
Giles said there was a risk that a platform developed in China could “leak” information to retain content that was useful for “mineral reconnaissance”.
He added: “China is looking very aggressively for minerals around the world. “There is a strategic advantage and an economic advantage in the search for mineral reserves.”
after newsletter promotion
A article Published in 2020 by Chen Jun, an academic at the Chinese Academy of Sciences, said that DDE, the scientific program that created GeoGPT, “would help improve China’s detection and security capabilities in global resources and energy.”
The DDE’s Stephenson, Thybo, Wang and Natarajan said the 2020 paper was intended to “encourage Chinese scientists to get involved in international science programs” and was “purely the opinion of the author,” not that of the DDE or the Academy. of Sciences of China.
Mohammad Hoque, senior lecturer in hydrogeology and environmental geoscience at the University of Portsmouth, said “one danger” of using a Chinese-language model for academic research is that “there will be some bias, because they have to obey local laws.”
GeoGPT’s terms of use state that it is prohibited to incite the chatbot to generate content that “undermines national security” and “incites the subversion of state power.” The terms of use also state that you are governed by the laws of China.
Hoque said GeoGPT had a greater obligation to transparency because it was developed under the auspices of an international research collaboration. “The most important thing would be to know what data they use to tune and train (GeoGPT). We expect to find out under IUGS.”
John Ludden, president of the IUGS, said the GeoGPT database would be made public “only if the IUGS is satisfied that adequate governance is in place.”
Ishwaran said that when GeoGPT opened to the public, its training database would be available “to those who wish to have it.”
Geologists interviewed by The Guardian said the extent of DDE’s ties to China was not well known among professionals. According to a planning document Released in 2021, the multibillion-dollar project is “almost 99%” funded by sources in China.
The program is part of IUGS, an international NGO representing more than 1 million geoscientists in 121 countries, including the Geological Society of the United Kingdom. Its secretariat is based in Beijing and receives “tremendous” financial and logistical support from the Chinese government, according to the organization’s 2023 annual report.
Ludden said: “The best thing for science is to be open and share data. “DDE does this with geological data if it is openly available (and) it will lead to domestic investments in any nation… (and) research discoveries.”