Artificial Intelligence (AI) is rapidly gaining ground in science. Whereas the use of algorithms was until recently limited to a few specific fields, the availability of generative AI tools such as ChatGPT offers attractive possibilities for virtually all disciplines. Generative AI (GAI) can automatically generate content, such as text, images, or programming code, at a user’s request. With the help of GAI, researchers can write articles faster, analyse larger datasets, and gather relevant literature more easily. Generative AI can also facilitate communication with parties outside the scientific community, for example, by producing accessible texts or animations.

However, GAI also frequently produces biased and inaccurate analyses. In an open letter, over 1,800 scientists based in the Netherlands warn of the dangers GAI poses to scientific integrity, privacy, sustainability, and other public values. Scientific journals are currently being flooded with fake articles, sometimes even listing authors who are unaware of their inclusion. Furthermore, the use of the most common GAI systems increases the dependence of research institutions on a handful of extremely powerful technology companies based in the United States. Finally, there are serious concerns about the amount of water and energy consumed by data centres, as well as the exploitation of workers in the Global South who are assigned the task of selecting disturbing texts and images into categories such as sexual abuse and violence.

Currently, a large responsibility rests with individual scientists to weigh these opportunities and risks and to assess if and how GAI can be used responsibly in science. Dutch universities advise scientists against using tools like ChatGPT but allow their staff the freedom to make their own decisions.

I wonder whether it is fair to ask individual researchers to handle this responsibility. The workload at universities is high, and competition is fierce. For young scientists, that one extra publication in a prestigious journal or that additional grant from NWO can make the difference between a permanent position and a temporary contract. Do scientists have sufficient knowledge of the technical limitations and sustainability impact of the use of large language models to make an informed decision? And can you trust that they will always prioritise public and scientific values, even when GAI can (seemingly) take a lot of work off their hands? Surveys among scientists worldwide show that many scientists are well aware of the errors and inaccuracies caused by the use of GAI, yet still use the available tools to generate summaries or have parts of their publications written.

Universities must therefore quickly establish standards or rules for the responsible use of GAI. This first and foremost requires a political assessment of conflicting values such as efficiency, autonomy, sustainability, and justice, on which the Dutch Parliament must decide: how important is it to accelerate and broaden scientific research, even in the current geopolitical landscape where data and AI are used as instruments of power? What is the moral bottom line in terms of sustainability and (global) justice, and at what cost are we willing to increase the efficiency of scientific research? In addition, a translation is needed from traditional scientific values to a new reality. What do integrity, reliability, and independence mean when using GAI, and how can we safeguard these values? For which applications is the use of GAI permissible, and what conditions must GAI tools meet? It is not certain that the Dutch science system can provide a single universal answer to this. It may be necessary for different disciplines to set out to formulate their own rules, because it is important to do justice to the great diversity of scientific practices and the varying ways in which scientists use GAI.

Drafting and enforcing new rules will, of course, not be easy. But scientific communities must do it, nonetheless. GAI challenges established mechanisms of scientific quality assurance, and we must find a response to this. My colleagues and I at the Rathenau Instituut are currently investigating how GAI is changing the nature of scientific knowledge, the demands its use places on scientific quality assurance, and what competencies scientists need to use GAI effectively.

A promising development is that the Dutch technology institute TNO is developing an alternative large language model, that will be more reliable, independent and transparent, called GPT-NL. Moreover, several Dutch universities are developing their own AI tools for science, which are delivering increasingly better performance. The EU also intends to invest heavily in this area. This offers the opportunity to develop alternative GAI technology that better aligns with European values and the rigor that forms the foundation of responsible science. But we cannot wait for that. As long as these alternatives remain insufficiently developed, scientists will continue to rely on GAI tools, which are fraught with problems. Setting limits on permissible use, therefore, requires our immediate attention.

This blogpost is a translation of a Dutch opinion piece published on ScienceGuide.nl and the Rathenau Institute website. The author has made use of DeepL (free version) to translate the text to English.

Header image by Anna Riepe & FARI on Better Images of AI.