This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility: Scientists are increasingly relying on powerful data sources and tools that they often cannot fully understand, inspect or verify, according to a new study. State-of-the-art tools and data like artificial intelligence (AI), satellite imagery, online data and digital sensors are revolutionizing the way scientists study the natural world.
But such systems can function as scientific "black boxes" that increasingly challenge trust in science. The new study, by an international team of scientists, addresses the problems of reproducibility, trust and the future of scientific research in an era when critical technologies can shape science and influence knowledge without being fully open to scrutiny. These technologies can process enormous amounts of information, monitor biodiversity and threats across continents and reveal patterns that would once have been out of reach.
"However, many of these tools represent true black boxes by keeping the processes behind those results largely hidden," said Ivan Jarić, a researcher at the University of Paris-Saclay and lead author of the study. "They are often owned by private companies that intentionally limit access to information about how their systems operate or process data, guided by proprietary constraints and commercial aims." The paper identifies several types of black boxes that are becoming widely used in ecology and conservation. Among the most prominent examples are large language models and other AI technologies, which are increasingly used to analyze massive data sets, interpret satellite imagery and model ecosystems.
However, researchers often have little or no access to the data used to train these systems, the underlying algorithms, direct system testing or an understanding of how and why they generate particular outputs. As AI becomes more capable and autonomous, this lack of transparency will make scientific findings harder to interpret and verify. This issue extends beyond AI.
Many remote sensing products rely on proprietary processing that researchers cannot fully access and verify, while some wildlife tracking devices provide only processed animal locations while withholding the underlying raw data. Online platforms such as search engines and social media, which have become valuable sources for studying biodiversity and human interactions with nature, are based on hidden algorithms and changing policies that can introduce unknown biases in such data. Similar problems are also affecting social surveys.
Scientists are increasingly relying on private companies to recruit participants and manage surveys, often with limited information about how respondents are selected, how data quality is maintained or whether responses may have been affected by AI agent interference. "This problem is not simply due to commercial and proprietary issues," said professor Karen Anderson of the University of Exeter, another author of the study. "Modern scientific tools are also becoming so technically complex that users, and in some cases even their developers, may struggle to fully scrutinize and understand how they operate." The growing dependence on black-box technologies is further strengthened by the publish-or-perish culture, which pressures scientists to increase productivity and remain competitive, and by the need to more effectively cope with growing data sets and urgent environmental crises.
Extract — continue reading at the source.
