Australia’s government inquiry process is supposed to help parliament make better decisions, hearing from experts and constituents alike. But Guardian Australia can reveal that the system is being flooded with AI-generated material, which is inventing studies and attributing nonexistent research to real academics and authors. In some cases, committee reports have cited submissions in which the majority of sources appear to be AI-generated “hallucinations”, when large language models (LLMs) invent content that looks real but doesn’t actually exist.
If the phenomenon continues, there is a significant risk of parliamentarians “making decisions based on evidence that doesn’t exist”, says Christian Downie, a professor in the Australian National University’s school of regulation and global governance. The situation is made more confusing by AI services including Google search. Previously, looking up an invented reference might produce results that would indicate the reference did not exist.
Now Google’s AI summary will sometimes create summaries of a fake reference as though it were genuine. In some instances, Google’s AI summary and ChatGPT will also cite the inquiry submission with the fake reference as a source, creating an ongoing cycle of misinformation. One document submitted to an inquiry into family violence and suicide included a hallucinated reference attributed to Divna Haslam, a University of Queensland associate professor and clinical psychologist, and misstated her team’s research findings.
Haslam, who researches child and family adversity and maltreatment, says the reference was “scary” because it looked so real that someone taking a cursory look could be convinced. Google’s AI summary summarised the reference as if it were a real paper. Documents that were flagged as having a high proportion of references not matching anything were then manually checked.
Using these methods, the Guardian found at least 39 submissions to political inquiries containing what appear to be hallucinated references. These findings represent a conservative estimate of the amount of AI-generated material in political processes, as it only identifies submissions with incorrect citations. This approach would not find documents that used AI to generate text without any references, for example.
In another indication of how often LLMs are being used, more than 100 papers included ChatGPT url tags in reference links, which are automatically added by the platform in its results. The submissions containing AI-generated material range from documents with a small number of incorrect citations to submissions in which every reference cited does not exist. They were authored by individuals and organisations across the political spectrum.
Extract — continue reading at the source.
