The recent revelation that a report supporting Australia's teen social media ban may contain AI-generated content has sparked concerns about the integrity of government-commissioned research. The Age Check Certification Scheme (ACCS), a UK-based organization, conducted a $3.48 million trial to test age assurance technologies for social media platforms, which led to the implementation of Australia's under-16 social media ban in December. However, the Senate inquiry has uncovered potential issues with the report's citations, raising questions about the reliability of AI-assisted writing.
The ACCS initially denied using AI in the report but later admitted to using it for rewriting paragraphs more succinctly. This admission comes after the Guardian Australia identified multiple citation errors, including non-existent Digital Object Identifiers (DOIs) and incorrect references. The errors were found in the section covering emerging technologies, which cited journal papers about potential new ways to check ages. The ACCS's response to these errors was not without further issues, as they provided clarifications and new references that also contained inaccuracies.
One of the most concerning findings was the citation of a journal article that was not publicly accessible until June 2025, yet the report claimed it was accessed in March 2025. The lead author of the study confirmed that the original citation was incorrect, and the paper's summary in the report was misleading. This highlights the potential for AI to inadvertently introduce inaccuracies, especially when used for rewriting rather than generating content from scratch.
The use of AI in government-commissioned reports is not without precedent. Deloitte, a consulting firm, had to refund part of its $440,000 contract with the Australian government after admitting to using AI. This incident, along with the current controversy, underscores the need for strict guidelines and penalties to prevent AI-generated errors in government submissions. As Prof. Christian Downie suggests, the government must take proactive measures to ensure the integrity of its research, as false citations can lead to poor decision-making and erode public trust in institutions.
The ACCS's admission of AI usage and the subsequent errors in the report serve as a reminder that while AI can be a powerful tool, it must be used responsibly and ethically. The government and its contractors need to be transparent about AI's role in research and take steps to mitigate the risks of AI hallucinations. The public's trust in institutions depends on the accuracy and reliability of the information they provide, and the use of AI in research should not compromise this fundamental principle.
