Will an AI guardrails approach really work?
This article is part of an ongoing series looking at artificial intelligence (AI) in relation to knowledge management (KM), and KM in relation to AI.
In September last year, the Australian Government Department of Industry, Science and Resources released the Introducing mandatory guardrails for AI in high-risk settings: proposals paper1 and invited feedback. This announcement was widely reported, and the public responses to the paper are generally supportive of the proposals.
The 10 proposed mandatory guardrails for AI in high-risk settings (Box 1) recognize that current regulatory frameworks are inadequate to respond to the distinct risks posed by artificial intelligence (AI), and aim to strike a balance between addressing these risks and supporting the benefits and innovation that AI can and has already started to bring. The mandatory guardrails complement 10 voluntary AI guardrails in the Australian Government’s Voluntary AI Safety Standard2 which was also published in September last year.
Box 1. Proposed mandatory guardrails for high-risk AI.
Organisations developing or deploying high-risk AI systems are required to:
- Establish, implement and publish an accountability process including governance, internal
capability and a strategy for regulatory compliance. - Establish and implement a risk management process to identify and mitigate risks.
- Protect AI systems, and implement data governance measures to manage data quality and
provenance. - Test AI models and systems to evaluate model performance and monitor the system once
deployed. - Enable human control or intervention in an AI system to achieve meaningful human oversight.
- Inform end-users regarding AI-enabled decisions, interactions with AI and AI-generated content.
- Establish processes for people impacted by AI systems to challenge use or outcomes.
- Be transparent with other organisations across the AI supply chain about data, models and
systems to help them effectively address risks. - Keep and maintain records to allow third parties to assess compliance with guardrails.
- Undertake conformity assessments to demonstrate and certify compliance with the guardrails.
Will this approach work?
While the the Australian Government’s AI guardrails approach can be expected to beneficially assist the safe and responsible use of AI, Australia’s robodebt AI scandal shows that it is very foolish to assume that such guardrails offer complete protection.
As I discussed in a previous RealKM Magazine article, the Robodebt AI system was designed to catch people exploiting welfare. However, it got many of the assessments seriously wrong to such an extent that it triggered numerous suicides3. The flaws in the scheme were ignored, even when concerns were raised multiple times4.
An outcome of the Royal Commission into the Robodebt Scheme has been an investigation5 into whether Australian public servants associated with the design and delivery of the Robodebt Scheme breached the Australian Public Service (APS) Code of Conduct6. The APS Code of Conduct is also a guardrails approach (Box 2).
Box 2. Australian Public Service (APS) Code of Conduct.
The Code of Conduct requires that an APS employee must:
- behave honestly and with integrity in connection with APS employment;
- act with care and diligence in connection with APS employment;
- when acting in connection with APS employment, treat everyone with respect and courtesy, and without harassment;
- when acting in connection with APS employment, comply with all applicable Australian laws;
- comply with any lawful and reasonable direction given by someone in the employee’s Agency who has authority to give the direction;
- maintain appropriate confidentiality about dealings that the employee has with any Minister or Minister’s member of staff;
- take reasonable steps to avoid any conflict of interest (real or apparent) and disclose details of any material personal interest of the employee in connection with the employee’s APS employment;
- use Commonwealth resources in a proper manner and for a proper purpose;
- not provide false or misleading information in response to a request for information that is made for official purposes in connection with the employee’s APS employment;
- not improperly use inside information or the employee’s duties, status, power or authority:
- to gain, or seek to gain, a benefit or an advantage for the employee or any other person; or
- to cause, or to seek to cause, detriment to the employee’s Agency, the Commonwealth or any other person.
- at all times behave in a way that upholds the APS Values and Employment Principles, and the integrity and good reputation of the employee’s Agency and the APS;
- while on duty overseas, at all times behave in a way that upholds the good reputation of Australia; and
- comply with any other conduct requirement that is prescribed by the regulations.
Highlighting the limitations of such approaches, the Robodebt Code of Conduct investigation found that 12 individuals had breached one or more elements of the APS Code of Conduct, including two former departmental heads, as reported in this article7 in The Conversation:
An article8 in The Mandarin contends that embedding integrity and ethics as core APS values goes beyond such compliance approaches. Author Matt Beard states that:
This shift marks the difference between a public service that relies solely on governance, guardrails, and risk mitigation strategies, and one that is genuinely driven by the positive values those processes are designed to protect. The former might, for instance, establish clear, transparent, and enforceable disclosure policies around conflicts of interest. The latter translates these policies into daily practice—where even non-conflicted staff routinely ask themselves (and each other), “What might get in the way of my impartiality here?”
They understand that even if they do not have a direct conflict, the public interest is not served if they approach policy issues with a one-sided perspective. As such, they create space within their teams and organisations for dialogue, reflection, and active debate—ensuring that biases and preferences are openly acknowledged. This fosters greater self-awareness, and an appreciation of diverse viewpoints, and ultimately leads to a more robust policymaking process and better outcomes for the community.
This shift is also needed in regard to the safe and responsible use of AI, or breaches of the AI guardrails can be similarly expected. The KM community can play a role in this shift, perhaps even a leading role, particularly as much of what Matt Beard puts forward sounds very much like KM for sustainable development’s highly progressive “multiple knowledges” approach.
However, for the KM community to be able to be such a leader, it first needs to greatly improve its currently seriously inadequate critical thinking capabilities.
Header image source: pasja1000 on Needpix, Public Domain.
References:
- Commonwealth of Australia. (2024, September). Safe and responsible AI in Australia: Proposals paper for introducing mandatory guardrails for AI in high-risk settings. ↩
- Commonwealth of Australia. (2024, September 5). Voluntary AI Safety Standard: Guiding safe and responsible use of artificial intelligence in Australia. ↩
- McPherson, E. (2020, August 17). Mothers who lost sons to suicide after Centrelink debts write heartbreaking letters to Senate. 9 News. ↩
- Henriques-Gomes, L. (2020, September 18). Robodebt court documents show government was warned 76 times debts were not legally enforceable. The Guardian. ↩
- Australian Public Service Commission. (2024, February 8). Robodebt Code of Conduct process. ↩
- Australian Public Service Commission. (2022, May 10). APS Code of Conduct. ↩
- Grattan, M. (2024, September 13). Two former federal departmental heads breached public service code 25 times in Robodebt scandal. The Conversation. ↩
- Beard, M. (2025, February 26). Embedding integrity as a core APS value goes beyond compliance. The Mandarin. ↩
