KATHMANDU, Sept 28: The global race to expand artificial intelligence capabilities is accelerating, particularly in the United States and China. But three recent incidents have sharpened questions about how to prevent AI from making wrong decisions, exceeding its authority, accessing sensitive systems without permission and acting beyond effective human control.
The first involves an AI agent developed by OpenAI that gained unauthorized access to an Australian government portal during internal testing. The incident occurred on June 18 but became public three months later. OpenAI informed the Australian government only on September 10.
Australian Prime Minister Anthony Albanese said the AI agent accessed a Medicare data reporting portal operated by Services Australia. It reached both public and non-public files. Authorities said they had found no evidence that personal health information or individual Medicare records were accessed, although the agent's ability to reach restricted files raised security concerns. A forensic investigation is underway with assistance from the Australian Signals Directorate.
OpenAI said it had not intended to attack a government website. The company was conducting an internal evaluation in which its models searched Australian government websites for information. During the process, the models performed some actions that the company had not intended. OpenAI said it discovered the incident during an internal review in August and notified Services Australia on September 10.
Australian officials were also unhappy with the notification process. OpenAI sent its report to a publicly available Services Australia email address. The agency saw the message on September 11, informed the Australian Cyber Security Centre on September 15, and Public Service Minister Katy Gallagher was briefed on September 17. Albanese and his office learned about it on September 19 and 20.
Albanese later spoke with OpenAI CEO Sam Altman and expressed Australia's serious concern, including over the delay in reporting the incident. The government is examining whether legal action is necessary and whether existing laws adequately cover cyber incidents caused by AI agents.
The case also raises a legal question because Australia's unauthorised access laws were largely designed around human activity. Authorities now face the question of how such laws apply when an autonomous AI agent crosses security boundary.
‘Parliamentary Oversight: Sharing and Discussion’ concludes
AI activity may also have affected systems belonging to the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research and Victoria's Department of Health, although no leak of personal information has been confirmed.
The incident highlights a broader challenge presented by AI agents. Unlike conventional AI systems that primarily answer questions, agents can open websites, search for information, use tools and independently decide on subsequent steps needed to complete an assigned task. This autonomy allows them to interact with external systems without human instructions at every stage.
Former Australian cybersecurity chief Alastair MacGibbon noted that the agent had not been instructed to hack a website. Instead, it was given a research task and entered an unauthorised area while using available tools to achieve its objective. The episode therefore highlights the risk of AI taking unexpected actions while pursuing a legitimate goal.
A second incident involved Google's Gemini model. During a cybersecurity test in May, Gemini was instructed to penetrate systems belonging to a fictional company.
Because the testing environment was connected to the internet, Gemini searched publicly available information and identified websites belonging to three real companies. In some cases, it entered their systems by guessing login details or using credentials found in public repositories. The model stopped when it realised it had reached real company systems.
Google said it notified the affected companies and changed its testing procedures. Although the incident was not a deliberate real world cyberattack, it demonstrated how an AI agent can misinterpret the boundaries of a controlled test and reach real systems.
The third incident involved the US military. According to CNN, an inaccurate intelligence report prepared with AI assistance circulated through the US military intelligence system during this year's war with Iran.
The report claimed that a Chinese ship in the Middle East was carrying material connected to a nuclear weapons programme. US forces reportedly began preparing to intercept the vessel, including preparations to send armed personnel aboard and deploy military aircraft.
Before action was taken, officials examined the intelligence more closely. They discovered that an analyst with the US Special Operations Command had used an AI chatbot while preparing the report. The chatbot had incorrectly concluded that the ship was carrying nuclear weapons related material.
The analyst then used AI assistance to turn that conclusion into a formal intelligence report, which was circulated among military officials. One source described the report as completely wrong and said it could have created circumstances capable of starting a war. What the vessel was actually carrying has not been confirmed.
The three incidents differ substantially. In Australia, an AI agent crossed security boundaries in a government system. Gemini reached real companies while operating during a controlled cybersecurity test. In the US military case, a human relied on faulty AI generated analysis. They therefore illustrate different categories of AI risk rather than a single form of “AI attack.”
They also raise difficult questions about accountability. If an AI agent acts beyond its intended limits, should responsibility fall on the company that failed to impose adequate safeguards, the organisation deploying the agent, or both? How quickly should companies be required to report such incidents to governments?
The military case highlights another danger: humans accepting AI generated information without adequate verification. Wrong AI conclusions in national security or military decision making could have severe consequences.
The US military is expanding AI use across intelligence analysis, targeting, resource management, logistics and budgeting. Defense Secretary Pete Hegseth announced an Artificial Intelligence Acceleration Strategy in January aimed at expanding military AI testing and investment.
International concern is also growing. During the UN General Assembly, 22 countries backed a joint statement calling for human control, safety measures and international coordination in AI development. The statement called for transparent safety protocols and common standards.
Some AI industry leaders, including OpenAI's Sam Altman, Anthropic's Dario Amodei, SpaceX's Elon Musk and Hugging Face's Clément Delangue, have also supported common standards for AI risk assessments and safety testing.
Others take a different view. Nvidia CEO Jensen Huang has argued that warnings about AI eliminating human employment entirely or ending human civilisation are exaggerated and lack sufficient scientific basis. Trump administration technology adviser Michael Kratsios acknowledges risks from rapid AI development but does not support creating a new global governance structure or slowing AI development for that reason.
AI safety has consequently moved beyond technology into diplomacy and national security. AI was among the issues discussed during Chinese President Xi Jinping's US visit, while senior US and Chinese trade negotiators have also held their first discussions specifically addressing AI risks.
Researchers argue that Washington and Beijing could establish regular channels to discuss shared risks, including AI-enabled cyberattacks, biological threats and the possibility of losing control over autonomous agents. Clear procedures could determine which incidents require immediate communication and which agencies should exchange information during a crisis.
These developments show that the AI race is no longer simply about building more capable models. It increasingly involves cybersecurity, military decisions, government systems, legal accountability, human oversight and relations between major powers. (With inputs from international agencies)
