The Distributed Query Attack
Why seemingly harmless AI requests can become an information-security risk when knowledge is assembled across interactions, and what that means for organisational safeguards.
Read article →AI Safety, Security & Governance
AI Safety, Security & Governance
Understand workplace AI risks, security, governance and human oversight, with practical guidance for responsible AI use in organisations.
As AI becomes more capable and connected to everyday work, organisations face questions about security, privacy, reliability and accountability. These challenges concern how people use AI, what it can access and how decisions are reviewed.
Understanding the Topic
AI safety is about more than the technology
AI systems can produce inaccurate information, misunderstand instructions or behave unexpectedly. The consequences depend on how the technology is used.
An incorrect response during brainstorming is different from an incorrect response informing a significant decision. An assistant working with public information poses different risks from an agent with access to sensitive records or connected systems.
Policies and technical controls are important safeguards, but they do not replace informed human judgement. Responsible use requires people, processes and technology to work together.
Building Understanding
What should organisations consider?
Information and privacy
What information is shared with AI systems, and is that use appropriate?
Accuracy and reliability
How are generated responses checked before they influence work or decisions?
Security and connected systems
What data, tools and permissions can AI access?
Human oversight
Who remains accountable, and where is review or approval required?
Governance in practice
Do people understand the policies and controls relevant to their work?
Responsibilities in Practice
Safety, security and governance: what is the difference?
AI safety
Question: Could AI produce an unreliable or harmful outcome?
Example: A staff member verifies an AI-generated recommendation before acting on it.
AI security
Question: Are information, systems and access protected?
Example: Confidential information is used only with approved tools and appropriate permissions.
AI governance
Question: Who sets the rules and remains accountable?
Example: The organisation defines approved uses, review requirements and escalation responsibilities.
These responsibilities overlap. Effective AI use requires people, processes and technical controls to work together. Training supports informed decisions but does not replace technical safeguards or organisational governance.
Further Reading
Explore the articles
The Blast Radius Is the Graph
Why organisations must consider the tools, information and permissions available to connected AI systems, not just the behaviour of an individual model.
Read article →Explore Further
Related topics
AI Adoption & Capability
Building the skills, confidence and judgement people need to use AI effectively.
AI in Practice
Moving beyond demonstrations to useful AI applications in real work.
For further discussion of AI failures and safeguards, explore dontfail.ai, a related field project. Its content is separate from AIBoK’s training and governance guidance.
Common Questions
Frequently asked questions
Common risks include inaccurate outputs, disclosure of confidential information, biased or misleading recommendations and inappropriate reliance on automated results. Connected AI systems can introduce additional security and operational risks.
AI safety concerns avoiding harmful or unreliable outcomes. Security focuses on protecting systems, access and information. Governance defines responsibilities, policies, oversight and processes for deciding how AI may be used.
That depends on the information, the tool's configuration, contractual protections and the organisation's policies. Staff should not assume that a consumer AI service is approved for confidential, personal or client information. Check authorised tools and data-handling rules first.
AI outputs can sound convincing even when they are wrong. People need to evaluate evidence, understand context and remain accountable for consequential decisions. The level of review should match the potential impact of an error.
A practical policy should identify approved tools and uses, restricted information, verification expectations, human approval points, incident reporting and ownership of decisions. It should be reviewed as tools and work practices change.
Training & Enablement
Build practical AI capability
Policies and technical safeguards are important, but people also need to understand how to use AI appropriately. Our practical training helps teams recognise common risks, handle information responsibly, check AI-generated outputs and exercise judgement within their organisation's requirements.