Assessing AI risks in cybersecurity helps an organization understand how models and autonomous agents affect system protection, service availability, and people’s safety. AI helps analyze threats and find vulnerabilities. The same capabilities can make it easier for malicious actors to prepare and automate attacks.
For hospitals, banks, energy grids, and water utilities, the cost of an error is especially high. An outage can affect not only data but also the continuity of critical services.
Start with critical assets and consequences
The risk map should show which systems support key processes, what data they handle, and what will happen if they are compromised or stopped. Include production and control systems, accounts with broad permissions, backups, remote-access channels, and external services connected to critical processes. For each asset, specify its owner, acceptable downtime, dependencies on other systems, and recovery order.
Assess not only the likelihood of an incident but also its consequences. A data breach and interference with the control of a technological process require different response scenarios, even if both cases begin with a single compromised account.
If an organization cannot determine which processes depend on a particular server or service, begin prioritizing protection by identifying that dependency. Choosing another AI tool will not replace this step.
Test defensive and offensive scenarios
An assessment of AI use should cover both sides: tasks in which the model helps defenders and ways similar capabilities could be used against the organization. Defensive scenarios include sorting alerts, analyzing event logs, correlating threat indicators, and assisting a specialist in finding vulnerabilities.
For each scenario, determine what the system can read and modify, which external tools it can access, and who confirms the consequences of its actions. A vendor’s security claims do not replace testing a configuration close to the real one. Account for connected data sources, access permissions, filters, network restrictions, and the ability to hand the task over to a person.
Test the system in a controlled environment. The set of scenarios should check routine requests and attempts to go beyond the task’s scope, use unnecessary tools, or send data outside the test environment. Check whether monitoring records such actions and whether a responsible employee can stop them before they affect production systems. At the same time, the result depends on the test conditions: which restrictions are enabled, which tools are available, and which actions are permitted.
In cybersecurity testing, internet access and disabling protective mechanisms change the test conditions. The result of such a test cannot be transferred to ordinary model use without qualification: record separately which restrictions were enabled, which tools were available, and which actions were permitted.
Restrict autonomous agent permissions
Give an autonomous agent only the permissions necessary for a specific task. Read access should not grant permission to change configuration, create accounts, run code, or send messages to external recipients.
For higher-risk tasks, separate action preparation from execution: the agent proposes a change, and an employee reviews and approves it. Restrict network connections to permitted destinations, store secrets separately from the model’s working context, and log requests, tool calls, and results.
An isolated environment helps reduce the consequences of an error, but it does not guarantee security by itself. Its settings must match the task. A test agent should not have access to real accounts, public repositories, or production systems unless the test requires such access and the organization has prepared separate protective measures.
Check in advance how to stop the agent. Specify who is authorized to terminate its execution, how tokens and network access are revoked, which actions are canceled, and which require manual recovery.
Set up detection, response, and recovery
Monitoring should detect not only known attack signatures but also actions that do not match the agent’s role or the system’s normal behavior. Warning signs include unexpected requests to external resources, unusual volumes of data exports, attempts to obtain new permissions, and the use of tools outside the approved scenario.
For each signal, assign a responsible person and define the procedure: verify the event, restrict access, preserve logs, and assess the impact on critical processes. AI can help analyze suspicious activity and prepare response recommendations. Decisions affecting the availability of a hospital, energy grid, or water utility should be made within the organization’s approved authorities and procedures.
Backups will not help without verified recovery. Make sure compromised accounts cannot modify the copies, and ensure that the recovery plan accounts for dependencies between systems and the order for safely returning services to operation.
Conduct exercises based on scenarios that simultaneously affect AI and ordinary infrastructure. For example, the team should practice a situation in which an agent produces a suspicious result, an external connection does not stop, and some logs are unavailable. After the exercise, record the gaps identified, assign responsibility for fixes, and determine how to verify the outcome.
Coordinate protection with partners
A cyberthreat may affect a supplier, contractor, or industry partner before it becomes apparent inside your network. Agree on what incident and vulnerability information may be shared, who receives reports, and how quickly participants will limit the consequences.
The material presented attributes calls to Bill Gates to increase investment in cybersecurity and coordinate protective measures, as well as to use AI to detect suspicious activity, analyze threats, and respond. This is the position set out in the material, not universal requirements for every organization. Align specific measures with the operator’s responsibilities and the nature of the infrastructure.
The assessment plan includes several stages: identify critical processes, test AI-use scenarios, restrict agent permissions, set up monitoring and a shutdown procedure, test response and recovery, and coordinate information sharing with partners. Review the plan when the model, its tools, access to data, or role in the process changes.
Frequently asked questions about AI risk assessment
Can AI be used to find vulnerabilities?
Yes, if the organization has defined the systems and testing methods that are permitted, restricted the model’s access, and controls actions that could affect production infrastructure. A specialist should review the search results before making fixes or changing configuration.
Is enabling the vendor’s filters enough?
No. Filters can be part of the protection, but the assessment should cover the agent’s permissions, connected tools, network access, data, and monitoring. Test the system in the configuration in which it will be used.
What should I do if the agent acts outside its task?
Terminate its access to tools and the external network, preserve the logs, and check which systems and data it affected. Then follow the approved response procedure and restore access only after determining the cause and verifying the restrictions.
Compare models before you start
The service sets its plans, limits and model catalog. If they differ from this article, contact us so we can update it and record a new review date.
Browse modelsAffiliate link: your price stays the same and the project earns a commission.