How to Safely Test an Autonomous AI Agent: Budget, Action Control, and Success Criteria

Piloting an autonomous AI agent is a limited experiment. The system independently plans and performs actions in a given environment, but operates with predefined permissions, a budget, and stopping conditions. The goal of the pilot is not to prove autonomy, but to measure useful results, costs, errors, and risks.

A large number of completed actions does not mean that the goal has been achieved. An agent can send hundreds of emails, repeat ineffective steps, spend money, and produce no useful results.

What Experiments with Autonomous Agents Show

In the experiments described, autonomous systems were assigned job searches, fact-checking, negotiations, bookings, and other tasks. Some agents were given their own servers, domains, and crypto wallets so they could act without constant human confirmation. According to the journalist 404 Media, one agent, which the author of the materials calls Kudom, sent more than 200 emails and spent 147 dollars without earning any money. This observation applies to a specific agent and a specific test scenario, not to all autonomous systems.

Such cases matter not because of the amount itself, but because of the risk structure. An agent may treat a sent email as progress, even though what matters to a business is a response from a suitable client, a scheduled meeting, a paid order, or another verifiable outcome. When a system is given access to money and external communication channels without restrictions, a planning error turns into real expenses and reputational consequences.

In the provided description of the SlowLab experiment, language models were compared with a domain-oriented numerical optimizer in a greenhouse-trial simulation. Three models completed the campaign close to the standard configuration and noticeably below the baseline optimizer. The authors attribute the gap not only to reading the data, but also to which actions the agent chose at the beginning, which experiments it launched, and when it moved to the next step. This result cannot be generalized to every task, but it clearly demonstrates a limitation: an autonomous agent may use available tools incorrectly even when enough tools are available.

First, Define the Task and Success Criterion

A pilot begins with one operational task, not with a broad formulation such as “let the agent make money.” The task should describe the work object, permissible actions, final result, and verification method.

• Object: for example, finding potential customers in an approved list of companies.
• Actions: analyzing public data, preparing drafts, classifying contacts, and assigning tasks to an employee.
• Result: confirmed interest, a scheduled call, or another predefined business metric.
• Constraints: a list of permitted sources, a ban on mass mailing, and a spending limit.

It is best to divide the success criterion into three levels. The first shows process completion: the agent processed the specified number of records and saved the results. The second records quality: the share of relevant contacts, classification accuracy, and the number of human corrections. The third reflects economic impact: leads received, paid orders, time saved, or the cost of one useful result.

And what counts as failure? The answer should be recorded before launch. For example, the pilot ends if the agent obtains no confirmed result after a set number of attempts, repeats the same scenario, or spends a defined share of the budget without improving the intermediate metric.

Divide Actions by Risk Level

Autonomy does not mean having the same rights for every operation. Each agent action should be assigned to an access class, linking it to potential harm and the need for confirmation.

Message preparation should be separated from message sending. An agent may compose an email, check the address, and explain the purpose of the contact, but the final sending should be performed by a human or a separate controller that checks the limit, recipient, frequency, and content. This arrangement is especially important for mass actions: one erroneous instruction can launch a series of similar messages.

Set the Budget and Limits Before Launch

The pilot budget includes more than the model cost. The calculation includes requests to external services, paid databases, domains, proxies, data storage, payment fees, and the cost of manual review. For the test, use a separate project, a separate payment profile, or a virtual account with an amount the team has agreed in advance that it can afford to lose.

• Limit on the total amount of expenses for the entire pilot.
• Limit per operation and per work cycle.
• Limit on the number of emails, calls, applications, or publications.
• Limit on the number of retries after a service error.
• Limit on continuous operating time.
• Limit on the number of actions without new confirmation of the result.

Limits must be controlled outside the model. If the agent maintains its own expense table and decides for itself that the budget has not yet been exhausted, that is not control but a declaration. The execution layer must reject an operation after the threshold is reached, even if the model gives a convincing explanation of why it should continue.

Set Up Logging and Emergency Shutdown

The log should make it possible to reconstruct the course of the experiment without guesswork. Record the initial goal, model version, settings, input data, tool called, call arguments, service response, cost, duration, verification result, and controller decision. For external messages, save the draft, recipient, sending time, and the basis on which permission to send was granted.

Mark repeated actions separately. Looping appears not only as a literal command repetition. An agent may change its wording but return to the same source, send emails to the same category of recipients, or launch new attempts without changing its hypothesis. The controller should count similar actions and stop the chain after a set number of repetitions.

Stopping conditions are defined in advance: exceeding expenses, a series of tool errors, lack of progress, accessing a prohibited resource, attempting to obtain a secret, changing the goal, or repeating an unsuccessful strategy. The stop button must disable execution authority, not merely stop displaying the interface. After shutdown, the team checks unfinished operations, message queues, and issued access tokens.

Conduct the Pilot in Several Stages

1. Dry run. The agent operates on synthetic data or a copy of the environment. External emails, payments, and changes to the primary database are replaced with simulations.
2. Observed mode. The agent proposes actions, but a human confirms every high-risk operation. The team evaluates decision quality and log completeness.
3. Limited autonomy. Only low-risk actions and a small series of similar operations are permitted. Limits are set by an external controller.
4. Comparative run. The agent’s result is compared with a manual process or a simple script using the same input data.
5. Review and decision. The team analyzes failures, the cost of a useful result, and the reasons for shutdowns, then decides whether to expand the agent’s permissions.

Comparison with manual execution is essential for an economic conclusion. If the agent performed 500 actions, while an employee achieved the same result in less time and without the risk of mass errors, autonomy provided no practical value. The report should include not only the number of operations, but also the cost per confirmed result, the share of manual corrections, and the time spent monitoring.

Which Metrics Show Real Effectiveness

A single final figure conceals important differences. An agent with fewer actions may be better than a system that works faster but creates more erroneous emails. Comparisons should use identical scenarios, identical limits, and identical verification rules. When the model, prompt, tools, or volume of input data changes, the result should be treated as a new experiment.

Why the Number of Actions Does Not Equal Autonomous Success

An autonomous agent optimizes the task formulation available to it and may choose a convenient proxy metric. The number of emails sent is easier to count than the quality of negotiations or the probability of payment, so the system may sometimes increase activity instead of producing useful results. The problem becomes worse when the agent does not receive timely feedback and cannot stop a strategy that is producing no results.

The SlowLab experiment described illustrates similar mechanics in another field: the models had access to intermediate measurements, but often organized their work in synchronous batches, so the data did not change the next choice. Within the scope of this assignment, the authors found no notable advantage from access to intermediate feedback. The conclusion is limited to the simulation and specific architecture, but it is useful for a pilot: the mere presence of data does not guarantee that the agent will incorporate it into its decision.

The agent should have an explicit strategy-review rule. After each series of actions, the controller checks whether new information has appeared, whether the intermediate metric has changed, and whether the next step differs from the previous one. If the answer is no, the system switches to pause mode and hands the decision to a human.

When Can the Pilot Be Expanded

Expansion is permitted only after several conditions are met: the agent achieves the predefined result, its expenses remain within the limit, the log makes its actions explainable, and critical operations are protected by confirmation. The team must also verify that the result holds on new data, rather than only on a convenient test sample.

The budget and permissions should not be increased simultaneously. First expand the volume of data while keeping the same permissions, then check quality, and only afterward consider a new class of actions separately. Access to money, mass messaging, and changes to user permissions should preferably remain outside early pilots, even if the system reliably performs lower-risk tasks.

Frequently Asked Questions

Can I immediately give the agent access to a real wallet or corporate card?

That is unjustified for a first pilot. Use an isolated account with a prelimited balance, a ban on transfers, and external control of every operation. Ideally, replace financial actions with a simulation until the agent has demonstrated usefulness on safe data.

Should an agent be allowed to send emails on its own?

In the initial scenario, the agent should prepare drafts and classify replies. A person or a separate gateway that checks the address, limit, frequency, content, and the existence of a basis for contacting the recipient must approve sending.

Which metric matters more: the number of actions or the number of results?

The primary metric is a useful result tied to the pilot's goal. The number of actions should be used as an indicator of workload, cost, and risk, but not as proof of effectiveness.

What should be done if the agent gets stuck in a loop?

Stop the execution layer, save the log, and identify the recurring pattern. Then change the stopping rule, restrict the tool, or narrow the task. Simply increasing the budget will not fix an unsuccessful strategy.

Conclusion

Safe piloting of an autonomous AI agent is built around constraints, not the promise of complete independence. The team defines the goal, budget, permissions, message limits, stopping conditions, and method for comparison with manual work in advance. The agent receives only the tools it needs, every action is logged, and financial and reputational operations go through an external controller.

Experiments with autonomous systems demonstrate one consistent practical lesson: an agent can be active, persistent, and technically independent, but that still does not make it economically useful. A successful pilot proves not the number of steps completed, but that the system achieves a measurable result at acceptable cost and risk.