What happens when an AI control goes offline?
AI controls depend on other systems. Map the dependencies, choose failure behavior by action, and test outages with synthetic data before production decides for you.
By Halo Security Engineering
A governance control depends on other systems. It may need an identity service, a policy decision, an approval response, or a place to record evidence. An evaluation should establish what happens when one of those dependencies becomes unavailable.
Failure behavior is a product requirement. It deserves a test, a named owner, and a written decision for each type of action.
Failure behavior belongs in the product contract. Choose it for each dependency and each action, and test it before production does.
A refund example
Consider an agent that reads orders and issues refunds. If the approval service times out, should the refund wait, fail, or continue? Should an unrelated read-only lookup still work?
These are operating decisions, and they deserve explicit rules before the system handles real work.
Build a dependency map
Identify where identity is resolved, where policy is loaded, where approvals are checked, and where decisions are logged. Then distinguish an unavailable service from a stale response. A component can answer successfully while supplying an old policy or an approval whose conditions have changed.
| Dependency | What it supplies | Failure to plan for |
|---|---|---|
| Identity service | Who is acting and who delegated the work | Timeout, expired session, missing delegator |
| Policy source | The rule and its version | Unreachable service, stale or unverified policy |
| Approval service | A person's decision | Timeout, expired approval, changed request |
| Content checks | Detection of sensitive data | Classifier unavailable or slow |
| Evidence store | A record of the decision | Write failures, dropped events |
Add the dependencies specific to your environment. Each component that a decision passes through belongs on this list. Our post on where AI policy runs explains how to name each execution point.
A stale answer deserves the same attention as an outage. Suppose a policy source stays reachable after a failed update and keeps returning a rule that is two days old.
The service is up, and its answer no longer matches the current rules. Put a maximum age on policy answers, and treat anything older as a control error.
Choose failure behavior by action and consequence
Blocking an unapproved financial operation may be appropriate even when it delays service. Allowing a low-impact operation under a verified local rule may also be appropriate. Document who accepts that tradeoff and what conditions make the fallback valid.
| Behavior | Fits | Watch for |
|---|---|---|
| Fail closed | High-impact actions such as refunds and deletions | Delays, and pressure to switch the control off |
| Hold for approval | Actions that can safely wait | Approvals that expire while queued |
| Use a local rule | Low-impact actions covered by a recent, verified policy version | A stale policy |
| Proceed with a declared gap | Low-impact actions where a record can be rebuilt later | Activity with no evidence |
How a team might decide for the refund agent
The hypothetical refund agent has three actions with different consequences. A team might set the rules like this.
- Order lookup. Low impact. If the policy source is unreachable, use the last verified local policy, and record that the decision used it.
- Refund. High impact. If the approval service is unreachable, hold the request and tell the requester that it is waiting. A timeout never releases the refund.
- Message to a customer. Moderate impact. If the content check is unavailable, hold external messages until it returns.
The team writes each choice down, names the owner who accepted it, and tests it.
Put bounds around the fallback
Define timeouts, policy freshness, and any limit on work that may proceed. If an emergency exception exists, give it an owner, an expiry, and a visible scope. A temporary workaround should not quietly become the permanent way the system operates.
A fallback needs five bounds.
- Timeout. How long the control waits before the fallback applies.
- Freshness. How old a local policy may be and still count as valid.
- Limit. How much work may proceed while the control is unavailable.
- Owner. Who accepted the tradeoff and can end the exception.
- Expiry. When the fallback stops applying without a new decision.
Test safely with synthetic data
Interrupt one dependency at a time and watch the real action, including the result in the downstream system. A green status page can sit beside a refund that went through.
Run each test as a short procedure.
- Choose one action and one dependency to interrupt, using synthetic data.
- Interrupt the dependency and send the action.
- Record whether the action ran, waited, or failed, and check the downstream system.
- Retry the request and confirm that no operation happens twice.
- Confirm that the alert reached the responsible team.
- Restore the dependency and confirm that recovery does not release an outdated approval.
Review the evidence separately
If logging fails, does the control stop, buffer records, or continue with a declared gap? Each approach has consequences. The eventual audit view should distinguish a blocked action, a control error, and a period when no complete record was available.
See what an AI audit log should contain for how to label those states.
Repeat the tests when things change
Repeat relevant cases when deployment or control behavior changes. Keep the tested configuration, the expected outcome, and the observed result together. An availability percentage alone cannot explain how a particular action behaves during failure.
Our post on when to review an AI system again lists the kinds of changes that should trigger a retest.
Questions to ask in an evaluation
Include one controlled outage in any evaluation. These questions turn reliability into a specific product requirement.
- Which dependencies can the control lose, and what does each loss do to a pending action?
- Which failure behaviors can be configured for each action type?
- How does the product show a control error in reports and exports?
- What happens to approvals that were pending during the outage?
- Which execution paths does the demonstration cover?
How Halo helps
You can reduce reliance on any single control by giving the workload a restrictive default. Halo's secure agent runtime applies default-deny network egress, so an agent reaches only the destinations that were explicitly allowed, and operators can quarantine and release agents (see how to respond to an AI agent incident). Budgets can pause new operations or stop active deployments automatically, which limits spend while a team investigates. Add one controlled outage to your Halo evaluation and ask which behavior is configurable, which is fixed, and which paths the test covers.