A policy is only useful if you can show where it runs
Turn an AI policy into a testable rule, name the point where it runs, and keep the evidence. This guide shows how to check each step before you enforce anything.
By Halo Research Team
"Agents must not send customer data to unapproved services" is a reasonable policy. It needs three more decisions before it can act as a control. Someone has to define customer data, identify the approved destinations, and decide where the rule will be checked.
The third decision deserves the most attention. A policy with no named execution point cannot be tested, and a policy that cannot be tested cannot be shown to a reviewer.
Can the vendor show your policy becoming a rule, that rule operating at a known boundary, and the resulting evidence explaining what happened?
Translate the policy into a decision
Start by turning the sentence into a reviewable decision with four parts. Specify the actor, the action, the resource, and the context.
| Part | Example |
|---|---|
| Actor | An agent acting for the support team |
| Action | Upload |
| Resource | An order record |
| Context | A destination outside the approved service list |
Read together, the example says that an agent acting for the support team may retrieve an order record and may not upload that record to an external tool outside the approved service list. A reviewer can now agree or disagree with something specific.
The approved service list is data that someone owns and updates, so name that owner. This is the core idea behind policy as code. The rule is written in a structured form that people can review, version, and test.
A rule is ready to test when it has five things.
- Owner. A named person who maintains the rule.
- Scope. The systems, users, and destinations it covers.
- Outcome. Allow, deny, or alert for each case.
- Version. A number or tag that appears in each decision record.
- Review date. The day someone will check that the rule still fits.
Pick the execution point
Next, identify where the rule runs. Different boundaries see different things.
- Model gateway. It can inspect traffic routed through it.
- Tool authorization layer. It can permit or deny a particular operation.
- Browser extension. It can see supported AI sites in the browser.
- Endpoint component. It can see local activity on a device.
- Lifecycle workflow. It can stop a system from being approved for release.
Each boundary has edges. Document which traffic, users, devices, and tools fall inside each one and which fall outside. A rule enforced at a model gateway has no view of a file that someone uploads in a browser tab that bypasses the gateway.
Label observation honestly
A system that receives a copy of an event can report a violation after the event occurred. An inline control can potentially stop or change the action before it completes. Both have value, and they answer different questions about the risk that remains.
Use plain labels in reports so readers know which one they are looking at.
- Observed. The event was seen and recorded after the action happened.
- Flagged. The event was seen and someone was alerted.
- Blocked. The control stopped the action before completion.
- Modified. The control changed the payload, for example by masking a value.
- Pending. The action is waiting for a person's decision.
A common approach is to start a new rule in audit mode, watch what it would have done, and move to enforcement once the results look right. For the rollout side, see our guide on rolling out AI controls without surprising your teams.
Test before you enforce
Test the rule before broad enforcement. Include allowed activity, prohibited activity, and an ambiguous case that should go to a person. Then test the failure cases.
| Case | Example | Result to confirm |
|---|---|---|
| Allowed | Retrieve an order record | Allowed, with actor and rule version |
| Prohibited | Upload the record to an unapproved tool | Blocked, with an explanation |
| Ambiguous | Send a summary to a partner domain nobody has reviewed | Pending, with a named approver |
| Missing identity | A request arrives with no delegating user | The outcome your failure rule defines |
| Expired approval | An approved action runs again after the window closes | A new approval request |
| Policy service unreachable | The policy check times out | The failure behavior you chose for this action |
Record the intended failure behavior for each type of action before testing begins. Our post on what happens when an AI control goes offline walks through that decision.
Keep the evidence connected
The resulting event should identify the system, the actor, the policy version, and the outcome. A reviewer should be able to tell whether the action was allowed, blocked, modified, or only flagged.
An error should never be counted silently as a successful policy decision. Keep errors as their own outcome so the numbers on the dashboard mean what they say. See what an AI audit log should contain for the fields that make this possible.
Treat exceptions as part of the policy
Give each exception a purpose, an owner, a scope, and an expiration date. Revisit it when the system changes. A temporary business decision should not stay in force indefinitely because nobody remembered to remove it.
An exception log also shows which rules cause the most friction. That is useful input when the policy owner decides whether a rule needs refining.
Use the sequence in your evaluation
The practical buying question is the one at the top of this post. Ask the vendor to show the three parts in order, using your own policy and synthetic data.
Technical and business reviewers can then assess the same demonstration and ask the same follow-up questions.
How Halo helps
Halo's policy engine uses versioned policy bundles with allow, deny, and alert outcomes, audit (observe) and enforce modes, and decision explanations, all authored as structured schema or YAML. Rules can check tool calls and model requests and responses at the model gateway and the MCP and tool gateway, and Halo Browser applies audit, warn, and block modes on supported AI sites. Decisions are recorded and export as CSV and JSON through audit and evidence records. Read more about AI policy enforcement.