Design guardrails and transparency practices
Content moderation screens responses for harm, so keeping your agent within your organization's policies depends on design decisions you make yourself. In this unit, you learn how to layer instructions, knowledge scope, tool boundaries, and approvals into guardrails, and how to design disclosures and oversight that keep users informed.
Scope set by focused instructions
Recall that a focused agent is safer and easier to maintain than a general-purpose one. Guardrails start from that focus, so document the scope and boundaries of your agent: what it should do and what it shouldn't.
Instructions are where that scope takes effect. They're natural language guidance that shapes the agent's tone, scope, and response style. They tell the agent what to do, how to respond, and what to avoid. On the GitHub Copilot harness, instructions are the primary mechanism for controlling agent behavior. Effective instructions cover:
- Role and purpose: The job the agent does and the people it serves.
- Tone and style: How the agent sounds, such as formal, friendly, or concise.
- Declined or redirected requests: Subjects or tasks the agent refuses or sends elsewhere.
- Unclear input: How the agent handles a request that's ambiguous or incomplete.
- Escalation and handoff: The situations that trigger an escalation or a handoff, such as a dispute the agent shouldn't try to resolve.
For your HR policy agent, instructions might state that it answers employees' benefits and leave questions, that it declines requests for another employee's records, and that it escalates disputes to the HR team. Start with a clear, simple statement of purpose, and refine the instructions as you test the agent's behavior.
Knowledge limited to approved sources
Instructions define what the agent covers, and knowledge sources determine what it draws on to answer. Without knowledge, the agent relies only on its general training and instructions. With knowledge, it grounds responses in your organization's content, so scope the agent to answer only from approved sources. For the HR agent, those sources might be the current benefits guide and leave policy that the HR team maintains.
Grounding also makes answers easier to check. Citations might be included to show where information came from, which lets an employee compare an answer with the policy itself. Because you might not control how a source changes over time, tell users to treat answers with caution and verify them when appropriate.
On the standard harness, the Allow ungrounded responses setting adds a stricter check. It's in the Knowledge section of the agent's Generative AI settings, and it requires generative orchestration. When you turn the setting off, the agent blocks any response in a turn where it used no knowledge source or tool. The check is strict: if the agent answers a follow-up question from earlier turns without calling a source again, that response is blocked too.
The setting also has a limit. Turning it off doesn't guarantee that the agent never uses general knowledge, because the model can still combine general knowledge with content it retrieves from a knowledge source or tool. With the setting off, the agent also returns an answer from a knowledge source only when the answer includes an in-text citation to that source. Instructions that ask the agent to always cite its sources make grounded answers more consistent.
Tool boundaries for high-impact actions
Knowledge shapes what the agent says, and tools determine what it can do. With tools, an agent can look up real-time data and take actions on a user's behalf, such as creating records, sending notifications, and updating data. Each tool extends the agent's reach, so each one needs its own boundary.
Start with least privilege. Grant each tool only the access its task requires, and validate the API permissions of any external integration. Then describe each tool precisely. When the agent decides whether to call a tool, it weighs the tool's name and description along with the conversation context, and your instructions also influence when it uses tools. Well-described tools are invoked more accurately, so state what each tool does and when it applies.
Impact sets the next boundary. Add approval steps for high-impact actions, such as financial transactions or HR changes, and limit automation to reversible tasks where possible. Suppose the HR agent later gains a tool that submits leave requests. Looking up a leave balance changes nothing, but a submitted request changes an employee's HR record. Design the process so that a person approves each submission before it takes effect, and give the tool only the access that submitting a request requires.
Authentication sets a related boundary, because it determines who can use the agent. Plan it as its own design decision alongside these guardrails.
Reflect: List the actions your agent can take today or might take later. Which of them can't be undone, and what should happen before the agent takes them?
Transparency practices across the agent's life
Guardrails limit what the agent does, and transparency helps people understand what they're working with. Communicate the presence and role of AI clearly. Tell users about the agent's purpose, capabilities, limitations, and potential risks so they can make informed decisions about its output.
The first message of a conversation is a natural place to start. On the GitHub Copilot harness, the Greeting message on the Greeting & prompts tab of agent settings is the first message users see when they start a conversation. Use it to disclose:
- That the agent uses AI.
- The agent's purpose and what it can help with.
- The agent's limitations, including that its output can be wrong.
- Where a person reviews output or makes the final decision.
For the HR agent, a greeting might explain that it's an AI agent that answers benefits and leave questions from HR policy, that its answers can be wrong, and that the HR team reviews leave requests and makes the final decision. A disclosure informs users, but it doesn't replace an approval step before a high-impact action. Short in-product labels, such as "AI-generated content may be incorrect," reinforce the same expectation where users read the output.
Transparency continues after launch. Tell users how the agent is monitored and updated, and give them a way to report inaccuracies. On the GitHub Copilot harness, turning on User feedback on the Safety & access tab shows thumbs up and down on each response so users can rate its quality. Pair that feedback with ongoing monitoring: track performance and error rates, collect feedback from users and stakeholders, and update the agent as business processes change.
Human oversight completes the design. AI output can be inaccurate, incomplete, or biased, and mistakes can be hard to detect, which creates the risk of overreliance. Design the experience so that users review responses and automated actions before they act on them in high-stakes situations. For the HR agent, that means an employee checks a cited policy answer before they plan leave, and an HR team member approves each request the agent submits.