Exercise - Build a knowledge-enabled agent

  • 25 minutes

In this exercise, you create a knowledge-enabled agent for Contoso Insurance that helps customer service representatives quickly find answers to common policy questions using the company's knowledge base.

Scenario

Contoso Insurance receives hundreds of customer inquiries daily about policy coverage, claims processes, and general insurance questions. The customer service team needs quick access to accurate information to provide consistent, helpful responses. You'll create an intelligent agent that can answer these questions using knowledge sources.

Learning objectives

After completing this exercise, you'll be able to:

  • Create an agent in Microsoft Copilot Studio using natural language prompts
  • Add multiple types of knowledge sources to enhance agent intelligence
  • Configure generative answers for optimal response quality
  • Test agent responses and refine knowledge source configurations
  • Manage agent settings for knowledge usage and web search

Prerequisites

  • Access to Microsoft Copilot Studio with appropriate permissions
  • A Power Apps developer environment
  • Basic familiarity with Microsoft Copilot Studio interface

Note

This exercise takes approximately 25 minutes to complete.

Task 1: Create your customer service agent

In this task, you'll create a new agent designed to help with customer service inquiries.

  1. Navigate to Copilot Studio and sign in with your credentials.

  2. On the Home page, locate the agent creation prompt field.

  3. Enter the following descriptive prompt to guide the agent creation:

    Create a customer service agent for an insurance company that helps representatives answer questions about policies, coverage, claims, and general insurance information

  4. Select the Send icon to create the agent.

  5. Wait for Copilot Studio to analyze your prompt and generate the agent. This process typically takes 30-60 seconds.

  6. Review the automatically generated components:

    • Instructions: Notice how the AI interpreted your prompt into specific behavioral guidelines
    • Knowledge suggestions: Look for any automatically suggested knowledge sources
    • Triggers: Review the suggested phrases that activate your agent
    • Tools: Check for recommended integrations

  7. In the agent details, update the Name field to "Contoso Insurance Helper" for clarity.

  8. Review the Description and modify it if needed to better reflect the agent's purpose.

Task 2: Configure knowledge sources

Now you'll add multiple knowledge sources to provide your agent with comprehensive information to answer customer questions.

Add a public website knowledge source

  1. Select the Knowledge tab in your agent.

  2. Select + Add knowledge to open the knowledge sources dialog.

  3. Select the Public websites tile.

  4. In the Public website link field, enter:

    https://learn.microsoft.com/en-us/microsoft-copilot-studio/

  5. Select Add.

  6. In the knowledge source configuration:

    • Name: Enter "Copilot Studio Documentation"
    • Description: Enter "Official Microsoft documentation for Copilot Studio features, configuration, and best practices"

  7. Select Add to agent to complete the addition.

Add a file-based knowledge source

  1. From the Knowledge tab, select + Add knowledge again.

  2. In the Upload file section, select Select to browse.

  3. For this exercise, create a simple text file on your computer with sample insurance information:

    • Create a new text file called "insurance-policies.txt"
    • Add sample content such as: 
      Auto Insurance Coverage Types:
- Liability: Covers damage to others
- Collision: Covers damage to your vehicle
- Comprehensive: Covers theft, vandalism, weather damage

Claims Process:
1. Report incident immediately
2. Document damage with photos
3. Contact claims department
4. Schedule vehicle inspection
5. Receive repair authorization

  4. Upload the file you created.

  5. Configure the knowledge source:

    • Name: Enter "Insurance Policy Guide"
    • Description: Enter "Basic information about auto insurance coverage types and claims processes for customer service reference"

  6. Select Add to agent.

  7. Wait for the file to be indexed. The status changes from "Indexing" to "Ready" when processing is complete.

Task 3: Configure generative answer settings

In this task, you'll optimize how your agent uses knowledge sources and handles unknown questions.

Configure general knowledge settings

  1. In your agent, select Settings from the navigation menu.

  2. Select Generative AI from the settings options.

  3. Review the Use general knowledge setting:

    • Turn this ON for this exercise to allow the agent to use broader knowledge when specific sources don't have answers
    • This setting helps provide comprehensive responses while still prioritizing your knowledge sources

  4. Review the Use information from the Web setting:

    • Turn this ON to allow real-time web searches for current information
    • This enables the agent to find up-to-date information when your knowledge sources are insufficient

Configure knowledge source usage

  1. Return to the Knowledge tab.

  2. Select one of your knowledge sources to edit it.

  3. Expand the Include/exclude options section.

  4. Verify that At any time is selected, allowing the knowledge source to be used for any user request.

  5. If you want to restrict usage, you could select Only when referenced by topics instead.

  6. Select Save if you made any changes.

Task 4: Test your agent

Now you'll test the agent with various questions to see how it uses your knowledge sources.

Test with knowledge source content

  1. In the Test your agent pane (usually on the right side), start a conversation by typing:

    What types of auto insurance coverage are available?

  2. Review the agent's response and notice:

    • Does it reference information from your uploaded file?
    • Is the response conversational and helpful?
    • Are the coverage types mentioned correctly?

  3. Ask a follow-up question:

    How do I file a claim?

  4. Observe how the agent responds using the claims process information from your knowledge source.

Test with Copilot Studio knowledge

  1. Ask a question related to the website knowledge source:

    How do I create topics in Copilot Studio?

  2. Notice how the agent draws from the Microsoft documentation you added as a knowledge source.

Test general knowledge fallback

  1. Ask a question outside your specific knowledge sources:

    What's the weather like today?

  2. Observe whether the agent uses general knowledge or web search to attempt an answer.

  3. Try another general question:

    What are some tips for safe driving?

Task 5: Refine and optimize

Based on your testing, make improvements to enhance the agent's performance.

Review and adjust responses

  1. If any responses seem irrelevant or off-topic, go to Settings > Generative AI.

  2. Consider adjusting the content moderation level:

    • Higher levels provide more answers but may be less precise
    • Lower levels provide fewer but more accurate answers

  3. Test the same questions again to see if responses improve.

Add more specific knowledge

  1. If you notice gaps in the agent's knowledge, consider adding another knowledge source:

    • Create another text file with additional insurance information
    • Or add another relevant public website URL

  2. Upload and configure any additional knowledge sources using the same process from Task 2.

Task 6: Review agent capabilities

Before completing the exercise, review what you've accomplished.

Examine knowledge integration

  1. Go to the Knowledge tab and review all added knowledge sources.

  2. Check that all sources show "Ready" status.

  3. Review the descriptions and verify they accurately describe each source's purpose.

Test conversational flow

  1. In the test pane, have a longer conversation to see how well the agent maintains context:

    User: "I need to file a claim"
Agent: [responds with claims process]
User: "What documentation do I need?"
Agent: [should provide relevant follow-up information]

  2. Observe how well the agent maintains conversation context and provides relevant responses.

Cleanup

When you've completed the exercise and testing, you can choose to keep or remove your agent.

If you want to continue experimenting:

  • Leave the agent active for future testing
  • Consider adding more knowledge sources
  • Explore additional Copilot Studio features

Delete your agent (optional)

  1. If you want to clean up your environment:

    • Go to the agent Settings
    • Look for a Delete option (usually in advanced settings)
    • Confirm deletion if you no longer need the agent

  2. Note that deleting the agent will also remove all associated knowledge sources and configurations.

Summary

Congratulations! You have successfully completed this exercise. You've accomplished the following:

  • Created an intelligent agent using natural language prompts
  • Added multiple knowledge sources including websites and file uploads
  • Configured generative answer settings for optimal performance
  • Tested the agent with various types of questions
  • Learned how knowledge sources integrate with agent responses
  • Experienced how agents handle different types of user inquiries

This exercise demonstrates the power of knowledge-enabled agents in Microsoft Copilot Studio. The agent you created can now help customer service representatives quickly find accurate information, improving response times and consistency in customer interactions.

Next steps

Consider exploring these additional capabilities:

  • Add SharePoint sites or Dataverse tables as knowledge sources
  • Create custom topics for specific conversation flows
  • Integrate the agent with Microsoft Teams or other channels
  • Configure advanced generative answer settings for specific use cases