Add a public website as a knowledge source

When you add public websites as knowledge sources at the agent level, you use publicly available information to improve the reach of your agent.

Add a public website

To add a public website as a knowledge source, complete the following steps:

  1. Open the agent.

  2. Select Add knowledge from either the Overview or Knowledge pages, or the Properties of a generative answers node.

  3. Select Public websites.

  4. Enter the URL to the public website. To use a variable instead of a static URL, select the variable picker icon next to the URL field and choose a Custom, System, or Environment variable.

  5. Add a name (by default, the website's URL is used).

  6. Add a description. Provide a detailed description, especially if generative AI is enabled, as the description aids generative orchestration.

  7. Select Add to agent to finish adding the knowledge source.

  8. Test your agent to see how well it responds to questions related to the content from your website. You might want to test edge case questions to decide if you need a lower moderation to be more inclusive.

URL considerations

The URL you use in your agent represents the scope of content for generating responses. Some URLs have requirements and restrictions.

Note

URL type and structure

  • The URL can have up to two levels of depth, with subpaths indicated by a forward slash /. However, a trailing forward slash is allowed.

    Valid Not valid
    www.contoso.com
    www.fabrikam.com/engines/rotary
    www.fabrikam.com/engines/rotary/    		 www.fabrikam.com/engines/rotary/dual-shaft

  • If the URL redirects to another top-level site, the content isn't included in results:

    For example, if www.fabrikam.com redirects to www.contoso.fabrikam.com, your agent doesn't generate responses from content on either of those URLs.

  • URLs that point to a website requiring authentication or ones not indexed by Bing aren't supported.

    For example, wikis and SharePoint sites require authentication, so you can't use them:

    • fabrikam.visualstudio.com/project/_wiki
    • fabrikam.sharepoint.com

URL domain structure

Your agent uses any publicly viewable content in the URL you specify, including subdomains under a top-level domain, to generate content.

Examples:

  • If you use www.fabrikam.com/engines/rotary, the agent also uses the content on www.fabrikam.com/engines/rotary/dual-shaft to generate responses.

    The agent doesn't use content on www.fabrikam.com/tools, since tools isn't a subdomain of rotary.

  • If you use www.fabrikam.com (the www exists), the agent doesn't use content on news.fabrikam.com (the www doesn't exist), since news. is a subdomain under the top-level domain fabrikam.com.

  • If you use fabrikam.com, the agent uses content on www.fabrikam.com and news.fabrikam.com, since they sit under the top-level domain fabrikam.com.

Social networks and forum URLs

Your agent might generate nonsensical, irrelevant, or inappropriate answers if you use a forum or social network site as your URL. Therefore, community content on social networks often increases the risk of more answers being rejected.

For more information, see the FAQ for generative answers. AI is trained to avoid generating malicious and offensive responses.

Search engine URLs

Don't include URLs of search engines like bing.com, as they don't provide useful responses.

Use variables as URLs

Use variables in public website or SharePoint knowledge source URLs to dynamically control the scope of content your agent uses.

Instead of creating multiple knowledge sources for different scenarios, define a single knowledge source and insert a variable into the URL. At runtime, Copilot Studio resolves the variable and uses the resulting URL for grounding.

Variable support

When you add or edit a public website or SharePoint knowledge source URL, you can insert a variable into the URL field.

Use a variable to:

  • Append a path to a base URL

    https://www.contoso.com/{variable}

  • Replace the entire URL

    {variable}

At runtime, the variable resolves to a string value. The knowledge source is scoped to the resolved URL.

Note

Using a variable changes which URL is used for grounding. It doesn't change URL structure requirements or depth limitations for public websites.

Create and set a variable

To use a variable in a public website or SharePoint knowledge source:

  1. Create a string variable with Global scope.

  2. Set the variable value using one of the following options:

  3. Insert the variable into the URL field by selecting {x} (Insert variable).

  4. Test the agent to confirm the variable resolves to the expected URL.

Example: Scope by product

You can scope knowledge to different product areas based on the user's query.

  1. Use a topic input to capture the product name as a variable named Product.

  2. Add a Set variable value node after the Trigger node, and create a new variable named ProductTable.

  3. Use a Power Fx formula to define and map the product names to URL paths.

     Table(
     { Product: "Copilot", Path: "microsoft-365-copilot" },
     { Product: "Teams", Path: "microsoft-teams" },
     { Product: "Surface", Path: "surface" }
 )

  4. Add another Set variable value node and use it to create a new string type global variable named GlobalProductURL.

  5. In the To value setting, select Formula and use a Power Fx function to perform a lookup into the table based on the value of Product input variable, Topic.Product, as follows:

    LookUp(Topic.ProductTable, Product = Topic.Product, Path)

  6. Use the variable in the knowledge source URL:

    https://www.microsoft.com/en-us/{GlobalProductURL}

The agent queries only the scoped section of the website at runtime.

Common scenarios

Use variables in public website knowledge sources for:

  • Product-based routing: Scope queries to specific product documentation.

  • Region or language filtering: Map User.Language to localized URLs (for example, en-us, it-it).

  • Environment-specific URLs: Switch between development, test, and production sites.

Considerations

  • Public website knowledge sources use Bing search and rely on indexed content.
  • URL depth limits still apply when you use variables.
  • Variables change the target URL only. They don't affect how content is indexed or retrieved.

Important

The variable must resolve to a valid public URL that meets the requirements for public website knowledge sources. Invalid or unsupported URLs might not return results.

  • Last updated on