Copilot for Data Factory overview
Important
This feature is in preview.
Copilot in Fabric enhances productivity, unlocks profound insights, and facilitates the creation of custom AI experiences tailored to your data. As a component of the Copilot in Fabric experience, Copilot in Data Factory empowers customers to use natural language to articulate their requirements for creating data integration solutions using Dataflow Gen2. Essentially, Copilot in Data Factory operates like a subject-matter expert (SME) collaborating with you to design your dataflows.
Copilot for Data Factory is an AI-enhanced toolset that supports both citizen and professional data wranglers in streamlining their workflow. It provides intelligent Mashup code generation to transform data using natural language input and generates code explanations to help you better understand earlier generated complex queries and tasks.
Before your business can start using Copilot capabilities in Fabric, your administrator needs to enable Copilot in Microsoft Fabric.
Note
- Your administrator needs to enable the tenant switch before you start using Copilot. See the article Copilot tenant settings for details.
- Your F64 or P1 capacity needs to be in one of the regions listed in this article, Fabric region availability.
- If your tenant or capacity is outside the US or France, Copilot is disabled by default unless your Fabric tenant admin enables the Data sent to Azure OpenAI can be processed outside your tenant's geographic region, compliance boundary, or national cloud instance tenant setting in the Fabric Admin portal.
- Copilot in Microsoft Fabric isn't supported on trial SKUs. Only paid SKUs (F64 or higher, or P1 or higher) are supported.
- Copilot in Fabric is currently rolling out in public preview and is expected to be available for all customers by end of March 2024.
- See the article Overview of Copilot in Fabric and Power BI for more information.
Supported capabilities
With Dataflow Gen2, you can:
- Generate new transformation steps for an existing query.
- Provide a summary of the query and the applied steps.
- Generate a new query that may include sample data or a reference to an existing query.
With Data pipelines, you can:
- Troubleshoot data pipeline issues with the error explaination capability with troubleshooting guidance.
Get started
Create a new Dataflows Gen2.
On the Home tab in Dataflows Gen2, select the Copilot button.
In the bottom left of the Copilot pane, select the starter prompt icon, then the Get data from option.
In the Get data window, search for OData and select the OData connector.
In the Connect to data source for the OData connector, input the following text into the URL field:
https://services.odata.org/V4/Northwind/Northwind.svc/
From the navigator, select the Orders table and then Select related tables. Then select Create to bring multiple tables into the Power Query editor.
Select the Customers query, and in the Copilot pane type this text:
Only keep European customers, then press Enter or select the Send message icon.Your input is now visible in the Copilot pane along with a returned response card. You can validate the step with the corresponding step title in the Applied steps list and review the formula bar or the data preview window for accuracy of your results.
Select the Employees query, and in the Copilot pane type this text:
Count the total number of employees by City, then press Enter or select the Send message icon. Your input is now visible in the Copilot pane along with a returned response card and an Undo button.Select the column header for the Total Employees column and choose the option Sort descending. The Undo button disappears because you modified the query.
Select the Order_Details query, and in the Copilot pane type this text:
Only keep orders whose quantities are above the median value, then press Enter or select the Send message icon. Your input is now visible in the Copilot pane along with a returned response card.Either select the Undo button or type the text
Undo(any text case) and press Enter in the Copilot pane to remove the step.
To leverage the power of Azure OpenAI when creating or transforming your data, ask Copilot to create sample data by typing this text:
Create a new query with sample data that lists all the Microsoft OS versions and the year they were releasedCopilot adds a new query to the Queries pane list, containing the results of your input. At this point, you can either transform data in the user interface, continue to edit with Copilot text input, or delete the query with an input such as
Delete my current query.
Limitations of Copilot for Data Factory
Here are the current limitations of Copilot for Data Factory:
- Copilot can't perform transformations or explanations across multiple queries in a single input. For instance, you can't ask Copilot to "Capitalize all the column headers for each query in my dataflow."
- Copilot doesn't understand previous inputs and can't undo changes after a user commits a change when authoring, either via user interface or the chat pane. For example, you can't ask Copilot to "Undo my last 5 inputs." However, users can still use the existing user interface options to delete unwanted steps or queries.
- Copilot can't make layout changes to queries in your session. For example, if you tell Copilot to create a new group for queries in the editor, it doesn't work.
- Copilot may produce inaccurate results when the intent is to evaluate data that isn't present within the sampled results imported into the sessions data preview.
- Copilot doesn't produce a message for the skills that it doesn't support. For example, if you ask Copilot to "Perform statistical analysis and write a summary over the contents of this query", it doesn't complete the instruction successfully as mentioned previously. Unfortunately, it doesn't give an error message either.