What's new in Microsoft Fabric?
This page is continuously updated with a recent review of what's new in Microsoft Fabric.
- To follow the latest in Fabric news and features, see the Microsoft Fabric Updates Blog.
- For community, marketing, case studies, and industry news, see the Microsoft Fabric Blog.
- Follow the latest in Power BI at What's new in Power BI?
- For older updates, review the Microsoft Fabric What's New archive.
New to Microsoft Fabric?
- Learning Paths for Fabric
- Get started with Microsoft Fabric
- End-to-end tutorials in Microsoft Fabric
- Definitions of terms used in Microsoft Fabric
Features currently in preview
The following table lists the features of Microsoft Fabric that are currently in preview. Preview features are sorted alphabetically.
Note
Features currently in preview are available under supplemental terms of use. Review for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability. Microsoft Fabric provides previews to give you a chance to evaluate and share feedback with the product group on preview features before they become generally available (GA).
| Feature | Learn more |
|---|---|
| Azure Data Factory item | You can now bring your existing Azure Data Factory (ADF) to your Fabric workspace. This new preview capability allows you to connect to your existing Azure Data Factory from your Fabric workspace. Select "Create Azure Data Factory" inside of your Fabric Data Factory workspace, and you can manage your Azure data factories directly from the Fabric workspace. |
| Capacity pools preview | Capacity administrators can now create custom pools (preview) based on their workload requirements, providing granular control over compute resources. Custom pools for Data Engineering and Data Science can be set as Spark Pool options within Workspace Spark Settings and environment items. |
| Code-First AutoML preview | In Synapse Data Science, the new AutoML feature enables automation of your machine learning workflow. AutoML, or Automated Machine Learning, is a set of techniques and tools that can automatically train and optimize machine learning models for any given data and task type. |
| Code-First Hyperparameter Tuning preview | In Synapse Data Science, FLAML is now integrated for hyperparameter tuning, currently a preview feature. Fabric's flaml.tune feature streamlines this process, offering a cost-effective and efficient approach to hyperparameter tuning. |
| Copilot in Fabric is available worldwide | Copilot in Fabric is now available to all customers, including Copilot for Power BI, Data Factory, Data Science & Data Engineering, and Real-Time Intelligence. Read more in our Overview on Copilot in Fabric. |
| Data Activator preview | We're thrilled to announce that Data Activator is now in preview and is enabled for all existing Microsoft Fabric users. |
| Apache Airflow job in Data Factory preview | Apache Airflow job (preview) in Data Factory, powered by Apache Airflow, offer seamless authoring, scheduling, and monitoring experience for Python-based data processes defined as Directed Acyclic Graphs (DAGs). For more information, see Quickstart: Create a Data workflow. |
| Data Wrangler for Spark DataFrames preview | Data Wrangler now supports Spark DataFrames in preview, users can now edit Spark DataFrames in addition to pandas DataFrames with Data Wrangler. |
| Copy job | The Copy job (preview) has advantages over the legacy Copy activity. For more information, see Announcing Preview: Copy Job in Microsoft Fabric. For a tutorial, see Learn how to create a Copy job (preview) in Data Factory for Microsoft Fabric. |
| Data Activator preview | Data Activator is in preview and is enabled for all existing Microsoft Fabric users. |
| Data Science AI skill (preview) | You can now build your own generative AI experiences over your data in Fabric with the AI skill (preview)! You can build question and answering AI systems over your Lakehouses and Warehouses. For more information, see Introducing AI Skills in Microsoft Fabric: Now in Preview. To get started, try AI skill example with the AdventureWorks dataset. |
| Delta column mapping in the SQL analytics endpoint | SQL analytics endpoint now supports Delta tables with column mapping enabled. For more information, see Delta column mapping and Limitations of the SQL analytics endpoint. This feature is currently in preview. |
| Domains in OneLake (preview) | Domains in OneLake help you organize your data into a logical data mesh, allowing federated governance and optimizing for business needs. You can now create sub domains, default domains for users, and move workspaces between domains. For more information, see Fabric domains. |
| High concurrency mode for Notebooks in Pipelines (preview) | High concurrency mode for Notebooks in Pipelines enables users to share Spark sessions across multiple notebooks within a pipeline. With high concurrency mode, users can trigger pipeline jobs, and these jobs are automatically packed into existing high concurrency sessions. |
| Introducing external data sharing (preview) | External Data Sharing (preview) is a new feature that makes it possible for Fabric users to share data from within their Fabric tenant with users in another Fabric tenant. |
| Fabric gateway enables OneLake shortcuts to on-premises data | Connect to on-premises data sources with a Fabric on-premises data gateway on a machine in your environment, with networking visibility of your S3 compatible or Google Cloud Storage data source. Then, you create your shortcut and select that gateway. For more information, see Create shortcuts to on-premises data. |
| Fabric Spark connector for Fabric Synapse Data Warehouse in Spark runtime (preview) | The Fabric Spark connector for Synapse Data Warehouse (preview) enables a Spark developer or a data scientist to access and work on data from a warehouse or SQL analytics endpoint of the lakehouse (either from within the same workspace or from across workspaces) with a simplified Spark API. |
| Fabric Spark Diagnostic Emitter (preview) | The Fabric Apache Spark Diagnostic Emitter (preview) allows Apache Spark users to collect logs, event logs, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. |
| Fabric workload dev kit (preview) | The Microsoft Fabric workload development kit extends to additional workloads and offers a robust developer toolkit for designing, developing, and interoperating with Microsoft Fabric using frontend SDKs and backend REST APIs. |
| Folder in Workspace preview | As an organizational unit in the workspace, folder addresses this pain point by providing a hierarchical structure for organizing and managing your items. For more information, see Create folders in workspaces (preview). |
| GraphQL API in Microsoft Fabric preview | The new API for GraphQL is a data access layer that allows us to query multiple data sources quickly and efficiently in Fabric. For more information, see What is Microsoft Fabric API for GraphQL? |
| Incremental refresh for Dataflow Gen2 (preview) | Incremental refresh for Dataflows Gen2 in Fabric Data Factory is designed to optimize data ingestion and transformation, particularly as your data continues to expand. For more information, see Announcing Preview: Incremental Refresh in Dataflow Gen2. |
| Invoke remote pipeline (preview) in Data pipeline | You can now use the Invoke Pipeline (preview) activity to call pipelines from Azure Data Factory or Synapse Analytics pipelines. This feature allows you to utilize your existing ADF or Synapse pipelines inside of a Fabric pipeline by calling it inline through this new Invoke Pipeline activity. |
| Kusto Cache consumption preview | The preview of Kusto Cache consumption means that you will start seeing billable consumption of the OneLake Cache Data Stored meter from the KQL Database and Eventhouse items. For more information, see KQL Database consumption. |
| Lakehouse schemas feature | The Lakehouse schemas feature (preview) introduces data pipeline support for reading the schema info from Lakehouse tables and supports writing data into tables under specified schemas. Lakehouse schemas allow you to group your tables together for better data discovery, access control, and more. |
| Lakehouse support for git integration and deployment pipelines (preview) | The Lakehouse now integrates with the lifecycle management capabilities in Microsoft Fabric, providing a standardized collaboration between all development team members throughout the product's life. Lifecycle management facilitates an effective product versioning and release process by continuously delivering features and bug fixes into multiple environments. |
| Managed virtual networks (preview) | Managed virtual networks are virtual networks that are created and managed by Microsoft Fabric for each Fabric workspace. |
| Microsoft 365 connector now supports ingesting data into Lakehouse (preview) | The Microsoft 365 connector now supports ingesting data into Lakehouse tables. |
| Microsoft Fabric Admin APIs | Fabric Admin APIs are designed to streamline administrative tasks. The initial set of Fabric Admin APIs is tailored to simplify the discovery of workspaces, Fabric items, and user access details. |
| Mirroring in Microsoft Fabric preview | With database mirroring in Fabric, you can easily bring your databases into OneLake in Microsoft Fabric, enabling seamless zero-ETL, near real-time insights on your data – and unlocking warehousing, BI, AI, and more. For more information, see What is Mirroring in Fabric?. |
| Native Execution Engine on Runtime 1.3 (preview) | Native Execution Engine for Fabric Runtime 1.3 is now available in preview, offering superior query performance across data processing, ETL, data science, and interactive queries. No code changes are required to speed up the execution of your Apache Spark jobs when using the Native Execution Engine. |
| Nested common table expressions (CTEs) (preview) | Fabric Warehouse and SQL analytics endpoint both support standard, sequential, and nested CTEs. While CTEs are generally available in Microsoft Fabric, nested common table expressions (CTE) in Fabric data warehouse are currently a preview feature. |
| Notebook debug within vscode.dev (preview) | You can now place breakpoints and debug your Notebook code with the Synapse VS Code - Remote extension in vscode.dev. This update first starts with the Fabric Runtime 1.3. |
| OneLake data access roles | OneLake data access roles for lakehouse are in preview. Role permissions and user/group assignments can be easily updated through a new folder security user interface. |
| OneLake SAS (preview) | Support for short-lived, user-delegated OneLake SAS is now in preview. This functionality allows applications to request a User Delegation Key backed by Microsoft Entra ID, and then use this key to construct a OneLake SAS token. This token can be handed off to provide delegated access to another tool, node, or user, ensuring secure and controlled access. |
| Prebuilt Azure AI services in Fabric preview | The preview of prebuilt AI services in Fabric is an integration with Azure AI services, formerly known as Azure Cognitive Services. Prebuilt Azure AI services allow for easy enhancement of data with prebuilt AI models without any prerequisites. Currently, prebuilt AI services are in preview and include support for the Microsoft Azure OpenAI Service, Azure AI Language, and Azure AI Translator. |
| Purview Data Loss Prevention policies have been extended to Fabric lakehouses | Extending Microsoft Purview's Data Loss Prevention (DLP) policies into Fabric lakehouses is now in preview. |
| Real-Time Dashboards and underlying KQL databases access separation (preview) | With separate permissions for dashboards and underlying data, administrators now have the flexibility to allow users to view dashboards without giving access to the raw data. |
| Real-Time hub | Real-Time hub is single, tenant-wide, unified, logical place for streaming data-in-motion. It enables you to easily discover, ingest, manage, and consume data-in-motion from a wide variety of sources. It lists all the streams and Kusto Query Language (KQL) tables that you can directly act on. It also gives you an easy way to ingest streaming data from Microsoft products and Fabric events. For more information, see Real-Time hub overview. |
| Reserve maximum cores for jobs (preview) | A new workspace-level setting allows you to reserve maximum cores for your active jobs for Spark workloads. For more information, see High concurrency mode in Apache Spark for Fabric. |
| REST APIs for Fabric Data Factory pipelines preview | The REST APIs for Fabric Data Factory Pipelines are now in preview. REST APIs for Data Factory pipelines enable you to extend the built-in capability in Fabric to create, read, update, delete, and list pipelines. |
| Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview) | By creating a Fabric Managed Private Endpoint, you can now securely connect Eventstream to your Azure services, such as Azure Event Hubs or IoT Hub, within a private network or behind a firewall. For more information, see Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview). |
| Session Expiry Control in Workspace Settings for Notebook Interactive Runs (preview) | A new session expiry control in Data Engineering/Science workspace settings allows you to set the maximum expiration time limit for notebook interactive sessions. By default, sessions expire after 20 minutes, but you can now customize the maximum expiration duration. |
| Share Feature for Fabric AI skill (preview) | "Share" capability for the Fabric AI skill (preview) allows you to share the AI Skill with others using a variety of permission models. |
| Share the Fabric AI skill (preview) | Share capability for the Fabric AI skill (preview) allows you to share the AI Skill with others using a variety of permission models. |
| Spark Run Series Analysis preview | The Spark Monitoring Run Series Analysis features allow you to analyze the run duration trend and performance comparison for Pipeline Spark activity recurring run instances and repetitive Spark run activities, from the same Notebook or Spark Job Definition. |
| Splunk add-on preview | Microsoft Fabric add-on for Splunk allows users to ingest logs from Splunk platform into a Fabric KQL DB using the Kusto python SDK. |
| Tags | Tags (preview) help admins categorize and organize data, enhancing the searchability of your data and boosting success rates and efficiency for end users. |
| Task flows in Microsoft Fabric (preview) | The preview of task flows in Microsoft Fabric is enabled for all Microsoft Fabric users. With Fabric task flows, when designing a data project, you no longer need to use a whiteboard to sketch out the different parts of the project and their interrelationships. Instead, you can use a task flow to build and bring this key information into the project itself. |
| varchar(max) and varbinary(max) support in preview | Support for the varchar(max) and varbinary(max) data types in Warehouse is now in preview. For more information, see Announcing public preview of VARCHAR(MAX) and VARBINARY(MAX) types in Fabric Data Warehouse. |
| Terraform Provider for Fabric (preview) | The Terraform Provider for Microsoft Fabric is now in preview. The Terraform Provider for Microsoft Fabric supports the creation and management of many Fabric resources. For more information, see Announcing the new Terraform Provider for Microsoft Fabric. |
| T-SQL support in Fabric notebooks (preview) | The T-SQL notebook feature in Microsoft Fabric (preview) lets you write and run T-SQL code within a notebook. You can use them to manage complex queries and write better markdown documentation. It also allows direct execution of T-SQL on connected warehouse or SQL analytics endpoint. To learn more, see Author and run T-SQL notebooks. |
| Warehouse restore points and restore in place | You can now create restore points and perform an in-place restore of a warehouse to a past point in time. Restore in-place is an essential part of data warehouse recovery, which allows to restore the data warehouse to a prior known reliable state by replacing or over-writing the existing data warehouse from which the restore point was created. |
| Warehouse source control (preview) | Using Git integration and/or deployment pipelines with your warehouse, you can manage development and deployment of versioned warehouse objects. You can use SQL Database Projects extension available inside of Azure Data Studio and Visual Studio Code. For more information on warehouse source control, see CI/CD with Warehouses in Microsoft Fabric. |
Generally available features
The following table lists the features of Microsoft Fabric that have recently transitioned from preview to general availability (GA).
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Notebook Git integration | Notebook Git integration now supports persisting the mapping relationship of the attached Environment when syncing to new workspace. For more information, see Notebook source control and deployment |
| October 2024 | Notebook in Deployment Pipeline | Now you can also use notebooks to deploy your code across different environments, such as development, test, and production. You can also use deployment rules to customize the behavior of your notebooks when they're deployed, such as changing the default Lakehouse of a Notebook. Get started with deployment pipelines, and Notebook shows up in the deployment content automatically. |
| September 2024 | Mirroring for Snowflake | With Mirroring for Snowflake in Fabric, you can easily bring your Snowflake data into OneLake in Microsoft Fabric. For more information, see Mirroring Snowflake. |
| September 2024 | Copilot for Data Factory | Copilot for Data Factory is now generally available and included in the Dataflow Gen2 experience. For more information, see Copilot for Data Factory. |
| September 2024 | Fast Copy in Dataflow Gen2 | The Fast Copy feature is now generally available. For more information, read Announcing the General Availability of Fast Copy in Dataflows Gen2. |
| September 2024 | Fabric Pipeline Integration in On-premises Data Gateway GA | On-premises connectivity for Data pipelines in Microsoft Fabric is now generally available. Learn How to access on-premises data sources in Data Factory for Microsoft Fabric. |
| September 2024 | Data Wrangler for Spark DataFrames | Data Wrangler is now generally available. A notebook-based tool for exploratory data analysis, Data Wrangler works for both pandas DataFrames and Spark DataFrames and arrives at general availability with new usability improvements. |
| September 2024 | Fabric Runtime 1.3 | Fabric Runtime 1.3 includes Apache Spark 3.5, Delta Lake 3.1, R 4.4.1, Python 3.11, support for Starter Pools, integration with Environment, and library management capabilities. For more information, see Fabric Runtime 1.3 is Generally Available!. |
| September 2024 | OneLake Shortcuts API | REST APIs for OneLake Shortcuts allow programmatic creation and management of shortcuts, now generally available. You can now programmatically create, read, and delete OneLake shortcuts. For example, see Use OneLake shortcuts REST APIs. |
| September 2024 | GitHub integration for source control | Fabric developers can now choose GitHub or GitHub Enterprise as their source control tool, and version their Fabric items there. For more information, see Intro to Git integration. |
| September 2024 | OneLake shortcuts to Google Cloud Storage | Create a Google Cloud Storage shortcut to connect to your existing data through a single unified name space without having to copy or move data. For more information, see Google Cloud Storage shortcuts shortcuts generally available. |
| September 2024 | OneLake shortcuts to S3-compatible data sources | Create an S3 compatible shortcut to connect to your existing data through a single unified name space without having to copy or move data. For more information, see S3 compatible shortcuts shortcuts generally available. |
| July 2024 | Update records in a KQL Database preview | The .update command is now generally available. Learn more about how to Update records in a Kusto database. |
| July 2024 | Warehouse queries with time travel (GA) | Warehouse in Microsoft Fabric offers the capability to query the historical data as it existed in the past at the statement level, now generally available. The ability to query data from a specific timestamp is known in the data warehousing industry as time travel. |
For older GA announcements, review the Microsoft Fabric What's New archive.
Community
This section summarizes new Microsoft Fabric community opportunities for prospective and current influencers and MVPs.
- Sign up for the Fabric Community Newsletter.
- Join a local Fabric User Group or join a local event.
- To learn about the Microsoft MVP Award and to find MVPs, see mvp.microsoft.com.
- Are you a student? Learn more about the Microsoft Learn Student Ambassadors program.
- Vote and contribute to Microsoft Fabric Ideas.
- Visit the Microsoft Fabric Career Hub for everything you need on your certification journey, including a 50% discount on exams.
- Watch and subscribe to Microsoft Fabric videos on YouTube.
- Ask and answer questions in the Microsoft Fabric community.
Note
The Microsoft Fabric and AI Learning Hackathon deadline is November 12, 2024. Whether you're an AI enthusiast, a cloud computing expert, or a database guru, this hackathon is the perfect platform to showcase your skills and creativity. Complete the Microsoft Learn AI Skills Challenge (Microsoft Fabric) and build a new Fabric solution that leverages Azure OpenAI services and falls into one of the following hackathon categories. Check out livestreamed events for the Microsoft Fabric & AI Learning Hackathon.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Fabric Influencers Spotlight October 2024 | Check out Microsoft MVPs & Fabric Super Users doing amazing work in October 2024 on all aspects of Microsoft Fabric. |
| October 2024 | Microsoft Fabric and AI Learning Hackathon: Copilot in Fabric | Part of the Microsoft Fabric and AI Learning Hackathon, read this guide of various capabilities that Copilot offers in Microsoft Fabric, empowering you to enhance productivity and streamline your workflows. |
| October 2024 | Get certified in Microsoft Fabric—for free! | For a limited time, the Microsoft Fabric Community team is offering 5,000 free DP-600 exam vouchers to eligible Fabric Community members. Complete your exam by the end of the year and join the ranks of certified experts. |
| October 2024 | DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric (beta) | The new Microsoft Certified: Fabric Data Engineer Associate certification will help demonstrate your skills with data ingestion, transformation, administration, monitoring, and performance optimization in Fabric. To learn more, see DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric (beta). |
| October 2024 | FabCon Europe 2024 | Read a recap of Europe's first Fabric Community Conference and a Recap of Data Factory announcements. |
| October 2024 | Fabric Influencers Spotlight September 2024 | The Fabric Influencers Spotlight September 2024 shines a bright light on the places on the internet where Microsoft MVPs & Fabric Super Users are doing some amazing work on all aspects of Microsoft Fabric. |
| September 2024 | Announcing: The Microsoft Fabric & AI Learning Hackathon | Get ready for the Microsoft Fabric & AI Learning Hackathon! We're calling all Data/AI Enthusiasts and Data/AI practitioners to join us for another exciting opportunity to upskill and build the next generation of Data + AI solutions with Microsoft Fabric! The Hackathon is open for a 7-week submission period and offers a total of $10,000 in prizes! |
| August 2024 | Fabric Influencers Spotlight August 2024 | The Fabric Influencers Spotlight August 2024 highlights and amplifies blog posts, videos, presentations, and other content related to Microsoft Fabric from members of Microsoft MVPs & Fabric Super Users from the Fabric community. |
| August 2024 | Winners of the Fabric Community Sticker Challenge | Congratulations to the winners of the Fabric Community Sticker Challenge! |
| July 2024 | Fabric Influencers Spotlight | Introducing the new Fabric Influencers Spotlight series of articles to highlight and amplify blog posts, videos, presentations, and other content related to Microsoft Fabric. Read blogs from Microsoft MVPs and Fabric Super Users from the Fabric community. |
For older updates, review the Microsoft Fabric What's New archive.
Power BI
Important
If you are accessing Power BI on a web browser version older than Chrome 94, Edge 94, Safari 16.4, Firefox 93, or equivalent, you need upgrade your web browser to a newer version by August 31, 2024. Using an outdated browser version after this date, may prevent you from accessing features in Power BI.
Updates to Power BI Desktop and the Power BI service are summarized at What's new in Power BI?
Microsoft Copilot in Microsoft Fabric
With Copilot and other generative AI features in preview, Microsoft Fabric brings a new way to transform and analyze data, generate insights, and create visualizations and reports. For more information, see Copilot in Fabric.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Microsoft Fabric and AI Learning Hackathon: Copilot in Fabric | Part of the Microsoft Fabric and AI Learning Hackathon, read this guide of various capabilities that Copilot offers in Microsoft Fabric, empowering you to enhance productivity and streamline your workflows. |
| October 2024 | Use Azure OpenAI to turn whiteboard sketches into data pipelines | Read this blog to learn how to turn whiteboard sketches into data pipelines, using the GPT-4o model through Azure OpenAI Service. |
| September 2024 | Creating a real time dashboard by Copilot | Copilot can review a table and automatically create a dashboard with insights and a profile of the data with a sample. |
| September 2024 | Copilot in Dataflow Gen2 GA | Copilot for Data Factory is now generally available and included in the Dataflow Gen2 experience. For more information, see Copilot for Data Factory. |
| September 2024 | Copilot for Data Warehouse | Copilot for Data Warehouse (preview) is now available, offering the Copilot chat pane, quick actions, and code completions. For more information and sample scenarios, see Announcing the Preview of Copilot for Data Warehouse in Microsoft Fabric. |
For older updates, review the Microsoft Fabric What's New archive.
Data Factory in Microsoft Fabric
This section summarizes recent new features and capabilities of Data Factory in Microsoft Fabric. Follow issues and feedback through the Data Factory Community Forum.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | New Features and Enhancements for Virtual Network Data Gateway | We're excited to announce several powerful updates to the Virtual Network (VNET) Data Gateway, designed to further enhance performance and improve the overall user experience. |
| October 2024 | Recap of Data Factory Announcements at Fabric Community Conference Europe | Read a recap of Data Factory announcements from Fabric Community Conference Europe 2024. |
| September 2024 | Copilot in Dataflow Gen2 GA | Copilot for Data Factory is now generally available and included in the Dataflow Gen2 experience. For more information, see Copilot for Data Factory. |
| September 2024 | Fast Copy in Dataflow Gen2 GA | The Fast Copy feature is now generally available. For more information, read Announcing the General Availability of Fast Copy in Dataflows Gen2. |
| September 2024 | Incremental refresh for Dataflow Gen2 (preview) | Incremental refresh for Dataflows Gen2 in Fabric Data Factory is designed to optimize data ingestion and transformation, particularly as your data continues to expand. For more information, see Announcing Preview: Incremental Refresh in Dataflow Gen2. |
| September 2024 | Certified connector updates | Updated Dataflow Gen2 connectors have been released, as well as an updated Data pipeline connectors for Salesforce and Vertica. For more information, see the Certified connector updates. |
| September 2024 | Fabric Pipeline Integration in On-premises Data Gateway GA | On-premises connectivity for Data pipelines in Microsoft Fabric is now generally available. Learn How to access on-premises data sources in Data Factory for Microsoft Fabric. |
| September 2024 | Invoke remote pipeline (preview) in Data pipeline | You can now use the Invoke Pipeline (preview) activity to call pipelines from Azure Data Factory or Synapse Analytics pipelines. This feature allows you to utilize your existing ADF or Synapse pipelines inside of a Fabric pipeline by calling it inline through this new Invoke Pipeline activity. |
| September 2024 | Spark Job environment parameters | You can now reuse existing Spark sessions with Session tags. In the Fabric Spark Notebook activity, tag your Spark session, then reuse the existing session using that same tag. |
| September 2024 | Azure Data Factory item in Fabric (preview) | You can now bring your existing Azure Data Factory (ADF) to your Fabric workspace. This new preview capability allows you to connect to your existing Azure Data Factory from your Fabric workspace. Select "Create Azure Data Factory" inside of your Fabric Data Factory workspace, and you can manage your Azure data factories directly from the Fabric workspace. |
| September 2024 | Copy job (preview) | The Copy job (preview) has advantages over the legacy Copy activity. For more information, see Announcing Preview: Copy Job in Microsoft Fabric. For a tutorial, see Learn how to create a Copy job (preview) in Data Factory for Microsoft Fabric. |
| September 2024 | Lakehouse Connector in Fabric Data Factory introduces Schema Support | Fabric Lakehouse supports the creation of custom schemas.When reading from a Lakehouse table with the Lakehouse Connector in Fabric Data Factory, custom schema information is now automatically included. |
| September 2024 | Storage Integration Support in Snowflake Connector for Fabric Data Factory | You can now connect Snowflake with external storage solutions (such as Azure Blob Storage) using a secure and centralized approach. For more information, see Snowflake SQL storage integration. |
| September 2024 | New Data Factory Connectors Released in Q3 2024 | New Data Factory Connectors include Salesforce, Azure MySQL Database, and Azure Cosmos DB for MongoDB. |
| August 2024 | Certified connector updates | Updated Dataflow Gen2 connectors have been released, as well as two new Data pipeline connectors for Salesforce and Vertica. For more information, see the August 2024 Certified connector updates. |
| August 2024 | Data Warehouse Connector Supports TLS 1.3 | The Data Warehouse connector now supports TLS 1.3, the latest version of the Transport Layer Security protocol. |
| August 2024 | Connect to your Azure Resources by Modern Get Data Experience in Data pipeline | You can easily browse and connect to your Azure resources automatically with the modern data experience of Data Pipeline. |
| July 2024 | Use existing connections from the OneLake Datahub integration | You can now select any existing connections from OneLake Datahub, not just your recent and favorite ones. This makes it easier to access your data sources from the homepage of modern get data in data pipeline. For more information, see Modern Get Data experience. |
| July 2024 | Snowflake storage integration | Connect and integrate Snowflake's storage integration to streamline data workflows and optimize performance across all staging scenarios, without the need to bring external storage to stage your dataset. For more information, see Snowflake connector. |
| July 2024 | Edit JSON code for Data pipelines | You can now edit the JSON behind your Data Factory pipelines in Fabric. When you design low-code pipeline workflows, directly editing the JSON code behind your visual pipeline canvas can increase your flexibility and improve your market time. |
| July 2024 | Dataflow Gen2 certified connector updates | New and updated Dataflow Gen2 connectors have been released, including two new connectors in Fabric Data Factory data pipeline: Azure MySQL Database Connector and Azure Cosmos DB for MongoDB Connector. For more information, see the July 2024 Certified connector updates. |
| July 2024 | Support for editing Navigation steps | Introducing a new experience to edit navigation steps within Dataflow, to connect to a different object, inside of the Applied steps section of the Query settings pane. For more information, see Editing Navigation steps. |
| July 2024 | Global view in Manage connections | The new Global view in Manage connections allows you to see all the available connections in your Fabric environment so you can modify them or delete them without ever having to leave the Dataflow experience. For more information, see Global view in Manage connections. |
| July 2024 | Fast Copy with On-premises Data Gateway Support in Dataflow Gen2 | Fast Copy (preview) in Dataflow Gen2 now supports on-premises data stores using a gateway to access on-premises stores like SQL Server with Fast Copy in Dataflow Gen2. |
| July 2024 | Fabric API for GraphQL (preview) pricing | API for GraphQL in Fabric starts billing on July 12, 2024, as part of your existing Power BI Premium or Fabric Capacity. Use the Fabric Capacity Metrics app to track capacity usage for API for GraphQL operations, under the name "Query". |
For older updates, review the Microsoft Fabric What's New archive.
Data Factory in Microsoft Fabric samples and guidance
| Month | Feature | Learn more |
|---|---|---|
| September 2024 | Integrate your SAP data into Microsoft Fabric | Learn more about an overview of SAP data options in Microsoft Fabric, along with some guidance on the respective use cases. |
| July 2024 | Connect to your Azure Resources from Fabric with the Data Pipeline Modern Get Data Experience | Learn how to connect to your Azure resources automatically with the modern get data experience of Data Pipelines. |
| July 2024 | Fabric Data Pipelines – Advanced Scheduling Techniques (Part 2: Run a Pipeline on a Specific Day) | This blog provides a tutorial on the ability to schedule a Pipeline on a specific day of the month, including both the start of the month along with the last day of the month. |
Synapse Data Engineering in Microsoft Fabric
This section summarizes recent new features and capabilities of the Data Engineering workload in Microsoft Fabric.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Native Execution Engine available at no additional cost | The Native Execution Engine is now available at no additional cost. The Native Execution Engine now supports Fabric Runtime 1.3, which includes Apache Spark 3.5 and Delta Lake 3.2. This upgrade enhances Microsoft Fabric's Data Engineering and Data Science workflows, offering boosts in performance and flexibility. |
| October 2024 | Use OneLake shortcuts to access data across capacities: Even when the producing capacity is paused | Learn how OneLake capacity consumption works when accessing data through a shortcut, particularly across capacities. |
| October 2024 | Purview Data Loss Prevention policies have been extended to Fabric lakehouses | Extending Microsoft Purview's Data Loss Prevention (DLP) policies into Fabric lakehouses is now in preview. |
| October 2024 | API for GraphQL support for Service Principal Names (SPNs) | Service Principal Names (SPN) support for API for GraphQL offers organizations looking to integrate their apps with API for GraphQL in Microsoft Fabric tie seamlessly with their enterprise identity and access management systems. For more information, see Service Principal Names (SPNs) in Fabric API for GraphQL. |
| October 2024 | Automatic code generation in API for GraphQL | Fabric API for GraphQL now adds the ability to automatically generate Python and Node.js code based on GraphQL queries tested in the API Explorer. |
| October 2024 | Notebook Git integration GA | Notebook Git integration now supports persisting the mapping relationship of the attached Environment when syncing to new workspace. For more information, see Notebook source control and deployment |
| October 2024 | Notebook in deployment pipeline GA | Now you can also use notebooks to deploy your code across different environments, such as development, test, and production. You can also use deployment rules to customize the behavior of your notebooks when they're deployed, such as changing the default Lakehouse of a Notebook. Get started with deployment pipelines, and Notebook shows up in the deployment content automatically. |
| October 2024 | Notebook in Org APP | The Notebook feature is now supported in Org APP. You can easily embed Notebook code and markdown cells, visuals, tables, charts, and widgets in OrgAPP, as a practical storytelling tool. |
| October 2024 | Notebook onboarding tour | The new Fabric Notebook Onboarding Tour is now available. This guided tour is designed to help you get started with the essential Notebook features and learn the new capabilities. |
| October 2024 | Notebook mode switcher | The Notebook mode switcher provides flexible access modes (Develop, Run Only, Edit, View) for your notebooks, which can help you easily manage the permissions to the notebook and the corresponding view. |
| October 2024 | Free selection support on display() table view | The free selection function on the rich dataframe preview in the notebook can improve the data analysis experience. To see the new features, read Free selection support on display() table view. |
| October 2024 | Filter, sort and search your Lakehouse objects | Sorting, Filtering, and Searching capabilities make data exploration and analysis more efficient by allowing you to quickly retrieve the information you need based on specific criteria, right within the Lakehouse environment. |
| September 2024 | Fabric Runtime 1.3 GA | Fabric Runtime 1.3, now generally available, includes Apache Spark 3.5, Delta Lake 3.1, R 4.4.1, Python 3.11, support for Starter Pools, integration with Environment, and library management capabilities. For more information, see Fabric Runtime 1.3 is Generally Available!. |
| September 2024 | Native Execution Engine on Runtime 1.3 (preview) | Native Execution Engine for Fabric Runtime 1.3 is now available in preview, offering superior query performance across data processing, ETL, data science, and interactive queries. No code changes are required to speed up the execution of your Apache Spark jobs when using the Native Execution Engine. |
| September 2024 | High concurrency mode for Notebooks in Pipelines (preview) | High concurrency mode for Notebooks in Pipelines enables users to share Spark sessions across multiple notebooks within a pipeline. With high concurrency mode, users can trigger pipeline jobs, and these jobs are automatically packed into existing high concurrency sessions. |
| September 2024 | Reserve maximum cores for jobs (preview) | A new workspace-level setting allows you to reserve maximum cores for your active jobs for Spark workloads. For more information, see High concurrency mode in Apache Spark for Fabric. |
| September 2024 | Session Expiry Control in Workspace Settings for Notebook Interactive Runs (preview) | A new session expiry control in Data Engineering/Science workspace settings allows you to set the maximum expiration time limit for notebook interactive sessions. By default, sessions expire after 20 minutes, but you can now customize the maximum expiration duration. |
| September 2024 | Fabric Spark Diagnostic Emitter (preview) | The Fabric Apache Spark Diagnostic Emitter (preview) allows Apache Spark users to collect logs, event logs, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. |
| September 2024 | Environment integration with Synapse VS Code extension | You can now create, configure and use an environment in Fabric in VS Code with the Synapse VS Code extension. |
| September 2024 | Notebook debug within vscode.dev (preview) | You can now place breakpoints and debug your Notebook code with the Synapse VS Code - Remote extension in vscode.dev. This update first starts with the Fabric Runtime 1.3. |
| September 2024 | Invoke Fabric User Data Functions in Notebook | You can now invoke User Defined Functions (UDFs) in your PySpark code directly from Microsoft Fabric Notebooks or Spark jobs. With NotebookUtils integration, invoking UDFs is as simple as writing a few lines of code. |
| September 2024 | Functions Hub | The new Functions Hub provides a single location to view, access and manage your User Data Functions. |
| September 2024 | Support for spaces in Lakehouse Delta table names | You can now create and query Delta tables with spaces in their names, such as "Sales by Region" or "Customer Feedback". All Fabric Runtimes and Spark authoring experiences support table names with spaces. |
| September 2024 | Enable/Disable Functionality in API for GraphQL | The Enable/Disable feature for queries and mutations in GraphQL API provides administrators and developers with granular control over API access and usage. |
| September 2024 | Public REST API of Livy endpoint | The Fabric Livy endpoint lets users submit and execute their Spark code on the Spark compute within a designated Fabric workspace, eliminating the need to create any Notebook or Spark Job Definition. |
| September 2024 | OneLake SAS (preview) | Support for OneLake SAS is now in preview. This functionality allows applications to request a User Delegation Key backed by Microsoft Entra ID, and then use this key to construct a short-lived, user-delegated OneLake SAS token. This token can be handed off to provide delegated access to another tool, node, or user, ensuring secure and controlled access. |
| September 2024 | Access Databricks Unity Catalog tables from Fabric (preview) | A mirrored Azure Databricks Unity Catalog in Fabric allows you to read data managed by Unity Catalog from Fabric workloads from the Lakehouse. In Fabric, you can now create a new data item called "Mirrored Azure Databricks Catalog". For more information, see Databricks Unity Catalog tables available in Microsoft Fabric. |
| September 2024 | T-SQL support in Fabric notebooks | The T-SQL notebook feature in Microsoft Fabric lets you write and run T-SQL code within a notebook. You can use them to manage complex queries and write better markdown documentation. It also allows direct execution of T-SQL on connected warehouse or SQL analytics endpoint. To learn more, see Author and run T-SQL notebooks. |
| September 2024 | OneLake shortcuts to Google Cloud Storage | Now a generally available feature, Create a Google Cloud Storage shortcut to connect to your existing data through a single unified name space without having to copy or move data. |
| September 2024 | OneLake shortcuts to S3-compatible data sources | Now a generally available feature, Create an S3 compatible shortcut to connect to your existing data through a single unified name space without having to copy or move data. |
| August 2024 | MsSparkUtils upgrade to NotebookUtils | The library MsSparkUtils has been rebranded as NotebookUtils. While NotebookUtils is backward compatible with MsSparkUtils, new features will only be added to the NotebookUtils namespace. For more information, see NotebookUtils (former MSSparkUtils) for Fabric. |
| August 2024 | Import Notebook UX improvement | The Import Notebook feature user interface has been enhanced - you can now effortlessly import notebooks, reports, or paginated reports using the unified entry in the workspace toolbar. |
| August 2024 | Lifecycle of Apache Spark runtimes in Fabric | The Lifecycle of Apache Spark runtimes document details the release cadence and versioning for the Azure-integrated platform based on Azure Spark. For more information, see the Fabric runtime lifecycle blog post. |
| July 2024 | MSSparkUtils API | The mssparkutils.runtime.context is a new API that provides context information of the current live session, including the notebook name, default lakehouse, workspace info, if it's a pipeline run, etc. For more information, see Microsoft Spark Utilities (MSSparkUtils) for Fabric. |
| July 2024 | Environment Resources folder | The new Environment Resources Folder is a shared repository designed to streamline collaboration across multiple notebooks. |
For older updates, review the Microsoft Fabric What's New archive.
Synapse Data Engineering samples and guidance
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Optimizing Spark Compute for Medallion Architectures in Microsoft Fabric | Learn how to optimize Spark Compute for Medallion architecture: a popular data engineering approach that emphasizes modularity. It organizes the data platform into three distinct layers: Bronze, Silver, and Gold. |
| August 2024 | Build a custom Sparklens JAR | In this blog, you will learn how to build the sparklens JAR for Spark 3.X, which can be used in Microsoft Fabric. |
| July 2024 | Create a shortcut to a VPC-protected S3 bucket | Learn how to create a shortcut to a VPC-protected S3 bucket, using the on-premises data gateway and AWS Virtual Private Cloud (VPC). |
| July 2024 | Move Your Data Across Workspaces Using Modern Get Data of Fabric Data Pipeline | The new modern get data experience of data pipeline now supports copying to Lakehouse and warehouse across different workspaces with an intuitive experience. |
Synapse Data Science in Microsoft Fabric
This section summarizes recent improvements and features for Data Science in Microsoft Fabric.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Enhancing Open Source: Fabric's Contributions to FLAML for Scalable AutoML | We have focused on enhancing FLAML's capabilities for Spark workloads. We've contributed several new Spark and non-Spark estimators to the FLAML project. Try these out with AutoML in Fabric (preview). |
| September 2024 | Data Wrangler for Spark DataFrames GA | Data Wrangler is now generally available. A notebook-based tool for exploratory data analysis, Data Wrangler works for both pandas DataFrames and Spark DataFrames and arrives at general availability with new usability improvements. |
| September 2024 | Share Feature for Fabric AI skill (preview) | "Share" capability for the Fabric AI skill (preview) allows you to share the AI Skill with others using a variety of permission models. |
| September 2024 | Session Expiry Control in Workspace Settings for Notebook Interactive Runs (preview) | A new session expiry control in Data Engineering/Science workspace settings allows you to set the maximum expiration time limit for notebook interactive sessions. By default, sessions expire after 20 minutes, but you can now customize the maximum expiration duration. |
| September 2024 | File editor in Notebook | The file editor feature in Fabric Notebook allows users to view and edit files directly within the notebook's resource folder and environment resource folder in notebook. Supported file types include CSV, TXT, HTML, YML, PY, SQL, and more. |
| August 2024 | Apply MLFlow tags on ML experiment runs and model versions | You can now apply MLflow tags directly on ML experiment runs and ML model versions from the user interface. |
| August 2024 | Track related ML Experiment runs in your Spark Application | You can now use an enhancement to the Monitoring Hub to track related ML experiment runs within Spark applications. You can also integrate Experiment items into the Monitoring Hub. |
| August 2024 | Use PREDICT with Fabric AutoML models | You can now move from training with AutoML to making predictions by using the built-in Fabric PREDICT UI and code-first APIs for batch predictions. For more information, see Machine learning model scoring with PREDICT in Microsoft Fabric. |
| August 2024 | Data Science AI skill (preview) | You can now build your own generative AI experiences over your data in Fabric with the AI skill (preview)! You can build question and answering AI systems over your Lakehouses and Warehouses. For more information, see Introducing AI Skills in Microsoft Fabric: Now in Preview. To get started, try AI skill example with the AdventureWorks dataset. |
| July 2024 | Semantic link preinstalled | Semantic Link in now included in the default runtime. If you use Fabric with Spark 3.4 or later, semantic link is already in the default runtime, and you don't need to install it. |
| July 2024 | Semantic Link Labs | Semantic Link Labs is a library of helpful python solutions for use in Microsoft Fabric notebooks. Semantic Link Labs helps Power BI developers and admins easily automate previously complicated tasks, as well as make semantic model optimization tooling more easily accessible within the Fabric ecosystem. For Semantic Link Labs documentation, see semantic-link-labs documentation. For more information and to see it in action, read the Semantic Link Labs announcement blog. |
For older updates, review the Microsoft Fabric What's New archive.
Synapse Data Science samples and guidance
| Month | Feature | Learn more |
|---|---|---|
| September 2024 | Using Microsoft Fabric for Generative AI: A Guide to Building and Improving RAG Systems | This tutorial includes three main notebooks, each covering a crucial aspect of building and optimizing RAG systems in Microsoft Fabric. |
| September 2024 | Harness Microsoft Fabric AI Skill to Unlock Context-Rich Insights from Your Data | This post demonstrates how you can extend the capabilities of Fabric AI Skill in Microsoft Fabric notebooks to deliver richer and more comprehensive responses using additional Large Language Model (LLM) queries. |
Synapse Data Warehouse in Microsoft Fabric
This section summarizes recent improvements and features for Synapse Data Warehouse in Microsoft Fabric.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Case insensitive collation support | By default, the collation of a warehouse is case sensitive (CS) with 'Latin1_General_100_BIN2_UTF8'. You can now Create a warehouse with case-insensitive (CI) collation. |
| October 2024 | varchar(max) and varbinary(max) support in preview | Support for the varchar(max) and varbinary(max) in data types in Warehouse is now in preview. For more information, see Announcing public preview of VARCHAR(MAX) and VARBINARY(MAX) types in Fabric Data Warehouse. |
| October 2024 | Concurrency performance improvements | We have recently optimized our task scheduling algorithm in our distributed query processing engine (DQP) to reduce contention when the workspace is under moderate to heavy concurrency. In testing we have observed that this optimization makes significant performance improvements in querying workloads. |
| October 2024 | JSON support enhancements | JSON functionalities in warehouse and SQL analytics endpoints for Lakehouse and mirrored databases have been improved. For details see, JSON support enhancements. |
| October 2024 | Nested Common Table Expressions (CTEs) (preview) | Fabric Warehouse and SQL analytics endpoint both support standard, sequential, and nested CTEs. While CTEs are generally available in Microsoft Fabric, nested common table expressions (CTE) in Fabric data warehousing (Transact-SQL) are currently a preview feature. |
| September 2024 | Mirroring for Snowflake GA | With Mirroring for Snowflake in Fabric, you can easily bring your Snowflake data into OneLake in Microsoft Fabric. For more information, see Mirroring Snowflake. |
| September 2024 | Copilot for Data Warehouse | Copilot for Data Warehouse (preview) is now updated and available as a preview feature, offering the Copilot chat pane, quick actions, and code completions. |
| September 2024 | Delta column mapping in the SQL analytics endpoint | SQL analytics endpoint now supports Delta tables with column mapping enabled. For more information, see Delta column mapping and Limitations of the SQL analytics endpoint. This feature is currently in preview. |
| September 2024 | Lakehouse schemas in SQL analytics endpoint | Lakehouse schemas allow delta tables in schemas to be queried in the SQL analytics endpoint. For more information, see Lakehouse schemas feature (preview). |
| September 2024 | Fabric Spark connector for Fabric Data Warehouse new features (preview) | The Fabric Spark connector for Fabric Data Warehouse (preview) now supports custom or pass-through queries, PySpark, and Fabric Runtime 1.3 (Spark 3.5). |
| September 2024 | New editor improvements | Editor improvements for Warehouse and SQL analytics endpoint items improve the consistency and efficiency. For more information, see New editor improvements. |
| September 2024 | T-SQL support in Fabric notebooks (preview) | The T-SQL notebook feature in Microsoft Fabric (preview) lets you write and run T-SQL code within a notebook. You can use them to manage complex queries and write better markdown documentation. It also allows direct execution of T-SQL on connected warehouse or SQL analytics endpoint. To learn more, see Author and run T-SQL notebooks. |
| September 2024 | Nested Common Table Expressions (CTEs) (preview) | Fabric Warehouse and SQL analytics endpoint both support standard, sequential, and nested CTEs. While CTEs are generally available in Microsoft Fabric, nested common table expressions (CTE) in warehouse are currently a preview feature. |
| September 2024 | Mirrored Azure Databricks (Preview) | A mirrored Azure Databricks Unity Catalog in Fabric allows you to read data managed by Unity Catalog from Fabric workloads from the Lakehouse. For more information, see Databricks Unity Catalog tables available in Microsoft Fabric. |
| August 2024 | Mirroring integration with modern get data experience | You can now use the Modern Get Data experience to choose from all the available mirrored databases in the OneLake data hub. |
| August 2024 | T-SQL DDL support in Azure SQL Database mirrored database | You can now run DDL operations on a Azure SQL Database mirrored database such as Drop Table, Rename Table, and Rename Column. |
| August 2024 | Delta Lake log publishing pause and resume | You can now pause and resume the publishing of Delta Lake Logs for Warehouses. For more information, see Delta Lake logs in Warehouse in Microsoft Fabric. |
| August 2024 | Managing V-Order behavior of Fabric Warehouses | You can now manage V-Order behavior at the warehouse level. For more information, see Understand V-Order for Microsoft Fabric Warehouse. |
| August 2024 | TRUNCATE T-SQL support | The TRUNCATE T-SQL command is now supported in Warehouse tables. |
| July 2024 | ALTER TABLE and nullable column support | We've added T-SQL ALTER TABLE support for some operations, as well as nullable column support to tables in the warehouse. For more information, see ALTER TABLE (Transact-SQL). |
| July 2024 | Warehouse queries with time travel (GA) | Warehouse in Microsoft Fabric offers the capability to query the historical data as it existed in the past at the statement level, now generally available. The ability to query data from a specific timestamp is known in the data warehousing industry as time travel. |
| July 2024 | Restore warehouse experience in the Fabric portal | You can now create restore points and perform a restore in-place of a warehouse item. For more information, see Seamless Data Recovery through Warehouse restoration. |
| July 2024 | Warehouse source control (preview) | Using Git integration and/or deployment pipelines with your warehouse, you can manage development and deployment of versioned warehouse objects. You can use SQL Database Projects extension available inside of Azure Data Studio and Visual Studio Code. For more information on warehouse source control, see CI/CD with Warehouses in Microsoft Fabric. |
| July 2024 | Time travel and clone table retention window expanded | The retention period for time travel queries and clone table is now 30 days. |
For older updates, review the Microsoft Fabric What's New archive.
Synapse Data Warehouse samples and guidance
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Ensuring Data Continuity in Fabric Warehouse: Best Practices for Every Scenario | Dive deep into the common recovery scenarios and features that help enable seamless end-to-end data recovery and discuss best practices to ensure data resilience. |
| August 2024 | Mirroring SQL Server database to Fabric | While SQL Server isn't currently supported for Fabric mirrored databases, learn how to extend Fabric mirroring to an on-premises SQL Server database as a source, using a combination of SQL Server Transactional replication and Fabric Mirroring. |
| July 2024 | Microsoft Entra authentication for Fabric Synapse Data Warehouse | For sample connection strings and more information on using Microsoft Entra as an alternative to SQL Authentication, see Microsoft Entra authentication as an alternative to SQL authentication. |
Real-Time Intelligence in Microsoft Fabric
This section summarizes recent improvements and features for Real-Time Intelligence in Microsoft Fabric.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview) | By creating a Fabric Managed Private Endpoint, you can now securely connect Eventstream to your Azure services, such as Azure Event Hubs or IoT Hub, within a private network or behind a firewall. For more information, see Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview). |
| October 2024 | Usage reporting for Data Activator is now live | The Data Activator team has rolled out usage reporting to help you better understand your capacity consumption and future charges. When you look at the Capacity Metrics App you'll now see operations for the reflex items included. |
| October 2024 | Real-Time Dashboards and underlying KQL databases access separation (preview) | With separate permissions for dashboards and underlying data, administrators now have the flexibility to allow users to view dashboards without giving access to the raw data. |
| October 2024 | Real-Time Dashboards Integration with GitHub | Fabric's Git integration is now available for Real-Time Dashboards. For more information, see What is Microsoft Fabric Git integration?. |
| October 2024 | Quickly visualize query results in KQL Queryset | You can now graphically visualize KQL Queryset results instantly and effortlessly and control the formatting without the need for re-run queries – all using a familiar UI. |
| October 2024 | Pin query to dashboard | You can now save the outcome of any query written in KQL Queryset directly to a new or existing Real-Time Dashboard. |
| September 2024 | Creating a real time dashboard by Copilot | Copilot can review a table and automatically create a dashboard with insights and a profile of the data with a sample. |
| September 2024 | New Real-Time Hub and KQL Database user experiences | The new user experience features new Real-Time hub navigation, a My Streams page, an enhanced database page experience, and more. |
| September 2024 | Eventhouse as a new Destination in Eventstream | Eventhouses, equipped with KQL Databases, can handle and analyze large volumes of data. With the Eventhouse destination in Eventstream, you can efficiently process and route data streams into an Eventhouse and analyze the data in near real-time using KQL. |
| September 2024 | Managed private endpoints for Eventstream | With a Fabric Managed Private Endpoint, you can now establish a private connection between your Azure services, such as Azure Event Hubs, and Fabric Eventstream. For more information, see Eventstream integration with managed private endpoint. |
| September 2024 | Data Activator alerts on KQL Querysets | Now you can set up Data Activator (preview) alerts directly on your KQL queries in KQL querysets. For more information and samples, see Create Data Activator alerts from a KQL Queryset (preview). |
| September 2024 | Real-Time Dashboards continuous or 10s refresh rate | The dashboard auto refresh feature now supports continuous and 10 second refresh rates, in addition to the existing options. This upgrade, addressing a popular customer request, allows both editors and viewers to set near real-time and real-time data updates. |
| September 2024 | Multivariate anomaly detection | A new workflow for multivariate anomaly detection of time series data is based on the algorithm that is used in the AI Anomaly Detector service (which is being retired as a standalone service). For a tutorial, see Multivariate Anomaly Detection. |
| September 2024 | Real-Time Intelligence Copilot conversational mode | The Copilot assistant, which translates natural language into KQL, now supports conversational mode, allowing you to ask follow-up questions that build on previous queries within the chat. |
| September 2024 | New connectors and UI in Real-Time hub | Four new connectors are released on September 24, 2024: Apache Kafka, Amazon Managed Streaming for Apache Kafka, Azure SQL Managed Instance CDC, SQL Server on VM DB CDC. The tabs in the main page of Real-Time hub are replaced with menu items on the left navigation menu. For more information, see Get started with Fabric Real-Time hub. You can connect to Azure streaming sources using private endpoints now. |
| September 2024 | Announcement: Eventhouse Standard Storage billing | Starting the week of September 16 you will start seeing billable consumption of the OneLake Storage Data Stored meter from the Eventhouse and KQL Database items. |
| August 2024 | Fabric Real-Time Hub Teaching Bubbles | New teaching bubbles provide a step-by-step guide through its major functionalities. These interactive guides allow you to seamlessly navigate each tab of the Real-Time Hub. For more information, see Fabric Real-Time Hub Teaching Bubble. |
| August 2024 | KQL Queryset REST API support | The new Fabric Queryset REST APIs allow you to create/update/delete KQL Querysets in Fabric, and programmatically manage them without manual intervention. For more information, see KQL Queryset REST API support. |
| July 2024 | Update records in a KQL Database preview | The .update command is now generally available. Learn more about how to Update records in a Kusto database. |
| July 2024 | Real-Time Dashboards 1s and 10s refresh rate | Real-time Dashboards now support ultra-low refresh rates of just 1 or 10 seconds. For more information, see Create a Real-Time Dashboard (preview). |
For older updates, review the Microsoft Fabric What's New archive.
Real-Time Intelligence samples and guidance
| Month | Feature | Learn more |
|---|---|---|
| August 2024 | Advanced Time Series Anomaly Detector in Fabric | Read an example using the time-series-anomaly-detector in Fabric to upload stocks change table to Fabric, train the multivariate anomaly detection model in a Python notebook using Spark engine, and predict anomalies by applying the trained model to new data using Eventhouse (Kusto) engine. |
| August 2024 | Acting on Real-Time data using custom actions with Data Activator | Learn how to monitor and acting on data is to use Data Activator, which is a no-code experience in Microsoft Fabric for taking action automatically when the condition of the package temperature is detected in the data. |
| July 2024 | Build real-time order notifications with Eventstream's CDC connector | Read about a a real-life example of how an online store used Eventstream's CDC connector from Azure SQL Database. |
| July 2024 | Automating Real-Time Intelligence Eventhouse deployment using PowerShell | Let's build a PowerShell script to automate the deployment of Eventhouse, KQL Database, Tables, Functions, and Materialized Views into a workspace in Microsoft Fabric. |
Microsoft Fabric core features
News and feature announcements core to the Microsoft Fabric experience.
| Month | Feature | Learn more |
|---|---|---|
| October 2024 | New Item panel in Workspace | Previously, by selecting +New in the workspace, you can access a drop-down menu with some pre-defined item types to get started. Now, the +New item button shows item types listed in a panel, categorized by tasks. |
| October 2024 | Enhanced Tenant Setting Delegation for Export Controls | Delegation of export settings is now available to workspaces via domain. This new capability provides more granular control over data export permissions, addressing the specific needs of tenant, domain and workspace administrators. |
| October 2024 | APIs for Managed Private Endpoint are now available | REST APIs for managed Private Endpoints are available. You can now create, delete, get, list Managed private endpoints via APIs. |
| October 2024 | Important billing updates coming to Copilot and AI in Fabric | Upcoming pricing and billing updates to make Copilot and AI features in Fabric more accessible and cost-effective. |
| September 2024 | Terraform Provider for Fabric (preview) | The Terraform Provider for Microsoft Fabric is now in preview. The Terraform Provider for Microsoft Fabric supports the creation and management of many Fabric resources. For more information, see Announcing the new Terraform Provider for Microsoft Fabric. |
| September 2024 | Announcing Service Principal support for Fabric APIs | You can now use service principal to access Fabric APIs. Service principal is a security identity that you can create in Microsoft Entra and assign permissions to it in Microsoft Entra and other Microsoft services, such as Microsoft Fabric. |
| September 2024 | Tag your data to enrich item curation and discovery | Tags (preview) help admins categorize and organize data, enhancing the searchability of your data and boosting success rates and efficiency for end users. |
| September 2024 | Trusted workspace access and Managed private endpoints in any Fabric capacity | Trusted workspace access and Managed private endpoints are available in any Fabric capacity. Previously, trusted workspace access and Managed private endpoints were available only in F64 or higher capacities. Managed Private endpoints are now available in Trial capacities. |
| September 2024 | Multitenant organization (MTO) (preview) | Fabric now supports Microsoft Entra ID Multitenant Organizations (MTO). The multitenant organizations capability in Microsoft Entra ID synchronizes users across multiple tenants, adding them as users of type external member. For more information, see Distribute Power BI content to external guest users with Microsoft Entra B2B. |
| September 2024 | Microsoft Fabric Achieves HITRUST CSF Certification | Microsoft Fabric is now certified for the HITRUST Common Security Framework (CSF) v11.0.1. |
| August 2024 | OneLake data access role improvements | Based on key feedback, we've updated data access roles with a user interface redesign. For more information, see Get started with OneLake data access roles (preview). |
| August 2024 | Workspace filter improvement to support nested folders | We have upgraded the filter experience to support filtering through the entire workspace or through a specific folder with all its nested folders. |
| August 2024 | Announcing the availability of Trusted workspace access and Managed private endpoints in any Fabric capacity | Use Trusted workspace access and Managed Private endpoints in Fabric with any F capacity and enjoy the benefits of secure and optimized data access and connectivity. |
| July 2024 | SOC certification compliance | We are excited to announce that Microsoft Fabric, our all-in-one analytics solution for enterprises, is now System and Organization Controls (SOC) 1 Type II, SOC 2 Type II, and SOC 3 compliant. |
| July 2024 | Microsoft Fabric .NET SDK | We are excited to announce the very first release of the Microsoft Fabric .NET SDK! For more information on the REST API documentation, see Microsoft Fabric REST API documentation. |
For older updates, review the Microsoft Fabric What's New archive.
Continuous Integration/Continuous Delivery (CI/CD) in Microsoft Fabric
This section includes guidance and documentation updates on development process, tools, source control, and versioning in the Microsoft Fabric workspace.
| Month | Feature | Learn more |
|---|---|---|
| September 2024 | GitHub integration for source control | Now generally available, Fabric developers can now choose GitHub or GitHub Enterprise as their source control tool, and version their Fabric items there. For more information, see Intro to Git integration. |
| September 2024 | New Deployment Pipelines design | A new and improved design for the Deployment Pipeline introduces a range of changes, additions, and improvements designed to elevate your deployment process. Read more about What's changed in deployment pipelines. |
| July 2024 | GitHub integration for source control (preview) | Fabric developers can now choose GitHub or GitHub Enterprise as their source control tool, and version their Fabric items there. For more information, see Get started with Git integration (preview). |
| July 2024 | Microsoft Fabric .NET SDK | We are excited to announce the very first release of the Microsoft Fabric .NET SDK! For more information on the REST API documentation, see Microsoft Fabric REST API documentation. |
For older updates, review the Microsoft Fabric What's New archive.
Continuous Integration/Continuous Delivery (CI/CD) samples
| Month | Feature | Learn more |
|---|---|---|
| August 2024 | Exploration of Microsoft Fabric's CI/CD Features | A guided tour of Microsoft Fabric's CI/CD features for data pipelines, lakehouse, notebooks, reports, and semantic models. |
Archive
For older updates, review the Microsoft Fabric What's New archive.