Ingestion (Bronze) layer

Important

Some or all of this functionality is available as part of a preview release. The content and the functionality are subject to change.

The bronze layer serves as the landing zone for raw data from multiple source systems. It captures data in its original form to create a complete audit trail and enable reprocessing if needed.

Data sources

Salesforce Nonprofit Success Pack (NPSP)

Resources: Salesforce connector on Fabric, bronze lakehouse, data pipeline

  • Data (NPSP Salesforce objects) is copied from Salesforce using a data pipeline connector to staging tables to enable incremental loading.
  • Data is merged into bronze layer tables in the Fundraising_BR (bronze) lakehouse.
  • Transformation notebooks process this data from the bronze layer to the silver layer.

Dynamics 365 Sales and the Common Data Model for Nonprofits

Resources: Dynamics 365 lakehouse (user-selected), transformation notebooks

  • Organizations using Dynamics 365 Sales with the Common Data Model for Nonprofits can connect their existing Dynamics 365 lakehouse.
  • During installation, users select their Dynamics 365 lakehouse from available lakehouses in the workspace.
  • Data is already in bronze format in the Dynamics 365 lakehouse and is transformed directly to the silver layer using specialized notebooks:
    • Fundraising_D365_Config: Configuration and base classes for Dynamics 365 synchronization
    • Fundraising_D365_Transform: Transforms Dynamics 365 data (option sets, contacts, accounts, opportunities, campaigns) from bronze to silver
  • The transformation uses incremental watermark-based synchronization to process only changed records.

Additional data sources

You can extend the ingestion layer by adding more data sources. For example, you can add a new connector for a new data source or to connect additional Dynamics 365 tables.

See also