Publish Error via Azure Data Factory after Terraform Deployment Pointing to existing GitHub Repo

Question

I have an error below when I tried to publish from Azure Data Factory after It is deployed via Terraform.

Publishing error: Invalid references or dependencies found. This is
likely due to publishing outside of Git mode or editing and deleting
linked services in other branches. To recover from this state, please
refer to the errors below and our Git troubleshooting guide
https://learn.microsoft.com/en-gb/azure/data-factory/source-control#stale-publish-branch

The repo settings in the Data Factory is below:

The Data Factory is created and deployed by Terraform. All pipelines in JSON files created in the DF are stored in a GitHub repo like below:

resource "azurerm_data_factory" "data-factory" {  
  name                = "data-factory"  
  location            = azurerm_resource_group.data-factory.location  
  resource_group_name = azurerm_resource_group.data-factory.name  
  github_configuration {  
    account_name = "xxxx"  
    branch_name = "main"  
    git_url = "https://github.com/xxxx/azure-data-factory-etl-demo"  
   repository_name = "azure-data-factory-etl-demo"  
    root_folder = "/"  
  }  

I publish from Data Factory after the deployment without making changes to the repo and anything in Data Factory.

I tried both Options 1&2 in the link below, but it is still not working.

https://learn.microsoft.com/en-gb/azure/data-factory/source-control#stale-publish-branch

The Github repository is public

Data Factory in Github for Authorized OAuth Apps is below (GitHub -> Settings -> Applications -> Authorized OAuth Apps)

I am able to create a new branch in my Github repo from Data Factory, which means it has write permission.

The example is from the link below:
https://medium.com/twodigits/deploy-your-azure-data-factory-through-terraform-2a9c2bd2d75d

The issue has been raised from below:

https://stackoverflow.com/questions/70854827/publish-error-via-azure-data-factory-after-terraform-deployment-pointing-to-exis/70862483

Answer 1

Hi @Song ,

Thank you for posting query in Microsoft Q&A Platform.

If any of our pipelines referring to datasets or linked services or other pipelines which are not existing in our pipeline, then in that case you usually end up with this error.

Could you please cross verify all your pipelines and see if they are referring to any of non existing datasets or linked services or triggers or pipelines ?

Kindly hit validate button as well in your ADF to validate your pipelines.

Hope this helps. Please let us know how it goes. Thank you.