How to configure GitHub to ADF

Question

Create a repo under org account and assigned myself as an admin. The visibility of the repo is Internal. Step 2 works fine with the repo with type public but not as an internal.

Step 1 : Connection to GitHub using the repo owner is successful

Step 2 : No repos are shown in the drop down.

Answer 1

Hi Sathya,

Thanks for reaching out to Microsoft Q&A.

Connecting to a Private GitHub Repository

1. Create a Private GitHub Repository:

• Create a new private repository under your GitHub organization account and assign yourself as an admin.

2. Connect ADF to the Private GitHub Repository:
   • In the Azure portal, navigate to your Azure Data Factory instance.
   • Click on "Set up code repository" and select "GitHub" as the repository type.
   • Authenticate with your GitHub account when prompted.
   • In the "GitHub repository owner" field, enter the name of your GitHub organization.
   • However, you will not see your private repository listed in the "Repository Name" dropdown.
3. Grant ADF Access to the Private Repository:
   • Go to your GitHub organization settings and navigate to the "Applications" section.
   • You should see the Azure Data Factory application listed there. Click on it.
   • In the application settings, grant the ADF application access to your private repository by selecting the appropriate permissions.
4. Retry Connecting ADF to the Private Repository:
   • After granting the necessary permissions, go back to the Azure Data Factory instance in the portal.
   • Click "Set up code repository" again and this time you should see your private repository listed in the "Repository Name" dropdown.
   • Select your private repository and complete the rest of the setup process.

The key steps are:

1. Create a private GitHub repository under your organization.
2. Grant the Azure Data Factory application access to your private repository through the GitHub organization settings.
3. Then you should be able to successfully connect your ADF instance to the private GitHub repository.

Let me know if you have any other questions!

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

Answer 2

Hi @Sathya

Thanks for the question and using MS Q&A platform.

The issue you're facing is that Azure Data Factory can't directly access private repositories during the initial setup through the dropdown menu. However, there is a workaround to connect ADF to your internal GitHub repo.

Steps that need to configure ADF with your private GitHub repo:

Create a Data Factory Instance without Git Integration: Instead of selecting the option for Git integration during ADF creation, choose to create a new instance without it.

Set Up Code Repository Later: Navigate to your newly created ADF instance in the Azure portal. Look for the option "Set up code repository" within the ADF settings.

Use "Link Repository" Option: During the code repository setup, you'll be presented with a choice between selecting a repository from a dropdown or using a link. Since your repo is private, the dropdown won't show it. Choose the "Use Link Repository" option instead. Manually Provide Repository URL: With the "Use Link Repository" option selected, you'll be able to manually enter the URL of your private GitHub repository. This URL should follow the format: https://{your_github_username_or_organisation}.github.com/{repository_name} Complete OAuth Authentication: After entering the URL, ADF will prompt you for GitHub OAuth credentials. Since you're the admin of the repository, provide your credentials with read/write permissions. This allows ADF to access and manage the code within your private repo.

By following these steps, you can establish a connection between your internal GitHub repo and Azure Data Factory without making the repository public.

Hope this helps! Let me know if you have any further questions.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.