Azure DevOps Dynamic Repository CheckOUT!

Question

Azure DevOps Dynamic Repository CheckOUT!

Anonymous

Trying to develop a multi-checkout tool in the pipeline that will enable to us to ghas scanning tool. Currently have a working pipeline for one repository at a time but would like to scale its capabilities. Looking for suggestions on what I can do to make this work efficiently?


# GHAS Security Scan for Databricks Notebook (.py) with Auto-Clean and Auto-Delete

trigger: none # Manual trigger to allow flexible input

parameters:
  
  - name: repository_name
    displayName: 'Repo to scan'
    type: string
    default: 'none'
    
  - name: branch_name
    displayName: 'Branch to scan'
    type: string
    default: 'none'

  - name: notebook_path
    displayName: 'Notebook Path (.py file)'
    type: string
    default: 'none'


variables:
  NOTEBOOK_PATH: ${{ parameters.notebook_path }}

pool:
  vmImage: 'ubuntu-latest'

steps:
  # 1. Checkout the code from the correct branch
  - checkout: self
    persistCredentials: true
    clean: true
    fetchDepth: 1

  # 2. Use Python 3
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.x'
    displayName: 'Use Python 3'

 
  # 5. Initialize CodeQL scan
  - task: AdvancedSecurity-Codeql-Init@1
    inputs:
      languages: 'python'
      sourceRoot: '.'
    displayName: 'Initialize CodeQL'

  # 7. Run the CodeQL static analysis
  - task: AdvancedSecurity-Codeql-Analyze@1
    displayName: 'Run CodeQL Scan'

  # 8. Publish results to Security tab
  - task: AdvancedSecurity-Publish@1
    displayName: 'Publish CodeQL Results'

0 comments

Answer accepted by question author

0 additional answers

Your answer

Answer 1

Gaurav Kumar 790 Moderator

Hi Ester, Sammy,

Azure DevOps Multi-Repo Checkout for GHAS Scanning using CodeQL

To dynamically scan multiple GitHub repository using Azure DevOps pipelines and GitHub Advanced Security (GHAS) with CodeQL, you can extend your pipeline to support multi-repo checkout, dynamic repo name/branch input, Reusable job templates for scalability and auto-cleanup post-analysis.

Follow the below workaround

Define Repository Parameters in Your Pipeline


parameters:

  - name: repositories

    type: object

    default:

      - name: 'repo1'

        branch: 'main'

        notebook_path: 'path/to/notebook1.py'

      - name: 'repo2'

        branch: 'dev'

        notebook_path: 'notebooks/demo.py'

Loop Over Repositories with Matrix-Style Jobs


jobs:

- ${{ each repo in parameters.repositories }}:

  - job: Scan_${{ repo.name }}

    displayName: Scan ${{ repo.name }}

    pool:

      vmImage: 'ubuntu-latest'

    steps:

      - checkout: git://${{ repo.name }}

        persistCredentials: true

        clean: true

      - task: UsePythonVersion@0

        inputs:

          versionSpec: '3.x'

      - task: AdvancedSecurity-Codeql-Init@1

        inputs:

          languages: 'python'

          sourceRoot: '.'

      - task: AdvancedSecurity-Codeql-Analyze@1

      - task: AdvancedSecurity-Publish@1

Please keep this in mind:

The git:// syntax works for GitHub repositories if a service connection is configured.
For Azure Repos, you'll need to use the checkout with repository syntax:

- checkout: git://MyProject/RepoName@refs/heads/my-branch

Use a Reusable Template (Recommended)

Create a template file named ghas-template.yml:


parameters:

  - name: repo_name

    type: string

  - name: branch_name

    type: string

  - name: notebook_path

    type: string

jobs:

- job: Scan_${{ parameters.repo_name }}

  displayName: Scan ${{ parameters.repo_name }}

  pool:

    vmImage: 'ubuntu-latest'

  steps:

    - checkout: git://${{ parameters.repo_name }}@${{ parameters.branch_name }}

      persistCredentials: true

      clean: true

    - task: UsePythonVersion@0

      inputs:

        versionSpec: '3.x'

    - task: AdvancedSecurity-Codeql-Init@1

      inputs:

        languages: 'python'

        sourceRoot: '.'

    - task: AdvancedSecurity-Codeql-Analyze@1

    - task: AdvancedSecurity-Publish@1

Then, Reference the template from main pipeline:


jobs:

- ${{ each repo in parameters.repositories }}:

  - template: ghas-template.yml

    parameters:

      repo_name: ${{ repo.name }}

      branch_name: ${{ repo.branch }}

      notebook_path: ${{ repo.notebook_path }}

For more details, please refer the following Microsoft documentation:
Multi-repo checkout in Azure Pipelines
Set up code scanning with GHAS
Reusable YAML templates in Azure Pipelines

I hope this information helps. Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Gaurav Kumar 790 Reputation points Moderator

2025-06-11T05:00:56.2133333+00:00

If this answer was helpful, please click "Accept the answer" and mark Yes, as this can help other community members.
Anonymous

2025-06-11T14:07:16.7933333+00:00

Thanks Gaurav Kumar, this looks good. Something that I am curious about is why do I not see all the repositories being touched when the CodeQL scanning is checking out the repositories listed in the pipeline?
Gaurav Kumar 790 Reputation points Moderator

2025-06-12T05:10:40.04+00:00
Hi Ester, Sammy,

The behavior you're seeing, CodeQL not appearing to touch all repositories even though they're checked out can stem from a few common causes in Azure DevOps pipelines with GHAS (GitHub Advanced Security).

Why You Might Not See All Repositories Scanned by CodeQL

CodeQL only analyzes the current working directory (sourceRoot)

If you’re checking out multiple repos, but all CodeQL tasks are running in the context of just one workspace (i.e., root folder), then CodeQL may not pick up source code in the other checked-out directories.

Incorrect or Static sourceRoot Setting

If your sourceRoot is hardcoded like this:

inputs: sourceRoot: '.'

Try the below Workaround to fix it

Update your sourceRoot dynamically for each scanned repo using the correct working directory, for example:

- task: AdvancedSecurity-Codeql-Init@1 inputs: languages: 'python' sourceRoot: '$(Build.SourcesDirectory)/${{ parameters.repo_name }}'

This ensures CodeQL scans the actual folder where the target repo is checked out, rather than the pipeline repo or root workspace.

Important Note on Multi-Repo Checkout Path

When using multi-repo checkout in Azure DevOps, each repo is checked out under its own subdirectory:

$(Build.SourcesDirectory)/<repo-alias>

So if you checked out:

- checkout: git://my-org/my-repo path: my-repo

then CodeQL must be pointed to:

sourceRoot: '$(Build.SourcesDirectory)/my-repo'

Tip: Enable Debug Logging

To verify what CodeQL is scanning, enable debug output in your pipeline:

variables: System.Debug: true

This can help you trace the folder paths and validate whether the expected files are being analyzed.

Please keep this in mind:

If CodeQL isn’t scanning all your listed repositories, make sure:

You set sourceRoot to each repo’s actual checkout path

You're using checkout: git://... or repository: with unique paths

Each job or template correctly maps the working directory to the target repo

Hope this helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Durga Reshma Malthi 11,595 Reputation points Microsoft External Staff Moderator

2025-06-13T07:49:37.63+00:00

Hi Ester, Sammy

Just checking in to see if you had a chance to review the response provided to your question.

Feel free to reach out if you have any further queries.

If you found the information helpful, please click "Upvote" on the post to let us know and consider accepting the answer as the token of appreciation. Thank You.
Durga Reshma Malthi 11,595 Reputation points Microsoft External Staff Moderator

2025-06-16T07:37:17.03+00:00

Hi Ester, Sammy

Just checking in to see if you had a chance to review the response provided to your question.

Feel free to reach out if you have any further queries.

If you found the information helpful, please click "Upvote" on the post to let us know and consider accepting the answer as the token of appreciation. Thank You.

Share via

Azure DevOps Dynamic Repository CheckOUT!

0 additional answers

Your answer