Pre-packaged SnpEff annotation pipeline
This documentation has been retired and might not be updated. The products, services, or technologies mentioned in this content are no longer supported.
The Databricks Genomics runtime has been deprecated. For open source equivalents, see repos for genomics-pipelines and Glow. Bioinformatics libraries that were part of the runtime have been released as a Docker container, which can be pulled from the ProjectGlow Dockerhub page.
For more information about the Databricks Runtime deprecation policy and schedule, see Supported Azure Databricks runtime releases and support schedule.
Run SnpEff (v4.3) as an Azure Databricks job.
The pipeline has been tested on 85.2 million variant sites from the 1000 Genomes project using the following cluster configurations:
- Driver: Standard_DS13_v2
- Workers: Standard_D32s_v3 * 7 (224 cores)
- Runtime: 2.5 hours
You must configure the reference genome using environment variables. To use GRCh37, set the environment variable:
To use GRCh38 instead, set the environment variable:
The pipeline accepts a number of parameters that control its behavior. The most important and commonly changed parameters are documented here; the rest can be found in the SnpEff Annotation pipeline notebook. After importing the notebook and setting it as a job task, you can set these parameters for all runs or per-run.
|inputVariants||n/a||Path of input variants (VCF or Delta Lake).|
|output||n/a||The path where pipeline output should be written.|
|exportVCF||false||If true, the pipeline writes results in VCF as well as Delta Lake.|
|exportVCFAsSingleFile||false||If true, exports VCF as single file|
The annotated variants are written out to Delta tables inside the provided output directory. If you configured the pipeline to export to VCF, they’ll appear under the output directory as well.
output |---annotations |---Delta files |---annotations.vcf
Submit and view feedback for