Imputation_develop

1.1.16

2025-02-24 (Date of Last Commit)

• Updated runtime parameters in some ImputationTasks, and added an explicit definition of a vcf_index.

ImputationBeagle_development_v0.0.4: TSPS-183 Beagle imputation hg38 wdl and associated support wdls (#1333)

* wip add beagle imputation stuff

* add 2 wdls to dockstore.yml

* fix docker gar url

* use the right path for jars

* wip on imputation wdl

* oops use correct jar

* missing equals

* fix java call again

* fix java call

* oops match file names

* update beagle jar to 01Mar24.d36

* debug GatherVcfs

* debug GatherVcfs 2

* try to resolve missing file issue

* don't impute over padding

* make the index again

* supply vcf_index input to SelectVariantsByIds

* update Imputation wdl too

* newlines

* update for hg38

* Revert "update for hg38"

This reverts commit 3757137a02a7d92686a5664596261a0fb91e2624.

* update for hg38

* liftover wdl

* remove GCP-specific vm commands

* use gatk

* fix suffix and basename

* fix more filenames

* remove missing contig stuff for now

* fix ref panel path

* another chr fix

* warn on missign contig

* do fail if missing contig

* more mem

* troubleshooting wld

* fixed plink path

* add select_first test

* cleanup

* add if block to test

* create and use ref panel interval list

* move interval list creation to ref panel wdl

* give default values for optional inputs, weird

* change CountVariants calls

* test

* add output to test

* next test

* more test

* another test

* update real task

* TSPS-226 presplit and prechunk beagle inputs (#1272)

*pre splitting and prechunking beagle imputation inputs to lower log numbers and storage account egress

---------

Co-authored-by: Jose Soto <[email protected]>

* TSPS-221 remove index input and add seed to make beagle tool deterministic (#1285)

* remove multi sample vcf index workflow input and add it to the PreSplitVcf task.
add seed number so that beagle is always deterministic. add comment to cpu input for PhaseAndImputeBeagle task

* change output_callset_name to output_base_name and remove optional outputs

* change n_failed_chunks ticket to an int

---------

Co-authored-by: Jose Soto <[email protected]>

* rename workflow

* TSPS-241 Clean up beagle wdl (#1288)

* clean up wdl with stuff from TSPS-241

* try to make fail fast work with double nested scatters

---------

Co-authored-by: Jose Soto <[email protected]>

* add specific gatk_docker

* TSPS-142 updates to help creating simulated reference panel and running imputation against it (#1296)

* add optional error count override for testing

* rename reference base prefix variable and make it more user friendly

---------

Co-authored-by: Jose Soto <[email protected]>

* add maxRetries 2 to all imputation beagle tasks

* add prechunk wdl to dockstore

* use acr for default ubuntu image

* add preemptible 3

* use acr gatk docker as default

* don't use preemptibles on GatherVcfs

* basename fix for imputation beagle ref panel generation (#1332)

* try auto specifying chr at end of basename

* both tasks

* add liftovervcfs to dockstore

* allow specifying max mem

* TSPS-269 Speed up CountVariantsInChunksBeagle by using bedtools (#1335)

* try creating bed files

* try again

* try again again

* a different thing

* use bedtools and bed ref panel files

* oops update the correct task

* fix

* use the right freaking file name

* remove comment

* update pipeline version to 0.0.2

* TSPS-293: Fix up streaming imputation beagle (#1347)

update ImputationBeagle

* add array imputation quota consumed wdl (#1425)

* add array imputation quota consumed wdl

* add changelogs for imputation array related workflows

---------

Co-authored-by: Jose Soto <[email protected]>

* TSPS-239 get wdl running on 400k sample ref panel (#1373)

* changes to help beagle imputation wdl run on a 400k sample reference panel

---------

Co-authored-by: Jose Soto <[email protected]>

* remove create imputation ref panel beagle wdl and changelog

* PR feedback

---------

Co-authored-by: Jose Soto <[email protected]>
Co-authored-by: M. Morgan Aster <[email protected]>

* add set -e -o pipefail to all relevant imputation tasks (#1434)

Co-authored-by: Jose Soto <[email protected]>

* TSPS-341 remove tasks for recovering variants not in the reference panel (#1468)

* remove tasks for recovering variants not in the reference panel and separate out beagle tasks from imputation tasks

* remove prechunk wdl and references to it
remove "Beagle" from task names in BeagleTasks.wdl

---------

Co-authored-by: Jose Soto <[email protected]>

* Updated pipeline_versions.txt with all pipeline version information

* [PR to feature branch] Add testing to imputation beagle (#1503)

* TSPS-239 get wdl running on 400k sample ref panel (#1373)

* changes to help beagle imputation wdl run on a 400k sample reference panel

---------

Co-authored-by: Jose Soto <[email protected]>

* remove create imputation ref panel beagle wdl and changelog

* PR feedback

---------

Co-authored-by: Jose Soto <[email protected]>
Co-authored-by: M. Morgan Aster <[email protected]>

* add new files for testing

* add test wdl to .dockstore.yml

* add test data json files, other updates

* version to 1.0.0, update changelog

* update beagle docker

* update beagle docker again

* fix call phase task

* re-deleting ImputationBeaglePreChunk.wdl

* temporarily try to run test on feature branch pr

* remove vault inputs

* update output basename for plumbing test

* remove feature branch from gha pr branches

* pr comments

* add quotes in VerifyTasks.CompareVcfs

* update dockers, move CreateVcfIndex to BeagleTasks

---------

Co-authored-by: jsotobroad <[email protected]>
Co-authored-by: Jose Soto <[email protected]>

* Updated pipeline_versions.txt with all pipeline version information

* remove newline at end of Utilities.wdl

* remove LiftoverVcfs, add README for imputation_beagle

* oops this commit adds the README for imputation_beagle

* rename test inputs files to reflect contents

* PR comments round 1

* Updated pipeline_versions.txt with all pipeline version information

* update changelog for BroadInternalImputation

* Updated pipeline_versions.txt with all pipeline version information

* add back newline to Utilities.wdl with -w flag on changed file check

* remove change to Minimac4 task

* revert change to tool command in OptionalQCSites

* fix fail task dependency, revert attempt to ignore newline in diff, other pr comments

* update README for ImputationBeagle

* rename test files

* Updated pipeline_versions.txt with all pipeline version information

* another commit for hashes

* Updated pipeline_versions.txt with all pipeline version information

* dummy commit

* pr comments

* Updated pipeline_versions.txt with all pipeline version information

* dummy commit

* dummy commit

---------

Co-authored-by: jsotobroad <[email protected]>
Co-authored-by: Jose Soto <[email protected]>
Co-authored-by: GitHub Action <[email protected]>
Co-authored-by: Nikelle Petrillo <[email protected]>
Co-authored-by: npetrill <[email protected]>

ImputationBeagle_develop

1.0.0

2025-02-26 (Date of Last Commit)

• Initial public release of the ImputationBeagle pipeline.
  • The ImputationBeagle pipeline imputes missing genotypes from a multi-sample VCF using a large genomic reference panel. It is based on the Michigan Imputation Server pipeline but uses the Beagle imputation tool instead of minimac. Overall, the pipeline filters, phases, and performs imputation on a multi-sample VCF. It outputs the imputed VCF.

BroadInternalImputation_develop

1.1.15

2025-02-24 (Date of Last Commit)

• Updated runtime parameters in some ImputationTasks, and added an explicit definition of a vcf_index.

ArrayImputationQuotaConsumed_develop

1.0.0

2025-02-24 (Date of Last Commit)

• Initial release of pipeline to calculate the number of samples, i.e. quota used by an imputation service that uses ImputationBeagle.wdl.

atac_develop

2.7.1

2025-02-25 (Date of Last Commit)

• Added a new warning for peak calling step if the probability_threshold is too low, resutling in a null matrix after doublet filtering
• Updated the probability threshold default to 0.5
• Updated the warp-tools docker image to include an update to the GroupQCs function in sctools; this does not affect the outputs of the pipeline
• Added reference information to the BAM header

SlideSeq_develop

3.4.9

2025-02-25 (Date of Last Commit)

• Updated the warp-tools docker image to include an update to the GroupQCs function in sctools; this does not affect the outputs of the pipeline
• Added reference information to the BAM header for Optimus and ATAC workflows; this does not impact Slideseq

PairedTag_develop

1.10.2

2025-02-25 (Date of Last Commit)

• Updated the SnapATAC2 docker image to the latest SnapATAC2, allowing for future peak calling implementation
• Updated the warp-tools docker image to include an update to the GroupQCs function in sctools; this does not affect the outputs of the pipeline
• Added reference information to the BAM headers

Optimus_develop

7.9.2

2025-02-25 (Date of Last Commit)

• Updated the warp-tools docker image to include an update to the GroupQCs function in sctools; this does not affect the outputs of the pipeline
• Added reference information to the BAM header

Multiome_develop

5.11.0

2025-02-25 (Date of Last Commit)

• Refactored the Peak Calling step of Multiome to use the JoinBarcodes output h5ad as the input for peak calling, ensuring the h5ad files have both GEX and ATAC barcodes
• Updated the warp-tools docker image to include an update to the GroupQCs function in sctools; this does not affect the outputs of the pipeline
• Added reference information to the BAM headers