Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update samples and cleanup unused #284

Merged
merged 1 commit into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ The `kubectl` program is only required when bootstrapping a new Galaxy instance,

### Credentials

You will need an [API key](https://training.galaxyproject.org/training-material/faqs/galaxy/preferences_admin_api_key.html) for every Galaxy instance you would like to intereact with. You will also need the *kubeconfig* file for each Kubernetes cluster. The `abm` script loads the Galaxy server URLs, API keys, and the location of the *kubeconfig* files from a Yaml configuration file that it expects to find in `$HOME/.abm/profile.yml` or `.abm-profile.yml` in the current directory. You can use the `profile-sample.yml` file as a starting point and it includes the URLs for all Galaxy instances we have used to date (December 22, 2021 as of this writing).
You will need an [API key](https://training.galaxyproject.org/training-material/faqs/galaxy/preferences_admin_api_key.html) for every Galaxy instance you would like to intereact with. You will also need the *kubeconfig* file for each Kubernetes cluster. The `abm` script loads the Galaxy server URLs, API keys, and the location of the *kubeconfig* files from a Yaml configuration file that it expects to find in `$HOME/.abm/profile.yml` or `.abm-profile.yml` in the current directory. You can use the `samples/profile.yml` file as a starting point and it includes the URLs for all Galaxy instances we have used to date (December 22, 2021 as of this writing).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current profile.yml does not seem to contain a list of all Galaxy instances as mentioned in the readme.


:bulb: It is now possible (>=2.0.0) to create Galaxy users and their API keys directly with `abm`.

Expand Down
1 change: 0 additions & 1 deletion abm/lib/experiment.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ def run(context: Context, args: list):
print(f"Starting with run number {argv.run_number}")

profiles = load_profiles()
# latch = CountdownLatch(len(config['cloud']))
threads = []
start = perf_counter()
for cloud in config['cloud']:
Expand Down
2 changes: 1 addition & 1 deletion abm/lib/history.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ def error_message(msg='Invalid command'):
else:
datasets = None
config = f'{os.path.dirname(os.path.abspath(__file__))}/histories.yml'
# First load the histories.yml file that is pacakged with abm
# First load the histories.yml file that is packaged with abm
if os.path.exists(config):
with open(config, 'r') as f:
datasets = yaml.safe_load(f)
Expand Down
20 changes: 0 additions & 20 deletions abm/lib/threads/Latch.py

This file was deleted.

11 changes: 0 additions & 11 deletions bootstrap-config/test.yaml

This file was deleted.

106 changes: 0 additions & 106 deletions rules/default.yml

This file was deleted.

File renamed without changes.
File renamed without changes.
14 changes: 14 additions & 0 deletions samples/benchmarks/rna-named.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
- workflow_id: d6d3c2119c4849e4
output_history_base_name: RNA-seq
reference_data:
- name: Reference Transcript (FASTA)
dataset_id: 50a269b7a99356aa
runs:
- history_name: 1
inputs:
- name: FASTQ RNA Dataset
dataset_id: 28fa757e56346a34
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should dataset_ids correspond with datasets.yml in the samples folder?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. The dataset_id in the benchmark file is the ID (or name) of the dataset in the Galaxy history. So something like ucsc.hg38.knownGene.gtf. That sample should likely be updated or removed. The datasets.yml (and histories.yml andworkflows.yaml) files are used to provide aliases to URLs that are used when importing datasets/histories/workflows into Galaxy.

abm main dataset import rna-1g
abm main workflow import rnaseq-pe

- history_name: 2
inputs:
- name: FASTQ RNA Dataset
dataset_id: 1faa2d3b2ed5c436
Empty file.
Empty file.
11 changes: 11 additions & 0 deletions samples/experiment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Benchmarking DNA
runs: 3
workflow_conf:
- benchmarks/example.yml
- benchmarks/dna-named.yml
- benchmarks/rna-named.yml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

benchmark samples need to be updated

cloud:
- iu2
job_configs:
- rules/4x8.yml
- rules/8x16.yml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rule files need sample content

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And probably more explanation as the format of the files will depend on what chart was used to install Galaxy. Since GalaxyKubeman installs Galaxy as a dependent chart there is a top level galaxy: key that is not needed if Galaxy is installed from the galaxy-helm chart.

I will also give some thought to rewriting this section. For the cloud costs we only update TPV rules, but any values from the Chart can be specified and updated here.