Minor updates to navigation and Side Quests index, orientation (#563)

vdauwera · web-flow · commit ce8595678680 · 2025-03-12T01:00:19.000-04:00
* Updated index and orientation pages for Side Quests 

* Updated navigation menu to follow home page listing

* Fix capitalization in genomics docs + renumber the placeholder for the config section to avoid confusion

* Rename Launch to Start on Home page to disambiguate starting a training vs launching the environment
diff --git a/docs/index.md b/docs/index.md
@@ -32,7 +32,7 @@ When you're ready to get down to work, click on the 'Open in GitHub Codespaces'
 
     Instructions for setting up your environment to work through training materials (all courses). Provides an orientation to GitHub Codespaces as well as alternate installation instructions for working on your own local machine.
 
-    [Launch the Environment Setup training :material-arrow-right:](envsetup/index.md){ .md-button .md-button--primary }
+    [Start the Environment Setup training :material-arrow-right:](envsetup/index.md){ .md-button .md-button--primary }
 
 ## Nextflow for Newcomers
 
@@ -48,7 +48,7 @@ These are foundational, domain-agnostic courses intended for those who are compl
 
     This is a course for newcomers who wish to learn how to develop their own pipelines. The course covers the core components of the Nextflow language in enough detail to enable developing simple but fully functional pipelines. It also covers key elements of pipeline design, development and configuration practices.
 
-    [Launch the Hello Nextflow training :material-arrow-right:](hello_nextflow/index.md){ .md-button .md-button--primary }
+    [Start the Hello Nextflow training :material-arrow-right:](hello_nextflow/index.md){ .md-button .md-button--primary }
 
 !!! info ""
 
@@ -63,7 +63,7 @@ These are foundational, domain-agnostic courses intended for those who are compl
 
     This is a course for newcomers who wish to learn how to run existing pipelines. The course covers the bare essentials of the Nextflow language in order to enable interpretation of existing pipelines, as well as the mechanics for configuring and running Nextflow pipelines from a command-line environment. It also covers important components of the Nextflow ecosystem, including the nf-core project, which offers a large number of community-curated pipelines, and the Seqera platform for managing pipeline execution at scale (operated by the creators of Nextflow).
 
-    [Launch the Nextflow Run training :material-arrow-right:](nextflow_run/index.md){ .md-button .md-button--primary }
+    [Start the Nextflow Run training :material-arrow-right:](nextflow_run/index.md){ .md-button .md-button--primary }
 -->
 
 ## Nextflow for Science
@@ -78,7 +78,7 @@ These are courses that demonstrate how to apply the concepts and components pres
 
     This is a course for researchers who wish to learn how to develop their own genomics pipelines. The course uses a variant calling use case to demonstrate how to develop a simple but functional genomics pipeline.
 
-    [Launch the Nextflow for Genomics training :material-arrow-right:](nf4_science/genomics/){ .md-button .md-button--primary }
+    [Start the Nextflow for Genomics training :material-arrow-right:](nf4_science/genomics/){ .md-button .md-button--primary }
 
 !!! exercise "Nextflow for RNAseq"
 
@@ -88,7 +88,7 @@ These are courses that demonstrate how to apply the concepts and components pres
 
     This is a course for researchers who wish to learn how to develop their own RNAseq pipelines. The course uses a bulk RNAseq processing use case to demonstrate how to develop a simple but functional RNAseq pipeline.
 
-    [Launch the Nextflow for RNAseq training :material-arrow-right:](nf4_science/rnaseq/){ .md-button .md-button--primary }
+    [Start the Nextflow for RNAseq training :material-arrow-right:](nf4_science/rnaseq/){ .md-button .md-button--primary }
 
 Let us know what other domains and use cases you'd like to see covered here by posting in the [Training section](https://community.seqera.io/c/training/) of the community forum.
 
@@ -104,7 +104,7 @@ These are courses that demonstrate how to use Nextflow features in more detail o
 
     This is a course for Nextflow developers who wish to widen their range and/or deepen their skills. Although the modules are presented linearly, learners are welcome to pick and choose topics in any order. Any dependencies on components/skills that go beyond the scope of the 'Hello Nextflow' course are indicated in the corresponding module overview.
 
-    [Launch the Side Quests training :material-arrow-right:](side_quests/){ .md-button .md-button--primary }
+    [Start the Side Quests training :material-arrow-right:](side_quests/){ .md-button .md-button--primary }
 
 !!! exercise "Fundamentals Training"
 
@@ -114,7 +114,7 @@ These are courses that demonstrate how to use Nextflow features in more detail o
 
     The fundamentals training material covers all things Nextflow. Intended as a reference material for anyone looking to build complex workflows with Nextflow.
 
-    [Launch the Fundamentals Training :material-arrow-right:](basic_training/index.md){ .md-button .md-button--primary }
+    [Start the Fundamentals Training :material-arrow-right:](basic_training/index.md){ .md-button .md-button--primary }
 
 !!! exercise "Advanced Training"
 
@@ -124,7 +124,7 @@ These are courses that demonstrate how to use Nextflow features in more detail o
 
     Advanced material exploring the more advanced features of the Nextflow language and runtime, and how to use them to write efficient and scalable data-intensive workflows.
 
-    [Launch the Advanced Training :material-arrow-right:](advanced/index.md){ .md-button .md-button--primary }
+    [Start the Advanced Training :material-arrow-right:](advanced/index.md){ .md-button .md-button--primary }
 
 ## Other/Experimental
 
diff --git a/docs/nf4_science/genomics/03_configuration.md b/docs/nf4_science/genomics/03_configuration.md
@@ -1,4 +1,4 @@
-# Part 3: Resource profiling and optimization
+# Part 5: Resource profiling and optimization
 
 THIS IS A PLACEHOLDER
 
diff --git a/docs/nf4_science/genomics/03_modules.md b/docs/nf4_science/genomics/03_modules.md
@@ -1,4 +1,4 @@
-# Part 3: moving code into modules
+# Part 3: Moving code into modules
 
 In the first part of this course, you built a variant calling pipeline that was completely linear and processed each sample's data independently of the others.
 
diff --git a/docs/nf4_science/genomics/04_testing.md b/docs/nf4_science/genomics/04_testing.md
@@ -1,4 +1,4 @@
-# Part 4: adding tests
+# Part 4: Adding tests
 
 In the first part of this course, you built a variant calling pipeline that was completely linear and processed each sample's data independently of the others.
 
diff --git a/docs/nf4_science/genomics/05_configuration.md b/docs/nf4_science/genomics/05_configuration.md
@@ -0,0 +1,89 @@
+# Part 3: Resource profiling and optimization
+
+THIS IS A PLACEHOLDER
+
+!!!note
+
+    This training module is under redevelopment.
+
+---
+
+TODO
+
+### 4.3. Run the workflow to generate a resource utilization report
+
+To have Nextflow generate the report automatically, simply add `-with-report <filename>.html` to your command line.
+
+```bash
+nextflow run main.nf -profile my_laptop -with-report report-config-1.html
+```
+
+The report is an html file, which you can download and open in your browser. You can also right click it in the file explorer on the left and click on `Show preview` in order to view it in VS Code.
+
+Take a few minutes to look through the report and see if you can identify some opportunities for adjusting resources.
+Make sure to click on the tabs that show the utilization results as a percentage of what was allocated.
+There is some [documentation](https://www.nextflow.io/docs/latest/reports.html) describing all the available features.
+
+<!-- TODO: insert images -->
+
+One observation is that the `GATK_JOINTGENOTYPING` seems to be very hungry for CPU, which makes sense since it performs a lot of complex calculations.
+So we could try boosting that and see if it cuts down on runtime.
+
+However, we seem to have overshot the mark with the memory allocations; all processes are only using a fraction of what we're giving them.
+We should dial that back down and save some resources.
+
+### 4.4. Adjust resource allocations for a specific process
+
+We can specify resource allocations for a given process using the `withName` process selector.
+The syntax looks like this when it's by itself in a process block:
+
+```groovy title="Syntax"
+process {
+    withName: 'GATK_JOINTGENOTYPING' {
+        cpus = 4
+    }
+}
+```
+
+Let's add that to the existing process block in the `nextflow.config` file.
+
+```groovy title="nextflow.config" linenums="11"
+process {
+    // defaults for all processes
+    cpus = 2
+    memory = 2.GB
+    // allocations for a specific process
+    withName: 'GATK_JOINTGENOTYPING' {
+        cpus = 4
+    }
+}
+```
+
+With that specified, the default settings will apply to all processes **except** the `GATK_JOINTGENOTYPING` process, which is a special snowflake that gets a lot more CPU.
+Hopefully that should have an effect.
+
+### 4.5. Run again with the modified configuration
+
+Let's run the workflow again with the modified configuration and with the reporting flag turned on, but notice we're giving the report a different name so we can differentiate them.
+
+```bash
+nextflow run main.nf -profile my_laptop -with-report report-config-2.html
+```
+
+Once again, you probably won't notice a substantial difference in runtime, because this is such a small workload and the tools spend more time in ancillary tasks than in performing the 'real' work.
+
+However, the second report shows that our resource utilization is more balanced now.
+
+<!-- **TODO: screenshots?** -->
+
+As you can see, this approach is useful when your processes have different resource requirements. It empowers you to right-size the resource allocations you set up for each process based on actual data, not guesswork.
+
+!!!note
+
+    This is just a tiny taster of what you can do to optimize your use of resources.
+    Nextflow itself has some really neat [dynamic retry logic](https://training.nextflow.io/basic_training/debugging/#dynamic-resources-allocation) built in to retry jobs that fail due to resource limitations.
+    Additionally, the Seqera Platform offers AI-driven tooling for optimizing your resource allocations automatically as well.
+
+    We'll cover both of those approaches in an upcoming part of this training course.
+
+That being said, there may be some constraints on what you can (or must) allocate depending on what computing executor and compute infrastructure you're using. For example, your cluster may require you to stay within certain limits that don't apply when you're running elsewhere.
diff --git a/docs/nf4_science/index.md b/docs/nf4_science/index.md
@@ -16,7 +16,7 @@ These are courses that demonstrate how to apply the concepts and components pres
 
     This is a course for researchers who wish to learn how to develop their own genomics pipelines. The course uses a variant calling use case to demonstrate how to develop a simple but functional genomics pipeline.
 
-    [Launch the Nextflow for Genomics training :material-arrow-right:](genomics/){ .md-button .md-button--primary }
+    [Start the Nextflow for Genomics training :material-arrow-right:](genomics/){ .md-button .md-button--primary }
 
 !!! exercise "Nextflow for RNAseq"
 
@@ -26,6 +26,6 @@ These are courses that demonstrate how to apply the concepts and components pres
 
     This is a course for researchers who wish to learn how to develop their own RNAseq pipelines. The course uses a bulk RNAseq processing use case to demonstrate how to develop a simple but functional RNAseq pipeline.
 
-    [Launch the Nextflow for RNAseq training :material-arrow-right:](rnaseq/){ .md-button .md-button--primary }
+    [Start the Nextflow for RNAseq training :material-arrow-right:](rnaseq/){ .md-button .md-button--primary }
 
 Let us know what other domains and use cases you'd like to see covered here by posting in the [Training section](https://community.seqera.io/c/training/) of the community forum.
diff --git a/docs/side_quests/index.md b/docs/side_quests/index.md
@@ -21,3 +21,15 @@ Nevertheless, they all assume some minimal familiarity with the following:
 - Foundational Nextflow concepts and tooling covered in the [Hello Nextflow](../../hello_nextflow/) beginner training course.
 
 For technical requirements and environment setup, see the [Environment Setup](../../envsetup/) mini-course.
+
+**If this is the first time you delve into the Side Quests, make sure to check out the [Orientation](./orientation.md) page first!**
+
+Otherwise, select a side quest from the menu below.
+
+## Menu of Side Quests
+
+- [Introduction to nf-core](./nf-core.md)
+- [Testing with nf-test](./nf-test.md)
+- [Workflows of workflows](./workflows_of_workflows.md)
+
+Let us know what other domains and use cases you'd like to see covered here by posting in the [Training section](https://community.seqera.io/c/training/) of the community forum.
diff --git a/docs/side_quests/nf-core.md b/docs/side_quests/nf-core.md
@@ -1,4 +1,4 @@
-# nf-core
+# Introduction to nf-core
 
 nf-core is a community effort to develop and maintain a curated set of analysis pipelines built using Nextflow. It was created by several core facilities wanting to consolidate their analysis development and is governed by community members from academia and industry. It is an open community that anyone can join and contribute to.
 
@@ -19,6 +19,8 @@ In this tutorial you will explore using and writing nf-core pipelines:
 - Section 2: Develop an nf-core-like pipeline
   In second section, you will use a simplified version of the nf-core template to write a nf-core-style pipeline. The pipeline consists of two modules to process FastQ data: `fastqe` and `seqtk`. It uses an input from a sample sheet, validates it, and produces a multiqc report.
 
+---
+
 ## 0. Warmup
 
 Let's move into the project directory.
diff --git a/docs/side_quests/orientation.md b/docs/side_quests/orientation.md
@@ -7,7 +7,7 @@ If you have not yet done so, please follow [this link](../../envsetup/) before g
 
 ## Materials provided
 
-Throughout this training course, we'll be working in the `tapas/` directory.
+Throughout this training course, we'll be working in the `side-quests/` directory.
 This directory contains all the code files, test data and accessory files you will need.
 
 Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the GitHub Codespaces workspace.
@@ -20,27 +20,30 @@ Here we generate a table of contents to the second level down:
 tree . -L 2
 ```
 
-If you run this inside `tapas`, you should see the following output: [TODO]
+If you run this inside `side-quests`, you should see the following output:
 
 ```console title="Directory contents"
 .
+├── nf-core
+├── nf-test
+├── solutions
+└── workflows_of_workflows
 ```
 
-!!!note
-
-    Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course.
-    This is just meant to give you an overview.
-
 **Here's a summary of what you should know to get started:**
 
-[TODO]
+- **Each directory corresponds to an individual side quest.**
+  Their contents are detailed on the corresponding side quest's page.
+
+- **The `solutions` directory** contains the completed workflow and/or module scripts that result from running through various steps of each side quest.
+  They are intended to be used as a reference to check your work and troubleshoot any issues.
 
 !!!tip
 
     If for whatever reason you move out of this directory, you can always run this command to return to it:
 
     ```bash
-    cd /workspaces/training/tapas
+    cd /workspaces/training/side-quests
     ```
 
 Now, to begin the course, click on the arrow in the bottom right corner of this page.
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -20,18 +20,13 @@ nav:
       - hello_nextflow/06_hello_config.md
       - hello_nextflow/survey.md
       - hello_nextflow/next_steps.md
-  - Side Quests:
-      - side_quests/nf-test.md
-      - side_quests/nf-core.md
-      - side_quests/workflows_of_workflows.md
   - Nextflow for Genomics:
       - nf4_science/genomics/index.md
       - nf4_science/genomics/00_orientation.md
       - nf4_science/genomics/01_per_sample_variant_calling.md
       - nf4_science/genomics/02_joint_calling.md
       - nf4_science/genomics/03_modules.md
       - nf4_science/genomics/04_testing.md
-
   - Nextflow for RNAseq:
       - nf4_science/rnaseq/index.md
       - nf4_science/rnaseq/00_orientation.md
@@ -40,6 +35,12 @@ nav:
       - nf4_science/rnaseq/03_multi-sample.md
       - nf4_science/rnaseq/survey.md
       - nf4_science/rnaseq/next_steps.md
+  - Side Quests:
+      - side_quests/index.md
+      - side_quests/orientation.md
+      - side_quests/nf-core.md
+      - side_quests/nf-test.md
+      - side_quests/workflows_of_workflows.md
   - Fundamentals Training:
       - basic_training/index.md
       - basic_training/orientation.md
@@ -173,33 +174,24 @@ plugins:
         - hello_nextflow/00_orientation.md
         - nf4_science/genomics/00_orientation.md
         - nf4_science/rnaseq/00_orientation.md
+        - side_quests/orientation.md
         - basic_training/orientation.md
         - advanced/orientation.md
 
       exclude:
         - index.md
         - help.md
-        - basic_training/index.md
-        - advanced/index.md
-        - nf_customize/index.md
-        - nf_develop/index.md
-        - troubleshoot/index.md
-        - hands_on/index.md
-        - hands_on/solutions/*md
         - envsetup/*.md
         - hello_nextflow/*.md
         - nf4_science/genomics/*.md
         - nf4_science/rnaseq/*.md
+        - side_quests/*.md
         - basic_training/index.md
         - basic_training/orientation.md
         - advanced/index.md
         - advanced/orientation.md
-        - side_quests/nf-test.md
-        - nf4_science/genomics/03_modules.md
-        - side_quests/workflows_of_workflows.md
-        - side_quests/nf-core.md
-        - nf4_science/genomics/03_modules.md
-        - nf4_science/genomics/04_testing.md
+        - nf_customize/index.md
+        - nf_develop/index.md
   - i18n:
       docs_structure: suffix
       fallback_to_default: true

Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-# Part 3: Resource profiling and optimization`
	`1`	`+# Part 5: Resource profiling and optimization`
`2`	`2`
`3`	`3`	`THIS IS A PLACEHOLDER`
`4`	`4`
Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-# Part 3: moving code into modules`
	`1`	`+# Part 3: Moving code into modules`
`2`	`2`
`3`	`3`	`In the first part of this course, you built a variant calling pipeline that was completely linear and processed each sample's data independently of the others.`
`4`	`4`
Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-# Part 4: adding tests`
	`1`	`+# Part 4: Adding tests`
`2`	`2`
`3`	`3`	`In the first part of this course, you built a variant calling pipeline that was completely linear and processed each sample's data independently of the others.`
`4`	`4`