Skip to content

Removals #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 7, 2025
Merged

Removals #315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6ef72e8
removals
branfosj Mar 29, 2025
ce7d9ec
more
branfosj Mar 29, 2025
adbf943
use `-r` in the example
branfosj Apr 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 5 additions & 123 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Configuring EasyBuild {: #configuring_easybuild }

This page discusses the recommended style of configuring
EasyBuild, which is supported since EasyBuild v1.3.0.
This page discusses the recommended style of configuring EasyBuild.

See [a demo on configuring EasyBuild][demo_configuring].

Expand Down Expand Up @@ -82,8 +81,7 @@ configuration files at `/etc/easybuild.d/*.cfg` and `$HOME/.config/easybuild/con
The configuration file located in `$XDG_CONFIG_HOME` will be listed *after* the ones obtained via `$XDG_CONFIG_DIRS`,
such that user-defined configuration settings have preference over system defaults.

A detailed overview of the list of default configuration files is available via `eb --show-default-configfiles`
(available since EasyBuild v2.1.0). For example::
A detailed overview of the list of default configuration files is available via `eb --show-default-configfiles`. For example:

``` console
$ XDG_CONFIG_DIRS=/tmp/etc:/tmp/moreetc eb --show-default-configfiles
Expand Down Expand Up @@ -184,7 +182,7 @@ See also [Controlling the robot search path][controlling_robot_search_path].

#### Generating a template configuration file

Since EasyBuild v1.10, a command line option `--confighelp` is
A command line option `--confighelp` is
available that prints out the help text as an annotated configuration
file. This can be used as an empty template configuration file:

Expand Down Expand Up @@ -316,28 +314,7 @@ A couple of selected configuration settings are discussed below,
in particular the mandatory settings.


### Mandatory configuration settings {: #configuration_mandatory_settings }

A handful of configuration settings are **mandatory**, and should be
provided using one of the supported configuration types.

The following configuration settings are currently mandatory
(more details in the sections below):

* [Source path (--sourcepath)][sourcepath]
* [Build path (--buildpath)][buildpath]
* [Software and modules install path (--installpath, --installpath-software, --installpath-modules)][installpath]
* [Easyconfigs repository (--repository, --repositorypath)][easyconfigs_repo]
* [Logfile format (--logfile-format)][logfile_format]


If any of these configuration settings is not provided in one way or another, EasyBuild will complain and exit.

In practice, all of these have reasonable defaults (see `eb --help` for the default settings).

!!! note
The mandatory path-related options can be tweaked collectively via `--prefix`, see [Overall prefix path (`--prefix`)][prefix] for more
information.
### Important configuration settings {: #important_configuration_settings }

#### Source path (`--sourcepath`) {: #sourcepath }

Expand Down Expand Up @@ -379,7 +356,7 @@ when the installation is completed (by default).
Using `/dev/shm` as build path can significantly speed up builds,
if it is available and provides a sufficient amount of space. Setting up
the variable `EASYBUILD_BUILDPATH` in your shell startup files makes this default.
However be aware that, fi., two parallel GCC builds may fill up `/dev/shm` !
However be aware that multiple parallel builds may fill up `/dev/shm` !


#### Software and modules install path {: #installpath }
Expand Down Expand Up @@ -463,101 +440,6 @@ so you do not need to adjust `$MODULEPATH` every time you start a new session.
runtime according to the modules install path it is configured with.


#### Easyconfigs repository (`--repository`, `--repositorypath`) {: #easyconfigs_repo }

*default*: `FileRepository` at `$HOME/.local/easybuild/ebfiles_repo`
(determined via [Overall prefix path (`--prefix`)][prefix])

EasyBuild has support for archiving (tested) `.eb` easyconfig files.
After successfully installing a software package using EasyBuild, the
corresponding `.eb` file is uploaded to a repository defined by the
`repository` and `repositorypath` configuration settings.

Currently, EasyBuild supports the following repository types (see also
`eb --avail-repositories`):

* `FileRepository('path', 'subdir')`: a plain flat file repository;
`path` is the path where files will be stored, `subdir` is an
*optional* subdirectory of that path where the files should be stored
* `GitRepository('path', 'subdir/in/repo'`: a *non-empty* **bare**
git repository (created with `git init --bare` or `git clone --bare`);
`path` is the path to the git repository (can also be a URL);
`subdir/in/repo` is optional, and specifies a subdirectory of the
repository where files should be stored in
* `SvnRepository('path', 'subdir/in/repo')`: an SVN repository;
`path` contains the subversion repository location (directory or
URL), the optional second value specifies a subdirectory in the repository

You need to set the `repository` setting inside a configuration file like this:

``` python
[config]
repository = FileRepository
repositorypath = <path>
```

Or, optionally an extra argument representing a subdirectory can be specified, e.g.:

``` shell
export EASYBUILD_REPOSITORY=GitRepository
export EASYBUILD_REPOSITORYPATH=<path>,<subdir>
```

You do not have to worry about importing these classes,
EasyBuild will make them available to the configuration file.

Using `git` requires the `GitPython` Python modules, using `svn`
requires the `pysvn` Python module (see [Dependencies][dependencies]).

If access to the easyconfigs repository fails for some reason
(e.g., no network or a missing required Python module), EasyBuild will
issue a warning. The software package will still be installed, but the
(successful) easyconfig will not be automatically added to the archive
(i.e., it is not considered a fatal error).


#### Logfile format (`--logfile-format`) {: #logfile_format }

*default*:
`easybuild, easybuild-%(name)s-%(version)s-%(date)s.%(time)s.log`

The `logfile format` configuration setting contains a tuple
specifying a log directory name and a template log file name.
In both of these values, using the following string templates
is supported:

* `%(name)s`: the name of the software package to install
* `%(version)s`: the version of the software package to install
* `%(date)s`: the date on which the installation was performed (in `YYYYMMDD` format, e.g. `20120324`)
* `%(time)s`: the time at which the installation was started (in `HHMMSS` format, e.g. `214359`)

!!! note
Because templating is supported in configuration files themselves (see
[Templates and constants supported in configuration files][configuration_file_templates_constants]), the
'`%`' character in these template values must be escaped when used in a configuration file (and only then),
e.g., '`%%(name)s`'. Without escaping, an error like `InterpolationMissingOptionError: Bad value
substitution` will be thrown by `ConfigParser`.

For example, configuring EasyBuild to generate a log file mentioning only the software name in a directory named
`easybuild` can be done via the `--logfile-format` command line option:

``` shell
eb --logfile-format="easybuild,easybuild-%(name)s.log" ...
```

or the `$EASYBUILD_LOGFILE_FORMAT` environment variable:

``` shell
export EASYBUILD_LOGFILE_FORMAT="easybuild,easybuild-%(name)s.log"
```

or by including the following in an EasyBuild configuration file (note the use of '`%%`' to escape the `name`
template value here):

``` ini
logfile-format = easybuild,easybuild-%%(name)s.log
```


### Optional configuration settings {: #configuration_optional_settings }

Expand Down
13 changes: 4 additions & 9 deletions docs/terminology.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ which is split up into different `steps`:
* unpacking sources
* configuration
* build
* testing
* installation
* module generation
* etc.
Expand Down Expand Up @@ -64,8 +65,8 @@ enabling code reuse across build procedure implementations.
For each software package being built, the EasyBuild framework will determine which easyblock
should be used, based on the name of the software package or the value of the `easyblock`
specification parameter (see [Easyblock specification][writing_easyconfigs_easyblock_spec]).
Since EasyBuild v2.0, an easyblock *must* be specified in case no matching easyblock is found based on the
software name (cfr. [Automagic fallback to ConfigureMake][depr_ConfigureMake_fallback_eb1]).
An easyblock *must* be specified in case no matching easyblock is found based on the
software name.


## Toolchains {: #toolchains }
Expand All @@ -90,7 +91,7 @@ Recent releases of EasyBuild include out-of-the-box toolchain support for:

* various compilers, including GCC, Intel, Clang, CUDA
* common MPI libraries, such as Intel MPI, MPICH, MVAPICH2, OpenMPI
* various numerical libraries, including ATLAS, Intel MKL, OpenBLAS, ScalaPACK, FFTW
* various numerical libraries, including Intel MKL, OpenBLAS, ScalaPACK, FFTW

Please see the [Common toolchains][common_toolchains] page for details about the two most common toolchains,
one for "free and open source software" (`foss`) based on GCC and one based on the Intel compilers
Expand All @@ -107,12 +108,6 @@ When the `system` toolchain is used, a corresponding `system` module file is not
environment is being defined.



### `dummy` toolchain *(DEPRECATED)* {: #dummy_toolchain }

The `dummy` toolchain has been deprecated in EasyBuild v4.0, and replaced by the [`system` toolchain][system_toolchain].


### Common toolchains

For more information on the concept of *common toolchains*, see [Common toolchains][common_toolchains].
Expand Down
14 changes: 4 additions & 10 deletions docs/using-easybuild.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,6 @@ These locations are only considered for easyconfig files that are specified only
*not* for easyconfig files that are specified via an absolute path. The dependencies are resolved using the
robot search path (see [robot search path][robot_search_path]).

!!! note

For easyconfig files specified on the `eb` command line, the *full* robot search path is only considered since
EasyBuild v2.0.0. Earlier versions only considered the local working directory and the easyconfig files that are
part of the active EasyBuild installation for *explicitly specified* easyconfig files.


### By providing a single easyconfig file {: #specifying_easyconfigs_single }

Expand Down Expand Up @@ -437,7 +431,7 @@ CFGS1=/home/example/easybuild-easyconfigs/easybuild/easyconfigs/m/Mesquite
* $CFGS1/Mesquite-2.3.0-GCCcore-8.3.0.eb
```

For more specific searching, a regular expression pattern can be supplied (since EasyBuild v2.1.1).
For more specific searching, a regular expression pattern can be supplied.

For example, to search which easyconfig files are available for GCC v11.x, without listing easyconfig files that use
GCC v11.x as a toolchain:
Expand Down Expand Up @@ -676,7 +670,7 @@ Matching setup:
You can do a "dry-run" overview by supplying `-D/--dry-run`:

```console
$ eb binutils-2.39.eb -D
$ eb binutils-2.39.eb -Dr
== Temporary log file in case of crash /tmp/eb-3lh496ml/easybuild-hxmie76g.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/Volumes/work/easybuild-easyconfigs/easybuild/easyconfigs
Expand All @@ -697,15 +691,15 @@ Note how the different status symbols denote distinct handling states by EasyBui

!!! note

Since EasyBuild v2.4.0, a detailed overview of the build and install procedure that EasyBuild
A detailed overview of the build and install procedure that EasyBuild
will be execute can be obtained using `--extended-dry-run` or `-x`, see [Extended dry run][extended_dry_run].


## Getting an overview of missing installations {: #eb_missing }

*(`--missing-modules` / `-M`)*

Since EasyBuild v3.9.1, you can obtain a list of missing installations (i.e. easyconfigs for which
You can obtain a list of missing installations (i.e. easyconfigs for which
no corresponding environment module file is available yet) using `eb --missing-modules` (or `-eb -M` for short):

```console
Expand Down