Skip to content

Commit 4d98e33

Browse files
committedMar 7, 2023
update README
1 parent 6baab37 commit 4d98e33

File tree

1 file changed

+9
-118
lines changed

1 file changed

+9
-118
lines changed
 

‎README.md

+9-118
Original file line numberDiff line numberDiff line change
@@ -8,124 +8,15 @@ An alternative would be the [NixOS](https://nixos.org/).
88

99
## Installation and Configuration
1010

11-
### Prerequisites
12-
13-
The bootstrap process will need a clean environment with C and C++ compilers (the system version of gcc and g++ will do) as well as the `make` command. It also is very sensitive to
14-
the environment, so setup a user with unset `CFFLAGS`, `CFLAGS`, `LDFLAGS`, `PKG_CONFIG_PATH` and the always harmful `LD_LIBRARY_PATH` variables.
15-
16-
EESSI provides a Docker container for this, which can be run via Singularity.
17-
18-
### Bootstrapping Gentoo Prefix
19-
Gentoo Prefix provides a bootstrap script to build the prefix, see [Gentoo Prefix Bootstrap](https://wiki.gentoo.org/wiki/Project:Prefix/Bootstrap).
20-
We forked [this version](https://gitweb.gentoo.org/repo/proj/prefix.git/tree/scripts/bootstrap-prefix.sh?id=e77fd01734f21ec2e9c985c28ba4eb30c1b2bc9d)
21-
and made some modifications. See issue [#8](https://github.com/EESSI/compatibility-layer/issues/8) for more details.
22-
23-
You can run our version of the bootstrap script (see `bootstrap-prefix.sh`) inside the Singularity container by executing:
24-
```
25-
singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
26-
```
27-
28-
If you want to run your own version of the bootstrap script, use:
29-
```
30-
singularity exec docker://ghcr.io/eessi/bootstrap-prefix:centos8 ./bootstrap-prefix.sh
31-
```
32-
Our version of the script allows you to pick a custom snapshot for the Portage tree. This can be done by setting `CUSTOM_SNAPSHOT_URL` to
33-
a URL that points to a directory, and setting `CUSTOM_SNAPSHOT_URL` to the name of a snapshot file (must be a bzip2 archive). For instance:
34-
```
35-
env CUSTOM_SNAPSHOT_URL="http://cvmfs-s0.eessi-hpc.org/snapshots" CUSTOM_SNAPSHOT_VERSION="20201209" singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
36-
```
37-
If you want to limit the supported/installed Python version(s), you can set the environment variable `PYTHON_TARGETS` before starting the bootstrap script. By only including a Python 3 version, you can prevent Python 2 from being installed, e.g.:
38-
```
39-
env PYTHON_TARGETS="python3_8" CUSTOM_SNAPSHOT_URL="http://cvmfs-s0.eessi-hpc.org/snapshots" CUSTOM_SNAPSHOT_VERSION="20201126" singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
40-
```
41-
42-
After starting the bootstrap have a long coffee...
43-
44-
Once the bootstrap is completed, run the script to replace some paths with symlinks into the host OS:
45-
46-
```
47-
scripts/prefix-symlink-host-paths.sh
48-
```
49-
50-
In order to generate the right locales for your installation, see [this item](https://wiki.gentoo.org/wiki/Project:Prefix/FAQ#Add_an_en_US.UTF-8_locale)
51-
of the Gentoo Prefix FAQ.
52-
53-
### Adding the EESSI overlay and packages
54-
Additional packages are added in the EESSI overlay, which is based on ComputeCanada.
55-
You can add them manually or in an automated way by using Ansible, being Ansible the preferred way. Below you can find the two options explained.
56-
57-
#### Ansible playbook (Option 1)
58-
The installation of the EESSI-specific parts can be automatically executed by running the Ansible playbook `install.yml` inside the folder `ansible/playbooks`.
59-
This playbook will install the [EESSI Gentoo overlay](https://github.com/EESSI/gentoo-overlay) and a set of packages, including `Lmod` and `archspec`. See the `README` in the `ansible/playbooks` folder for more details.
60-
61-
#### Manually (Option 2)
62-
First, set `EPREFIX` to the path containing your Gentoo Prefix installation, and start the prefix:
63-
```
64-
export EPREFIX=/path/to/your/prefix
65-
${EPREFIX}/startprefix
66-
```
67-
Ensure that the configuration directory for repositories exists:
68-
```
69-
mkdir -p ${EPREFIX}/etc/portage/repos.conf
70-
```
71-
If you used `${PYTHON_TARGETS}` during the bootstrap, be sure to set it to the same value now, e.g.:
72-
```
73-
export PYTHON_TARGETS="python3_7"
74-
```
75-
76-
Next, configure and sync the overlay:
77-
```
78-
emerge eselect-repository
79-
eselect repository add eessi git https://github.com/EESSI/gentoo-overlay.git
80-
emerge --sync eessi
81-
```
82-
83-
After synchronizing the overlay, add the EESSI package set(s) that you would like to install, e.g. for set `2020.08`:
84-
```
85-
mkdir ${EPREFIX}/etc/portage/sets/
86-
ln -s ${EPREFIX}/var/db/repos/eessi/etc/portage/sets/2020.08 ${EPREFIX}/etc/portage/sets/
87-
```
88-
89-
Finally, install the package set(s) defined at `${EPREFIX}/etc/portage/sets/`, e.g.:
90-
```
91-
emerge @2020.08
92-
```
93-
94-
### Updating the Prefix
95-
#### Packages
96-
Updating packages can be as easy as
97-
```
98-
emerge --sync
99-
emerge
100-
```
101-
If you run into problems, usually a newer ebuild is not suited to build in a prefix environment.
102-
Try to mask latest versions:
103-
104-
Create a mask file if not existing and mask newer versions from thin provisioning tools greater or equal to 0.7.6:
105-
```
106-
echo ">=sys-block/thin-provisioning-tools-0.7.6" >> ${EPREFIX}/etc/portage/package.mask
107-
```
108-
109-
#### Portage
110-
Updating Portage requires the kernel source which corresponds to your running kernel on the host. Emerge will detect it in `/usr/src/linux`.
111-
112-
Check your running kernel version with:
113-
```
114-
cat /proc/version
115-
Linux version 4.20.0-1.el7.elrepo.x86_64 (mockbuild@Build64R7)
116-
```
117-
118-
On a Centos 7 host kernel sources are installed in `/usr/src/kernels`. Link `/usr/src/linux` to the appropiate kernel source after installation. Example for an `elrepo` kernel:
119-
```
120-
rpm -ivh kernel-ml-devel-4.20.0-1.el7.elrepo.x86_64.rpm
121-
cd /usr/src ; ln -s kernels/4.20.0-1.el7.elrepo.x86_64 linux
122-
```
123-
124-
When ready update Portage from the Prefix environment:
125-
```
126-
startprefix
127-
emerge --oneshot sys-apps/portage
128-
```
11+
The installation of the compatibility layer is implemented with an Ansible playbook and Ansible role,
12+
which you can find in the `ansible` subdirectory. It will do the Gentoo Prefix bootstrap (which usually takes several hours to complete!),
13+
add a lot of EESSI configurations and customizations, install a bunch of packages that we require for EESSI, and finally it will
14+
run a ReFrame test suite to check the installation.
15+
16+
To make the installation even easier, we provide a script `install_compatibility_layer.sh` that can be used on basically any host that has Apptainer installed,
17+
without requiring special privileges.
18+
The script will execute the Ansible playbook inside an Apptainer build container, ensuring that all dependencies (including Ansible itself) are available.
19+
In order to be able to write to `/cvmfs`, the container will bind mount a directory from the host as `/cvmfs` inside the container.
12920

13021
# License
13122

0 commit comments

Comments
 (0)