|
1 | 1 | # CCR Software Layer |
2 | 2 |
|
3 | | -This is the CCR software layer. |
4 | | - |
5 | | -_NOTE: these docs are currently very much a WIP_ |
6 | | - |
7 | 3 | ## About |
8 | 4 |
|
9 | | -All software at CCR is built using [easybuild](https://docs.easybuild.io/en/latest/), |
10 | | -available via modules using [Lmod](https://lmod.readthedocs.io/en/latest/), and |
11 | | -distributed using [CernVM-FS](https://cvmfs.readthedocs.io/en/stable/). This repository |
12 | | -contains scripts to help build software with easybuild using containers. To ensure all |
13 | | -software runs on any compute node regardless of the underlying Linux distro, we |
14 | | -use [Gentoo Prefix](https://wiki.gentoo.org/wiki/Project:Prefix) as the |
15 | | -compatibility layer. For more information on CCR's compatibility layer see our |
16 | | -[gentoo-overlay](https://github.com/ubccr/gentoo-overlay). |
17 | | - |
18 | | -## Building Software |
19 | | - |
20 | | -Software at CCR is distributed via CernVM-FS, a read-only file system designed |
21 | | -to deliver scientific software. CCR's workflow for building new software uses |
22 | | -singularity containers with a writable overlay on top of the CCR CernVM-FS |
23 | | -repository. Any physical compute node can be used to run the container and |
24 | | -build software. |
25 | | - |
26 | | -### Quick start |
27 | | - |
28 | | -``` |
29 | | -# Clone repo and start container |
30 | | -user@host$ git clone https://github.com/ubccr/software-layer.git |
31 | | -user@host$ cd software-layer |
32 | | -user@host$ ./start-container.sh /scratch/username_somepackage |
33 | | -
|
34 | | -# Inside load easybuild module and run easybuild |
35 | | -user@container$ module load easybuild |
36 | | -
|
37 | | -# Show easybuild config |
38 | | -user@container$ eb --show-config |
39 | | -
|
40 | | -# Search for package showing short names |
41 | | -user@container$ eb -S OpenMPI |
42 | | -
|
43 | | -# Search for package showing full path to eb file |
44 | | -user@container$ eb --search OpenMPI |
45 | | -
|
46 | | -# Show the deps that will be built without actually building |
47 | | -user@container$ eb OpenBLAS-0.3.9-GCC-9.3.0.eb -M |
48 | | -
|
49 | | -# Build package and all deps |
50 | | -user@container$ eb OpenBLAS-0.3.9-GCC-9.3.0.eb --robot |
51 | | -
|
52 | | -user@container$ module load openblas |
53 | | -user@container$ exit |
54 | | -
|
55 | | -# Exit container and create tarball for ingestion into CernVM-FS |
56 | | -user@host$ ./create-tarball.sh easybuild /scratch/username_somepackage |
57 | | -user@host$ cp /scratch/ccr-xxx-easybuild-user_somepackage-xxx.tar.gz /path/to/mirrors |
58 | | -``` |
59 | | - |
60 | | -### Workflow in detail |
61 | | - |
62 | | -1. Clone this repo (or clone your own fork if you plan on submitting PRs): |
63 | | - |
64 | | -``` |
65 | | -$ git clone https://github.com/ubccr/software-layer.git |
66 | | -$ cd software-layer |
67 | | -``` |
68 | | - |
69 | | -2. Start a singularity container. We use the helper script `start-container.sh` |
70 | | -which takes a single argument: the path to a working directory where the |
71 | | -writable-overlay will write your files. This should be somewhere in `/scratch`. |
72 | | -It's recommended to follow a basic naming convention, for example if your |
73 | | -building OpenMPI, then use `/scratch/username_OpenMPI`. |
74 | | - |
75 | | -``` |
76 | | -$ ./start-container.sh /scratch/username_somepackage |
77 | | -``` |
78 | | - |
79 | | -NOTE: The first time you run this command it may fetch the build-node docker |
80 | | -image from docker hub, convert it to singularity sif format, then start the |
81 | | -container. |
82 | | - |
83 | | -3. After the container starts you have a shell with the current directory |
84 | | -(this software-layer git repo where you ran the start-container.sh script) |
85 | | -bind mounted into `/srv/software-layer` inside the container. So cd into this |
86 | | -directory: |
87 | | - |
88 | | -``` |
89 | | -$ cd /srv/software-layer |
90 | | -``` |
91 | | - |
92 | | -4. Ensure you enviroment is setup: |
93 | | - |
94 | | -``` |
95 | | -# This should now show the latest software modules |
96 | | -$ module avail |
97 | | -``` |
98 | | - |
99 | | -NOTE: You can make changes to the config/init scripts and test them inside the |
100 | | -container. Then commit and submit a PR's etc. |
101 | | - |
102 | | - |
103 | | -5. Load the easybuild module: |
| 5 | +Software at CCR is built using [easybuild](https://docs.easybuild.io/en/latest/) and available via modules using [Lmod](https://lmod.readthedocs.io/en/latest/). CCR's [standard software environments](https://docs.ccr.buffalo.edu/en/latest/software/releases/) provide a method for users to switch between versions of toolchains, compilers, libraries, and other packages compiled with specific optimizations enabled to take advantage of the various features of each CPU microarchitecture available in CCR's HPC environment. For more information about using software modules, refer to the [documentation](https://docs.ccr.buffalo.edu/en/latest/software/modules/). |
104 | 6 |
|
105 | | -``` |
106 | | -$ module load easybuild |
107 | | -``` |
108 | 7 |
|
109 | | -6. Build software, test software: |
| 8 | +## CCR Software Policy |
110 | 9 |
|
111 | | -``` |
112 | | -$ eb OpenBLAS-0.3.9-GCC-9.3.0.eb --robot |
113 | | -[grab a cup of coffee] |
114 | | -... |
115 | | -[hours later] |
116 | | -.. |
117 | | -$ module load openblas |
118 | | -$ exit |
119 | | -``` |
| 10 | +Please refer to the [CCR documentation](https://docs.ccr.buffalo.edu/en/latest/policies/software) for information about what software is provided by CCR, instructions for requesting software installations, and details on what restrictions or limitations are enforced. |
120 | 11 |
|
121 | | -7. Once you exit the container terminated and any easybuild files (modules, |
122 | | -software) that were written to `/cvmfs` were captured in your workdir in a |
123 | | -directory named `overlay-upper`. We can now run a script to tar up these files |
124 | | -for eventual importing into the CCR cvmfs repo. The tarball will be created |
125 | | -with the following naming convention: |
| 12 | +## Installing Software Yourself |
126 | 13 |
|
127 | | -``` |
128 | | - ccr-<version>-{compat,easybuild}-[some tag]-<timestamp>.tar.gz |
129 | | -``` |
| 14 | +CCR users are able to install software themselves in their project and home directories or utilize container environments for custom installations. For more information, please refer to the Easybuild documentation [here](https://docs.ccr.buffalo.edu/en/latest/software/building/), an Easybuild how-to tutorial [here](https://docs.ccr.buffalo.edu/en/latest/howto/easybuild/), and container documentation [here](https://docs.ccr.buffalo.edu/en/latest/howto/containerization/). We also have Python-specific documentation [here](https://docs.ccr.buffalo.edu/en/latest/howto/python/). If you're unsure which method makes the most sense for your workflow, contact [CCR Help](https://docs.ccr.buffalo.edu/en/latest/help/) for guidance. |
130 | 15 |
|
131 | | -To create a tarball after building software with easybuild run: |
| 16 | +## Submitting Software Build Requests and Bug Reports |
132 | 17 |
|
133 | | -``` |
134 | | -$ ./create-tarball.sh easybuild /scratch/username_somepackage |
135 | | -``` |
| 18 | +**Installation Requests:** |
| 19 | +CCR users should submit build requests for staff to install software using [GitHub issues](https://github.com/ubccr/software-layer/issues). Before submitting a new request, please search the issues to see if another user has already requested an installation of the software you're interested in. If there is, rather than submitting a new request, please "like" the GitHub issue and comment on it to let us know which research group you're work for. This helps us to prioritize the build requests. When creating a new issue you should select the `Software Request` template which asks for the following information: |
| 20 | + - Your last name so we can lookup your CCR account (this is a public repo so please don't share your username) |
| 21 | + - The last name of the faculty member in charge of the research group you're working with |
| 22 | + - A list of any other research groups that you may know of that are interested in this software |
| 23 | + - A link to the software website so we can ensure we're building the software you want. There are many packages with the same name for different scientific disiplines! |
| 24 | + - The preferred version of the software you'd like us to install |
| 25 | + - Which [CCR software environment](https://docs.ccr.buffalo.edu/en/latest/software/releases/) you'd like the software published to |
| 26 | + - Any additional information you'd like to share about your request |
| 27 | + |
| 28 | +**Please enter a new request for each piece of software you'd like installed.** |
136 | 29 |
|
137 | | -8. Copy the resulting tarball to mirrors so that others can test out your work |
138 | | -before publishing to our production CernVM-FS stratum0 server. |
| 30 | +**Bug Reports:** |
| 31 | +If you experience an issue with an installed module on CCR's systems, please submit a bug report. Bug reports should only be submitted for problems with the software installation or the Easybuild system itself. If you need assistance with using a particular software module, running jobs, or anything else related to CCR's systems, please submit a [help request](https://www.buffalo.edu/ccr/support.html) using our ticketing system. |
139 | 32 |
|
140 | 33 | ## See Also |
141 | 34 |
|
|
0 commit comments