This repository contains all components that make up the CMS DQM Run Registry service.
The process of monitoring and certifying data in the CMS experiment consists of several stages. Each through which new decision-making information regarding the quality of data is revealed. Run Registry is an application designed to document and aggregate each decision made -and by which actor, either human, automatic or machine learning agent- at every stage. It is then responsible for aggregating and exposing the results of data deemed good or ‘usable for analysis’ to the CMS collaboration in what is known as the golden json.
From a technical standpoint, run registry is a full-stack javascript application. With a frontend built with React and a backend built with Node.js, it uses a PostgreSQL database instance running in CERN DB on demand and a redis microservice to handle the job queue for backend processing.
-
The frontend of run registry is written in JavaScript using React.js and Next.js. You can read more about it here.
-
The backend of run registry is written in JavaScript using Node.js. You can read more about it here.
-
The API client is a Python PIP package that interfaces via HTTP requests with the API of run registry and allows users to read data easily from the application. You can read more about it here.
-
They are in charge of building and pushing the frontend and microservices into Dockerhub on every push to the master branch. They are located in the .github folder.
This repository contains all the code from Run Registry. However, it doesn't contain the commit log of how the application was developed. To view them go to the individiual repositories (which are no longer used) of every component mentioned above. To modify the application you must do so however in this repository only.
Run registry is designed using loosely-coupled microservices. This architecture allows for future maintainers to only need to know how a given microservice works either to maintain it or replace it.
Run registry encompasses 5 microservices, each with its respective Dockerfile and dockerhub repository, here is a list of them with its respective dockerhub repository:
- The API. cmssw/runregistry-backend
- The service in charge of fetching runs & lumsiections from the OMS API. cmssw/runregistry-workers-oms-fetching
- The service in charge of pinging the DQM GUI API. cmssw/runregistry-workers-dqm-gui-pinging
- The service in charge of processing the jsons. cmssw/runregistry-workers-json-processing
- A helper redis service which serves as a transport queue between service 1 and 4.
There are two ways that Run Registry uses Event Sourcing: for configuration, and for data.
- For configuration
Run registry has multiple sources of configuration, some of them can be changed on the fly via the web interface, other configurations need to be changed via a change in the database, and the least commonly modified configurations need to be changed in code in one of the config.js files. Configurations updated via web interface are all event-sourced: The dataset classifier, class classifier, component classifier, datasets accepted in GUI classifier (datasets accepted), JSON classifier, offline dataset classifier. This means that all versions are tracked in the database, when a classifier is deleted it is not really deleted forever, there is a copy of it still hanging in the database. When it is edited, the previous version is also saved.
- For data
There is a great amount of effort in making Run Registry event-sourced on a run and per-lumisection basis. If there is a human error, for example if a user sometimes batch-updates runs and sets wrong values, there will always be a way to undo it. And there will be a track of what was done, by whom and when.
Both the production and development instances are deployed on Openstack. You should have been given access to the projects automatically by the DQM-DC egroups.
To handle authentication/authorization, a proxy is deployed on PaaS/OpenShift, one for each instance. The proxy is running custom code found here. To access the proxies' deployment, you will need to login to PaaS with the cmsdqm service account.
TODO: This proxy uses the old CERN SSO with ADFS, to be deprecated in Summer 2023.
flowchart TD
user[fa:fa-user User] -->|HTTPS|proxy("SSO Proxy")
subgraph OpenShift
proxy
end
subgraph OpenStack
fe[Frontend:7001]
be[Backend:9500]
end
proxy-->|HTTP|fe
proxy-->|HTTP|be
fe-->be
From your computer:
ssh -tCY [email protected] ssh -CY [email protected]On dev-runregistry.cern.ch:
cd /srv/node && export HOME=$PWD
export PATH=/srv/node/bin/:$PATH
cd /srv/
export NODE_ENV=production
export ENV=staging
export CLIENT_SECRET= #Ask DQM core conveners
sudo chmod -R 777 /srv/
forever stopall
rm -rf node
sudo wget https://nodejs.org/dist/v12.20.0/node-v12.20.0-linux-x64.tar.xz
sudo tar -xf node-v12.20.0-linux-x64.tar.xz
sudo mv node-v12.20.0-linux-x64 node
sudo rm node-v12.20.0-linux-x64.tar.xz
sudo chmod -R 777 /srv/
cd node && export HOME=$PWD
cd ..
export PATH=/srv/node/bin/:$PATH
npm install forever -g
npm i yarn -g
rm -rf runregistry
git clone https://github.com/cms-DQM/runregistry
cd runregistry/runregistry_frontend
sudo chmod -R 777 /srv/
mkdir certs
yarn
sudo chmod -R 777 /srv/
yarn build
sudo chmod -R 777 /srv/
forever start server.js
cd ../runregistry_backend
## how to generate key and certificate, check the section below
<!--
cp your cert usercert.pem certs/usercert.pem
cp cp your key certs/userkey.pem
-->
yarn
forever start app.jsAfter starting the project, it's important to check logs, is everything working fine!
Logs can be found by running a command: forever logs
On your computer:
ssh -tCY [email protected] ssh -CY [email protected]On runregistry.cern.ch:
cd /srv/node && export HOME=$PWD
export PATH=/srv/node/bin/:$PATH
cd /srv/
export NODE_ENV=production
export ENV=production
export CLIENT_SECRET= #Ask DQM core conveners
sudo chmod -R 777 /srv/
forever stopall
rm -rf node
sudo wget https://nodejs.org/dist/v12.20.0/node-v12.20.0-linux-x64.tar.xz
sudo tar -xf node-v12.20.0-linux-x64.tar.xz
sudo mv node-v12.20.0-linux-x64 node
sudo rm node-v12.20.0-linux-x64.tar.xz
sudo chmod -R 777 /srv/
cd node && export HOME=$PWD
cd ..
export PATH=/srv/node/bin/:$PATH
npm install forever -g
npm i yarn -g
rm -rf runregistry
git clone https://github.com/cms-DQM/runregistry
cd runregistry/runregistry_frontend
sudo chmod -R 777 /srv/
yarn
sudo chmod -R 777 /srv/
yarn build
sudo chmod -R 777 /srv/
forever start server.js
cd ../runregistry_backend
yarn
mkdir certs
## how to generate key and certificate, check the section below
<!--
cp your cert usercert.pem certs/usercert.pem
cp cp your key certs/userkey.pem
-->
forever start app.jsAfter starting the project, it's important to check logs, is everything working fine!
Logs can be found by running a command: forever logs
You need to have grid certificate, provoded by CERN, https://ca.cern.ch/ca/
After, execute following commands: Geberate a private key:
openssl pkcs12 -in myCertificate.p12 -nokeys -out usercert.pem -nodesGeberate a certificate:
openssl pkcs12 -in myCertificate.p12 -nocerts -out userkey.pem -nodesThe person, who has sudo rights to dev-runregistry and runregistry machines has to add you as a user:
sudo useraddcern usernameTo give sudo rights (optional):
sudo usermod -aG wheel usernameCheck, can you connect to the machine(-s):
ssh -tCY [email protected] ssh -CY [email protected]ssh -tCY [email protected] ssh -CY [email protected]Check do you have sudo rights (optional):
sudo whoamiCheck out the dedicated FAQ.