Update Technical informations authored by Moreau Nicolas's avatar Moreau Nicolas
This page is intended to provide technical informations hopefully useful for anybody in charge of running the yafits containers. The commands indicated below were run under SL Linux. Feel free to introduce examples of commands used on different flavours of Linux or on Mac OSX where it's relevant. This page is intended to provide technical informations hopefully useful for anybody in charge of running the yafits containers. The commands indicated below were run under SL Linux. Feel free to introduce examples of commands used on different flavours of Linux or on Mac OSX where it's relevant.
# Prerequisites # Prerequisites
You have cloned/downloaded the `yafits` project and you can run the `docker-compose` command You have cloned/downloaded the `yafits` project and you can run the `docker-compose` command
# Starting docker daemon (possibly) # Starting docker daemon (possibly)
### Manually on demand ### Manually on demand
Before trying to launch any container, it's advised to ensure that the docker daemon is running. E.g. it may happen that the daemon has not been registered to be launched during a (re)boot sequence. Before trying to launch any container, it's advised to ensure that the docker daemon is running. E.g. it may happen that the daemon has not been registered to be launched during a (re)boot sequence.
``` ```plaintext
ps aux|grep dockerd ps aux|grep dockerd
``` ```
If the above command returns nothing then start a session on an account belonging to sudoers and start the daemon. Example : If the above command returns nothing then start a session on an account belonging to sudoers and start the daemon. Example :
``` ```plaintext
[caillat-adm@juliette ~]$ sudo service docker start [caillat-adm@juliette ~]$ sudo service docker start
[sudo] Mot de passe de caillat-adm : [sudo] Mot de passe de caillat-adm :
Redirecting to /bin/systemctl start docker.service Redirecting to /bin/systemctl start docker.service
...@@ -23,49 +25,66 @@ root 14212 0.2 0.0 2064984 65392 ? Ssl 11:59 0:00 /usr/bin/docke ...@@ -23,49 +25,66 @@ root 14212 0.2 0.0 2064984 65392 ? Ssl 11:59 0:00 /usr/bin/docke
``` ```
### Having `dockerd` automatically started during the future boot sequences ### Having `dockerd` automatically started during the future boot sequences
```
```plaintext
sudo systemctl enable docker.service sudo systemctl enable docker.service
``` ```
# Starting the yafits containers # Starting the yafits containers
## Change the current directory to the yafits project's : ## Change the current directory to the yafits project's :
``` ```plaintext
cd <..>/yafits cd <..>/yafits
``` ```
## Build and run the containers ## Build and run the containers with docker-compose
``` ```plaintext
docker-compose build docker-compose build
docker-compose up -d docker-compose up -d
``` ```
# Starting portainer
## Build and run the containers with the control script
Yafits provides a bash control script called yafits installed in the root directory. It accepts the following arguments :
```plaintext
- help Show this help
- build Build the application
- buildrun Build and run the application
- prune Prune existing containers
- run Run the application without build
- updatespectro Will build and run yafits, spectroscopy database will be updated
``` ```
# Starting portainer
```plaintext
docker volume create portainer_data docker volume create portainer_data
docker run -d -p 8000:8000 -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer docker run -d -p 8000:8000 -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
``` ```
## Activating `jupyterhub` sur `romeo` ## Activating `jupyterhub` sur `romeo`
Start a session under the `jupyterhub` account and then type : Start a session under the `jupyterhub` account and then type :
```
```plaintext
nohup jupyterhub & nohup jupyterhub &
``` ```
and logout. and logout.
or in a more system admin way : or in a more system admin way :
``` ```plaintext
sudo systemctl start jupyterhub.service sudo systemctl start jupyterhub.service
``` ```
and possibly check the status of the service : and possibly check the status of the service :
``` ```plaintext
sudo systemctl status jupyterhub.service sudo systemctl status jupyterhub.service
● jupyterhub.service - Jupyterhub ● jupyterhub.service - Jupyterhub
Loaded: loaded (/usr/lib/systemd/system/jupyterhub.service; disabled; vendor preset: disabled) Loaded: loaded (/usr/lib/systemd/system/jupyterhub.service; disabled; vendor preset: disabled)
...@@ -90,39 +109,44 @@ déc. 15 10:20:12 romeo.obspm.fr jupyterhub[115277]: [I 2020-12-15 10:20:12.529 ...@@ -90,39 +109,44 @@ déc. 15 10:20:12 romeo.obspm.fr jupyterhub[115277]: [I 2020-12-15 10:20:12.529
``` ```
## Having `jupyterhub` active concurrently with `docker` on `romeo` ## Having `jupyterhub` active concurrently with `docker` on `romeo`
By default `jupyterhub`'s proxy uses the port 8000 which the port used by `docker`'s. See below a way to have `jupyterhub`'s proxy using a different port ( e.g. 8500 ) :
* In `jupyterhub`'s account on `romeo` ( username : jupyterhub, password: askme) edit the file `jupyterhub_config.py` in the home directory and type `c.JupyterHub.bind_url = 'http://:8500'` at line 94 By default `jupyterhub`'s proxy uses the port 8000 which the port used by `docker`'s. See below a way to have `jupyterhub`'s proxy using a different port ( e.g. 8500 ) :
* Start `jupyterhub`. In the same account type `nohup jupyterhub &` . Be careful that a `nohup.out` file logging the activity is going to grow.
* In `jupyterhub`'s account on `romeo` ( username : jupyterhub, password: askme) edit the file `jupyterhub_config.py` in the home directory and type `c.JupyterHub.bind_url = 'http://:8500'` at line 94
* Start `jupyterhub`. In the same account type `nohup jupyterhub &` . Be careful that a `nohup.out` file logging the activity is going to grow.
* As root or from a sudo enabled account open the port 8500 once forever : * As root or from a sudo enabled account open the port 8500 once forever :
```
```plaintext
sudo firewall-cmd --zone=public --permanent --add-port=8500/tcp sudo firewall-cmd --zone=public --permanent --add-port=8500/tcp
sudo firewall-cmd --reload sudo firewall-cmd --reload
``` ```
At this point you should be able to access `jupyterhub` on `romeo` via the link [https://romeo.obspm.fr:8500](https://romeo.obspm.fr:8500) At this point you should be able to access `jupyterhub` on `romeo` via the link <https://romeo.obspm.fr:8500>
# Hardware configuration # Hardware configuration
This section describes the network, storage and computing resources dedicated to ARTEMIX and YAFITS on their development site i.e. the LERMA/Observatoire de Paris. This section describes the network, storage and computing resources dedicated to ARTEMIX and YAFITS on their development site i.e. the LERMA/Observatoire de Paris.
## juliette[.obspm.fr] ## juliette\[.obspm.fr\]
The host for artemix ( artemix.obspm.fr ) and yafits ( artemix.obspm.fr/fits ). The host for artemix ( artemix.obspm.fr ) and yafits ( artemix.obspm.fr/fits ).
* machine : DELL R740, 2 processors, 32 cores, 384GB RAM * machine : DELL R740, 2 processors, 32 cores, 384GB RAM
* disks : * disks :
- /datartemix ( 83TB ) - /datartemix ( 83TB )
- /datartemix2 ( 101TB ) - /datartemix2 ( 101TB )
# House cleaning # House cleaning
The execution of YAFITS servers generates a lot of temporary files amongst which are the FITS files generated a The execution of YAFITS servers generates a lot of temporary files amongst which are the FITS files generated a
# Network considerations # Network considerations
## Open a port ## Open a port
Example : Example :
```
```plaintext
[caillat-adm@juliette ~]$ sudo sudo firewall-cmd --zone=public --permanent --add-port=3023/tcp [caillat-adm@juliette ~]$ sudo sudo firewall-cmd --zone=public --permanent --add-port=3023/tcp
[sudo] Mot de passe de caillat-adm : [sudo] Mot de passe de caillat-adm :
success success
... ...
......