|
|
|
Requirements
|
|
|
|
============
|
|
|
|
|
|
|
|
python 3 >= 3.7
|
|
|
|
|
|
|
|
|
|
|
|
Initial steps
|
|
|
|
=============
|
|
|
|
|
|
|
|
1. Install docker-compose
|
|
|
|
2. Add the user that will run the command to the docker group (sudo usermod -aG docker ${USER})
|
|
|
|
3. Start a new session or su - ${USER}
|
|
|
|
4. Get the Yafits archive (https://zenodo.org/record/3696974#.Ya-L633ftdB)
|
|
|
|
5. Untar the archive in a directory
|
|
|
|
6. Go in the newly created directory
|
|
|
|
7. Copy yafits.bashrc.dist in .yafits.bashrc
|
|
|
|
8. Fill .yafits.bashrc configuration file according to your needs ( the file is self-documented )
|
|
|
|
|
|
|
|
By default the application will use 3 ports :
|
|
|
|
* 3022 for the UI
|
|
|
|
* 4252 for the fits file server
|
|
|
|
* 30000 for the spectral lines server
|
|
|
|
|
|
|
|
Installation without spectroscopic data
|
|
|
|
========================================
|
|
|
|
|
|
|
|
1. Start from Initial step -> 8
|
|
|
|
2. Check that ./yafits scripts is executable
|
|
|
|
3. Execute ./yafits buildrun
|
|
|
|
|
|
|
|
Installation with spectroscopic data
|
|
|
|
====================================
|
|
|
|
|
|
|
|
1. Start from Initial step -> 8
|
|
|
|
2. Download spectroscopic data zip files
|
|
|
|
3. Moves them in YAFITS_HOME/spectro/mongo-seed/data
|
|
|
|
4. Comment/Uncomment content of YAFITS_HOME/spectro/mongo-seed/Dockerfile
|
|
|
|
|
|
|
|
```
|
|
|
|
# example where only local.json file is enabled
|
|
|
|
# metadata is required
|
|
|
|
COPY ./data/local.json.zip /local.json.zip
|
|
|
|
#COPY ./data/cdms.json.zip /cdms.json.zip
|
|
|
|
#COPY ./data/jpl.json.zip /jpl.json.zip
|
|
|
|
#COPY ./data/ISM_cdms.json.zip /ISM_cdms.json.zip
|
|
|
|
#COPY ./data/ISM_CSM_cdms.json.zip /ISM_CSM_cdms.json.zip
|
|
|
|
COPY ./data/metadata.json.zip /metadata.json.zip
|
|
|
|
|
|
|
|
RUN unzip /local.json.zip
|
|
|
|
#RUN unzip /cdms.json.zip
|
|
|
|
#RUN unzip /jpl.json.zip
|
|
|
|
#RUN unzip /ISM_cdms.json.zip
|
|
|
|
#RUN unzip /ISM_CSM_cdms.json.zip
|
|
|
|
RUN unzip /metadata.json.zip
|
|
|
|
|
|
|
|
RUN rm /local.json.zip
|
|
|
|
#RUN rm /cdms.json.zip
|
|
|
|
#RUN rm /jpl.json.zip
|
|
|
|
#RUN rm /ISM_cdms.json.zip
|
|
|
|
#RUN rm /ISM_CSM_cdms.json.zip
|
|
|
|
RUN rm /metadata.json.zip
|
|
|
|
```
|
|
|
|
|
|
|
|
4. Comment/Uncomment content of YAFITS_HOME/spectro/mongo-seed/import.sh
|
|
|
|
|
|
|
|
```
|
|
|
|
# example where only local.json file is enabled
|
|
|
|
# metadata is required
|
|
|
|
# removes old databases
|
|
|
|
mongo spectroscopy --eval "db.dropDatabase()"
|
|
|
|
# import available json files
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection cdms --file /cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection ism --file /ISM_cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection ismcsm --file /ISM_CSM_cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection jpl --file /jpl.json
|
|
|
|
mongoimport --host mongodb --db spectroscopy --collection local --file /local.json
|
|
|
|
mongoimport --host mongodb --db spectroscopy --collection metadata --file /metadata.json
|
|
|
|
|
|
|
|
# create indexes
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ism.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ism.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ismcsm.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ismcsm.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.jpl.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.jpl.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.cdms.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.cdms.createIndex({frequency:1})"
|
|
|
|
```
|
|
|
|
|
|
|
|
5. Go back to YAFITS_HOME
|
|
|
|
6. Execute ./yafits updatespectro
|
|
|
|
|
|
|
|
|
|
|
|
Running the application in background
|
|
|
|
======================================
|
|
|
|
|
|
|
|
1. Execute nohup ./yafits buildrun & ( same goes for updatespectro option )
|
|
|
|
|
|
|
|
|
|
|
|
Stopping the application
|
|
|
|
========================
|
|
|
|
|
|
|
|
1. Execute ./yafits down
|
|
|
|
|
|
|
|
Performing tests
|
|
|
|
================
|
|
|
|
|
|
|
|
Running the tests
|
|
|
|
-----------------
|
|
|
|
|
|
|
|
1. Execute ./yafits test
|
|
|
|
|
|
|
|
This will generate a report file in the Yafits home directory
|
|
|
|
|
|
|
|
|
|
|
|
Configuring the tests
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
The configuration is done through the .yafits.bashrc file.
|
|
|
|
|
|
|
|
```
|
|
|
|
# contains reference json files against which the tests will be performed
|
|
|
|
export YAFITS_REFDIR=/obs/partemix/yafits_tmp/REF
|
|
|
|
|
|
|
|
# contains the test json files
|
|
|
|
export YAFITS_OBJDIR=/obs/partemix/yafits_tmp/OBJ
|
|
|
|
|
|
|
|
# if true, a json data file will be created in YAFITS_OBJDIR for each opened data cube
|
|
|
|
export YAFITSV_TESTMODE=true
|
|
|
|
```
|
|
|
|
|
|
|
|
How does it work
|
|
|
|
----------------
|
|
|
|
|
|
|
|
All the files existing in YAFITS_REFDIR will be compared to the file with the same name in YAFITS_OBJDIR
|
|
|
|
The result of the comparison will be stored in a report file in the home directory of Yafits
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Requirements
|
|
|
|
============
|
|
|
|
|
|
|
|
python 3 >= 3.7
|
|
|
|
|
|
|
|
|
|
|
|
Initial steps
|
|
|
|
=============
|
|
|
|
|
|
|
|
1. Install docker-compose
|
|
|
|
2. Add the user that will run the command to the docker group (sudo usermod -aG docker ${USER})
|
|
|
|
3. Start a new session or su - ${USER}
|
|
|
|
4. Get the Yafits archive (https://zenodo.org/record/3696974#.Ya-L633ftdB)
|
|
|
|
5. Untar the archive in a directory
|
|
|
|
6. Go in the newly created directory
|
|
|
|
7. Copy yafits.bashrc.dist in .yafits.bashrc
|
|
|
|
8. Fill .yafits.bashrc configuration file according to your needs ( the file is self-documented )
|
|
|
|
|
|
|
|
By default the application will use 3 ports :
|
|
|
|
* 3022 for the UI
|
|
|
|
* 4252 for the fits file server
|
|
|
|
* 30000 for the spectral lines server
|
|
|
|
|
|
|
|
Installation without spectroscopic data
|
|
|
|
========================================
|
|
|
|
|
|
|
|
1. Start from Initial step -> 8
|
|
|
|
2. Check that ./yafits scripts is executable
|
|
|
|
3. Execute ./yafits buildrun
|
|
|
|
|
|
|
|
Installation with spectroscopic data
|
|
|
|
====================================
|
|
|
|
|
|
|
|
1. Start from Initial step -> 8
|
|
|
|
2. Download spectroscopic data zip files
|
|
|
|
3. Moves them in YAFITS_HOME/spectro/mongo-seed/data
|
|
|
|
4. Comment/Uncomment content of YAFITS_HOME/spectro/mongo-seed/Dockerfile
|
|
|
|
|
|
|
|
```
|
|
|
|
# example where only local.json file is enabled
|
|
|
|
# metadata is required
|
|
|
|
COPY ./data/local.json.zip /local.json.zip
|
|
|
|
#COPY ./data/cdms.json.zip /cdms.json.zip
|
|
|
|
#COPY ./data/jpl.json.zip /jpl.json.zip
|
|
|
|
#COPY ./data/ISM_cdms.json.zip /ISM_cdms.json.zip
|
|
|
|
#COPY ./data/ISM_CSM_cdms.json.zip /ISM_CSM_cdms.json.zip
|
|
|
|
COPY ./data/metadata.json.zip /metadata.json.zip
|
|
|
|
|
|
|
|
RUN unzip /local.json.zip
|
|
|
|
#RUN unzip /cdms.json.zip
|
|
|
|
#RUN unzip /jpl.json.zip
|
|
|
|
#RUN unzip /ISM_cdms.json.zip
|
|
|
|
#RUN unzip /ISM_CSM_cdms.json.zip
|
|
|
|
RUN unzip /metadata.json.zip
|
|
|
|
|
|
|
|
RUN rm /local.json.zip
|
|
|
|
#RUN rm /cdms.json.zip
|
|
|
|
#RUN rm /jpl.json.zip
|
|
|
|
#RUN rm /ISM_cdms.json.zip
|
|
|
|
#RUN rm /ISM_CSM_cdms.json.zip
|
|
|
|
RUN rm /metadata.json.zip
|
|
|
|
```
|
|
|
|
|
|
|
|
4. Comment/Uncomment content of YAFITS_HOME/spectro/mongo-seed/import.sh
|
|
|
|
|
|
|
|
```
|
|
|
|
# example where only local.json file is enabled
|
|
|
|
# metadata is required
|
|
|
|
# removes old databases
|
|
|
|
mongo spectroscopy --eval "db.dropDatabase()"
|
|
|
|
# import available json files
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection cdms --file /cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection ism --file /ISM_cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection ismcsm --file /ISM_CSM_cdms.json
|
|
|
|
#mongoimport --host mongodb --db spectroscopy --collection jpl --file /jpl.json
|
|
|
|
mongoimport --host mongodb --db spectroscopy --collection local --file /local.json
|
|
|
|
mongoimport --host mongodb --db spectroscopy --collection metadata --file /metadata.json
|
|
|
|
|
|
|
|
# create indexes
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ism.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ism.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ismcsm.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.ismcsm.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.jpl.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.jpl.createIndex({frequency:1})"
|
|
|
|
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.cdms.createIndex({sourcefile:1})"
|
|
|
|
#mongo --host mongodb spectroscopy --eval "db.cdms.createIndex({frequency:1})"
|
|
|
|
```
|
|
|
|
|
|
|
|
5. Go back to YAFITS_HOME
|
|
|
|
6. Execute ./yafits updatespectro
|
|
|
|
|
|
|
|
|
|
|
|
Running the application in background
|
|
|
|
======================================
|
|
|
|
|
|
|
|
1. Execute nohup ./yafits buildrun & ( same goes for updatespectro option )
|
|
|
|
|
|
|
|
|
|
|
|
Stopping the application
|
|
|
|
========================
|
|
|
|
|
|
|
|
1. Execute ./yafits down
|
|
|
|
|
|
|
|
|
