Data management
Introduction
ANIS provides scientific projects and for each project datasets extracted from databases. ANIS allows to associate files (fits, png, xml, ...) to the projects and datasets. We will see here how to associate data with projects and datasets.
Server data path
First of all you have to specify the directory where the data will be stored.
This directory will become the root directory for ANIS server.
Go to the docker-compose.yml
file to configure the DATA_PATH
environment variable.
Here you can see that the data directory is set to /data
Admin files explorer
An administrator can list files or folders contained in the root directory. Please note that if authentication is enabled you will need to send a token to perform this action.
If you want to explore the ANIS root
directory type:
curl http://localhost:8080/admin-file-explorer
If you want to explore the DEFAULT
directory type:
curl http://localhost:8080/admin-file-explorer/DEFAULT
Instance data path
You can associate a data directory for each of your scientific projects.
The directory path will start from the data root directory of anis-server.
To configure the data path of a project you have to fill in the property data_path
in the metamodel.
Example:
curl -d '{"name":"default","label":"Default instance","description":"Instance for the test","scientific_manager":"M. Durand","instrument":"Multiple","wavelength_domain":"Visible imaging / Spectroscopy","display":10,"data_path":"\/DEFAULT","files_path":"\/INSTANCE_FILES","public":true,"portal_logo":"","design_color":"#7AC29A","design_background_color":"#ffffff","design_logo":"/logo.png","design_favicon":"/favicon.ico","samp_enabled":true,"back_to_portal":true,"search_by_criteria_allowed":true,"search_by_criteria_label":"Search","search_multiple_allowed":false,"search_multiple_label":"Search multiple","search_multiple_all_datasets_selected":false,"documentation_allowed":false,"documentation_label":"Documentation"}' --header 'Content-Type: application/json' -X POST http://localhost/instance
You can also configure a directory for storing files related to the project : files_path
.
The directory path will start from the data path directory of the project.
Files related to the project such as the logo, images will be stored in this directory.
Instance files explorer
A user can list files or folders contained in the instance files path directory. Please note that if the instace is private you will need to send a token to perform this action.
If you want to explore the files for default project directory type:
curl http://localhost:8080/instance/default/file-explorer
Dataset data path
You can associate a data directory for each of your datasets. The directory path will start from the scientific project root directory (instance data_path). To configure the data_path of a dataset you have to fill in the property in the metamodel.
Example:
curl -d '{"name":"vipers_dr2_w1","table_ref":"aspic_vipers_dr2_w1","label":"VIPERS-W1 (DR2)","description":"VIPERS W1 dataset","display":10,"data_path":"\/ASPIC\/VIPERS_DR2","public":true,"download_json":true,"download_csv":true,"download_ascii":true,"download_vo":false,"server_link_enabled":false,"datatable_enabled":true,"datatable_selectable_rows":false,"id_database":1}' --header 'Content-Type: application/json' -X POST http://localhost/dataset-family/1/dataset
Here you can see that the data directory for this dataset is set to /ASPIC/VIPERS_DR2
Dataset files explorer
A user can list files or folders contained in the dataset directory. Please note that if the dataset is private you will need to send a token to perform this action.
Example:
curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer
If you want explore the spec1d directory you can execute :
curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D
If a user want to download the file VIPERS_101121877_bis.fits
you can execute :
curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D/VIPERS_101121877_bis.fits