Data management

Introduction

ANIS provides scientific projects and for each project datasets extracted from databases. ANIS allows to associate files (fits, png, xml, ...) to the projects and datasets. We will see here how to associate data with projects and datasets.

Server data path

First of all you have to specify the directory where the data will be stored. This directory will become the root directory for ANIS server. Go to the docker-compose.yml file to configure the DATA_PATH environment variable.

anis_server_data_path

Here you can see that the data directory is set to /data

Admin files explorer

An administrator can list files or folders contained in the root directory. Please note that if authentication is enabled you will need to send a token to perform this action.

If you want to explore the ANIS root directory type:

curl http://localhost:8080/admin-file-explorer

admin_file_explorer_root

If you want to explore the DEFAULT directory type:

curl http://localhost:8080/admin-file-explorer/DEFAULT

admin_file_explorer_default_directory

Instance data path

You can associate a data directory for each of your scientific projects. The directory path will start from the data root directory of anis-server. To configure the data path of a project you have to fill in the property data_path in the metamodel.

Example:

curl -d '{"name":"default","label":"Default instance","description":"Instance for the test","scientific_manager":"M. Durand","instrument":"Multiple","wavelength_domain":"Visible imaging / Spectroscopy","display":10,"data_path":"\/DEFAULT","files_path":"\/INSTANCE_FILES","public":true,"portal_logo":"","design_color":"#7AC29A","design_background_color":"#ffffff","design_logo":"/logo.png","design_favicon":"/favicon.ico","samp_enabled":true,"back_to_portal":true,"search_by_criteria_allowed":true,"search_by_criteria_label":"Search","search_multiple_allowed":false,"search_multiple_label":"Search multiple","search_multiple_all_datasets_selected":false,"documentation_allowed":false,"documentation_label":"Documentation"}' --header 'Content-Type: application/json' -X POST http://localhost/instance

You can also configure a directory for storing files related to the project : files_path. The directory path will start from the data path directory of the project. Files related to the project such as the logo, images will be stored in this directory.

Instance files explorer

A user can list files or folders contained in the instance files path directory. Please note that if the instace is private you will need to send a token to perform this action.

If you want to explore the files for default project directory type:

curl http://localhost:8080/instance/default/file-explorer

instance_file_explorer_default_directory

Dataset data path

You can associate a data directory for each of your datasets. The directory path will start from the scientific project root directory (instance data_path). To configure the data_path of a dataset you have to fill in the property in the metamodel.

Example:

curl -d '{"name":"vipers_dr2_w1","table_ref":"aspic_vipers_dr2_w1","label":"VIPERS-W1 (DR2)","description":"VIPERS W1 dataset","display":10,"data_path":"\/ASPIC\/VIPERS_DR2","public":true,"download_json":true,"download_csv":true,"download_ascii":true,"download_vo":false,"server_link_enabled":false,"datatable_enabled":true,"datatable_selectable_rows":false,"id_database":1}' --header 'Content-Type: application/json' -X POST http://localhost/dataset-family/1/dataset

Here you can see that the data directory for this dataset is set to /ASPIC/VIPERS_DR2

Dataset files explorer

A user can list files or folders contained in the dataset directory. Please note that if the dataset is private you will need to send a token to perform this action.

Example:

curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer

dataset_file_explorer

If you want explore the spec1d directory you can execute :

curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D

dataset_file_explorer_spec1d_directory

If a user want to download the file VIPERS_101121877_bis.fits you can execute :

curl http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D/VIPERS_101121877_bis.fits