Corral is a collection of storage and data management resources located at TACC. Corral services provide high-reliability, high-performance storage for research requiring persistent access to large quantities of structured or unstructured data. Such data could include data used in analysis or other computational and visualization tasks on other TACC resources, as well as data used in collaborations involving many researchers who need to share large amounts of data.
For the SHADE-AI Gym, we have created workspace on Corral to allow for sharing of data, scripts, etc. within and between performer teams. Currently, there is 5TB provisioned for the program. Additionaly storage can be added as needed.
Basic command-line access
Similar to compute systems, Corral can be accessed directly through the login node via SSH:
localhost$ ssh firstname.lastname@example.org
The login node is provided for use in transferring and organizing data through command-line utilities. The DARPA-SHADE directory is located at
Within this project workspace, there are two subdirectories:
Shared/ #intended to store and share data, scripts, etc. across program Performers/ #intended to store data, scripts, etc. to share within teams
Within Performers/ are 7 directories reserved for each performer team. Access to these directories is governed by Access Control Lists. Only users from respective teams will be able to access these directories.
Corral is mounted on the login and compute nodes of Frontera and Lonestar6. For I/O intensive tasks, it may be more efficient to stage data by copying from Corral to the $SCRATCH or $WORK file systems on specific machines. Data must be staged on Longhorn as Corral is not directly mounted.
The DARPA-SHADE workspace on Corral is mounted to the container running JupyterNotebooks. You will be able to access to the Shared/ and Performers/ directories.
To transfer a file from your local machine the Shared directory:
localhost$ scp filename email@example.com:/corral/projects/DARPA-SHADE/Shared
To transfer a file from Corral to your $SCRATCH directory on Lonestar6:
login1$ cp /corral-repl/projects/DARPA-SHADE/Shared/myfile.txt $SCRATCH/job_directory/