Home / Grid tools & services / Data access & analysis

Data access at Castor2

Page under construction

Castor is the storage management system used at CERN. All interaction with the file system is managed by a stager. Files are presented to the user in a Unix-like directory hierarchy.

Useful links
Castor2 user guide
Environment variables
ATLAS group settings 
export RFIO_USE_CASTOR_V2=YES     (rfio version)
				export STAGE_HOST=castoratlas       (stage host)
				export STAGE_SVCCLASS=default       (disk pool)
File names
Experiments and users with an lxplus account have space reserved on Castor. For the Grid enabled space note the "grid/" in the path. 
/castor/cern.ch/atlas/project/			     ATLAS files
				  /castor/cern.ch/user/<u>/<userId>/		  user file
				  /castor/cern.ch/grid/atlas/users/<userId>/		user file

Castor2 stager commands

In the text below shorts like castorfile1 should be replaced by full Castor file names like /castor/cern.ch/user/t/test/file1. The requestid is a complicated identifier used internally by Castor to identify a file; The usertag is a user-defined name for a put or get request, and can be re-used when inquiring the status of a transaction.

Man pages also User guide
>   man -k castor
Use: <command> --help or -h.
stager_qry
Querying the status and progress of files and stager requests stager_get/put, rfcp. 
>   stager_qry  -M cfile1  -r requestid
			  >   stager_qry  -U usertag    (usertag previously defined in stager_get/put command)
			  >   stager_qry  -s            (display statistics)
stager_put
Starts a put/update session. The client must take care of ending the transaction using stager_putdone. 
>   stager_put  -M castorfile1  -U usertag
			  >   rfcp  myfile1  castorfile1
			  >   stager_qry  -U usertag
			  >   stager_putdone  -M castorfile1
stager_putdone
Terminate a CASTOR stager session and flag files for tape migration 
>   stager_putdone  -M castorfile1  -M castorfile2
stager_get
Get (stage) existing files to disk. Use the usertag to query the progress of staging. 
>   stager_get  -M castorfile1  -M castorfile2  -U usertag 
		     >   stager_qry  -U usertag
stager_rm
Remove file(s) from the CASTOR stager catalog. This includes marking all replicas of the file for garbage collection. 
>   stager_rm  -M castorfile1  -M castorfile2

RFIO user commands

rfmkdir   <directory>
Create remote directory with your userId $USER 
>   rfmkdir /castor/cern.ch/grid/atlas/users/$USER/
rfcp   <file>   <directory>
remote file copy to the Castor/Grid user space 
>   rfcp my.file /castor/cern.ch/grid/atlas/users/$USER/
rfdir   <directory>
List the content of the Castor directory, if it exists 
>   rfdir /castor/cern.ch/grid/atlas/users/$USER/
nsls   <file>
can only list CASTOR files; lists also the tape residence (-T option) and supports a special mode-bit 'm' flagging that the file has been migrated to tape 
>   nsls -l /castor/cern.ch/user/h/hansl
				>   nsls /castor/cern.ch/grid/atlas/tzero/prod2/perm/fdr08_run2/physics_Express/0052280
rfstat   <file1> <file2> …
Displays information about the specified file(s). For testing the existence of a directory, it is better to use rfstat rather than rfdir. 
>   rfstat /castor/cern.ch/grid/atlas/users/$USER/myfile
rfrm   [-r]   <file1> <file2> …
Remove a file or directory; be careful with the "-r" option, which removes recursively files and directories. 
>   rfrm /castor/cern.ch/grid/atlas/users/$USER/myfile

Support

↑ Top