Transfer Files and Directories

For storage mounted directly on the Wynton cluster, files can be transferred to and from the cluster using regular file copying, e.g. cp and rsync. For non-mounted storage, files may be transferred to and from the cluster via secure copying, e.g. scp and sftp, which can be utilized via rsync. For file downloads from online resources, tools such as curl, wget, and rsync may be used.

From your local machine to the Wynton HPC file system

Copy a single file on your local file system to your Wynton home directory

To copy a single file to your home directory (~/) on the cluster, use

{local_machine}$ scp one_file.tsv alice@dt2.wynton.ucsf.edu:~/

Copy one or more files to a folder on the cluster

To copy multiple files to Wynton so they appear directly under ~/study/files/, use

{local_machine}$ scp *.txt *.R ../some/path/another_file.tsv alice@dt2.wynton.ucsf.edu:study/files/

Recursively copy a directory to a folder on the cluster

To copy all content of directory dataset/ to Wynton so that it appears as ~/study/dataset/, use

{local_machine}$ scp -r dataset/ alice@dt2.wynton.ucsf.edu:study/

From the Wynton HPC file system to your local machine

Copy a single file from your Wynton home directory to your local machine

To copy a single file in your Wynton home directory to the working directory of your local machine, use

{local_machine}$ cd /path/project
{local_machine}$ scp alice@dt2.wynton.ucsf.edu:one_file.tsv .

Note: Don’t forget that period (.) at the end - it indicates copy [the file] “to the current directory”.

Copy one or more files from the cluster

To copy multiple files from ~/study/files/ on the cluster to ~/study/ on your local machine, do:

{local_machine}$ scp alice@dt2.wynton.ucsf.edu:study/files/*.txt alice@dt2.wynton.ucsf.edu:study/files/*.R ~/study/

Recursively copy a folder from the cluster

To copy all content of directory dataset/ on the cluster so that it appears as dataset/ in your local working directory, use

{local_machine}$ cd /path/project
{local_machine}$ scp -r alice@dt2.wynton.ucsf.edu:dataset/ .

Note: Don’t forget that period (.) at the end - it indicates copy [the folder] “to the current directory”.