Skip to main content
Log inGet free account

Large File Transfers with rsync and SFTP

Transfer large files using an SFTP client or rsync.

Discuss in Slack
Edit this page on GitHub Report an issue with this doc

This section provides information on how to use your SFTP client or rsync to transfer large files.

File Size Limits

The Pantheon Filesystem supports files up to 256 MiB. Attempting to write a file larger than that will fail.

  • Drupal: sites/default/files
  • WordPress: wp-content/uploads

An SFTP client or rsync allows you to transfer unlimited data server-to-server, which is faster than transferring from your workstation. Files can be transferred to and from any Pantheon site environment (Dev, Test, and Live).

Notes

  • This section covers copying files, excluding database files. You cannot directly access the database files. Refer to Use the Pantheon WebOps Workflow for more information on how code moves up and content moves down.

  • You will not be able to use SFTP or rsync to add any file or directory listed in a .gitignore file to your Git repository. Any file uploaded in this way cannot be committed and will not be available for deployment.

There are two mechanisms for transferring files: SFTP and rsync.

SFTP

There are a number of GUI SFTP clients available, such as WinSCP and Cyberduck. Do not use simultaneous connections.

  1. Go to the Site Dashboard.

  2. Click Connection Info to access your credentials.

  3. Connect to your SFTP client. The example belows uses a command-line SFTP client to connect to a site environment's file directory.

    Note

    You must replace [env] with the target environment and [uuid] with the Site UUID to connect. The values are case sensitive and must be lower case (for example, dev, test, live).

    export ENV=[env]
# Usually dev, test, or live
export SITE=[uuid]
# Site UUID from dashboard URL: https://dashboard.pantheon.io/sites/[uuid]

sftp -oPort=2222 $ENV.$SITE@appserver.$ENV.$SITE.drush.in
Connected to appserver.$ENV.$SITE.drush.in
sftp> cd files
sftp> put [your file or files]

rsync

rsync is an advanced tool that requires experience with the command line. You can also use the Terminus rsync plugin as a shortcut to rsync files to your Pantheon sites. rsync is highly customizable. Refer to rsync documentation to learn more.

Note

Your files must be in the /files directory for both Drupal and WordPress sites. This directory maps to sites/default/files for Drupal and wp-content/uploads for WordPress. Adjust paths as needed to include web (for example, web/wp-content/uploads) for sites configured to use a nested docroot.

Transfer Files with rsync

  1. Ensure that source or the destination is a local file or directory. The source and destination cannot both be remote.

  2. Replace [env] with the target environment and [uuid] with the Site UUID before you run the code below to connect. The values are case sensitive and must be lower case (for example, dev, test, live).

    export ENV=[env]
# Usually dev, test, or live
export SITE=[uuid]
# Site UUID from dashboard URL: https://dashboard.pantheon.io/sites/[uuid]

# To Upload/Import
rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' . --temp-dir=~/tmp/ $ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/

# To Download
rsync -rvlz --copy-unsafe-links --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/ ~/files
    # -r: Recurse into subdirectories
    # -v: Verbose output
    # -l: copies symlinks as symlinks
    # -L: transforms symlinks into files.
    # -z: Compress during transfer
    # --copy-unsafe-links: transforms symlinks into files when the symlink target is outside of the tree being copied
    # Other rsync flags may or may not be supported
    # (-a, -p, -o, -g, -D, etc are not).
    ```

Examples

Exports

The examples below use the variables $ENV and $SITE. You must replace these variables with your site UUID and environment, or set them before you begin:

export ENV=dev
export SITE=3ef6264e-51d9-43b9-a60b-6cc22c3129308as83

Replace the example values above with the environment you're working with and your site UUID. You can find the UUID in your Site Dashboard URL:

https://dashboard.pantheon.io/sites/**3ef6264e-51d9-43b9-a60b-6cc22c3129308as83**

Download a Drupal Directory from Pantheon

Download the contents of the sites/default/files directory into a folder on your local environment in the files home folder:

rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:code/sites/default/files/ ~/files

Download a WordPress Directory from Pantheon

Download the contents of the wp-content/uploads directory into a folder on your local environment in the files home folder:

rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:code/wp-content/uploads ~/files

Download a Drupal File from Pantheon

Download the sites/default/settings.php file into a Drupal installation called Foo on your local environment in a sites/default/files folder:

rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:code/sites/default/settings.php ~/Foo/sites/default

Download a WordPress File from Pantheon

Download the index.php file into a WordPress installation called Foo on your local environment in a wp-content/uploads folder:

rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' $ENV.$SITE@appserver.$ENV.$SITE.drush.in:code/wp-content/uploads/index.php ~/Foo/sites/wp-content/uploads

Upload a Directory to Pantheon

If you need to upload the file directory from a local installation called Foo in your home directory to a Pantheon site's Test environment sites/default/files directory, use the command below. If you are migrating a site or otherwise overwriting an existing site, remove --ignore-existing before running the command from the code example below.

Warning

Always use the temp-dir flag when using rsync for uploads. Removing the flag will result in broken files after cloning from one environment to another.

rsync --ignore-existing -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' ~/files/. --temp-dir=~/tmp/ 
$ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/

Upload a Single File to Pantheon

This example shows how to upload the logo.png file into a Pantheon site's theme folder.

rsync -rLvz --size-only --checksum --ipv4 --progress -e 'ssh -p 2222' ~/Foo/sites/all/themes/foo/logo.png --temp-dir=~/tmp/ $ENV.$SITE@appserver.$ENV.$SITE.drush.in:code/sites/all/themes/foo

Empty a Folder Recursively Using rsync

The rm -r command is not available over SFTP on Pantheon. An alternative way to recursively empty a folder is to use the rsync --delete flag. This example shows how to empty the remote folder files/remote_folder_to_empty (change this to match the remote directory you want to empty).

Create an empty folder with mkdir empty_folder on your local machine. The folder can be named anything, as long as it's empty.

export ENV=env # Replace with the site environment, usually dev, test, or live
export SITE=uuid # Replace with the site UUID, which you can find from the Site Dashboard URL or terminus site:info $sitename --field=id

rsync -rLvz --size-only --checksum --ipv4 --progress -a --delete -e 'ssh -p 2222' empty_folder/ --temp-dir=~/tmp/ $ENV.$SITE@appserver.$ENV.$SITE.drush.in:files/remote_folder_to_empty

Now you can use rmdir over SFTP to remove the empty directory itself.

Known Issues

Uploading a large number of files into a multi-container Live environment may fail silently. Follow the steps below if you're uploading many files, and your Live environment has multiple application containers.

  1. Upload to an environment other than Live (for example, Dev).

  2. Use the clone operation in the Dashboard or Terminus to move the files to Live.

More Resources