How to: Migrate website files over SSH with RSYNC

What this will cover

This will cover a useful method of migrating website files between two machines using the command line interface, RSYNC (Remote Sync) the command we will use is a versatile command with great efficiency when transferring and syncing files which makes it fantastic too for website/file migration, backing up sites and can be useful for initially setting up dev, staging and production environments for website development projects.

Requirements

• Command-line tool – For this, I use a handy tool called Termius this software allows you to create and save multiple server shortcuts just like you would for FTP connections in software like FileZilla. Alternatively, you could use the terminal software native to your operating system, such as Terminal on macOS or a 3rd party app.
• Users on both servers with sudo or root access to run the commands (might not be required for all the commands)
• Server SSH access enabled – Make sure that both of the servers you’re using both have SSH enabled, sometimes hosting providers might restrict access so you may need to raise a support ticket for access.
• Rsync installed on the source server, some servers come with this preinstalled you can check this by running rsync –version if it’s not installed, you can install it using: sudo apt-get install rsync
• Access to a DNS control panel – Make sure that you have access to the DNS control panel for your domain, in this example the DNS for the domain used is hosted in Cloudflare.

What commands will be used

SSH – This will allow us to create a secure connection to the server you intend to do work on, to use this command you will need a username, IP address, and password when prompted.

LL (LS -L) – This is a very useful command which allows you to list all files and folders with their permissions and owners/groups in the current working directory.

CD – This command will allow you to change the working directory you’re in so that you can navigate to the files and folders you wish to migrate.

PWD – This command will print the path of the current working directory you’re in for example it will print /var/www/site1/ in the terminal if you’re in the folder site1.

RSYNC – This command along with some useful options will allow you to sync files from one server to another and will also move any newer files if it’s run again.

CHOWN – This command allows you to change the owner and group of files and folders within a directory, this might be used to ensure the website runs correctly after migration.

For further reading if you’re interested in the more detailed explanation of these commands check explainshell out, this site will explain the command and its options and you can also enter commands into the search for it to explain what the command will do which is very useful!

Step by step guide

Step 1 – Preparation

Collect all the login information you need; this will include both server IP addresses, usernames, and also passwords.

Step 2 – SSH Access

Make sure you can successfully connect via SSH using the information collected in step 1, you can create an ssh connection using the following command ssh username@ipaddress this will try to connect and ask for a password if successful you should be in terminal of the server, you might see a welcome type message at the top.

If this doesn’t work you might need to do some troubleshooting, Cpanel will automatically allow SSH access but in Plesk it doesn’t, you will need to go to FTP Access > click the username > change the option in the ‘Access to the server over SSH’ dropdown to ‘/bin/bash’ apply the settings and try again in the command line, if this doesn’t work check your credentials are correct, if it still doesn’t work and the credentials are correct you may need to raise a support ticket requesting SSH access with your hosting provider because the server might have firewall rules in place to block incoming connections.

Step 3 – Finding the correct folder paths

Now we need to work out the paths of where the files are currently stored and where they will be migrated to, for this we need to use a combination of commands LL (LS -S) which lists files and directories, CD which changes directory and PWD which will print the working directory.

First list the contents of the directory you log into using LL or LS -L, on Cpanel it’s generally /home/accountname/ and on plesk it’s generally /var/www/vhosts/accountname this depends on the user you log in with, root user might be different and you would have to change directory to the similar directories as listed above.

Once you’re in the correct directory use the PWD command to print the directory and make a copy of it so you can construct the RSYNC command later.

After doing this there should be two paths ready to be used in the RSYNC command and should look similar to these:

Step 4 – Using Rsync to perform file transfer

With the all the relevant information you can now create the rsync command that you will run on the source server, this will attempt to connect to the destination where the files are to be transferred to so you will have to provide the password for the connection, once running the command will print out the files that are being transferred in the terminal and once this is complete it all files will be on the server you intended to migrate them to.

rsync -avh /home/accountname/public_html/ username@127.0.0.1:/var/www/vhosts/accountname/plesksite.domainname.com

For this example, I will be recursively syncing everything in the public_html directory into the plesksite.domainname.com directory on the destination server the root directory on the destination server will be:

/var/www/vhosts/accountname/plesksite.domainname.com/

Step 5 (Optional) – Using CHOWN command to change ownership

You may also need to update the owner/group recursively throughout the document root if you experience any issues when performing certain tasks such as installing plugins or uploading images. To do this you will need to use the CHOWN command and set the user and group used for directory and file ownership on the destination server

chown -R user:group /var/www/vhosts/accountname/plesksite.domainname.com/

Step 6 Changing DNS to the destination server

Once the files have been transferred, to finish the migration you would need to change the DNS record from the old server IP to the new server IP, you would need to log into your Domain control panel or DNS provider to do this, after it has been changed it could take up to 24 hours to full propagate. TIP: Using services like Cloudflare usually propagates pretty quick so it might be worth setting this up.

Pros/Cons

Pros

1. Efficient
   Compared to alternative methods of migration this takes out some of the processes and bottlenecks, such as archiving the site, downloading it locally, and uploading it and then extracting it as it transfers directly to another server with end to end compression and decompression with correct permissions/ownership. Another area where this method is efficient is with storage space, other conventional ways would require storage on the server to store the zip file and if it’s moving because of lack of storage you’d need to find an alternative method!
2. Minimal Downtime
   This method allows you to be prepared well in advance of a switchover, you could move the bulk of a large website then a week later resync to pick up any new or modified files ready to be switched live in a given timeframe.
3. Versatile

One of the options with rsync allows you to exclude files, in the past, I have started a rsync command when its got to a large file and it would appear to have paused but really it was still transferring, I was able to stop the transfer, exclude the file or files then continue from where I left off without having to delete what had been transferred and start again the same thing would apply if you lost connection to one or both servers meaning if you were in the middle of a 10gb transfer and the connection dropped out you could continue and not lose data.

Cons

1. Adapting
   At first, without command-line experience, this method didn’t seem right to me as I’ve always been used to performing tasks like this using a GUI with software such as FileZilla, and transitioning away from the GUI to command line was difficult.
2. Setup and Access
   Like any hardware and software combination, there’s not a standardised setup for how a server is set up which means the initial setup using RSYNC with SSH might not be available out of the box, so some manual setup/configuring might be required same goes for SSH access, in this example both servers were setup using Digital Ocean and Cpanel had SSH access for the website account but Plesk required some extra configuration.
3. Trial and error
   Getting this right the first time I’d say would be an achievement! Especially if you’re used to using FTP where you can see exactly which files and folders you’re transferring and where you intend them to be stored because it’s like dragging and dropping files between folders on our machines. So there’s a bit of trial and error to perfect it and for the files to end up where you intend.

Alternative method

SCP (Secure Copy) is an alternative way of transferring files using the command line both locally and between multiple machines, this method works similar to RSYNC but the difference is SCP doesn’t have the ability to check what has been updated and only copy the relevant files, as a result, it will just create a copy of the entire directory regardless of changes so it would be less efficient. This way is probably best suited for single files such as databases or files that are zipped ready to be transferred.

TL;DR

For those that are just here for the commands and not the explanation here’s the TL;DR version.

1. Gather credentials and ensure SSH access to both servers is possible by using the SSH command: ssh username@ipaddress
2. Login into the source server and use the ll or ls -l command to list directories then change directory into the root directory using the cd command and then print the path using the pwd command.
3. Repeat step 2 on the destination server.
4. Enter the Rsync command on the source server using the credentials and paths gathered in steps 1, 2, and 3. It should look something like: rsync -avh /home/accountname/public_html/ username@127.0.0.1:/var/www/vhosts/accountname/plesksite.domainname.comTIP: Adding a trailing slash on the source root directory will omit the public_html folder but recursively copy its contents
5. Press enter and allow time for the transfer to complete