5 Ways to Synchronise WordPress Uploads Across Environments

When building WordPress sites for clients I generally try to use different environments for development, testing and deployment to live. This means that there are three areas of the site that need to be synchronised across environments; the code, the database and any media uploads for the site.

Managing code deployments can be a smooth operation with the help of an IDE or deployments based on a Git repository. Keeping the database in sync is extremely easy with a plugin like WP Migrate DB Pro, which allows you to push or pull databases across environments. However, media uploads can prove more of a problem.

Consider this scenario. A site relies heavily on user uploaded images or other media and has been out in production for some time. Your client asks for some amendments that include front end design changes. You fire up the development site. At this point the code should be inline with production. However, the database will not be. So you use WP Migrate DB Pro to ‘pull’ the remote production database, which replaces the local database. You view your local site and you see a whole bunch of broken links to images that are in your live site’s wp-content/uploads/ folder but of course are not in the local site’s.

The solution? Well here are five different ways to solve this problem, with varying degrees of time, hassle and elegancy.

1. FTP

I have to admit I did this recently. It takes time and it is not the best option presented here, but it works. Using your favourite FTP client, connect to the live site and download the content’s of wp-content/uploads/ to your local install. Do not overwrite existing to save a bit of time. Then grab yourself a cup of tea as, depending on the site, this could take some time.

2. rsync

I was recently introduced to rsync to help copy only the missing images between environments. Wikipedia describes it well:

rsync is a utility software and network protocol for Unix-like systems (with a port to Microsoft Windows) that synchronizes files and directories from one location to another while minimizing data transfer by using delta encoding when appropriate.

rysnc needs to be run in the command line and connecting to your remote site needs to be with SSH.

First of all navigate to your local WordPress site:
cd wordpress-site

Move to your uploads folder:
cd wp-content/uploads

To perform the synchronisation (replacing the SSH credentials):
rsync -avz --rsh=ssh user@host:/path/to/site/wp-content/uploads/* .

rsync is a lot quicker as it uses an SSH connection and is a very powerful tool. Read more about the different options and configuration here.

3. Media Addon for Migrate DB Pro

WP Migrate DB Pro already does a great job of handling database migrations, but as of version 1.3 they introduced the Media Addon which:

In a nutshell, this update will allow you to pull down the media files from your remote WordPress installation into your local environment. To accomplish this we compare the attachment posts found within your remote database to the ones in your local database and only pull updated or missing media files.

This is a brilliant feature making synchronising media files now as easy as migrating databases. The add-on comes with the Developer and higher licenses of WP Migrate DB Pro.

4. .htaccess

The last two solutions are my favourites as they don’t require any form of copying of files across environments.

This involves adding an .htaccess file to the wp-content/uploads/ directory of your site with this code:

<IfModule mod_rewrite.c>
  RewriteEngine On

  RewriteBase /wp-content/uploads/
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^(.*) http://yourlivesite.com/wp-content/uploads/$1 [L,P]

</IfModule>

So for any file requested within wp-content/uploads/, that does not exist, it will serve the image from http://yourlivesite.com.

You may need to edit the RewriteBase to suit your site’s base path in your various environments.

5. WordPress Filters

This last solution is the equivalent of the .htaccess code but using core WordPress filters to serve the images. To make this work on any environment you need to add some extra config to your site’s wp-config.php:

Define your site’s url used by WordPress to be dynamic:

define('WP_SITEURL', 'http://' . $_SERVER['HTTP_HOST'] .
    str_replace(DIRECTORY_SEPARATOR, '/', str_replace(realpath($_SERVER['DOCUMENT_ROOT']), '', dirname(__FILE__))));

Define the url of your production site where most up to date media will be uploaded:

define('LIVE_SITEURL', 'http://yourlivesite.com');

Then in your theme’s functions.php, a plugin or better still a site-wide functions plugin, add:

add_action('init', 'my_replace_image_urls' );
function my_replace_image_urls() {
    if ( defined('WP_SITEURL') && defined('LIVE_SITEURL') ) {
        if ( WP_SITEURL != LIVE_SITEURL ){
            add_filter('wp_get_attachment_url', 'my_wp_get_attachment_url', 10, 2 );
        }
    }
}

function my_wp_get_attachment_url( $url, $post_id) {
    if ( $file = get_post_meta( $post_id, '_wp_attached_file', true) ) {
        if ( ($uploads = wp_upload_dir()) && false === $uploads['error'] ) {
            if ( file_exists( $uploads['basedir'] .'/'. $file ) ) {
                return $url;
            }
        }
    }
    return str_replace( WP_SITEURL, LIVE_SITEURL, $url );
}

This will work for uploaded media that is being served by WordPress’ standard image functions such as the_post_thumbnail() or wp_get_attachment_image().

The function my_wp_get_attachment_url() does the work, but will only be added if it is not the live site, as it will already have all the images.

The function then hooks into the WordPress filter wp_get_attachment_url, so for each time WordPress is serving an attachment url our function jumps in and checks if that attachment file exists in our site, and if not swaps out the site url with the live url.

Which synchronisation method do you use? How do you keep your WordPress environments aligned?

About Iain

I am a WordPress and PHP developer building my own plugins and working with Delicious Brains. I like to blog about things, especially WordPress.

  • Pingback: Synchronize WordPress uploads across environments : Post Status()

  • Pingback: WordPress links, March 24 2014, cost, plugins to avoid, backup plugins()

  • Pingback: The Best Way to Synchronize WordPress Uploads - ohryan.ca - A Web Developer in Winnipeg()

  • Thanks for this! I’ve been using method #4 and it works pretty well, but #5 is a good addition to my toolkit. Very clever technique 🙂

  • donkeyjong

    Interesting read! I’ve never worked on a project long enough to have the need to keep media in sync but they’re starting to appear in my workload so this came in the nick of time!

    What’s your favorite?

    • Currently it is #4, I use it a lot when a project is in its early stages, when the client is adding content/media to the staging site and I can keep my local in sync with it.

  • Pingback: WordPress links, Import/Export Widgets, Church of WordPress()

  • To be honest #4 and #5 don’t feel like “synchronization” to me, as it’s basically still connected to the place the files are. I don’t have them over at the local testinstallation – and (more important maybe) i got no backup of the files. So RSync or FTP are the way to go, even if it might take some time to get the files physically over to the second installation.

    Also: If I understand that right the uploads remain on the server, so upload works only on one side of the system (the live site in your examples).

    On the other side: very good solution if you do “only” designing work on the site.

    • Andy

      This is a very good thing if you need to make a few changes and the live site has a 2GB uploads directory. You don’t want to wait all morning to download a bunch of files. That’s where .htaccess and filter method comes in handy.

  • Christian Matthew

    I am confused… I am trying to do this function: I am using a load balanced server set with a Lamp setup on a WordPress site distribution. I have successfully place the database to a third standalone server and am looking to do the same with the “uploads” folder. Is there anyway to do this with the methods you are referring to? 4 looks awesome but I am unsure of what you are doing and why this would work and or solve my problem.

  • Christian Matthew

    found a solution… Unison… the very best way to get going… it is a master to master sync which can literally be done completely from one server no matter what happens on either side… Freaking awesome.

  • Jon Kensy

    I have a sort of different situation – I am using a load balancer in front of 3 web front ends for WordPress. I can put persistent connections on so I can log into /wp-admin and make posts, but when I install a plugin or add media the web frontends are out of sync from a file perspective. Database is good, since I run that on a remote MySQL server, but how can I sync web servers in a load balancer?

    • DevDonkey

      use a shared mounted storage location across all 3.

  • Pingback: Point Local WordPress Uploads at Live URL | Multiple States Knowledge Base()

  • Craig Berry

    Good roundup post. I just purchased the Media add-on for Migrate DB Pro, will be trying it out soon. I also just found out about ‘Uploads by Proxy’ (https://wordpress.org/plugins/uploads-by-proxy/) which appears to be a variation of #4 and #5 above – the local installation will pull media from the production site. Again, I haven’t tried it out yet, but it looks promising.

  • Ryan Taylor

    If you’re dealing with two wordpress installs, you can use

    define(‘UPLOADS’, “wp-content/other_uploads”);

    in your wp-config.php with a soft link from the command line

    ln -s /path/to/wordpress/source/uploads /path/to/local/new/folder/wp-content/other_uploads

    or alternatively just use the full path in wp-config.php

  • Love method 4. The only time it doesn’t work for me is if I add an image size and need to regenerate thumbnails. Any ideas how to get around that? I could always just add the image size on production and regenerate there, but ideally I’d be able to see how everything looks in development before deploying any of it to production.

    • At that point, might as well download all the images to dev and test there.

  • Pingback: Using Remote WordPress Uploads on nginx & vvv - Kevin McKernan()

  • Josh

    rsync all the way!