We first tried to use Jekyll, as it was lighter, and easier to integrate in a software development process. But we reverted to a WordPress based website in order to allow non technical members of the team to edit the website content, and also to benefit of the huge WordPress ecosystem, providing lots of themes, plugins and widgets.
WordPress is a very powerful tool to create nice, easy-to-edit websites. A static part of the site (for instance theme and plugins) is stored as files, and the dynamic part (pages, posts, user management) are stored in the database. Once our WordPress website is live, you might start wondering how to develop, test and deploy changes. Here are a few questions that we’ve asked ourselves
- How to backup my website and make sure I’ll be able to restore it if ever my server would crash?
- How to make changes on my website, working in a sandbox isolated from the production environment?
- How do I smoothly report those changes on the production environment after testing?
- How can I version the modifications made on my website in order to be able to revert back to a previous revision if something goes wrong?
A lot of blog posts or support questions about these concerns can be found on the web, but none were satisfying to us, so here is what we decided to setup.
1- Version WordPress files with GitHub
A WordPress website has a large part of file based content, so let’s apply the standard version control that we apply for every other technical project!
First, we have created a private repository on GitHub (whereas this contains no secret, we don’t think the content of our website is worth sharing with the community). As our website was already up and running at that time, we created an empty repo, and used git remote add origin command on the website file system to commit and push the current state.
We have used gitignore.io to create the appropriate .gitignore file. Let’s see the result:
# Created by https://www.gitignore.io ### WordPress ### *.log .htaccess sitemap.xml sitemap.xml.gz wp-config.php wp-content/advanced-cache.php wp-content/backup-db/ wp-content/backups/ wp-content/blogs.dir/ wp-content/cache/ wp-content/upgrade/ wp-content/uploads/ wp-content/wp-cache-config.php
First of all, we can see that wp-config is excluded from version control. We agree with that as we try as much as possible to not keep under version control files containing passwords (even in a private repo), and, moreover, the database connection info will much likely be different between a development environment and a production environment.
Then, we can see that some directories that may contain temporary data are also excluded (advanced-cache, cache, backup, upgrade,…).
Finally, we can see that the uploads directory is also excluded from source control. There may be very good reasons for that, as explained in Steve Grunwell’s blog:
- As soon as a client starts uploading media on production, your repo will be out of date
- By default, WordPress generates multiple copies of every uploaded image (thumbnail, medium, large). If all of those images are in your repo, everyone who clones it will be downloading every version of every image that’s been uploaded. That means more bandwidth, larger repos, and no real benefit
- Remember: version control !== site backup. Just like anything else, repos can be deleted, corrupted, or generally messed up.
Although all the reasons mentioned here are exact, we decided not to apply the advice. In our case, we need to be able to setup an exact copy of the production site on a local development environment, and make sure that the resources can be loaded in order to render the website exactly as the production version. Moreover, if our images are not versionned in Git, nor stored in the database that we will backup, then how do we guarantee that we’ll be able to restore everything if ever we had to restore from a backup? For all those reasons, we then decided to remove wp-content/uploads/ from our gitignore file.
2 – Create database backup
In order to restore the full live website on a development environment, you not only need the file structure, but you also need to copy the content of the database to get all pages, articles, and all the plugins data. But dumping the database on the production server, and restoring it on the local environment is not enough: you need to perform some search & replace to change the values that are hard coded in the DB, such as the website URL and the WordPress installation directory.
To make the backup simpler, we use WP Migrate DB plugin. It allows you to perform a SQL dump, and the appropriate search & replace in one single step.
To make the backup procedure even simpler, you can save migration profile. In our case, we have defined two profiles: one to migrate DB from production to a local environment for development, and the other one to create a backup of the database that could be restored in production in case of failure.
In the development backup profile, we took advantage of the search and replace feature to remove all Google Analytics tracking code from the local copy, so that we stop reporting what happens on the local website to Google Analytics.
3 – Prepare a local environment for development
Once the backup of the database has been done, you are ready to copy the website on your local development environment. First of all, you need to install all the prerequisites for WordPress on your machine. You can find the official wordpress list of requirements on the wordpress website, or follow one of the tutorials that can be found on the web, like https://www.digitalocean.com/community/tutorials/how-to-install-wordpress-on-ubuntu-14-04 for instance.
If you are working on Mac, we advise you to install MAMP and to configure the ports to standard values (80 for Apache, and 3306 for MySQL).
After installing the prerequisites, clone the Git repository to get the file structure. Note that this directory path must match the replace string used in WP Migrate DB. To simplify the process, you can also decide to install it at the same root path as on the production server. Then, change permissions on the wordpress directory to give full access to www-data user and group.
Then, create a wp-config.php file in wordpress directory containing your local database access details.
Finally, import the database dump that has been performed on the production website. To do this import, you can either use phpMyAdmin, or type the following command line:
mysql -uroot -p wordpress < wordpress-migrate.sql
That’s it! You’re all set to perform any modifications on your local copy of the website, without fearing to break anything on your production environment!
4 – Report changes to production
Once you have performed all the required modifications on your local copy of the website, and test that everything is ok, we need to report those changes to the production website. Most of the tutorials or articles we have found on the web pretty much explained the first 3 steps that we just went through. But none of them went through how to report the changes made locally to the production.
First of all, you will need to make sure that no modifications were applied on the production environment while you were working locally. To do so, simply run a git pull and do a new extract of your production DB to synchronise your local environment, and run new tests on your local copy to make sure that everything is ok.
Then, you can start pushing changes back. As the files are versioned with Git, applying the modifications is pretty straight forward: simply use git commit and git push on your local environment, and finally git pull back on the production environment.
The tricky part is about database modifications. We did not want to use WP Migrate DB in this case, because we found too risky to overwrite the whole content of the production database with a development copy. So, what we do, is we dump the database on the local development copy, using WP Migrate DB, but without specifying any substitution. You can then compare the exported SQL file with the file used to import the content of the DB with a diff tool, like Meld for instance if you are working on Ubuntu. If you find any modifications, you can report them on the production environment, either by doing the same modification via the WordPress dashboard on the production website, or, if you feel lucky, apply updates on the database via command line (this is not what we recommend!).
Note that if you have performed very big changes to the website, you could test applying changes to a staging environment before really pushing to production. A staging environment can be setup by following the exact same procedure as creating a development environment.
5 – Tag versions and backup
After releasing the changes to production, we perform a snasphot of the result for backup purpose. We do this by adding a tag on our Git repository, and create a database backup, in order to be able to revert to the current state later in the future in case of problem. For database backup, we use WP Migrate DB without any string substitution. The resulting SQL file name will be renamed with an indication of the version tag on the Git repo, and will be saved to an external backup server.
Most of the articles that we’ve found on the web were about how to duplicate a running website on a development or staging environment, but none of them resolved the matter of how to validate the changes made on a local environment to the production environment. We have managed to describe a development process for our WordPress website, and push back to production. Following this process, you can even introduce an intermediary integration environment, in order to test applying the changes on a server before pushing to production. Unfortunately, a quite important part of this process is still manual. In the next step, we will work on how we could automate pushing changes changes back to production, and automatically run tag and database backup on a regular basis, to make sure that we’ll be able to revert back to a as-recent-as-possible version of our live web site in case of crash. For doing so, we have already identified some tools that could be helpfull, such as WP CLI.