Agile web app deployment with SVN, rsynch, Phing and more
We are a very small development team, handling multiple tasks throughout the day, across different environments and projects. It makes for an exciting challenge. Our roles/ skills are clearly defined and our small size means we can react quickly to change, with little margin for communication errors and so on (see “Less is more: Jumpstarting Productivity with Small Teams). But, when it comes to deploying to a staging or production server, I am ashamed to admit that we often introduce a huge potential for human error by doing much of the work manually. Each project is so different and so there is no consistent deployment process across the board. With more agile development projects, where deployments may be required several times a day, we obviously cannot afford the time nor the potential mistakes that may be made by manually deploying a project.
At its simplest level, deployment may just involve making sure that the target server (i.e. staging or production) has the latest revision of files from the project’s source control repository (we use SVN). There are several ways to achieve even this simple step. You could run a Subversion client on the target server and checkout a tagged release or just the latest version of the source code. Or you could export the source code and upload it to the target server. But imagine doing this several times a day, then factor in database changes and so on - you’ll quickly go mad! You need to automate as many steps in the process as possible, thereby eliminating any manual tasks that introduce potential error and duplication.
Over the last year or so, we’ve been working with symfony - an excellent PHP framework - that includes a few tools to help simplify the deployment process (Phing, Pake and Rsync) . These tools allow for remote syncronisation of files via SSH (with the shell command: e.g. symfony sync staging) and semi-automate the build process (including database schema changes, data dumps etc: e.g. symfony propel-build-all myApp) by using Pake on the target machine. But there is still a lot of room for improvement, particularly when it comes to database migrations and SVN integration. Plus, symfony is not our only set-up. We need an automated deployment process that can be applied across all our projects. This is where I’ve previously seen shell scripts being used but Phing promises to do much of the work for us, as it has many standard deplyment tasks built-in to it. According to the Phing blurb:
Features include file transformations (e.g. token replacement, XSLT transformation, Smarty template transformations), file system operations, interactive build support, SQL execution, CVS operations, tools for creating PEAR packages, and much more.
Deployment is often the last step in the process to get any attention and it is one that can be unnecessarily tedious and prone to error. Here, I’ve outlined the steps required to implement a simple automated deployment process (for both releases and updates) that builds on the tools we use at Harvest and combines them so that they can be applied to a variety of different projects (mainly for our own sanity but if it helps you - great):
Tagged releases in source control
What constitutes a “release”? Whenever we reach a point in a project where the code can be considered stable, we should tag it in SVN as a release. This gives you the ability to deploy a particular release based on its tag, instead of just the latest revision in the repository. By convention you never create revisions in the tag folder - tags are simply named snapshots of the source repository. We typically use version numbers as tags, e.g. ‘v3′ or some scheme based on feature-set or milestone and release date that can be ordered alpha-numerically. This allows us to say with certainty what is deployed at any one time, without having to scan logs etc. Tags give context to a release name (you can easily associate the release with project milestones) that revision numbers and timestamps do not. Tags are usually kept in a ‘tags’ subfolder, in the top level of the repository.
File synchronisation with rsynch
The benefits of synchronisation over standard FTP are much touted elsewhere. Suffice to say - we should do it. Whether we synch with a tagged release or our working trunk will depend on the nature of the project. Very often with more agile projects, this will be against the trunk in SVN but for a stable release deployment we should create a tag in SVN and synch against that.
Rsynch is a unix tool that can be made available to a windows environment in one of the following ways:
- Via Cygwin with openssh and rsync
- By installing cwRsync - available from sourceforge
- By using standalone binaries (not recommended - outdated)
- By simply dropping the contents of this zip into your windows32 directory (not recommended - potentially insecure)
Database deltas
Automating tasks with Phing
Phing (PHing Is Not GNU make) is a project build system based on Apache Ant and is available through the PEAR installer. If you use --alldeps on Phing, it’ll grab, funnily enough, all the dependencies. But most are good packages (e.g. PHPUnit, PHPdoc and VersionControl_SVN). It can be used in a multitude of ways, including unit testing, creating docs, running SQL - which allows us to keep database schema changes in version control. But that’s getting ahead of ourselves. Let’s start with something simple first. Many of the steps below can be achieved in isolation but the beauty of Phing is everything can be configured easily through the build.xml file (and then further with properties files) - so that the project’s deployment configuration can be kept in version control too.