At Nucleus we have, besides a lot of servers in our data centers, a number of fairly complex software systems we develop ourselves. Besides the things everyone can see, like our control panel and order wizard, there are a number of internal systems that are constantly in action and being used by our employees.

Of course it’s of vital importance to us that these systems are bug free and stable, but also that updates can be made transparently and without downtime. That’s where Continuous integration and deployment spring into action. In this blog post we shed some light on these two techniques and how we apply certain aspects of it in our daily workflow at Nucleus.

What is continuous integration?

Continuous integration is a technique in which you submit the code of your software to a couple of tests such as Unittest suite or Integration tests with every update (or at other key moments). By writing tests for certain important classes or other pieces of code, you can increase the quality and robustness of your code and gain more trust amongst developers (developer confidence).

Continuous integration makes sure that the parts test were written for are tested at regular intervals so that they don’t break by updates on other parts. It is for example quite possible that because of updates to module A, module B stops working. Continuous integration won’t prevent this from happening, but will report it at the first sign of failure (someone broke the build), so it can be responded to quickly and the proper functioning of the vital systems can be guaranteed.

What is deployment?

In this case deployment refers to the automatic deploying of an update to certain systems. Deployment describes the process wherein the code that was written by the developer goes to an update of your website or project on your staging or production servers. Each of us has done some form of manual deployment before. Who hasn’t uploaded a full website directory using FTP before, only to have to re-upload everything again when the next update comes along? It’s one (good or bad) way of deploying. But we obviously want to use a safer and more robust method.

Why Bamboo?

Why do we use Bamboo? We already use various Atlassian tools, like Jira and Confluence, in our workflow and are very happy about them. From there, going to Bamboo is easy. We could have selected another solution, like Jenkins, Travis or PHPUnderControl, but we chose to make use of the integration between Bamaboo and Jira since they are both Atlassian tools.

Let’s build

Configuring a build in Bamboo starts with creating a build script. A build script is a script wherein you make a working product out of your code in an automated way and then upload it to your production/staging server(s).

A Build script usually consists of 2 or more steps (or targets), in our case that’s testing and deployment. Each of these steps consists of a number of smaller steps that each have their own purpose.

In the testing phase all test suites are run and in the deploy phase all steps are taken to move your system to the servers you appointed and are made ready for use.

There are many systems available for making build scripts. A few popular solutions are e.g. Apache, Ant, Phing and Capistrano. We choseApache Ant, which finds its origin in Java. Phing is a PHP alternative which is based on Ant when it comes to syntax (xml) and the way it is built up, so it’s a good alternative if you’d rather not go with a Java based solution.

Testing

A robust software product always contains a couple of Unit tests for testing vital parts and a couple of User Acceptance tests that test the user experience (on e.g. failing JavaScript functionality). These tests can easily be executed via the command line. In an Ant script this is the Exec command, it allows you to easily run CLI scripts like PHPUnit:

<target name="phpunit">
    <exec dir="tests" executable="/usr/bin/phpunit" failonerror="true">
        <arg line="--log-junit ${config.builddir}/phpunit/log/phpunit.xml --coverage-html ${config.builddir}/phpunit/coverage" />
    </exec>
</target>

Mostly the –log-junit parameter is important with regard to Bamboo integration, because it is needed in Bamboo to evaluate the results of your Unittests. Besides that we also generate a Code Coverage report which can be found in the Artifacts of the Bamboo build later on. Using those we can evaluate how much of our code is encompassed by the PHPUnit tests.

Besides the previously mentioned test suites, a couple of Quality Assurance Tools are run in this step, such as php copy/paste detector (phpcpd), PHP Mess Detector, PHP_Depend and PHP CodeSniffer. The results of which can be found with every build. This way you can easily judge the quality of a build after it’s been run.

Deploying

Turning your codebase into a working product for PHP projects isn’t too difficult. There are no complex compilations or modifications necessary. Usually deploying involves the following steps:

1. Making a copy of the code you want to deploy.

Making a copy of the code you want to deploy is quite simple if you use git as Version Control System (VCS). For our project that’s a simple matter of one git command:

    git clone user[at]foobar[dot]git -b develop ./deploy

This command takes a clone of the develop branch of the foobar project to the deploy directory. Now you have all the code you need to starting deploying.

2. Installing dependencies

Installing dependencies is very easy nowadays when you PHP Composer.

    php composer.phar install --no-dev --optimize-autoloader

3. Execute database migrations

Those that think about the setup of their project and what libraries to use in advance, can save a lot of time with this step. We use Doctrine as Database Abstraction Layer and ORM. Because of this we can use the Doctrine Migrations.

By default Doctrine offers a command-line script to manage your database schema based on the entities you’ve defined in the ORM layer. It also offers a specific migration library to execute more advanced migrations.

But be careful using the ORM library command-line script as using it wrong can lead to data-loss, because the library does not take the presence of data in the tables into account. It only manages the database schema. It can very well be that out of the blue a column is removed from a table. That’s why you should always run your migrations on a testing or staging server before unleashing them on the production server.

4. Clearing the cache and warming it.

With updates that render certain parts of the cache invalid it is important that you (partially) empty the cache and reload it. This is often a manual process. We use a self-written command-line script to do this. Depending on the framework you use this can be done more easily.

5. Creating a backup of the previous deploy on the target server.

This step is not necessary. Much depends on how the next step is executed, but it is advised to NEVER remove the current production code when deploying. Make sure you always retain the previous version of your production code. In the next it’ll become clear that this can be done quite easily.

6. Copy everything to the desired server (staging, production, …)

Uploading everything to production can be done in different ways. You can e.g. use the Ant FTP command to upload everything to the server.SSH or a custom Rsync command are better alternatives.

To facilitate deploys and rollbacks we use a symlink strategy. This means that when we execute a deploy we don’t remove or overwrite the current deploy on the server but use symlinks to make the server use the correct version.

Step two is twofold: The first part is uploading the new version to a location on the server where all versions are stored. Next the symlink that refers to the previous version is altered so that it refers to the new uploaded version.

deploy_structure

This way you can easily roll back to a previous version (refer the symlink to v1.3) and you have a backup of the codebase.

In our case we have a shared directory where files like uploads, caches and logs are stored. This directory is also symlinked to the shared directory in the current map.

/var/www/versions/
/var/www/versions/1.1
/var/www/versions/1.2
/var/www/versions/1.3
/var/www/current -> versions/1.3
/var/www/shared -> current/shared

That’s how deploys are done at Nucleus. If you’d like more information on this, on deployments and lifecycles or devops in general, let’s get together and talk about it!

Related posts
Nucleus - Laravel

What is the best way to host Laravel?

What is the best way to host Laravel? By combining Laravel, Forge and user-friendly deployments with managed hosting. Find out why!

Read more

Spectre en Meltdown

What impact do Spectre and Meltdown really have?

What impact do Spectre and Meltdown really have? Was this one of the most dangerous leaks? And how about loss of performance?

Read more

Blog Uptime

How do I improve my uptime: a step-by-step plan

Uptime has become crucial in our “always on, always connected” society. I already wrote about the impact of downtime in an earlier blog post. But per permanent uptime, which is considered a given by end users, can have a serious impact on a company.

Read more