We're getting closer and closer to full test automation and continuous integration. The last article introduced you to how to write PHPUnit tests for SeleniumRC so that we can test website frontends through PHPUnit. After that we wrote our own phing build script that ran our backend as well as the frontend tests automatically, generated an code coverage report based on Xdebug as well as an comprehensive APIDoc based on PHPDoc comments.

The phing build script we used for this - test.xml - was executed manually by calling

phing -f test.xml

on the command line. Everything happened on the development virtual host.

The next logical step is to automate this and perform the additional tasks required to run continuous integration.

The first step is to do a manual svn checkout on the integration virtual host, so we have setup that is identical to the development virtual host.

Now, what the automation will have to do is a Subversion update of the integration virtual host in order to retrieve the latest copy. If the revision changes in that process (that is, if we indeed fetched newer files from Subversion) we again run the tests and generate the reports. If all tests succeed we can safely assume that we have a running system, and we can decide to create a tarball from it or update the staging system for our project manager / client to check it out. On the other hand, if a test fails we'd like to be notified by email.

This is exactly what a second phing build script will do.

Introducing more flexibility

Before we can do all that, we need to make a small but very important change to the Selenium RC test case. The setUp() method of out tests/TestForm.php file configures SeleniumRC so that it points to http://integration.local which is our development environment. This means that no matter where we run the frontend tests, this test would always test our development environment! Clearly not what we want.

If the developer runs the test manually (by executing phing -f test.xml on the commandline), the development environment has to be used. For the automation, the integration host http://integrate.integration.local has to be used.

We accomplish this by changing the setUp() method to this:

function setUp() {
  // determine hostname based on phing property
  $startDir = Phing::getProperty('application.startdir');
  preg_match('/\/(default|integration|demo)/', $startDir, $vhostMainDir);

  switch($vhostMainDir[1]) {
    case 'default':
      $browserUrl = 'http://integration.local/';
      break;

    case 'integration':
      $browserUrl = 'http://integrate.integration.local/';
      break;

    case 'demo':
      $browserUrl = 'http://demo.integration.local/';
      break;

    default:
  } // end: switch

  $this->screenshotUrl    = $browserUrl . 'reports/selenium';
  $this->screenshotPath    = $_ENV['PWD'] . '/../reports/selenium';
  $this->setBrowser('*firefox');
  $this->setBrowserUrl($browserUrl);
} // end: function setUp()

We retrieve the application.startdir property and with this information we can decide in which environment we run to set the browser URL and the screenshot path correctly.

Committing and retrieve everything to Subversion.

With that flexibility issue in the test out of the way we can commit the whole development environment /var/www/vhosts/default to Subversion:

svn import /var/www/vhosts/default https://your.subversion.server/repository/path/trunk
svn checkout --force https://your.subversion.server/repository/path/trunk /var/www/vhosts/default

Your development environment now is connected to the subversion repository.

Next, you have to do another svn checkout from the same repository URL, this time to the integration environment.

svn checkout --force https://your.subversion.server/repository/path/trunk /var/www/vhosts/integration

Now the integration environment is identical to the development environment.

Note that the virtual host configuration file for apache will not be overwritten, since the filenames are different. However the integration environment now contains two .conf files for apache. Since only the correct one is linked to /etc/apache2/sites-available, no harm is done though.

The continuous integration build script.

The single addition between our manual tests from the test.xml phing build file and the continuous integration is another phing build file that you will find at buildScripts/updateAndBuild.xml. If you open it in an editor it will look like this.

<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="runTests" name="updateWorkingCopy">
  <property name="demoDir"      value="/var/www/vhosts/demo" />
  <property name="baseDir"      value="/var/www/vhosts/integration" />
  <property name="htdocsDir"    value="${baseDir}/htdocs" />
  <property name="testDir"      value="${baseDir}/tests/" />
  <property name="reportDir"    value="${htdocsDir}/reports" />
  <property name="seleniumDir"  value="${reportDir}/selenium" />
  <property name="masterFile"   value="updateAndBuild" />
  <property name="buildVersion" value="1.6.1" />
  <property name="srcDir"       value="/var/www/vhosts/integration/htdocs" />
  <property name="svnUser"      value="your_svn_user" />
  <property name="svnPass"      value="your_svn_password" />

  <!--fetch current revision of working copy -->
  <target name="getOriginalRevision">
    <svnlastrevision workingcopy="${baseDir}" propertyname="svnOriginalRevision" />
    <echo msg="Current revision of this working copy: ${svnOriginalRevision}" />
  </target>

  <!-- update local working copy -->
  <target name="svnUpdate">
    <svnupdate username="${svnUser}" password="${svnPass}" nocache="true" todir="${baseDir}" />
  </target>

  <!-- fetch revision of working copy again -->
  <target name="getCurrentRevision" depends="svnUpdate">
    <svnlastrevision workingcopy="${baseDir}" propertyname="svnCurrentRevision" />
    <echo msg="Revision of this working copy after svn update: ${svnCurrentRevision}" />
  </target>

  <!-- create the tarball from the integration VHost-->
  <target name="tar">
    <echo msg="Creating archive..." />
    <tar destfile="${baseDir}/builds/build-${buildVersion}.r${svnCurrentRevision}.tar.gz" compression="gzip">
      <fileset dir="${htdocsDir}">
        <include name="**" />
        <exclude name="reports/**" />
      </fileset>
    </tar>
    <echo msg="Build copied and compressed into directory!" />
  </target>

  <!-- updates the demo VHost after all tests have succeeded -->
  <target name="updateDemo">
    <!-- change baseDir to demoDir -->
    <property name="baseDir" value="${demoDir}" override="true" />
    <phingcall target="svnUpdate" />
  </target>

  <!-- compare original and current working copy revisions -->
  <target name="runTests"  depends="getOriginalRevision,getCurrentRevision">
    <if>
      <equals arg1="${svnOriginalRevision}" arg2="${svnCurrentRevision}" />
      <then>
        <echo msg="Working copy is up to date." />
      </then>
      <else>
        <echo msg="Updated working copy from SVN, starting integration tests." />
        <phing phingfile="test.xml" haltonfailure="true" />
        <phingcall target="tar" />
        <phingcall target="updateDemo" />
      </else>
    </if>
  </target>
</project>

Let's go through this script step by step.

1. The target <getOriginalRevision> gets called first.
   Thankfully phing provides a ready-made task to retrieve the Subversion revision of a given path. We store that revision in the property ${svnOriginalRevision}.
2. Next, the <svnUpdate> target performs a Subversion update on the whole integration environment, ensuring we get any updates that have been committed since the last run. Since this environment will never be edited there will never be any merge conflicts.
3. Then the< svnCurrentRevision> target fetches the Subversion revision number of the integration environment again and stores it in a property, this time in ${svnCurrentRevision}. Note that this target is identical to <getOriginalRevision>, except for the property used.
4. After we have ensured that the integration environment is up to date, the main target <runTests> compares the ${svnOriginalRevision} with ${svnCurrentRevision} to check if a newer version has arrived. If not, this continuous integration loop stops. No need to test again what was already tested.
5. If the execution did not stop - that is, if a newer source code version has arrived - we use the <phing> task to execute the test.xml build file we introduced in the third part of this series.
   The buildAndUpdate.xml uses the same properties as the test.xml and because of that the latter will now run in the integration environment.
6. In case the test.xml script runs through without error a tarball is created automatically, containing everything in the htdocs directory.
7. Last not least we perform a SVN update on the demo environment.

Note that the test.xml build file will now send an email if one of the tests fails!

This is due to the fact that the updateAndBuild.xml sets the ${masterFile} property to a value that causes an <if> task in test.xml (line 112) to pass. This way the developers will automatically be alerted if the current revision in the integration environment is broken. If the test.xml is executed manually the if-condition will not apply and no mail is sent.

Testing the phing script.

You can direct your SSH shell to /var/www/vhosts/integration and execute

phing -f buildScripts/updateAndBuild.xml

but this will not do much good since the integration environment is up to date.

What I would recommend is to create a new file in the development environment /var/www/vhosts/default and fill it with some arbitrary text, only to add and commit that file to Subversion. Then return to the integration environment and run the same command.

Voila! The phing script updated the integration environment, recognized the update and started all tests (this time in the integration environment). Once more, a reports folder has been created (again in the integration environment) and filled. A new folder /var/www/vhosts/integration/builds will have been created, containing a .tar.gz file with the latest build.

Note that this folder may get huge very fast, depending on project complexity. It is probably advisable to deactivate the automatic tarball generation task for a real life environment.

If you add an error to one of the tests, you should get an email. Double check the ${errorMail} property in test.xml and make sure the server can successfully sent mails if you don't.

The last step for you to do is to set up a cronjob to run the phing command periodically. Make sure the cron environment is forwarding the X server display and otherwise is as close as your shell environment as possible, because otherwise you will run into all sorts of issues. This is the nature of cron...

This concludes my tutorial on continuous integration for PHP.

Leave me a comment if you liked the series, if you have questions or suggestions for improvements. I will work on a fifth and last article about my own thoughts for possible improvements, to be published in a bit.

In this article I want to introduce you to the way I run the continuous integration tests, what to run when, and where. If you're unsure how to write unittests, this will not be covered in this article but phpunit.de has an excellent documentation section just covering how to write tests. Another section is there covering how to write tests for SeleniumRC.

Let me start this article with a short summary of what we have installed in the second part: We have a pretty standard LAMP server based on Ubuntu Linux. Then there is PHPUnit for unit and regression testing as well as phing, our integration server. We added SeleniumRC as a frontend test tool, then Xvfb and Firefox to facilitate the frontend tests. xdebug is used by PHPUnit for code coverage reports and phpDocumentor will be used by phing for automatic APIdoc generation.

A Subversion client to fetch latest updates out of the repository completes the setup.

Ok, let's dig a little deeper into the filesystem structure and how to use it, shall we?

We installed three virtual hosts in apache, one for development, one for the integration and one as staging system.

My continuous integration environment

The whole setup is merely to demonstrate the workflow. In reality the integration and staging environment would most likely run on a separate machine, or two.

Each virtual host contains the same initial directory structure, like you see on the left. The image shows the structure of the default server (used for development).

You see the ./conf folder, which stores the Apache configuration file. You can add your own configuration files to this folder if you like.

The ./htdocs folder is what you think it is, the webserver root.

The ./tests and ./buildScripts folders will contain the test classes, as well as all files required by phing to build the project.

This layout is simple, yet effective. It cleanly separates the utilities used for testing and continuous integration from the rest of the project, thus making packaging and deployment comparatively easy.

In the screenshot you also see that the whole default virtual host is under revision control. Experience shows that in the end, you will want everything versioned no matter what you think initially.

Last not least the screenshot shows the other two virtual hosts 'integration' and 'demo', both being under version control as well - but more on that later.

Ok, are you ready to write some tests?

Some of the scripts discussed below are quite long, so I will not post them here in the blog in their full length. You can get a continuous integration test file setup (.zip file) for ease of use.

We start with a simple file which we put in default/htdocs/class.hello.php:

<?php
 class HelloWorld {
   public function hello() {
     return 'Hello World';
   } // end: function hello()
 } // end: class HelloWorld
?>

As you see the class HelloWorld contains a single method 'hello' without any parameters that returns the famous 'Hello World'. The corresponding unittest is equaly simple. We create a second file in default/tests/Test.HelloWorld.php:

<?php
require_once('PHPUnit/Framework.php');
include_once('../htdocs/class.hello.php');

class TestHelloWorld extends PHPUnit_Framework_TestCase {
  private $ciTestObj = null;

  public function setUp() {
    $this->ciTestObj = new HelloWorld();
  } // end: function setUp()

  public function testHello() {
    $this->assertEquals($this->ciTestObj->hello(), 'Hello World');
  } // end: function testHello()
} // end: class TestHelloWorld
?>

Now, the property $ciTestObj contains an instance of the test object which is our HelloWorld class from the first script. The only test we perform is an assertEquals() call that compares the return value of the HelloWorld::hello() method with the static string 'Hello World'.

Save the file and connect to the continuous integration server using your favorite SSH client. Change into the tests directory and execute phpunit:

phpunit Test.HelloWorld.php

You should get a result similar to this:

PHPUnit 3.4.2 by Sebastian Bergmann.
.
Time: 1 second
OK (1 test, 1 assertion)

Hooray, the test passed. Our class behaves as expected.

The second step will be to run a frontend test in SeleniumRC.

Usually you would create a test in Firefox with SeleniumIDE set to write PHPUnit test files instead of the html default. However for a quick success I recommend copy-pasting the test that I prepared.

Before we attempt to run SeleniumRC in the continuous integration server, configure your SSH client to allow X forwarding. If it is currently inactive, enable it. The test won't run without that. Double check your settings are correct.

Ok, once more we start with our test subject. This time it is a very simple html page with a form. The form doesn't do anything except filling the form fields with the values you entered before submit. Create a file default/htdocs/form.php with the following content:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
<head>
  <title>Testable form</title>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  <style type="text/css">
    body {background-color: #fff;}
    label {
      display: block;
      float: left;
      min-width: 140px;
      padding-right: 5px;
    }
    input,textarea {
      border: 1px inset;
      margin: 5px 0;
    }
    textarea {height: 150px;}
    input[type=submit] {border: 1px outset;}
   .long {width: 300px;}
  </style>
</head>
<body>
  <h2>Selenium RC test form</h2>
  <p>
    This form does have absolutelly no purpose other than being a test subject for selenium RC.
  </p>
  <form action="<?php echo $PHP_SELF; ?>" method="post">
    <label for="firstname">firstname, lastname: *</label><input type="text" name="firstname" value="<?php echo $_POST['firstname']; ?>" /> <input type="text" name="lastname" value="<?php echo $_POST['lastname']; ?>" /><br />
    <label for="email">email: *</label><input type="text" name="email" value="<?php echo $_POST['email']; ?>" /><br />
    <label for="url">homepage:</label><input type="text" name="url" value="<?php echo $_POST['url']; ?>" /><br />
    <label for="subject">subject: *</label><input type="text" name="subject" value="<?php echo $_POST['subject']; ?>" /><br />
    <label for="comment">your message: *</label><textarea name="comment"><?php echo $_POST['comment']; ?></textarea><br />
    <label for="">&nbsp;</label><input type="submit" value="submit" />
  </form>
</body>
</html>

Open it in your browser, just to confirm that it is working. If you followed this howto step by step, http://integration.local/form.php should work just fine. You should do the same from the command line on the server to see if local access is also correct:

lynx http://integration.local/form.php

If that is not working, go back to the second part of this series and check the settings in the hosts files. However if it does, we can write our test for the subject. Open a new file in default/tests/TestForm.php and paste this:

<?php
require_once('PHPUnit/Extensions/SeleniumTestCase.php');

class TestForm extends PHPUnit_Extensions_SeleniumTestCase {
  protected $captureScreenshotOnFailure = TRUE;
  protected $screenshotUrl = 'http://integration.local/reports/selenium';

  function setUp() {
    $this->screenshotPath = $_ENV['PWD'] . '/../reports/selenium';
    $this->setBrowser('*firefox');
    $this->setBrowserUrl('http://integration.local/');
  } // end: function setUp()

  function testFormSubmit() {
    $this->open('/form.php');
    $this->waitForPageToLoad('30000');
    $this->type('firstname', 'SeleniumRC');
    $this->type('lastname', 'Testautomation');
    $this->type('email', 'selenium@example.com');
    $this->type('url', 'http://seleniumhq.com');
    $this->type('subject', 'Test for Selenium RC');
    $this->type('comment', 'This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.');

    $this->click('//input[@value="submit"]');
    $this->waitForPageToLoad('30000');
    $this->assertEquals('SeleniumRC', $this->getValue('firstname'));
    $this->assertEquals('Testautomation', $this->getValue('lastname'));
    $this->assertEquals('selenium@example.com', $this->getValue('email'));
    $this->assertEquals('http://seleniumhq.com', $this->getValue('url'));
    $this->assertEquals('Test for Selenium RC', $this->getValue('subject'));
    $this->assertEquals('This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.', $this->getValue('comment'));
  } // end: function testFormSubmit()
} //end: class TestForm
?>

Note that before 30th of November 2009 the above code contained the wrong hostnames for both TestFor::screenShotUrl and the setBrowserUrl() call. Sorry for this! The .zip file has been corrected, as well.

This PHPUnit test will start a firefox browser on the server, hit the form.php on the server and fill out the form. It will submit the form and compare the values in the form fields with those entered previously with assertions.

To check this we run PHPUnit again. Since this is a frontend test it required Xvfb (our X server) and SeleniumRC to run.

Xvfb :1 -screen 0 1280x1024x24 &
env DISPLAY=:1 java -jar /var/www/SeleniumRC/selenium-server-1.0.1/selenium-server.jar  &

Ignore all errors and warnings that Xvfb will probably throw for now - those will be interesting only if the tests fail. It takes some time for both daemons to start up, but after a bit of patience we can  run PHPUnit:

phpunit TestForm.php

The first thing you will notice is that it takes some time before PHPUnit is doing anything - that is while firefox starts - but then it is cluttering the screen with output (actually most of it doesn't come from PHPUnit but from SeleniumRC but you get the point).

The important part is that the output ends with something like this:

.
Time: 24 seconds
OK (1 test, 6 assertions)

You already know what this means... the test passed, the form works as expected. Just to clarify once again: We just tested a website in Firefox by writing and executing a PHPUnit test class.

Now you will want to stop both Xvfb and SeleniumRC in order to continue:

kill -TERM %2
kill -TERM %1

Step three: Automate all tests with phing.

Now that the PHPUnit tests run independently it is time for some more automation with phing, our integration server. Check out the test.xml file in the buildScripts folder contained in the .zip file that contains the test setup. It is over 100 lines long so I will not post it here. If you're familiar with the file structure of a phing build script, you will see that the file consists of a couple of targets which will implement the following workflow:

1. Removing a htdocs/report folder (more on that in a bit)
   <target name="prepare">
2. Prepare a (temporary) xdebug code coverage database
   <target name="coveragePrepare">
3. Start Xvfb and SeleniumRC, just like we did above
   <target name="setUp">
4. Generate an APIdoc using phpDocumentor
   <target name="doc">
5. Running all PHPUnit tests inside the tests folder
   <target name="unittest">
6. Generating the code coverage report
   <target name="coverageReport">
7. Shutting down Xvfb and SeleniumRC again
   (inside the "main" target)
8. In some condition (which we discuss in a bit) the build script will send a mail notification that the build tests have failed.

But first let's do the fun thing, run the script. Once piece of advice though: You might want to go to line no 23 of the test.xml script and set your own mail Id. While running the script in this constellation the mail will not be used but better cover this now than wonder why mails are not reaching you later.

So go ahead... change into the default/buildScripts directory and execute

phing -f test.xml

Phing will read in the .xml file and perform all the tasks mentioned above in that order. Once the script has finished, check the htdocs folder. Now there is a reports subfolder that hasn't been there before. This is where all test results are stored. To be specific:

1. Your PHPUnit test results

   http://integration.local/reports/phpunit/

2. A code coverage report

   http://integration.local/reports/coverage/

3. PHP APIDoc

   http://integration.local/reports/apidocs/

There is an additional folder http://integration.local/reports/selenium/ which contains the STDOUT and STDERR logs generated by SeleniumRC but that is really only of interest if something goes wrong in the execution of the frontend tests.

Open each of the URLs above in your browser to see what you got. I believe it is pretty impressive.

Final note on the text.xml file.

To avoid this article getting even longer, I'll cut things short. The test.xml phing build script is meant to be run by the developer, and manually so. It will only work in the 'default' virtual host unless you change the properties on top. But please keep them as they are for now, as in the fourth article of this series I'll explain how to use another phing build file to automate all things once and for all.

The idea behind the test.xml build file is to make it easy for the developer to get a thorough overview of his working copy - without the need to commit everything just to see what is breaking.

I can only motivate you to try and play around with things. One cool detail for example is that PHPUnit will automatically make a screenshot from the website in firefox whenever a SeleniumRC test assertion fails. So break one assertion and visit the phpunit report file in your browser and you will find a report on what failed along with a URL (sadly it is not linked).

This concludes the third part of the continuous integration for PHP series. Come back in a couple of days for details on how to automate the build process in the integration virtual host.

Here I will give you a step-by-step guide how to set up a LAMP server with all the necessary tools to run a thorough set of tests and reports.

As of this writing Ubuntu Linux version 9.10 (Karmic Koala) has just been released. Personally I like Ubuntu for its ease of use, so this howto will focus on that. I assume that the general setup will be slightly different if you use another flavor of Linux, though.

If you don't have a spare computer lying around just yet, install VMWare Server. That will get you going well enough to convince your manager that a few hundred bucks for a medium-spec server will be well invested money.

This step of the tutorial assumes that you're at least somewhat familiar with the Linux command line. If you are not, do yourself a favor and get in touch with one of your system administrators. Make him understand what you want and convince him to divert from the typical company guidelines for server installation. You might want to show him this:

This will be a test server. It will not run in production. It will never be accessible by the public. It'll be a proof of concept and will be replaced by a server compliant to all guidelines once this proof of concept is approved by management.

For those of you who do this on their own: The above paragraph is for you. All aspects of creating a secure, well administered server with backup are ignored in this tutorial. Do not attempt to run this box online.

One more comment. Even if you're convinced that you'll not be using one of the packages or reports described below, just install them for now. The whole setup is well under 2GB and it works. Removing a package might break things.

This being said, are you ready for the installation?

Step one: Set up a LAMP Server.

Go to the Ubuntu Homepage and download the .iso CD image of 9.10 Karmic Koala. I grabbed the 32-bit version but I assume the 64-bit will run just fine. VMWare users: You can directly mount the .iso image as CD in the virtual machine. No need to burn a disc.

Follow the graphical installer, it is pretty straight forward. Write down the user name for the unprivileged user you will be asked to create during the process, we will need it later.

When you reach the package selection, check 'LAMP', 'OpenSSH' and 'Samba', you will need it.

Now go for lunch, the download of the additional packages will take a bit.

Step two: Basic setup.

You should set up your OpenSSH public key authorization for root and one unprivileged user, any bash aliases that you like as well as any Samba shares that you require. Since I can't know your requirements, I won't even try to give a one-size-fits-all description. Push your question in the comments below and I will answer asap if I can.

You will want one Samba share pointing to where the apache environments will be:

vi /etc/samba/smb.conf

Activate the line reading

security = user

Append this section to the end of the file.

#---- start webserver vhost main directory
comment = webserver main directory
read only = no
create mask = 0664
directory mask = 0775
force group = www-data
#---- end webserver vhost main directory

After that, restart Samba

/etc/init.d/samba restart

Step three: Install basic packages.

A few essentials are missing after the initial installation of Ubuntu. For example we need to install quite a few additional PHP packages as well as Subversion (my personal favorite in revision control).

Make sure you're logged in as root user.

apt-get install php5 php5-dev php5-cli php5-curl php5-gd php5-mysql
apt get install php5-sqlite php5-xmlrpc php5-imagick php5-mcrypt
apt-get install php-pear php5-xsl php5-xdebug
apt-get install subversion
apt-get install zip unzip mailutils

I assume that you might need additional PHP packages. Feel free to add them. However the list above is what we'll need for continuous integration.

Step four: Install continuous integration software.

This is where the fun starts. Luckily next to everything we need can be installed thru the handy PEAR installer, so the complexity is down to a minimum.

pear channel-discover pear.phpunit.de
pear channel-discover pear.phing.info
pecl install xdebug
pear install phpunit/PHPUnit-3.4.2
pear install Image_GraphViz PhpDocumentor
pear install http://pear.phing.info/get/phing-2.4.0RC2.tgz
pear install channel://pear.php.net/VersionControl_SVN-0.3.3
pear install channel://pear.php.net/XML_Serializer-0.20.0
pear install PEAR_PackageFileManager_Plugins PEAR_PackageFileManager

Please note that there is a death-grip like dependency between PHPUnit and phing. phing before v2.4 does not run with PHPUnit 3.4.x because one class was replaced in the latter. On the other hand, PHPUnit before version 3.4.x is not able to perform screenshots for failing Selenium tests, something that you'll definitely want if you plan to automate your frontend testing as well.

You might want to check the phing website if 2.4.0 final is released by the time you read this. I didn't run into any difficulties with the RC2 of phing though.

Since almost everything is based on PEAR, we need to adapt the include_path directive in the php.ini files:

Do a

vi /etc/php5/apache2/php.ini

and a

vi /etc/php5/cli/php.ini

and in both files find the line

include_path = ".:/usr/share/php"

and remove the comment.

Step five: Install SeleniumRC.

SeleniumRC or Selenium Remote Control is a solution to automate front end testing without the need for a display. For its installation, follow me:

apt-get install openjdk-6-jre-headless xvfb lynx firefox imagemagick

wget -O /var/www/seleniumRC.zip http://release.seleniumhq.org/selenium-remote-control/1.0.1/selenium-remote-control-1.0.1-dist.zip
unzip /var/www/seleniumRC.zip -d /var/www/
mv /var/www/selenium-remote-control-1.0.1 /var/www/SeleniumRC
rm /var/www/seleniumRC.zip

We now have the SeleniumRC server (a java program) installed in /var/www/SeleniumRC. Also, we installed the Xvfb virtual frame buffer X server and Firefox 3.5, the former enabling the latter to run without an attached monitor, mouse or keyboard.

Step six: Create the 'dev' environment.

As I described in the first article, I typically use three different environments for continuous integration. A 'dev' environment for the developer, an 'int' environment where the continuous integration takes place as well as a third 'stage' environment to show the client the latest working revision.

In order to try out continuous integration we will simply install all three virtual hosts on one and the same machine.

With Ubuntu the default host for apache is in /var/www. While that is nice and well, I thrive for a little more consistency, hence we begin by moving the default host to its new home for its purpose as the 'dev' environment.

mkdir -p /var/www/vhosts/default/conf
mkdir /var/www/vhosts/default/htdocs
mkdir /var/www/vhosts/default/tests
mkdir /var/www/vhosts/default/buildScripts

mv /var/www/index.html /var/www/vhosts/default/htdocs/.

mv /etc/apache2/sites-available/default /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default /etc/apache2/sites-available/default

mv /etc/apache2/sites-available/default-ssl /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default-ssl /etc/apache2/sites-available/default-ssl

vi /var/www/vhosts/default/conf/default

Now change the DocumentRoot directive to

/var/www/vhosts/default/htdocs

and also change

<Directory /var/www>

to

<Directory /var/www/vhosts/default/htdocs>

Step seven: Create the 'int' and 'stage' environments.

The setup for both the integration and staging environments is almost identical to the 'dev' setup. We start with 'int':

mkdir -p /var/www/vhosts/integration/conf
mkdir /var/www/vhosts/integration/htdocs
mkdir /var/www/vhosts/integration/tests
mkdir /var/www/vhosts/integration/buildScripts

cp /var/www/vhosts/default/conf/default /var/www/vhosts/integration/conf/vhost.conf
ln -s /var/www/vhosts/integration/conf/vhost.conf /etc/apache2/sites-available/integration.conf
ln -s /etc/apache2/sites-available/integration.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/integration/conf/vhost.conf

Now add

ServerName    integrate.integration.local

and once more change the DocumentRoot directive, this time to

/var/www/vhosts/integration/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/integration/htdocs>

As said, for the staging environment it is almost the same:

mkdir -p /var/www/vhosts/demo/conf
mkdir /var/www/vhosts/demo/htdocs
mkdir /var/www/vhosts/demo/tests
mkdir /var/www/vhosts/demo/buildScripts

# create virtual host conf file
cp /var/www/vhosts/default/conf/default /var/www/vhosts/demo/conf/vhost.conf
ln -s /var/www/vhosts/demo/conf/vhost.conf /etc/apache2/sites-available/demo.conf
ln -s /etc/apache2/sites-available/demo.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/demo/conf/vhost.conf

Now add

ServerName    demo.integration.local

and one last time change the DocumentRoot directive, this time to

/var/www/vhosts/demo/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/demo/htdocs>

Step eight: Setting permissions, restarting Apache

We simply want to correct the file system permissions so that both the unprivileged user and the webserver have write permissions everywhere. You do remember that I said this server will not be secure, right?

Do you remember the user name for the unprivileged user you sat up during the installation procedure? Fill in that user name below

chown -R YOUR_USERNAME:www-data /var/www
chmod -R g+w /var/www

We need to restart apache in order to 'activate' the three virtual hosts we just created in step seven.

/etc/init.d/apache2 restart

Last step: Making the host names known.

You may have wondered about the unusual host names, ending in .local. Well you could name them whatever you want but I prefer .local personally because that makes it absolutely clear that this is a non public machine.

In order to make them accessible you may ask your system administrators to configure your local DNS zone. Or you simply add the host names to the hosts file of your development machine (probably Windows or Mac) and to the Ubuntu server we're currently configuring. For the latter one all you need to do is

vi /etc/hosts

and append this section

127.0.0.1    integration.local
127.0.0.1    integrate.integration.local
127.0.0.1    demo.integration.local

Note that on your Windows machine you will probably want to do the same, but with the IP address of the virtual machine. Your hosts file for Windows usually sits in c:\WINDOWS\system32\drivers\etc

Alright, that's it!

You've just set up the whole continuous integration environment.

Before I let you wander through the (required, really) manuals and user guides of phing, PHPUnit, Selenium ([1], [2]) and what not, bare with me for a few more moments though.

Let us just summarize what we did.

We've set up an Ubuntu Linux server that runs on your computer, right along with whatever operating system you use to see this website.

That server has Apache, PHP, MySQL, Samba and OpenSSH and thus can be considered a full fledged LAMP server. However it is highly insecure.

We installed a PHP build tool (phing), a regression testing tool (PHPUnit), Xdebug for code coverage. We can generate API docs on the fly (phpDocumentor) and we a graphical environment (Xvfb) and Firefox for frontend tests with Selenium.

All three Apache environments run on the same machine which is different what I mentioned to be my ideal setup in the first article of this series. However, for the purpose of this demo this is perfectly alright. If you want to go into production with continuous integration, simply repeat all the steps above for your two (or three) machines, once for each environment. You can and should leave out the Samba configuration for your integration and staging system however. Oh and get back to your system admin, thank him for his work so far and kindly ask him to secure those boxes!

Okay that concludes the second part in this series. Come back here in a bit to continue with the third part: Automating tests and reports with phing.

This will be a series of several posts where I will try to cover all basic aspects of continuous integration for PHP:

An informational article about what continuous integration is all about.

After that I'll give you a step-by-step guide on how to set up a continuous integration environment using a virtual environment.

I will provide you with an example build script for automated local tests in the third article.

The fourth article will put all things together and establish continuous integration.

This article will cover the first topic "What is continuous integration all about?".

Continuous integration is an important aspect in agile development methodologies and test driven development.

The whole idea behind continuous integration is to have a process running in the background that integrates and tests the various pieces of source code that you generated continuously and automatically. In a nutshell, that's it.

Why would we want to automate our testing? Well I believe that is obvious:

1. Testing is hard.
2. Testing is boring.
3. Testing is time consuming.

All three aspects - and there might be more - make it impractical to achieve complete test coverage manually even in a medium sized project. It becomes much more feasible to invest the effort of setting up an automated test environment and writing the tests once which from then on will run day after day after day.

Which components are required for continuous integration with PHP?

There are a few alternatives to what I'm going to discuss here (I give some of them in parenthesis) but roughly you'll want:

• Unittest scripts that will cover your regression tests. PHPUnit is the tool of choice here (SimpleTest also does a good job)
• A build tool such as phing. It will automate the building of your test environment.
• For the continuity aspect a cronjob is enough. (CruiseControl & phpUnderControl are much more sophisticated. Check them out if a cronjob is not cutting it for you)
• You will need some sort of revision control. My tool of choice is Subversion. (CVS, and many others are available)
• I strongly recommend using your own integration environment over testing on your development machine. VMWare is a great free virtualizer.

Is there more that we can do with continuous integration?

Hell yes! Here are a few examples:

• Frontend testing (yes, automated) with Selenium IDE and Selenium Remote Control. Note that you will also need some sort of graphical environment for that. Xvfb is usually good enough.
• You could generate your API doc for PHP with phpDocumentor.
• You certainly can generate an API doc for JavaScript as well.
• You could perform unittests for JavaScript or your jQuery plugins.
• and last not least why don't you push everything on a staging system for your client to see when everything goes well?

Now that we have that covered...

How does an environment with continuous integration look like?
How do we have to work in order to get the most out of it?

As usual, there are many ways to do this. My personal approach consists of two, better three environments:

First, the development machine(s). That is where we develop our code.
Second, the integration machine, where the continuous integration is happening.
Third, optionally a system for staging. This gives you constant feedback from the client.

Overall the workflow looks like this:

My continuous integration environment

The 'devel' environment is local on the developers' machine. A virtual machine as I pointed out earlier but that is not relevant at this moment. If there is more than one developer, each will have his own 'devel' environment. The development (1) is done locally. All developers commit to and update  from a central Subversion repository (2).

The 'int' (integration) environment usually sits on a different piece of hardware. I prefer the 'int' and the 'stage' system to be on two different virtual hosts on one machine. However your setup might be, on the 'int' environment a cronjob is running continuously and pulls the latest revision of the product out of Subversion (3). If a new revision is found, the integration (4) starts, running all tests, generating all reports, writing API docs and so on.

If an error occurs during the integration test, a mail is being sent to all developers. I've heard of teams who change the color of an ambient orb to red in this case, or switch on a lava lamp. Do as you please. A mail is a good start, however. The team can then utilize all reports on 'int' (available thru the apache virtual host) to inspect the problem and fix it.

Because the integration is continuous, the feedback about a failing test is fast.

Not much code has to be revised in order to find and fix the bug.

Only when the integration runs without errors, the step (5) starts: Deploying that revision to the 'stage' environment where the client can test it. If you think that might probably not be a good idea in your case / with your client, you might be right. Releasing every other week, at the end of a sprint works just as well. This last step (5) is optional.

That concludes the first part in my series on setting up continuous integration for PHP. Continue with the second part: A installation howto.

Anyways... originally I just wanted to tap a bit into test automation [1, 2] but I ended up downloading the 32bit server version of Ubuntu Koala. With my crappy Tata Photon+ it took a while until the CD .iso file was there but after that it went smooth. The installation system is pretty straight forward and did a decent job of detecting my keyboard.

I chose the LAMP, OpenSSH, Samba packages and let it install.

After that it was just a matter of installing a few more php packages, along with PEAR. This enabled me to set up everything I needed to get a glimpse of what test automation is all about.

# install PHPUnit (make sure you catch the dependencies)
> pear channel-discover pear.phpunit.de
> pear install phpunit/PHPUnit

# install PhpDocumentor
> pear install PhpDocumentor

# install Phing (once more, cover the dependencies)
> pear channel-discover pear.phing.info
> pear install phing/phing
 

In case you're in doubt what all this means:

PHPUnit is the unittesting tool for PHP, a clone of JUnit basically...
PhpDocumentor generates a Html (.pdf, ...) API Doc out of your source code comments...
and finally Phing is a build tool written in and for PHP.

The whole setup tool me less than two hours after the .iso file was there, and that includes additional things like setting up a Samba share and configuring two VirtualHosts for Apache. Really cool and really really smooth.

Now with a short but to-the-point article on Linux.com I got a test scenario running. What joy! you simply enter 'phing' on the commandline and the tools clear up the build filesystem, fill the build again with the latest version of the software, run PhpDocumentor to generate the up-to-date APIDoc, then run all unittests and in case those succeed they generate a tarball with the whole system.

How cool is that?! Really impressive. Those who know me also know that it's hard to impress me .
[Update: By now there is a much more in depth series of articles online here in my blog]