Hope you find them as useful as I do.

If you're irritated about what pen & paper Scrum is refer to the original article.

However in the case that you're just starting with Scrum I would argue that you might be better off without tools. All tools force you into some kind of process that works for somebody else, but might not be optimal for you.

So instead of jumping to a fancy tool that comes with all the bells and whistles instantly, I'd recommend starting as low-tech as possible, learn the basics and discover your individual needs. After that you are enabled to get the Scrum tool that does exactly what you need it to do.

How do you start low-tech? Use pen and paper.

Experience over tools

Scrum as an agile methodology has a very shallow complexity. It doesn't require many artefacts in the forms of documents, lists, reports or charts.

The artefacts defined in Scrum are the Product Backlog, Sprint Backlog and a Sprint Burndown Chart. Personally I find two more "artefacts" to be extremely useful: User Story Cards and one set of Planning Poker Cards for each team member.

No more than three documents plus two helpers for all your project management needs. Sounds like thin ice? Believe me, I know how you feel but hear me out.

I started off doing Scrum in a waterfall environment, using tools that where developed for waterfall projects. It gave me the advantage of familiarity with the tools (false security?) but at the price of bad usability and cumbersome overhead.

Only since my ScrumMaster Certification I see the benefit of using low-tech "tools" like pen and paper. It is faster, more fun, breeds involvement and identification, can be distributed without permission and data security problems and most important: You will get a much better understanding how Scrum actually works.

Granted, the "high-tech" tools mandatory for my company are still in use, but I already wrote an article how to utilize SDLC tools best for Scrum so let's keep that aside for a moment.

Doing Scrum "hands-on" with pen and paper is easier than you might think. There are no complex metrics or calculations to perform. You will see that your planning meetings get a totally different dynamic if you work with a stack of user story cards and pen & paper. People interact physically with each other rather than staring at the pale image of the projector.

The Product Backlog

My product backlog template is as simple as it gets. One column with the user story number, to make identification easy. A title and a complexity column is all that is filled out in the beginning, usually by the product owner.

In the planning sessions the team will use its planning poker cards to agree upon a complexity which we fill out together in the second column to the right.

Last not least I added a column for dependencies. This will contain the user story number(s) that need to be completed before the one at hand can be filled out.

The User Story Cards

The title of a feature in the product backlog serves only as a short summary of the feature itself. It will not explain the task, only give it a name.

User stories fill that gap. My User Story Card template is half of an A-4 size paper, print it on a duplex printer (so that page 2 of the template is printed on the back) and cut it in half.

The head of the front page of the user story contains the user story number and the priority, both also found in the product backlog. Another header entry, the estimation is left empty. Here we will the required time (not points!) for the task, in the sprint planning meeting.

Note that I estimate the features in the product backlog in story points through planning poker, but the tasks of a sprint in hours. One user story is usually comprised of several tasks. The "estimation" value on the user story card is the sum of hours for all tasks of this story.

The main part of the front of the user story card is pretty much standard, it has to be filled with the role of the user ("as a ..."), a feature description ("I would like to ...") and the purpose of the feature ("in order to...").

For example "As a logged in user I would like to be able to edit and store my delivery address in order to have it pre filled in the shopping basket."

In short, the front of the card will give a good overview of a feature. The back contains additional notes and the acceptance criteria as given in the planning meeting.

The Planning Poker Cards

Planning poker is easy to perform but I find it notoriously difficult to describe. I will leave it to the ever helpful Wikipedia to give a good run-down of the process of planning poker.

I can provide you with a template to cut out your own set of Planning Poker Cards. Bring a stack of uncut prints to your next meeting along with a bunch of scissors and cut them out together. Play your first couples of rounds on non-critical features or even features that do not exist in reality.

The Sprint Backlog

In the sprint planning meeting I bring the product backlog along, which has the complexity column already filled out. It contains the complexity in terms of story points, derived through planning poker.

During sprint planning the team picks features from the product backlog that have the highest priority to the product owner. The priority is usually based on business value so it makes perfect sense to finish the most valuable features first.

The user story that belongs to a product backlog feature is read aloud and a discussion starts. Questions are raised and clarified. This continues for a couple of minutes until the feature is commonly understood. Then, the team breaks the feature down in smaller tasks, which are written into the sprint backlog.

The sprint backlog template once again contains the user story number to easily identify and group the tasks. The sprint backlog also contains a column for the name of the task and a whole lot of columns for numbers.

In the sprint planning session we discuss the overall complexity of each task of a feature and estimate an amount of time (in hours). This value is being written into the "Est." column, marking the initial estimation.

Note that some teams prefer to estimate both product backlog features and sprint backlog tasks in story points (planning poker) while others prefer to estimate both in time. You'll have to find out what works best for you.

The other columns marked "owner" and "1" to "15" are not filled out yet. During the sprint the team members will pick tasks for them to implement, and write their name into the "owner" column as soon as they do so.

Every evening before leaving the office the team members do a re-assessment how much time on their current task is left and write that into the column marking the Nth day of the sprint.

Let me say that again: The columns "1" to "15" contain the amount of time that is left. It is irrelevant how much time someone spent on a task. The criticality of a delay derives from too much work remaining.

The Sprint Burndown Chart

Finally there is the sprint burndown chart which will display the amount of work remaining on an every day basis.

At the end of the sprint planning meeting the sum of time in the "Est." column of the sprint backlog is transferred to the "day 0" column in the sprint burndown chart.

Note that the vertical axis doesn't have numbers assigned to it. Those vary wildly depending on team size, velocity and sprint length. Fill it out yourself.

During the sprint I will take the sprint backlog every morning and update the sum of time at it's bottom. This number is X'ed on the corresponding "day N" column of the burndown chart and the joining X's connected. Done before each daily scrum this is a good information radiator telling everybody how well the progress is without even talking about it.

It is a good idea to draw a line of "ideal progress" onto the burndown chart as well. This is a straight line, starting at the sum of values of the "Est." column in the sprint backlog, going straight to zero at the column of the last sprint day. This way you can easily real progress with ideal progress.

Conclusion

So there you are, a complete set of templates for your pen & paper based Scrum management needs. For your convenience I've also prepared a .zip file of all .pdfs that will also contain a set of Office 2000 compatible .doc and .xls files of the same templates.

Give it a try.

You can always come back to technical solutions or add a column here and there if you think you'll need them. The KISS principle applies. Keep it simple. I haven't really found a need for more columns in my projects yet. But your mileage may vary.

I will compare PERT and COCOMO, both being traditional techniques with a long history and Planning Poker, probably the most prevalent agile estimation technique.

All techniques have their pros and cons and it will be up to you to determine which technique will work best in your environment.

My comparison is purely based on experience, which is broader for PERT and more narrow for COCOMO and to some extend Planning Poker. Your mileage may vary. This article does not intend to be the definite guide on estimation techniques.

That being said, let's start shall we?

PERT

The PERT estimation is the standard estimation technique at my company. It is time-based and probability-based. The basic formula is to estimate an optimistic time (Topt), realistic time (Treal) and pessimistic time (Tpess) for an activity and generate a mean value (Tm) based on the formula Tm = (Topt + 4 * Treal + Tpess) / 6.

The realistic time Treal is weighted four times higher than the optimum time and worst case time, the sum of all three is divided by six.

For example if a task might complete in two days if everything runs fine, in three days under normal conditions and in five days if we run into complications the PERT value would be (2 + 4*3 + 5) / 6 ~= 3.2 days.

If you estimate all tasks of a project with PERT you can generate the variance and get a buffer value out of it which can be pretty handy. Don't trust that buffer blindly though.

One of the pros for me is that PERT acknowledges the fact that humans are particularly bad in estimating a task down to a single number. The thought of optimistic, realistic and pessimistic cases alone works wonders in discussions. The formula usually does a decent job to come up with a mean value.

Also a pro for PERT is that it is reasonably fast. Estimating a 3 month project on a level of abstraction needed for an offer usually takes no longer than a couple of hours.

My biggest con against PERT is that it is easy to misuse it. What I see frequently is that a discussion starts whether or not a particular task has a risk. Often that leads to optimum and realistic estimation being the same, let's say 2-2-5 instead of 2-3-5 as shown above. Or in a similar manner, realistic and pessimistic estimation are the same: 2-3-3.

Instead of the above mentioned 3.2 days for 2-3-5 we get 2.5 days for 2-2-3 and 2.8 days for 2-3-3. In a nutshell that ruins your buffer. You do not want that.

Another more general issue I have is that PERT is time-based but I'll get to that in a bit.

COCOMO

For me COCOMO is the rocket science approach for estimations. It is complexity-based and rooted on calibration through historic data. The algorithm behind COCOMO will get fine tuned over time, while your "database" of historic data grows.

The key figure in COCOMO is the lines of code (LOC or SLOC), more frequently thousand lines of code (KSLOC). You need to know - or in the first project, estimate - how long it takes to produce 1000 lines of functional code. Analysis, design, coding, testing, ... included.

Then you need to do an estimation of how many lines of code will have to be written or changed for the new project.

A set of factors or "cost drivers" such as project complexity, required quality, team experience etc. is applied to the estimations and after putting all this into a formula you get the estimated amount of time.

By far my biggest issue with COCOMO is that it is slow and too complex. You can't get decent estimation quality with COCOMO in a short manner of time. Even intermediate COCOMO takes into account no less than fifteen cost drivers, all of which affect each and every task. The bottomline is that you need a tool for estimating.

Disclaimer: There is a variant of COCOMO that is faster than traditional COCOMO. Personally I don't find that to be very exact, though and hence not very useful.

The second con against COCOMO is that it is fragile. Change your team constellation from one team to another and your historic data is useless. The same applies more or less for all other cost drivers.

I like the idea that COCOMO gets calibrated with time through your historic estimations. But in the reality of changing teams, hardly comparable projects and time pressure that leads to historic data not being available in a clean form frankly I never saw that work. Granted, that is not a failure of COCOMO but rather the methodology not working for us.

What I generally like in COCOMO is that it is complexity-based rather than time-based.

Planning Poker

Planning poker is an agile estimation technique. As the name implies it is based on playing cards (my .pdf for download) . The estimations in planning poker are relative, complexity-based and consensus-oriented.

Each planning poker card will show a number: "0.5-1-2-3-5-8-13-20-40-100-?-infinity" are a common set but there are other variations. The card "?" indicates that the estimating person can't estimate the task and "infinity" translates into "really, really huge" and can be seen as a sign that the task should be broken down into several smaller tasks.

The estimation works as follows: Each team member has a set of cards on his own. A task is explained, usually by the product owner / customer. A discussion takes place to clarify doubts from the team. After the task is clear, each team member picks a card out of his stack secretly and in unison they show the cards to each other.

If the numbers on each team members' card are reasonably close together (i.e. 8-13-8-8-5-13) a rough mean value is "calculated" (probably 8 in this case) and the next task comes up for estimation.

In case the numbers vary wildly the team members with the lowest and highest numbers explain why they picked their number. This is to reveal misunderstandings and generate a broader understanding for all team members. After that a second estimation round is performed. The process repeats until all numbers are reasonably close.

By now you may ask what the numbers are indicating? They do not indicate time. The card "5" does not mean "five days" or "five hours". The cards indicate complexity. There is no unit for the numbers, they are commonly referred to as story points or points for short.

What points show is that a task with a complexity of 2 is twice as complex as a task with 1 point and roughly half as complex as a 5 point task.

When estimating with planning poker it is absolutely mandatory to understand that each tasks complexity is only relative to another task. So what you do is you pick a comparable task that has already been developed (from your last project or the last sprint in the current project) and assign a medium sized number to it, for example "eight". Now for the current bunch of tasks you ask your team to compare the new task to that one and pick a card from their stack based on its relative size.

What you get is a consensus-based estimation how big or small a current task is related to one that is well known and understood. Since a task with complexity "eight" is almost twice as big as complexity "five", and you know how much time "eight" took to plan, develop, document and test, you have a rough idea how long the new task with complexity "five" will take.

My biggest pro for Planning Poker is that it will enable your team to estimate their own work. Usually this will take a few iterations as they need to get to know their own pitfalls but the long term benefits clearly outweigh the initial pains. Once the team knows how to estimate, you will not find anyone outside the team who can estimate the teams own work.

Another big pro for Planning Poker is that it is by far the fastest approach and that it is fun. Calculating numbers for hours and hours is boring, planning poker is more of a focused discussion.

One negative aspect of Planning Poker is that it is somewhat fragile. If your team size changes often, getting good estimations out of Pllanning Poker might be hard. But in agile the team constellation should stay constant even more than in traditional methodologies.

Bottomline

That's it, the three estimation techniques I'm most familiar with.

If you work in a traditional environment, give PERT a try. Use COCOMO if you can afford the overhead and if you have comparable projects in a very stable environment.

In any case I urge you to try out Planning Poker. While the methodology itself is agile, I don't see why Planning Poker should not work for you, even if you're in a traditional waterfall of SDLC environment. It will give your team a much better understanding of the upcoming work and enable it to participate in a project at a much earlier stage than usual. Through that, the team will feel involved and motivation will go up.

He states that

There are now two definitions of Scrum. One is maintained and sustained by Jeff Sutherland and myself at www.scrum.org. Another is an old copy that is posted at www.scrumalliance.org, by the ScrumAlliance.

His article continues mentioning what sounds like the start of a copyright dispute over the Chinese version of the Scrum Guide (English version) which in its original form is written and maintained by Ken Schwaber and Jeff Sutherland. Apparently now the ScrumAlliance claims ownership to the Chinese translation of the Scrum Guide (again the English version). As Ken points out

Any of you familiar with copyright law know that a derivative of the original is still owned by the original copyright holder.

Ken Schwaber recommends

that you refer to the Scrum Guide created and sustained by
the authors of Scrum, Jeff and myself.

This is serious news.

Do we see the beginning of a split up of Scrum?

Well I don't know about that but the way I read the article we are seeing a split up between Ken Schwaber and Jeff Sutherland with the ScrumAlliance.

Already scrum.org, maintained by Ken Schwaber and Jeff Sutherland offers a Scrum Assessment program that looks similar to the one offered by the ScrumAlliance.

What good will come out of it?

Well again, I don't know. My personal opinion is "not so much".

I see Ken's point and his desire to keep his intellectual property his (and Jeff's).

I also agree with Ken that it is required to have one (!) formal description of what Scrum is. As he points out in application Scrum is mixed up with other agile approaches such as Kanban, XP and others. This makes it important to have one (!) "master copy" of what is Scrum and what is not Scrum. A benchmark is required.

Paul Oldfield of Capgemini may be right that in reality, in the scope of your project it does not matter whether your Scrum is 100% pure or whether you mix it with other methodologies in order to make it work for you.

The importance of a formal definition is on the global scale, not on project scale.

If Scrum minus Burndown Chart, plus 30% Kanban and estimations done through COCOMO works in your project, keep doing that! But don't call it Scrum.

There is only one definition for each of software design pattern. Differences in implementation in your project don't matter. But if I ask in a job interview what the "goal" of the Factory design pattern is and the answer is not something along the lines of "it deals with the problem of creating objects without specifying the exact class of object that will be created" ( (c) Wikipedia) the candidate fails. He doesn't know the Factory design pattern.

If you tell me your estimation method is PERT and the formular is "(opt + 3*real + pess) / 6" you are not using PERT. The estimation technique you're using might work perfectly well for you but it is not PERT.

Same for Scrum. The importance of one formal definition lies in what you know and what everybody (!) agrees to, not in what you do.

We need one formal definition

We are able to benchmark people against a principle only if there is one commonly agreed upon definition.

If there are several definitions of one principle the whole idea gets diluted and harder to assess. The principle in itself might - but doesn't have to - become less important.

I have no background information on what lead to the confusion as Ken calls it. But surely I hope that all involved parties get back on one table and keep Scrum unique. Keep it simple. Don't dilute it.

In this second part I will focus more on the virtual environment, the tools typically used by a company following the traditional waterfall approach / SDLC and how you might still be able to use these tools for Scrum.

Bug tracking system

Let me start with the bug tracking system. Most likely your organization will have one of those. Typically issues can be categorized, prioritized and assigned to a person as well as tagged or grouped into releases. To make this tool work for you in a more agile way you could create a couple of additional categories and put everything (!) into that system: Don't put only issues into it. Also add things like feature requests, enhancements, documentation, testing and refactoring tasks. I'm sure you can come up with more with a bit of thinking. Just be careful not to make this more complicated than required...

Well, that looks pretty much like a product backlog to me. You can do a printout of the overview and pin that to the board for better clarity. Stick it to the window with Scotch Tape if you don't have a pin board. In any case you should make this virtual backlog physical, otherwise the transparency will suffer. Only with a physical product backlog the team will have a clear idea about the big picture.

The overhead to maintain that printout would be minimal. You effectively avoid having two independent lists (in my experience always a recipe for disaster) and your not-so-agile management will probably be pleasantly surprised by the transparency you introduce.

Time tracking system

Next on my list is time tracking software. You know, that thing where you book how much time you spend on each project and / or task.

Well, there are two major ways to handle these in an agile way.

First, you (or the project manager) can create one bookable item in the tracking software for each of the tasks in the sprint. That'd mean one bookable item for each issue in the Bug tracking system (see above).

The benefit of this approach comes with reports - if that time tracking software allows custom reports based on Excel or similar, you can create a sprint burndown chart quite easily. Each item has a planned amount of effort (from Sprint planning) and the actual effort booked by all team members. That's all you need for a burndown. Calculate the sum of each and have Excel calculate a bar chart for each day - the sum of the planned efforts minus daily booked time is the sum of currently remaining efforts... at least until a task is re-evaluated. In this case you'd have to change your estimation.

The downside of this approach I'd say is efficiency. Depending on how userfriendy the interface is, tracking the time as a member of the agile team in a halfway precise manner can take quite a bit of time. Time better spent elsewhere. However, your company might demand such fine grained tracking. In that case it is best to get the most of it and get the burndown chart out of it as well.

The other way to handle this is on the other extreme of the bandwidth. Create one single task for the project and have all members of the team book all times onto that. They will love you. Higher management however may not, so use this with care. If you decide to go with this, you should encourage the team members to update the story cards with the current estimations, to keep track of where you are in the sprint and to generate the sprint burndown chart manually.

As always, the truth probably is somewhere in between. You could create one task for each type of work, i.e. one each for coding, testing, documentation, technical design, ... This will give much more clarity without too much overhead.

Conclusion

Your mileage may vary. The ideas presented here work for me, in the company I work, with the expectations that my management has. This may or may not hold true for you, and in your environment.

Let me know how you handle these situations! Use the comments!

We're getting closer and closer to full test automation and continuous integration. The last article introduced you to how to write PHPUnit tests for SeleniumRC so that we can test website frontends through PHPUnit. After that we wrote our own phing build script that ran our backend as well as the frontend tests automatically, generated an code coverage report based on Xdebug as well as an comprehensive APIDoc based on PHPDoc comments.

The phing build script we used for this - test.xml - was executed manually by calling

phing -f test.xml

on the command line. Everything happened on the development virtual host.

The next logical step is to automate this and perform the additional tasks required to run continuous integration.

The first step is to do a manual svn checkout on the integration virtual host, so we have setup that is identical to the development virtual host.

Now, what the automation will have to do is a Subversion update of the integration virtual host in order to retrieve the latest copy. If the revision changes in that process (that is, if we indeed fetched newer files from Subversion) we again run the tests and generate the reports. If all tests succeed we can safely assume that we have a running system, and we can decide to create a tarball from it or update the staging system for our project manager / client to check it out. On the other hand, if a test fails we'd like to be notified by email.

This is exactly what a second phing build script will do.

Introducing more flexibility

Before we can do all that, we need to make a small but very important change to the Selenium RC test case. The setUp() method of out tests/TestForm.php file configures SeleniumRC so that it points to http://integration.local which is our development environment. This means that no matter where we run the frontend tests, this test would always test our development environment! Clearly not what we want.

If the developer runs the test manually (by executing phing -f test.xml on the commandline), the development environment has to be used. For the automation, the integration host http://integrate.integration.local has to be used.

We accomplish this by changing the setUp() method to this:

function setUp() {
  // determine hostname based on phing property
  $startDir = Phing::getProperty('application.startdir');
  preg_match('/\/(default|integration|demo)/', $startDir, $vhostMainDir);

  switch($vhostMainDir[1]) {
    case 'default':
      $browserUrl = 'http://integration.local/';
      break;

    case 'integration':
      $browserUrl = 'http://integrate.integration.local/';
      break;

    case 'demo':
      $browserUrl = 'http://demo.integration.local/';
      break;

    default:
  } // end: switch

  $this->screenshotUrl    = $browserUrl . 'reports/selenium';
  $this->screenshotPath    = $_ENV['PWD'] . '/../reports/selenium';
  $this->setBrowser('*firefox');
  $this->setBrowserUrl($browserUrl);
} // end: function setUp()

We retrieve the application.startdir property and with this information we can decide in which environment we run to set the browser URL and the screenshot path correctly.

Committing and retrieve everything to Subversion.

With that flexibility issue in the test out of the way we can commit the whole development environment /var/www/vhosts/default to Subversion:

svn import /var/www/vhosts/default https://your.subversion.server/repository/path/trunk
svn checkout --force https://your.subversion.server/repository/path/trunk /var/www/vhosts/default

Your development environment now is connected to the subversion repository.

Next, you have to do another svn checkout from the same repository URL, this time to the integration environment.

svn checkout --force https://your.subversion.server/repository/path/trunk /var/www/vhosts/integration

Now the integration environment is identical to the development environment.

Note that the virtual host configuration file for apache will not be overwritten, since the filenames are different. However the integration environment now contains two .conf files for apache. Since only the correct one is linked to /etc/apache2/sites-available, no harm is done though.

The continuous integration build script.

The single addition between our manual tests from the test.xml phing build file and the continuous integration is another phing build file that you will find at buildScripts/updateAndBuild.xml. If you open it in an editor it will look like this.

<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="runTests" name="updateWorkingCopy">
  <property name="demoDir"      value="/var/www/vhosts/demo" />
  <property name="baseDir"      value="/var/www/vhosts/integration" />
  <property name="htdocsDir"    value="${baseDir}/htdocs" />
  <property name="testDir"      value="${baseDir}/tests/" />
  <property name="reportDir"    value="${htdocsDir}/reports" />
  <property name="seleniumDir"  value="${reportDir}/selenium" />
  <property name="masterFile"   value="updateAndBuild" />
  <property name="buildVersion" value="1.6.1" />
  <property name="srcDir"       value="/var/www/vhosts/integration/htdocs" />
  <property name="svnUser"      value="your_svn_user" />
  <property name="svnPass"      value="your_svn_password" />

  <!--fetch current revision of working copy -->
  <target name="getOriginalRevision">
    <svnlastrevision workingcopy="${baseDir}" propertyname="svnOriginalRevision" />
    <echo msg="Current revision of this working copy: ${svnOriginalRevision}" />
  </target>

  <!-- update local working copy -->
  <target name="svnUpdate">
    <svnupdate username="${svnUser}" password="${svnPass}" nocache="true" todir="${baseDir}" />
  </target>

  <!-- fetch revision of working copy again -->
  <target name="getCurrentRevision" depends="svnUpdate">
    <svnlastrevision workingcopy="${baseDir}" propertyname="svnCurrentRevision" />
    <echo msg="Revision of this working copy after svn update: ${svnCurrentRevision}" />
  </target>

  <!-- create the tarball from the integration VHost-->
  <target name="tar">
    <echo msg="Creating archive..." />
    <tar destfile="${baseDir}/builds/build-${buildVersion}.r${svnCurrentRevision}.tar.gz" compression="gzip">
      <fileset dir="${htdocsDir}">
        <include name="**" />
        <exclude name="reports/**" />
      </fileset>
    </tar>
    <echo msg="Build copied and compressed into directory!" />
  </target>

  <!-- updates the demo VHost after all tests have succeeded -->
  <target name="updateDemo">
    <!-- change baseDir to demoDir -->
    <property name="baseDir" value="${demoDir}" override="true" />
    <phingcall target="svnUpdate" />
  </target>

  <!-- compare original and current working copy revisions -->
  <target name="runTests"  depends="getOriginalRevision,getCurrentRevision">
    <if>
      <equals arg1="${svnOriginalRevision}" arg2="${svnCurrentRevision}" />
      <then>
        <echo msg="Working copy is up to date." />
      </then>
      <else>
        <echo msg="Updated working copy from SVN, starting integration tests." />
        <phing phingfile="test.xml" haltonfailure="true" />
        <phingcall target="tar" />
        <phingcall target="updateDemo" />
      </else>
    </if>
  </target>
</project>

Let's go through this script step by step.

1. The target <getOriginalRevision> gets called first.
   Thankfully phing provides a ready-made task to retrieve the Subversion revision of a given path. We store that revision in the property ${svnOriginalRevision}.
2. Next, the <svnUpdate> target performs a Subversion update on the whole integration environment, ensuring we get any updates that have been committed since the last run. Since this environment will never be edited there will never be any merge conflicts.
3. Then the< svnCurrentRevision> target fetches the Subversion revision number of the integration environment again and stores it in a property, this time in ${svnCurrentRevision}. Note that this target is identical to <getOriginalRevision>, except for the property used.
4. After we have ensured that the integration environment is up to date, the main target <runTests> compares the ${svnOriginalRevision} with ${svnCurrentRevision} to check if a newer version has arrived. If not, this continuous integration loop stops. No need to test again what was already tested.
5. If the execution did not stop - that is, if a newer source code version has arrived - we use the <phing> task to execute the test.xml build file we introduced in the third part of this series.
   The buildAndUpdate.xml uses the same properties as the test.xml and because of that the latter will now run in the integration environment.
6. In case the test.xml script runs through without error a tarball is created automatically, containing everything in the htdocs directory.
7. Last not least we perform a SVN update on the demo environment.

Note that the test.xml build file will now send an email if one of the tests fails!

This is due to the fact that the updateAndBuild.xml sets the ${masterFile} property to a value that causes an <if> task in test.xml (line 112) to pass. This way the developers will automatically be alerted if the current revision in the integration environment is broken. If the test.xml is executed manually the if-condition will not apply and no mail is sent.

Testing the phing script.

You can direct your SSH shell to /var/www/vhosts/integration and execute

phing -f buildScripts/updateAndBuild.xml

but this will not do much good since the integration environment is up to date.

What I would recommend is to create a new file in the development environment /var/www/vhosts/default and fill it with some arbitrary text, only to add and commit that file to Subversion. Then return to the integration environment and run the same command.

Voila! The phing script updated the integration environment, recognized the update and started all tests (this time in the integration environment). Once more, a reports folder has been created (again in the integration environment) and filled. A new folder /var/www/vhosts/integration/builds will have been created, containing a .tar.gz file with the latest build.

Note that this folder may get huge very fast, depending on project complexity. It is probably advisable to deactivate the automatic tarball generation task for a real life environment.

If you add an error to one of the tests, you should get an email. Double check the ${errorMail} property in test.xml and make sure the server can successfully sent mails if you don't.

The last step for you to do is to set up a cronjob to run the phing command periodically. Make sure the cron environment is forwarding the X server display and otherwise is as close as your shell environment as possible, because otherwise you will run into all sorts of issues. This is the nature of cron...

This concludes my tutorial on continuous integration for PHP.

Leave me a comment if you liked the series, if you have questions or suggestions for improvements. I will work on a fifth and last article about my own thoughts for possible improvements, to be published in a bit.

In this article I want to introduce you to the way I run the continuous integration tests, what to run when, and where. If you're unsure how to write unittests, this will not be covered in this article but phpunit.de has an excellent documentation section just covering how to write tests. Another section is there covering how to write tests for SeleniumRC.

Let me start this article with a short summary of what we have installed in the second part: We have a pretty standard LAMP server based on Ubuntu Linux. Then there is PHPUnit for unit and regression testing as well as phing, our integration server. We added SeleniumRC as a frontend test tool, then Xvfb and Firefox to facilitate the frontend tests. xdebug is used by PHPUnit for code coverage reports and phpDocumentor will be used by phing for automatic APIdoc generation.

A Subversion client to fetch latest updates out of the repository completes the setup.

Ok, let's dig a little deeper into the filesystem structure and how to use it, shall we?

We installed three virtual hosts in apache, one for development, one for the integration and one as staging system.

My continuous integration environment

The whole setup is merely to demonstrate the workflow. In reality the integration and staging environment would most likely run on a separate machine, or two.

Each virtual host contains the same initial directory structure, like you see on the left. The image shows the structure of the default server (used for development).

You see the ./conf folder, which stores the Apache configuration file. You can add your own configuration files to this folder if you like.

The ./htdocs folder is what you think it is, the webserver root.

The ./tests and ./buildScripts folders will contain the test classes, as well as all files required by phing to build the project.

This layout is simple, yet effective. It cleanly separates the utilities used for testing and continuous integration from the rest of the project, thus making packaging and deployment comparatively easy.

In the screenshot you also see that the whole default virtual host is under revision control. Experience shows that in the end, you will want everything versioned no matter what you think initially.

Last not least the screenshot shows the other two virtual hosts 'integration' and 'demo', both being under version control as well - but more on that later.

Ok, are you ready to write some tests?

Some of the scripts discussed below are quite long, so I will not post them here in the blog in their full length. You can get a continuous integration test file setup (.zip file) for ease of use.

We start with a simple file which we put in default/htdocs/class.hello.php:

<?php
 class HelloWorld {
   public function hello() {
     return 'Hello World';
   } // end: function hello()
 } // end: class HelloWorld
?>

As you see the class HelloWorld contains a single method 'hello' without any parameters that returns the famous 'Hello World'. The corresponding unittest is equaly simple. We create a second file in default/tests/Test.HelloWorld.php:

<?php
require_once('PHPUnit/Framework.php');
include_once('../htdocs/class.hello.php');

class TestHelloWorld extends PHPUnit_Framework_TestCase {
  private $ciTestObj = null;

  public function setUp() {
    $this->ciTestObj = new HelloWorld();
  } // end: function setUp()

  public function testHello() {
    $this->assertEquals($this->ciTestObj->hello(), 'Hello World');
  } // end: function testHello()
} // end: class TestHelloWorld
?>

Now, the property $ciTestObj contains an instance of the test object which is our HelloWorld class from the first script. The only test we perform is an assertEquals() call that compares the return value of the HelloWorld::hello() method with the static string 'Hello World'.

Save the file and connect to the continuous integration server using your favorite SSH client. Change into the tests directory and execute phpunit:

phpunit Test.HelloWorld.php

You should get a result similar to this:

PHPUnit 3.4.2 by Sebastian Bergmann.
.
Time: 1 second
OK (1 test, 1 assertion)

Hooray, the test passed. Our class behaves as expected.

The second step will be to run a frontend test in SeleniumRC.

Usually you would create a test in Firefox with SeleniumIDE set to write PHPUnit test files instead of the html default. However for a quick success I recommend copy-pasting the test that I prepared.

Before we attempt to run SeleniumRC in the continuous integration server, configure your SSH client to allow X forwarding. If it is currently inactive, enable it. The test won't run without that. Double check your settings are correct.

Ok, once more we start with our test subject. This time it is a very simple html page with a form. The form doesn't do anything except filling the form fields with the values you entered before submit. Create a file default/htdocs/form.php with the following content:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
<head>
  <title>Testable form</title>
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  <style type="text/css">
    body {background-color: #fff;}
    label {
      display: block;
      float: left;
      min-width: 140px;
      padding-right: 5px;
    }
    input,textarea {
      border: 1px inset;
      margin: 5px 0;
    }
    textarea {height: 150px;}
    input[type=submit] {border: 1px outset;}
   .long {width: 300px;}
  </style>
</head>
<body>
  <h2>Selenium RC test form</h2>
  <p>
    This form does have absolutelly no purpose other than being a test subject for selenium RC.
  </p>
  <form action="<?php echo $PHP_SELF; ?>" method="post">
    <label for="firstname">firstname, lastname: *</label><input type="text" name="firstname" value="<?php echo $_POST['firstname']; ?>" /> <input type="text" name="lastname" value="<?php echo $_POST['lastname']; ?>" /><br />
    <label for="email">email: *</label><input type="text" name="email" value="<?php echo $_POST['email']; ?>" /><br />
    <label for="url">homepage:</label><input type="text" name="url" value="<?php echo $_POST['url']; ?>" /><br />
    <label for="subject">subject: *</label><input type="text" name="subject" value="<?php echo $_POST['subject']; ?>" /><br />
    <label for="comment">your message: *</label><textarea name="comment"><?php echo $_POST['comment']; ?></textarea><br />
    <label for="">&nbsp;</label><input type="submit" value="submit" />
  </form>
</body>
</html>

Open it in your browser, just to confirm that it is working. If you followed this howto step by step, http://integration.local/form.php should work just fine. You should do the same from the command line on the server to see if local access is also correct:

lynx http://integration.local/form.php

If that is not working, go back to the second part of this series and check the settings in the hosts files. However if it does, we can write our test for the subject. Open a new file in default/tests/TestForm.php and paste this:

<?php
require_once('PHPUnit/Extensions/SeleniumTestCase.php');

class TestForm extends PHPUnit_Extensions_SeleniumTestCase {
  protected $captureScreenshotOnFailure = TRUE;
  protected $screenshotUrl = 'http://integration.local/reports/selenium';

  function setUp() {
    $this->screenshotPath = $_ENV['PWD'] . '/../reports/selenium';
    $this->setBrowser('*firefox');
    $this->setBrowserUrl('http://integration.local/');
  } // end: function setUp()

  function testFormSubmit() {
    $this->open('/form.php');
    $this->waitForPageToLoad('30000');
    $this->type('firstname', 'SeleniumRC');
    $this->type('lastname', 'Testautomation');
    $this->type('email', 'selenium@example.com');
    $this->type('url', 'http://seleniumhq.com');
    $this->type('subject', 'Test for Selenium RC');
    $this->type('comment', 'This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.');

    $this->click('//input[@value="submit"]');
    $this->waitForPageToLoad('30000');
    $this->assertEquals('SeleniumRC', $this->getValue('firstname'));
    $this->assertEquals('Testautomation', $this->getValue('lastname'));
    $this->assertEquals('selenium@example.com', $this->getValue('email'));
    $this->assertEquals('http://seleniumhq.com', $this->getValue('url'));
    $this->assertEquals('Test for Selenium RC', $this->getValue('subject'));
    $this->assertEquals('This is just to test Selenium RC in a continuous integration environment with Phing and PHPUnit.', $this->getValue('comment'));
  } // end: function testFormSubmit()
} //end: class TestForm
?>

Note that before 30th of November 2009 the above code contained the wrong hostnames for both TestFor::screenShotUrl and the setBrowserUrl() call. Sorry for this! The .zip file has been corrected, as well.

This PHPUnit test will start a firefox browser on the server, hit the form.php on the server and fill out the form. It will submit the form and compare the values in the form fields with those entered previously with assertions.

To check this we run PHPUnit again. Since this is a frontend test it required Xvfb (our X server) and SeleniumRC to run.

Xvfb :1 -screen 0 1280x1024x24 &
env DISPLAY=:1 java -jar /var/www/SeleniumRC/selenium-server-1.0.1/selenium-server.jar  &

Ignore all errors and warnings that Xvfb will probably throw for now - those will be interesting only if the tests fail. It takes some time for both daemons to start up, but after a bit of patience we can  run PHPUnit:

phpunit TestForm.php

The first thing you will notice is that it takes some time before PHPUnit is doing anything - that is while firefox starts - but then it is cluttering the screen with output (actually most of it doesn't come from PHPUnit but from SeleniumRC but you get the point).

The important part is that the output ends with something like this:

.
Time: 24 seconds
OK (1 test, 6 assertions)

You already know what this means... the test passed, the form works as expected. Just to clarify once again: We just tested a website in Firefox by writing and executing a PHPUnit test class.

Now you will want to stop both Xvfb and SeleniumRC in order to continue:

kill -TERM %2
kill -TERM %1

Step three: Automate all tests with phing.

Now that the PHPUnit tests run independently it is time for some more automation with phing, our integration server. Check out the test.xml file in the buildScripts folder contained in the .zip file that contains the test setup. It is over 100 lines long so I will not post it here. If you're familiar with the file structure of a phing build script, you will see that the file consists of a couple of targets which will implement the following workflow:

1. Removing a htdocs/report folder (more on that in a bit)
   <target name="prepare">
2. Prepare a (temporary) xdebug code coverage database
   <target name="coveragePrepare">
3. Start Xvfb and SeleniumRC, just like we did above
   <target name="setUp">
4. Generate an APIdoc using phpDocumentor
   <target name="doc">
5. Running all PHPUnit tests inside the tests folder
   <target name="unittest">
6. Generating the code coverage report
   <target name="coverageReport">
7. Shutting down Xvfb and SeleniumRC again
   (inside the "main" target)
8. In some condition (which we discuss in a bit) the build script will send a mail notification that the build tests have failed.

But first let's do the fun thing, run the script. Once piece of advice though: You might want to go to line no 23 of the test.xml script and set your own mail Id. While running the script in this constellation the mail will not be used but better cover this now than wonder why mails are not reaching you later.

So go ahead... change into the default/buildScripts directory and execute

phing -f test.xml

Phing will read in the .xml file and perform all the tasks mentioned above in that order. Once the script has finished, check the htdocs folder. Now there is a reports subfolder that hasn't been there before. This is where all test results are stored. To be specific:

1. Your PHPUnit test results

   http://integration.local/reports/phpunit/

2. A code coverage report

   http://integration.local/reports/coverage/

3. PHP APIDoc

   http://integration.local/reports/apidocs/

There is an additional folder http://integration.local/reports/selenium/ which contains the STDOUT and STDERR logs generated by SeleniumRC but that is really only of interest if something goes wrong in the execution of the frontend tests.

Open each of the URLs above in your browser to see what you got. I believe it is pretty impressive.

Final note on the text.xml file.

To avoid this article getting even longer, I'll cut things short. The test.xml phing build script is meant to be run by the developer, and manually so. It will only work in the 'default' virtual host unless you change the properties on top. But please keep them as they are for now, as in the fourth article of this series I'll explain how to use another phing build file to automate all things once and for all.

The idea behind the test.xml build file is to make it easy for the developer to get a thorough overview of his working copy - without the need to commit everything just to see what is breaking.

I can only motivate you to try and play around with things. One cool detail for example is that PHPUnit will automatically make a screenshot from the website in firefox whenever a SeleniumRC test assertion fails. So break one assertion and visit the phpunit report file in your browser and you will find a report on what failed along with a URL (sadly it is not linked).

This concludes the third part of the continuous integration for PHP series. Come back in a couple of days for details on how to automate the build process in the integration virtual host.

While you can buy professional card decks for planning poker online from many sources [1], [2], I decided to try a little more subtle start and go with our own cut-out version.

So that's my post for the day... click on the link to get the probably most minimalistic Planning Poker cards wordwide . Don't expect anything flashy though... it will get the job done, but that's about it.

Psssssst... the third article in my series on continuous integration for PHP will be out in less than 48 hours...

wordwide .

Here I will give you a step-by-step guide how to set up a LAMP server with all the necessary tools to run a thorough set of tests and reports.

As of this writing Ubuntu Linux version 9.10 (Karmic Koala) has just been released. Personally I like Ubuntu for its ease of use, so this howto will focus on that. I assume that the general setup will be slightly different if you use another flavor of Linux, though.

If you don't have a spare computer lying around just yet, install VMWare Server. That will get you going well enough to convince your manager that a few hundred bucks for a medium-spec server will be well invested money.

This step of the tutorial assumes that you're at least somewhat familiar with the Linux command line. If you are not, do yourself a favor and get in touch with one of your system administrators. Make him understand what you want and convince him to divert from the typical company guidelines for server installation. You might want to show him this:

This will be a test server. It will not run in production. It will never be accessible by the public. It'll be a proof of concept and will be replaced by a server compliant to all guidelines once this proof of concept is approved by management.

For those of you who do this on their own: The above paragraph is for you. All aspects of creating a secure, well administered server with backup are ignored in this tutorial. Do not attempt to run this box online.

One more comment. Even if you're convinced that you'll not be using one of the packages or reports described below, just install them for now. The whole setup is well under 2GB and it works. Removing a package might break things.

This being said, are you ready for the installation?

Step one: Set up a LAMP Server.

Go to the Ubuntu Homepage and download the .iso CD image of 9.10 Karmic Koala. I grabbed the 32-bit version but I assume the 64-bit will run just fine. VMWare users: You can directly mount the .iso image as CD in the virtual machine. No need to burn a disc.

Follow the graphical installer, it is pretty straight forward. Write down the user name for the unprivileged user you will be asked to create during the process, we will need it later.

When you reach the package selection, check 'LAMP', 'OpenSSH' and 'Samba', you will need it.

Now go for lunch, the download of the additional packages will take a bit.

Step two: Basic setup.

You should set up your OpenSSH public key authorization for root and one unprivileged user, any bash aliases that you like as well as any Samba shares that you require. Since I can't know your requirements, I won't even try to give a one-size-fits-all description. Push your question in the comments below and I will answer asap if I can.

You will want one Samba share pointing to where the apache environments will be:

vi /etc/samba/smb.conf

Activate the line reading

security = user

Append this section to the end of the file.

#---- start webserver vhost main directory
comment = webserver main directory
read only = no
create mask = 0664
directory mask = 0775
force group = www-data
#---- end webserver vhost main directory

After that, restart Samba

/etc/init.d/samba restart

Step three: Install basic packages.

A few essentials are missing after the initial installation of Ubuntu. For example we need to install quite a few additional PHP packages as well as Subversion (my personal favorite in revision control).

Make sure you're logged in as root user.

apt-get install php5 php5-dev php5-cli php5-curl php5-gd php5-mysql
apt get install php5-sqlite php5-xmlrpc php5-imagick php5-mcrypt
apt-get install php-pear php5-xsl php5-xdebug
apt-get install subversion
apt-get install zip unzip mailutils

I assume that you might need additional PHP packages. Feel free to add them. However the list above is what we'll need for continuous integration.

Step four: Install continuous integration software.

This is where the fun starts. Luckily next to everything we need can be installed thru the handy PEAR installer, so the complexity is down to a minimum.

pear channel-discover pear.phpunit.de
pear channel-discover pear.phing.info
pecl install xdebug
pear install phpunit/PHPUnit-3.4.2
pear install Image_GraphViz PhpDocumentor
pear install http://pear.phing.info/get/phing-2.4.0RC2.tgz
pear install channel://pear.php.net/VersionControl_SVN-0.3.3
pear install channel://pear.php.net/XML_Serializer-0.20.0
pear install PEAR_PackageFileManager_Plugins PEAR_PackageFileManager

Please note that there is a death-grip like dependency between PHPUnit and phing. phing before v2.4 does not run with PHPUnit 3.4.x because one class was replaced in the latter. On the other hand, PHPUnit before version 3.4.x is not able to perform screenshots for failing Selenium tests, something that you'll definitely want if you plan to automate your frontend testing as well.

You might want to check the phing website if 2.4.0 final is released by the time you read this. I didn't run into any difficulties with the RC2 of phing though.

Since almost everything is based on PEAR, we need to adapt the include_path directive in the php.ini files:

Do a

vi /etc/php5/apache2/php.ini

and a

vi /etc/php5/cli/php.ini

and in both files find the line

include_path = ".:/usr/share/php"

and remove the comment.

Step five: Install SeleniumRC.

SeleniumRC or Selenium Remote Control is a solution to automate front end testing without the need for a display. For its installation, follow me:

apt-get install openjdk-6-jre-headless xvfb lynx firefox imagemagick

wget -O /var/www/seleniumRC.zip http://release.seleniumhq.org/selenium-remote-control/1.0.1/selenium-remote-control-1.0.1-dist.zip
unzip /var/www/seleniumRC.zip -d /var/www/
mv /var/www/selenium-remote-control-1.0.1 /var/www/SeleniumRC
rm /var/www/seleniumRC.zip

We now have the SeleniumRC server (a java program) installed in /var/www/SeleniumRC. Also, we installed the Xvfb virtual frame buffer X server and Firefox 3.5, the former enabling the latter to run without an attached monitor, mouse or keyboard.

Step six: Create the 'dev' environment.

As I described in the first article, I typically use three different environments for continuous integration. A 'dev' environment for the developer, an 'int' environment where the continuous integration takes place as well as a third 'stage' environment to show the client the latest working revision.

In order to try out continuous integration we will simply install all three virtual hosts on one and the same machine.

With Ubuntu the default host for apache is in /var/www. While that is nice and well, I thrive for a little more consistency, hence we begin by moving the default host to its new home for its purpose as the 'dev' environment.

mkdir -p /var/www/vhosts/default/conf
mkdir /var/www/vhosts/default/htdocs
mkdir /var/www/vhosts/default/tests
mkdir /var/www/vhosts/default/buildScripts

mv /var/www/index.html /var/www/vhosts/default/htdocs/.

mv /etc/apache2/sites-available/default /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default /etc/apache2/sites-available/default

mv /etc/apache2/sites-available/default-ssl /var/www/vhosts/default/conf/.
ln -s /var/www/vhosts/default/conf/default-ssl /etc/apache2/sites-available/default-ssl

vi /var/www/vhosts/default/conf/default

Now change the DocumentRoot directive to

/var/www/vhosts/default/htdocs

and also change

<Directory /var/www>

to

<Directory /var/www/vhosts/default/htdocs>

Step seven: Create the 'int' and 'stage' environments.

The setup for both the integration and staging environments is almost identical to the 'dev' setup. We start with 'int':

mkdir -p /var/www/vhosts/integration/conf
mkdir /var/www/vhosts/integration/htdocs
mkdir /var/www/vhosts/integration/tests
mkdir /var/www/vhosts/integration/buildScripts

cp /var/www/vhosts/default/conf/default /var/www/vhosts/integration/conf/vhost.conf
ln -s /var/www/vhosts/integration/conf/vhost.conf /etc/apache2/sites-available/integration.conf
ln -s /etc/apache2/sites-available/integration.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/integration/conf/vhost.conf

Now add

ServerName    integrate.integration.local

and once more change the DocumentRoot directive, this time to

/var/www/vhosts/integration/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/integration/htdocs>

As said, for the staging environment it is almost the same:

mkdir -p /var/www/vhosts/demo/conf
mkdir /var/www/vhosts/demo/htdocs
mkdir /var/www/vhosts/demo/tests
mkdir /var/www/vhosts/demo/buildScripts

# create virtual host conf file
cp /var/www/vhosts/default/conf/default /var/www/vhosts/demo/conf/vhost.conf
ln -s /var/www/vhosts/demo/conf/vhost.conf /etc/apache2/sites-available/demo.conf
ln -s /etc/apache2/sites-available/demo.conf /etc/apache2/sites-enabled/

vi /var/www/vhosts/demo/conf/vhost.conf

Now add

ServerName    demo.integration.local

and one last time change the DocumentRoot directive, this time to

/var/www/vhosts/demo/htdocs

and also change

<Directory /var/www/vhosts/default/htdocs>

to

<Directory /var/www/vhosts/demo/htdocs>

Step eight: Setting permissions, restarting Apache

We simply want to correct the file system permissions so that both the unprivileged user and the webserver have write permissions everywhere. You do remember that I said this server will not be secure, right?

Do you remember the user name for the unprivileged user you sat up during the installation procedure? Fill in that user name below

chown -R YOUR_USERNAME:www-data /var/www
chmod -R g+w /var/www

We need to restart apache in order to 'activate' the three virtual hosts we just created in step seven.

/etc/init.d/apache2 restart

Last step: Making the host names known.

You may have wondered about the unusual host names, ending in .local. Well you could name them whatever you want but I prefer .local personally because that makes it absolutely clear that this is a non public machine.

In order to make them accessible you may ask your system administrators to configure your local DNS zone. Or you simply add the host names to the hosts file of your development machine (probably Windows or Mac) and to the Ubuntu server we're currently configuring. For the latter one all you need to do is

vi /etc/hosts

and append this section

127.0.0.1    integration.local
127.0.0.1    integrate.integration.local
127.0.0.1    demo.integration.local

Note that on your Windows machine you will probably want to do the same, but with the IP address of the virtual machine. Your hosts file for Windows usually sits in c:\WINDOWS\system32\drivers\etc

Alright, that's it!

You've just set up the whole continuous integration environment.

Before I let you wander through the (required, really) manuals and user guides of phing, PHPUnit, Selenium ([1], [2]) and what not, bare with me for a few more moments though.

Let us just summarize what we did.

We've set up an Ubuntu Linux server that runs on your computer, right along with whatever operating system you use to see this website.

That server has Apache, PHP, MySQL, Samba and OpenSSH and thus can be considered a full fledged LAMP server. However it is highly insecure.

We installed a PHP build tool (phing), a regression testing tool (PHPUnit), Xdebug for code coverage. We can generate API docs on the fly (phpDocumentor) and we a graphical environment (Xvfb) and Firefox for frontend tests with Selenium.

All three Apache environments run on the same machine which is different what I mentioned to be my ideal setup in the first article of this series. However, for the purpose of this demo this is perfectly alright. If you want to go into production with continuous integration, simply repeat all the steps above for your two (or three) machines, once for each environment. You can and should leave out the Samba configuration for your integration and staging system however. Oh and get back to your system admin, thank him for his work so far and kindly ask him to secure those boxes!

Okay that concludes the second part in this series. Come back here in a bit to continue with the third part: Automating tests and reports with phing.