Diversity and Inclusion - best laid plans
Out-Law Analysis | 21 Nov 2016 | 10:14 am | 5 min. read
There is a lot to take from this judgment. It offers a reminder of why it is so important to agree and document the value of changes and variations contemporaneously and serves as a cautionary tale for employers considering whether to withhold or deduct sums from a contractor prior to it having established and tested the robustness of what is said to form the basis for the deduction. However, perhaps the most interesting part of this judgment can be found in the court's commentary on the County Council’s use of statistical sampling and extrapolation for the purposes of converting what was, on one level, a fairly standard and modest defects counterclaim into a very significant claim.
The County Council's objective was simple: it wanted to be able to claim substantial damages based on a review of a relatively small sample of alleged defects by extrapolating the results of that review over a much larger population.
Whilst such claims might be more common where processes are repetitive and the product is generally intended to be identical or close to it (i.e. where any defect identified has a high chance of recurring such as claims relating to manufacturing processes), the court's observations may well prove to be significant in terms of the viability of claims arising under term maintenance or similar contracts where there is a high volume of repeat work, even where that work may be highly variable in nature, where it is uneconomic, disproportionate or otherwise unrealistic to proceed in any way other than by a sampling approach. However, in order to produce any representative sample capable of being extrapolated, the sampling process should be based on a logical and robust statistical methodology (ideally supported by expert advice from the outset), something that was clearly lacking in this case.
The contractor agreed to provide highways maintenance and associated services for Cumbria County Council for a term of 7 years starting on 1 April 2005. By the time the contract expired in 2012 a number of claims and counterclaims had already been intimated. The County Council had, by then, made substantial deductions from the contractor’s final monthly payment applications.
Neither the final account process nor the subsequent pre-action protocol process resulted in an overall settlement of the claims and counterclaims, with the result that proceedings commenced in December 2013.
Advancing an extrapolated claim
Given the very substantial quantities of individual patching and surfacing works undertaken by the contractor over the duration of the contract, the County Council asserted that a sampling approach was the only sensible way of advancing its case. It said that it would have been impractical for it to have inspected each individual item of work undertaken and to have pleaded and proved a case in relation to each allegedly defective item separately.
Unsurprisingly, the Judge had sympathy with these points and indeed the contractor had always recognised that such claims are permissible in principle but only if they are presented on the basis of a representative sample. There were extensive arguments between the parties relating to the use of 'random' sampling, however the key point is, regardless of whether the sampling is random or not, it must be representative.
The parties' statistical experts agreed that "when it is difficult to study all the elements of a population, a sample can provide a practical and efficient means to collect data. However, for a researcher to extend a study’s findings from the sample to the population, the sample must be representative of the population".
They also helpfully agreed on the steps required to produce a reliable representative sample which was capable of being extrapolated across a wider population. In general terms, a party seeking to rely on extrapolation as a basis for a claim in damages must:
Finally, the data resulting from the sample must be properly analysed to produce the estimated characteristic for the sample.
Building on the third point, a sample can be drawn using probability sampling or non-probability sampling methods. Where probability sampling is based on a random sample with a confidence level (typically 95%) and a margin of error, non-probability sampling relies on subjective judgment for which elements / parts of the overall population are sampled. Non-probability samples might still be considered representative if they are not compromised by bias which affects the subjective selection process, something which is not easy to avoid in non-probability sampling. Non-probability sampling does not permit the calculation of a margin of error or confidence interval for a population estimate.
The experts' joint statement also considered the importance of an appropriate sample size and the risks associated with combining samples taken from different selection methods to arrive at a single population without carefully correcting the sample.
The County Council’s extrapolated claim
The County Council’s patching and surfacing counterclaim was pleaded on the basis of random probability sampling. The County Council said that it would rely on expert evidence to establish that its findings, based on its actual inspections of sample sites, could be extrapolated with a 95% confidence rate across the whole of the patching and surfacing works undertaken by the contractor and across the full term of the contract.
However, in the course of oral opening submissions, the County Council accepted that its sampling exercise could not be shown to be statistically random. Nevertheless, the County Council maintained that a representative sample had still been produced and that the sample could therefore be relied on for the purposes of extrapolation. The judge accepted that it was open to the County Council to seek to establish its counterclaim by reference to representative sampling rather than random probability sampling.
The judge's decision
The judge dismissed the County Council’s extrapolated claim concluding that there were a number of errors in the development of the process for choosing the samples. He said that the County Council "set itself the ambitious target of constructing a very substantial edifice of a claim, but failed at every stage to undertake the necessary groundwork to support it".
After noting that only 7% of the total population of patches had been made available for selection in the sample, the Judge found that there was deliberate and clear bias by year and by geographic area, borne from a conscious decision to focus on the patches laid in the final three years of the contract on heavily trafficked roads.
The judge also criticised the County Council’s decision to exclude pre-surface dressing patches because this skewed the sample and no attempt had been made by the County Council to explore the consequences of that in terms of its effect on the reliability of the sample. Finally, whilst the judge accepted that the risk of bias could have been addressed by appropriate weighting procedures to arrive at a defensible outcome, he noted that this was never attempted by the County Council.
Despite this criticism and the County Council’s failure to succeed with its extrapolated claim, this judgment provides helpful guidance to parties who elect to use statistical sampling and extrapolation as a means to demonstrate entitlement to substantial damages.
John Williams and David Greenwood are construction disputes experts at Pinsent Masons, the law firm behind Out-Law.com. They were part of the legal team that advised the successful contractor in this case.
Diversity and Inclusion - best laid plans