Case Studies, Features

Soil put to the test: A proficiency testing program

soil

Proficiency Testing Australia has completed the 31st round of a soils proficiency testing program on the determination of California Bearing Ratio (CBR) in soil.

Proficiency testing is a means of determining the performance of individual laboratories for specific tests or measurement and represents an external audit of their quality control procedures.

A standard proficiency testing program compares the measuring results of different laboratories on identical samples.

Proficiency Testing Australia (PTA) is one of Australia’s largest proficiency testing providers, services the construction materials industry. It offers proficiency testing programs for aggregate, soils, concrete, rocks, cement, asphalt, and bitumen.

PTA has recently coordinated and analysed the results from its thirty-first Soils Proficiency Testing Program, for the determination of the California bearing ratio (CBR) of a soil.

A total of 70 construction materials laboratories participated soils proficiency testing program.

The program is approved by International Accreditation New Zealand and was organised with the technical assistance and sample supply of Douglas Partners laboratory quality manager David Evans in New South Wales. 

The aim of the program was to assess the laboratories abilities to competently perform the prescribed analyses.

The CBR is a penetration test used for evaluating the mechanical strength of soil. 

It was developed in the 1930s by the California Department of Transportation and has since been modified into various standards internationally. 

The most common methods used in construction materials testing laboratories within Australia currently are “AS 1289.6.1.1:2014 Methods of testing soils for engineering purposes – Soil strength and consolidation tests – Determination of the California Bearing Ratio of a Soil – Standard laboratory method for a remoulded specimen” and “Q113C:2021 California Bearing Ratio of soil at nominated levels of dry density and moisture content”.

PTA’s Soils CBR program was designed to cover the two methods outlined.

In this round, sixty-nine of the seventy participating laboratories were in Australia, with one overseas participant in Kenya. 

Each laboratory was provided with a sample of around 7kg of soil. 

All but four of the laboratories submitted their results by the due date. 

Each laboratory was allocated a unique code number for the program to ensure confidentiality of results. 

Prior to sample distribution, pre-testing and homogeneity testing was performed on the sample material. 

The pre-testing was performed to determine the maximum dry density (MDD) and optimum moisture content (OMC) of the material. 

Once established, these assigned values were included in the Instructions to participants. 

A number of randomly selected samples were analysed for homogeneity and based on the results of this testing; the homogeneity of the samples was established. 

Once receiving their samples, laboratories were requested to perform the tests as per PTA instructions and to record their findings on a PTA-supplied results sheet. 

Statistical analysis of participants’ results

PTA used a robust statistical approach to assess the laboratories’ testing performance. 

A list of summary statistics appeared at the bottom of each of the tables of results and consisted of:

The number of results for that test/sample (No. of results)

The median of these results, i.e., the middle value (median)

The uncertainty of the median; a robust estimate of the standard deviation of the median

The normalised interquartile range of the results (normalised IQR)

The robust coefficient of variation, expressed as a percentage (robust CV); i.e., 100 x normalised IQR/median

The target coefficient of variation (target CV) is a fixed value used to calculate the z-score when the spread of results is too large or too small. Its value is determined based on previous rounds of the program 

The target standard deviation (target SD) is calculated as the target CV multiplied by the median

The minimum and maximum laboratory results

The range (maximum – minimum).

Z-scores measure how far a result is from the median consensus value and create a “score” for each result relative to the others in the group.

A z-score close to zero indicates the result is in agreeance with those from other laboratories. 

A z-score with an absolute value greater than or equal to 3.0 is considered an outlier and is marked by the symbol §.

On the ordered z-score charts, each laboratory’s robust z-score was shown in order of magnitude and marked with its code number. 

From these charts, each laboratory can compare its performance relative to its peers.

Table A summarises the results submitted by the program participants (see above).

Results analysis

There were 18 outliers (or 4.2 per cent) from the 430 results analysed across the 70 laboratories. 

All laboratories with outliers or an absolute z-score between 2.0 and 3.0 have been encouraged to review their procedures.

Achieving high accuracy in CBR is difficult and therefore high variability in results is expected. 

In this round, there was high variability observed in all CBR results, except for those for CBR 5 mm corrected Q (Q113C method). 

Therefore, a target coefficient of variation (CV) was used in this round to calculate z-scores.

Most of the CBR outliers in the AS group (Method AS 1289.6.1.1) were identified in the 2.5 mm uncorrected results, followed by the 2.5 mm corrected and 5.0 mm corrected. 

In the Queensland method group, CBR outliers identified were more evenly distributed among the given tests. 

Besides CBR, in the AS group one outlier was also identified in the results for achieved moisture content, with the recorded value exceeding the limit described in the standard. 

According to AS 1289.6.1.1 the achieved moisture content should not differ by more than the 0.5 per cent of the target moisture content (SOMC). 

It should be noted that for the type of material provided, a penetration correction would be expected, though it is dependent on the curve. 

However, it was observed that some laboratories did not record a penetration correction value, despite a curve correction being applied. 

Participants have been recommended to investigate whether the correction was applied per the requirements of the standard, and in future to submit the value of the penetration correction applied. 

Several laboratories were also observed to have lower values for corrected results than the uncorrected, which should not be possible once the correction has been applied. 

This suggests a possible software calculation issue or misinterpretation of the standard among some participants. 

A few laboratories submitted results that were considered extreme, and these results were therefore excluded from statistical analysis. 

For example, one laboratory reported a value as low as 2.4 for the CBR uncorrected 2.5 mm AS test, where the median for the test was 22.0.

Overall, a satisfactory level of performance was achieved in Soils Round 31. 

Laboratories seeking to improve their scores in future rounds were advised to take note of the advice below.

Advice to participants 

Participants should take care when recording results and note the units and calculations when testing the samples.

Other possible sources of error that might have affected the outcome of the proficiency testing program included but were not limited to:

  • Compaction patterns and procedures 
  • Seating load at time of penetration 
  • Thickness of each layer 
  • Calibration of equipment 
  • Tester competence 
  • Software program for CBR 
  • Calculations, especially units of measurements, examination of application to any correction applied to penetration graph 
  • Reporting and training. 

The final program report for Soils Round 31 can be found on the PTA website: pta.asn.au. 

Round 34 of the Soils Proficiency Testing Program (California Bearing Ratio) is scheduled for December 2022, registration will commence 2-3 months prior to sample dispatch.

For more information about the program and to register contact PTA on ptaenquiry@pta.asn.au. 

Send this to a friend