Heat release rate: Precision - ISO 5725 Round-Robin Analysis

by Dr. Vytenis Babrauskas, Fire Science and Technology Inc.

What is precision?

Laboratory measurements have error associated them, and for a user to make intelligent use of a measurement its precision should be known. Precision of a measurement has two aspects: (1) how well can the same laboratory repeat the measurement on multiple samples of the same material; and (2) how different will the results be if conducted in a different laboratory. The former issue is termed repeatability, while the latter is reproducibility. Laboratories regularly conduct exercises termed round-robins (or interlaboratory trials) which have the objective of quantifying the precision achievable with a particular test apparatus or test protocol.

In addition to precision, the user also normally wants to know about the accuracy of a measurement. If 10 laboratories make a measurement and the 95% confidence interval encompasses a scatter of only +1%, then the precision would extraordinarily good (this does not happen in fire testing!). However, it can also happen that the mean value they establish is wrong by a factor of 5, with respect to the true value. To know the true value of a measurement is generally impossible. In most situations, if the best-available measuring technology is used, then the true value is declared to be identical to the mean value obtained from a competently-conducted round-robin. In a few cases, true values can be established by other independent means, but this is not common in fire testing. For HRR testing of solids or liquids, no independent means currently exist for assessing the true values.

Recent results establishing precision of HRR tests

In a recent study [1], Prof. Marc Janssens compiled results for four commonly used HRR test methods. These are shown in the table below.

Test method	Year	Labs	Levels	Peak HRR		Total HR
Test method	Year	Labs	Levels	r (%)	R (%)	r (%)	R (%)
Cone Calorimeter	2000	4	16	17	23	8	15
SBI	1997	16	30	38	54	47	71
ICAL	1999	3	8	56	67	72	118
Room/corner test	1994	12	5	65	79	25	41

The results were analyzed according to ISO 5725 [2], which is the international standard for round-robins. This standard has been through some major revisions in the last cycle, and because of the plethora of fire tests run under the previous (1986) edition, this was used for all of the analysis, instead of the new (1994) edition. A brief explanation of the results obtained is as follows.

Test method
Cone Calorimeter: ASTM E 1354, also published by ISO as ISO 5660, parts 1 and 2.

SBI: this is the ‘Single Burning Item’ test mandated by the European Commission; a standard describing its details is expected to be published shortly.

ICAL: ASTM E 1623.

Room/Corner test: This was run in a similar, but not identical way, to that prescribed in ISO 9705.

Labs
The number of labs participating.

Levels
In ISO terminology, this denotes the number of different test conditions explored. The number of levels is identical to the number of materials tested, unless some materials are tested in more than one way (for example, at two different heat fluxes).

Peak HRR
The peak HRR (kW m-2) reported from the test, using the standard protocol for the particular test method.

Total HR
The total heat release (MJ m-2) reported from the test, using the standard protocol for the particular test method.

r (%)
The repeatability of the measurement, expressed as a 95% confidence interval. Numerically this is equal to 2.8 times the repeatability standard deviation. Note that in the 1994 version of ISO 5725, r is now to be reported as the repeatability standard deviation, without multiplying by 2.8. This is the only major difference between the 1994 and the 1986 versions of the ISO 5725 standard, thus if it were desired to obtain expressions for r in the new way, this could (approximately) be done by dividing all r values by 2.8.

R(%)
The reproducibility of the measurement, expressed as a 95% confidence interval. Numerically this is equal to 2.8 times the reproducibility standard deviation. Note that in the 1994 version of ISO 5725, Ris now to be reported as the reproducibility standard deviation, without multiplying by 2.8. This is the only major difference between the 1994 and the 1986 versions of the ISO 5725 standard, thus if it were desired to obtain expressions for R in the new way, this could (approximately) be done by dividing all R values by 2.8.

Example

Suppose a laboratory conducts Cone Calorimeter tests on a product and determines that the peak HRR is 200 kW m-2. Then if the same product was sent out to a large number of other labs, the estimate is that 95% of the test reports would come back with a peak HRR value being reported between 154 and 246 kW m-2. Note that in general it is easier to achieve tighter reproducibility for test methods that use smaller specimen sizes. The reported reproducibility may be higher than expected if a very small number of laboratories participates in a round robin. Thus, it might be expected that an improved precision would be found, for example, if more laboratories were able to run ICAL tests (only 3 labs participated, since only 3 labs in North America had the capability at the time of the round robin).

References

[1] Janssens, M., Heat Release Rate (HRR), Ohlemiller, T. J., Johnsson, E. L., and Gann, R. G., eds., Measurement Needs for Fire Safety: Proc. of an Intl. Workshop (NISTIR 6527), Nat. Inst. Stand. and Technol., Gaithersburg MD (2000).

[2] Precision of test methods – Determination of repeatability and reproducibility for a standard test method by inter-laboratory tests (ISO 5725). International Organization for Standardization, Geneva (1986).