##### Document Text Contents

Page 1

Evaluation of measurement data –

The role of measurement uncertainty

in conformity assessment

Évaluation des données de mesure –

Le rôle de l’incertitude de mesure dans

l’évaluation de la conformité

O

IM

L

G

1

-1

06

E

di

tio

n

20

12

(E

)

OIML G 1-106

Edition 2012 (E)

ORGANISATION INTERNATIONALE

DE MÉTROLOGIE LÉGALE

INTERNATIONAL ORGANIZATION

OF LEGAL METROLOGY

GUIDE

Page 2

OIML G 1-106:2012 (E)

2

Contents

OIML Foreword ........................................................................................................................................................... 3

Note .................................................................................................................................................................................. 4

JCGM 106:2012 ............................................................................................................................................................ 5

Page 30

JCGM 106:2012

7.6.4 There is a close connection betweenCm and other derived parameters that have been used to characterize

measurement quality in various contexts. Among these are gauging ratio, gauge maker’s rule, test uncertainty ratio

(TUR) [32] and test accuracy ratio (TAR) [1]. Such parameters are typically stated in ratio form such as a 10-to-1

rule or a TUR of 4:1. Care has to be taken when such rules are encountered because they are sometimes ambiguously

or incompletely de�ned. De�nition (12), on the other hand, makes clear that a statement such asCm ≥ 4 means

that um ≤ T/16.

7.6.5 In the calibration or veri�cation of a measuring instrument, a speci�ed requirement is often expressed in terms

of a maximum permissible error (of indication) (see 3.3.18). Such a requirement means that when the instrument

is used to measure a quantityY , the error of indication must lie in an interval de�ned by speci�ed upper and lower

limits. In the common case of a symmetric interval [−Emax, Emax], the tolerance isT = 2Emax and the measurement

capability index is

Cm =

2Emax

2U

=

Emax

U

.

In this expression,U is the expanded uncertainty, for a coverage factork = 2, associated with a measurement of the

error of indication of the instrument.

7.7 Measurement capability index and conformance probability

7.7.1 For a normal PDF, expression (11) gives the conformance probabilitypc in terms of a particular pair of

tolerance limits (TL, TU) and a particular measurement result summarized by (y, u). Taking y = ηm and u = um, this

expression can be re-written in a form suitable for a general measurement problem by de�ning a quantity

ỹ =

ηm − TL

T

. (13)

For an estimate ηm in the tolerance interval, ỹ lies in the interval 0 ≤ ỹ ≤ 1.

7.7.2 For a normal post-measurement PDFϕ

(

η; ηm, u

2

m

)

, expression (11) can then be written, using expressions

(12) and (13),

pc = � [4 Cm(1− ỹ)] − �( −4Cmỹ) = pc(ỹ, Cm), (14)

so that pc is completely determined by the two quantities ỹ and Cm.

7.7.3 It is often the case that the standard uncertainty um associated with an estimateηm has a �xed value

that depends on the design of the measuring system, but is independent ofηm. A series of water samples, for

example, might be inspected to determine, for each sample, a concentration of dissolved mercury, using a measurement

procedure that yields di�erent estimates, each having the same associated standard uncertaintyum. In such a case the

measurement capability indexCm = T/4um is �xed, and a question as to whether or not a measured property conforms

to speci�cation with an acceptable probability can be decided based on the estimateηm, using expressions (13) and

(14) with Cm �xed.

NOTE A case where the standard uncertainty um is proportional to the estimate ηm is treated in [13, Appendix A].

18 © JCGM 2012— All rights reserved

Page 31

JCGM 106:2012

Figure 7 – Measurement capability index Cm = T/(4um) versus ỹ = (ηm − TL)/T , showing the locus of

constant 95 % conformance probability pc. The curve separates regions of conformity and non-conformity

at a 95 % level of confidence. The post-measurement distribution for the measurand Y is taken to be the

normal PDF ϕ

(

η; ηm, u

2

m

)

.

7.7.4 There is an infinite number of pairs (ỹ, Cm) that yield a given conformance probability pc via expression

(14). Figure 7 shows Cm versus ỹ along a curve of constant 95 % conformance probability for estimates ηm within the

tolerance interval 0 ≤ ỹ ≤ 1. The curve separates regions of conformity (unshaded) and non-conformity (shaded) at a

95 % level of confidence.

7.7.5 The horizontal axis in figure 7 corresponds to Cm = 1, or um = T/4. For this relatively large standard

uncertainty it can be seen that pc ≥ 95 % only for 0.45 ≤ ỹ ≤ 0.55. If the measured property was required to conform

to specification with at least a 95 % level of confidence, an acceptable estimate ηm would thus have to lie in the central

approximately 10 % of the tolerance interval.

8 Acceptance intervals

8.1 Acceptance limits

8.1.1 A decision to accept an item as conforming, or reject it as non-conforming, to specification is based on a

measured value ηm of a property of the item in relation to a stated decision rule that specifies the role of measurement

uncertainty in formulating acceptance criteria. An interval of measured values of a property that results in acceptance

of the item is called an acceptance interval (see 3.3.9), defined by one or two acceptance limits (see 3.3.8).

8.1.2 As suggested in the Introduction, acceptance limits and corresponding decision rules are chosen in such a way

as to manage the undesired consequences of incorrect decisions. There are a number of widely used decision rules that

are simple to implement. They can be applied when knowledge of a property of interest is summarised in terms of a

best estimate and corresponding coverage interval. Two such decision rules are described in the following subclauses.

8.2 Decision rule based on simple acceptance

8.2.1 An important and widely used decision rule is known as simple acceptance [2] or shared risk [20]. Under

such a rule, the producer and user (consumer) of the measurement result agree, implicitly or explicitly, to accept

as conforming (and reject otherwise) an item whose property has a measured value in the tolerance interval. As

© JCGM 2012— All rights reserved 19

Page 59

JCGM 106:2012

[24] International Organization for Standarization . ISO 3650 Geometrical Product Specifications (GPS) — Length

standards — Gauge blocks, 2nd ed. Geneva, 1998.

[25] International Organization of Legal Metrology . OIML R 111-1 Edition 2004(E) Weights of classes

E1,E2,F1,F2,M1,M1−2,M2,M2−3,M3 — Part 1: Metrological and technical requirements. Paris.

[26] Jaynes, E. T. Probability Theory: The Logic of Science. Cambridge University Press, 2003.

[27] Jeffreys, H. Theory of Probability, 3rd ed. Clarendon Press, Oxford, 1983.

[28] Johnson, N. L., Kotz, S., and Balakrishnan, N. Continuous Univariate Distributions, Volume 1, 2nd ed. John

Wiley & Sons, New York, NY, 1994.

[29] K �allgren, H., Lauwaars, M., Magnusson, B., Pendrill, L., and Taylor, P. Role of measurement uncertainty in

conformity assessment in legal metrology and trade. Accred. Qual. Assur. 8 (2003), 541–47.

[30] M. Evans, N. H., and Peacock, B. Statistical Distributions, 3rd ed. Wiley, 2000.

[31] Modarres, M., Kaminskiy, M., and Krivtsov, V. Reliability and Risk Analysis. Marcel Dekker, New York, 1999.

[32] NCSL International . ANSI/NCSL Z540-3:2006 Requirements for the Calibration of Measuring and Test Equipment.

Boulder, Colorado USA, 2006.

[33] Oakland, J. S. Statistical Process Control, 6th ed. Butterworth-Heinemann, 2007.

[34] Pendrill, L. R. Optimised measurement uncertainty and decision-making when sampling by variables or by attribute.

Measurement 39 (2006), 829–840.

[35] Pendrill, L. R. Optimised measurement uncertainty and decision-making in conformity assessment. NCSLI Measure

2, 2 (2007), 76–86.

[36] Pendrill, L. R., and K �allgren, H. Exhaust gas analysers and optimised sampling, uncertainties and costs. Accred.

Qual. Assur. 11 (2006), 496–505.

[37] Possolo, A., and Toman, B. Assessment of measurement uncertainty via observation equations. Metrologia 44 (2007),

464–475.

[38] Rossi, G. B., and Crenna, F. A probabilistic approach to measurement-based decisions. Measurement 39 (2006),

101–19.

[39] Sivia, D. S. Data Analysis - A Bayesian Tutorial. Clarendon Press, Oxford, 1996.

[40] Sommer, K.-D., and Kochsiek, M. Role of measurement uncertainty in deciding conformance in legal metrology. OIML

Bulletin XLIII, 2 (April 2002), 19–24.

[41] Titterington, D. M. Statistical analysis of finite mixture distributions. Wiley, 1985.

[42] van der Grinten, J. G. M. Confidence levels of measurement-based decisions. OIML Bulletin XLIV, 3 (July 2003),

5–11.

[43] Wheeler, D. J., and Chambers, D. S. Understanding Statistical Process Control, 2nd ed. SPC Press, 1992.

[44] Williams, E., and Hawkins, C. The economics of guardband placement. In Proceedings of the 24th IEEE International

test Conference (Baltimore, 1993).

[45] W �oger, W. Probability assignment to systematic deviations by the principle of maximum entropy. IEEE Trans. Inst.

Meas. IM-20, 2 (1987), 655–8.

© JCGM 2012— All rights reserved 47

Page 60

Alphabetical index

acceptance interval, viii, 1, 6, 19, 22, 23, 25, 27, 28,

30{32

acceptance limit(s), viii, 1, 6, 8, 19{23, 28{32

Bayes’ theorem, 10, 11, 36, 37

conformance probability, viii, 1, 6, 13{19, 22, 23, 25,

27, 30

conformity assessment, vii, viii, 1, 2, 5, 6, 9, 10, 23{31,

33, 34, 37, 39

consumer’s risk

global, 7, 25, 27, 29, 31

speci�c, 6, 25

coverage interval, vii, 1, 4, 12, 16, 17, 35

coverage probability, vii, 1, 4, 12, 16, 17

decision rule, 6, 8, 9, 19{25, 27, 30, 31, 37

distribution function, 1, 2, 12, 14, 34, 38

expectation, 3, 12, 29, 30, 36, 37, 42

guard band, 6, 20{22, 28, 31, 32

guarded acceptance, 21, 22, 28, 31

guarded rejection, 22

indication, vii, 5, 7, 8

inspection, 5

item, 1, 4{10, 12{14, 19, 20, 22{28, 35, 39{41

maximum permissible error, 7, 10, 18

measurand, vii, 4, 7, 9{11, 17, 20, 35, 39

measured value,see quantity value, measured

measurement capability index, 7, 17, 18, 20, 32, 33

measurement result, vii, 1, 4, 7, 8, 11, 16, 19

non-conforming item, 1, 7, 21, 25

PDF, see probability density function

probability density function, vii, 1{3, 10, 34

gamma, 29, 30, 42, 43

joint, 26, 27, 37

normal, 13{15, 17, 23, 27{30, 34{38, 41

posterior, 11, 36, 37

prior, 10, 11, 24, 29, 32, 35{37, 39

probability distribution, 2

producer’s risk

global, 7, 25, 27, 31

speci�c, 6, 25

property, 4, 5, 7, 8, 10, 12, 14, 19, 23, 24

quantity, vii, 1, 3

quantity value, 4

measured, vii, viii, 1, 4, 6, 7, 11, 19, 23, 25{29,

34{37, 39

true, 4

rejection interval, 6

shared risk, 19, 20, 31

speci�ed requirement, vii, viii, 1, 5, 6, 8, 9, 11{13, 18

standard deviation, 3, 12, 23, 27, 39, 41

sample, 29, 39, 41

statistical process control, 39

tolerance, 6, 7, 9, 15, 18, 32

tolerance interval, vii, viii, 5, 8, 9, 12{16, 19, 21{23, 27,

28, 30, 32

tolerance limit(s), vii, viii, 1, 5, 6, 8{10, 14{17, 20, 21

uncertainty

expanded, 7, 12, 20, 21, 32, 35

measurement, vii, viii, 4, 8, 9, 19, 27{29

standard, 7, 11{13, 15, 16, 23, 29, 34{37, 39{41

variance, 3, 12, 37, 40{43

sample, 39{41

48

Evaluation of measurement data –

The role of measurement uncertainty

in conformity assessment

Évaluation des données de mesure –

Le rôle de l’incertitude de mesure dans

l’évaluation de la conformité

O

IM

L

G

1

-1

06

E

di

tio

n

20

12

(E

)

OIML G 1-106

Edition 2012 (E)

ORGANISATION INTERNATIONALE

DE MÉTROLOGIE LÉGALE

INTERNATIONAL ORGANIZATION

OF LEGAL METROLOGY

GUIDE

Page 2

OIML G 1-106:2012 (E)

2

Contents

OIML Foreword ........................................................................................................................................................... 3

Note .................................................................................................................................................................................. 4

JCGM 106:2012 ............................................................................................................................................................ 5

Page 30

JCGM 106:2012

7.6.4 There is a close connection betweenCm and other derived parameters that have been used to characterize

measurement quality in various contexts. Among these are gauging ratio, gauge maker’s rule, test uncertainty ratio

(TUR) [32] and test accuracy ratio (TAR) [1]. Such parameters are typically stated in ratio form such as a 10-to-1

rule or a TUR of 4:1. Care has to be taken when such rules are encountered because they are sometimes ambiguously

or incompletely de�ned. De�nition (12), on the other hand, makes clear that a statement such asCm ≥ 4 means

that um ≤ T/16.

7.6.5 In the calibration or veri�cation of a measuring instrument, a speci�ed requirement is often expressed in terms

of a maximum permissible error (of indication) (see 3.3.18). Such a requirement means that when the instrument

is used to measure a quantityY , the error of indication must lie in an interval de�ned by speci�ed upper and lower

limits. In the common case of a symmetric interval [−Emax, Emax], the tolerance isT = 2Emax and the measurement

capability index is

Cm =

2Emax

2U

=

Emax

U

.

In this expression,U is the expanded uncertainty, for a coverage factork = 2, associated with a measurement of the

error of indication of the instrument.

7.7 Measurement capability index and conformance probability

7.7.1 For a normal PDF, expression (11) gives the conformance probabilitypc in terms of a particular pair of

tolerance limits (TL, TU) and a particular measurement result summarized by (y, u). Taking y = ηm and u = um, this

expression can be re-written in a form suitable for a general measurement problem by de�ning a quantity

ỹ =

ηm − TL

T

. (13)

For an estimate ηm in the tolerance interval, ỹ lies in the interval 0 ≤ ỹ ≤ 1.

7.7.2 For a normal post-measurement PDFϕ

(

η; ηm, u

2

m

)

, expression (11) can then be written, using expressions

(12) and (13),

pc = � [4 Cm(1− ỹ)] − �( −4Cmỹ) = pc(ỹ, Cm), (14)

so that pc is completely determined by the two quantities ỹ and Cm.

7.7.3 It is often the case that the standard uncertainty um associated with an estimateηm has a �xed value

that depends on the design of the measuring system, but is independent ofηm. A series of water samples, for

example, might be inspected to determine, for each sample, a concentration of dissolved mercury, using a measurement

procedure that yields di�erent estimates, each having the same associated standard uncertaintyum. In such a case the

measurement capability indexCm = T/4um is �xed, and a question as to whether or not a measured property conforms

to speci�cation with an acceptable probability can be decided based on the estimateηm, using expressions (13) and

(14) with Cm �xed.

NOTE A case where the standard uncertainty um is proportional to the estimate ηm is treated in [13, Appendix A].

18 © JCGM 2012— All rights reserved

Page 31

JCGM 106:2012

Figure 7 – Measurement capability index Cm = T/(4um) versus ỹ = (ηm − TL)/T , showing the locus of

constant 95 % conformance probability pc. The curve separates regions of conformity and non-conformity

at a 95 % level of confidence. The post-measurement distribution for the measurand Y is taken to be the

normal PDF ϕ

(

η; ηm, u

2

m

)

.

7.7.4 There is an infinite number of pairs (ỹ, Cm) that yield a given conformance probability pc via expression

(14). Figure 7 shows Cm versus ỹ along a curve of constant 95 % conformance probability for estimates ηm within the

tolerance interval 0 ≤ ỹ ≤ 1. The curve separates regions of conformity (unshaded) and non-conformity (shaded) at a

95 % level of confidence.

7.7.5 The horizontal axis in figure 7 corresponds to Cm = 1, or um = T/4. For this relatively large standard

uncertainty it can be seen that pc ≥ 95 % only for 0.45 ≤ ỹ ≤ 0.55. If the measured property was required to conform

to specification with at least a 95 % level of confidence, an acceptable estimate ηm would thus have to lie in the central

approximately 10 % of the tolerance interval.

8 Acceptance intervals

8.1 Acceptance limits

8.1.1 A decision to accept an item as conforming, or reject it as non-conforming, to specification is based on a

measured value ηm of a property of the item in relation to a stated decision rule that specifies the role of measurement

uncertainty in formulating acceptance criteria. An interval of measured values of a property that results in acceptance

of the item is called an acceptance interval (see 3.3.9), defined by one or two acceptance limits (see 3.3.8).

8.1.2 As suggested in the Introduction, acceptance limits and corresponding decision rules are chosen in such a way

as to manage the undesired consequences of incorrect decisions. There are a number of widely used decision rules that

are simple to implement. They can be applied when knowledge of a property of interest is summarised in terms of a

best estimate and corresponding coverage interval. Two such decision rules are described in the following subclauses.

8.2 Decision rule based on simple acceptance

8.2.1 An important and widely used decision rule is known as simple acceptance [2] or shared risk [20]. Under

such a rule, the producer and user (consumer) of the measurement result agree, implicitly or explicitly, to accept

as conforming (and reject otherwise) an item whose property has a measured value in the tolerance interval. As

© JCGM 2012— All rights reserved 19

Page 59

JCGM 106:2012

[24] International Organization for Standarization . ISO 3650 Geometrical Product Specifications (GPS) — Length

standards — Gauge blocks, 2nd ed. Geneva, 1998.

[25] International Organization of Legal Metrology . OIML R 111-1 Edition 2004(E) Weights of classes

E1,E2,F1,F2,M1,M1−2,M2,M2−3,M3 — Part 1: Metrological and technical requirements. Paris.

[26] Jaynes, E. T. Probability Theory: The Logic of Science. Cambridge University Press, 2003.

[27] Jeffreys, H. Theory of Probability, 3rd ed. Clarendon Press, Oxford, 1983.

[28] Johnson, N. L., Kotz, S., and Balakrishnan, N. Continuous Univariate Distributions, Volume 1, 2nd ed. John

Wiley & Sons, New York, NY, 1994.

[29] K �allgren, H., Lauwaars, M., Magnusson, B., Pendrill, L., and Taylor, P. Role of measurement uncertainty in

conformity assessment in legal metrology and trade. Accred. Qual. Assur. 8 (2003), 541–47.

[30] M. Evans, N. H., and Peacock, B. Statistical Distributions, 3rd ed. Wiley, 2000.

[31] Modarres, M., Kaminskiy, M., and Krivtsov, V. Reliability and Risk Analysis. Marcel Dekker, New York, 1999.

[32] NCSL International . ANSI/NCSL Z540-3:2006 Requirements for the Calibration of Measuring and Test Equipment.

Boulder, Colorado USA, 2006.

[33] Oakland, J. S. Statistical Process Control, 6th ed. Butterworth-Heinemann, 2007.

[34] Pendrill, L. R. Optimised measurement uncertainty and decision-making when sampling by variables or by attribute.

Measurement 39 (2006), 829–840.

[35] Pendrill, L. R. Optimised measurement uncertainty and decision-making in conformity assessment. NCSLI Measure

2, 2 (2007), 76–86.

[36] Pendrill, L. R., and K �allgren, H. Exhaust gas analysers and optimised sampling, uncertainties and costs. Accred.

Qual. Assur. 11 (2006), 496–505.

[37] Possolo, A., and Toman, B. Assessment of measurement uncertainty via observation equations. Metrologia 44 (2007),

464–475.

[38] Rossi, G. B., and Crenna, F. A probabilistic approach to measurement-based decisions. Measurement 39 (2006),

101–19.

[39] Sivia, D. S. Data Analysis - A Bayesian Tutorial. Clarendon Press, Oxford, 1996.

[40] Sommer, K.-D., and Kochsiek, M. Role of measurement uncertainty in deciding conformance in legal metrology. OIML

Bulletin XLIII, 2 (April 2002), 19–24.

[41] Titterington, D. M. Statistical analysis of finite mixture distributions. Wiley, 1985.

[42] van der Grinten, J. G. M. Confidence levels of measurement-based decisions. OIML Bulletin XLIV, 3 (July 2003),

5–11.

[43] Wheeler, D. J., and Chambers, D. S. Understanding Statistical Process Control, 2nd ed. SPC Press, 1992.

[44] Williams, E., and Hawkins, C. The economics of guardband placement. In Proceedings of the 24th IEEE International

test Conference (Baltimore, 1993).

[45] W �oger, W. Probability assignment to systematic deviations by the principle of maximum entropy. IEEE Trans. Inst.

Meas. IM-20, 2 (1987), 655–8.

© JCGM 2012— All rights reserved 47

Page 60

Alphabetical index

acceptance interval, viii, 1, 6, 19, 22, 23, 25, 27, 28,

30{32

acceptance limit(s), viii, 1, 6, 8, 19{23, 28{32

Bayes’ theorem, 10, 11, 36, 37

conformance probability, viii, 1, 6, 13{19, 22, 23, 25,

27, 30

conformity assessment, vii, viii, 1, 2, 5, 6, 9, 10, 23{31,

33, 34, 37, 39

consumer’s risk

global, 7, 25, 27, 29, 31

speci�c, 6, 25

coverage interval, vii, 1, 4, 12, 16, 17, 35

coverage probability, vii, 1, 4, 12, 16, 17

decision rule, 6, 8, 9, 19{25, 27, 30, 31, 37

distribution function, 1, 2, 12, 14, 34, 38

expectation, 3, 12, 29, 30, 36, 37, 42

guard band, 6, 20{22, 28, 31, 32

guarded acceptance, 21, 22, 28, 31

guarded rejection, 22

indication, vii, 5, 7, 8

inspection, 5

item, 1, 4{10, 12{14, 19, 20, 22{28, 35, 39{41

maximum permissible error, 7, 10, 18

measurand, vii, 4, 7, 9{11, 17, 20, 35, 39

measured value,see quantity value, measured

measurement capability index, 7, 17, 18, 20, 32, 33

measurement result, vii, 1, 4, 7, 8, 11, 16, 19

non-conforming item, 1, 7, 21, 25

PDF, see probability density function

probability density function, vii, 1{3, 10, 34

gamma, 29, 30, 42, 43

joint, 26, 27, 37

normal, 13{15, 17, 23, 27{30, 34{38, 41

posterior, 11, 36, 37

prior, 10, 11, 24, 29, 32, 35{37, 39

probability distribution, 2

producer’s risk

global, 7, 25, 27, 31

speci�c, 6, 25

property, 4, 5, 7, 8, 10, 12, 14, 19, 23, 24

quantity, vii, 1, 3

quantity value, 4

measured, vii, viii, 1, 4, 6, 7, 11, 19, 23, 25{29,

34{37, 39

true, 4

rejection interval, 6

shared risk, 19, 20, 31

speci�ed requirement, vii, viii, 1, 5, 6, 8, 9, 11{13, 18

standard deviation, 3, 12, 23, 27, 39, 41

sample, 29, 39, 41

statistical process control, 39

tolerance, 6, 7, 9, 15, 18, 32

tolerance interval, vii, viii, 5, 8, 9, 12{16, 19, 21{23, 27,

28, 30, 32

tolerance limit(s), vii, viii, 1, 5, 6, 8{10, 14{17, 20, 21

uncertainty

expanded, 7, 12, 20, 21, 32, 35

measurement, vii, viii, 4, 8, 9, 19, 27{29

standard, 7, 11{13, 15, 16, 23, 29, 34{37, 39{41

variance, 3, 12, 37, 40{43

sample, 39{41

48