Conclusion - Northern Virginia Section 0511 ASQ

advertisement

The Odds Are Against Auditing

Statistical Sampling Plans

Steven Walfish

Statistical Outsourcing Services

Olney, MD

301-325-3129 steven@statisticaloutsourcingservices.com

1

Topics of Discussion

 The Paradox

 Different types of sampling plans.

 Types of Risk

 Statistical Distribution

Normal

Binomial

Poisson

 When to Audit.

April 21, 2010 ASQ Section 511 2

The Paradox

 During an audit you increase the sample size if you have a finding…

 But, no findings might be because your sample size is too small to find errors.

April 21, 2010 ASQ Section 511 3

Common Sampling Strategies

 Simple random sample.

 Stratified sample.

 Systematic sample.

 Haphazard

 Probability proportional to size

April 21, 2010 ASQ Section 511 4

Types of Risk

Decision

April 21, 2010

Accept

Reject

Reality

Accept Reject

Correct Decision Type II Error ( b

)

Consumer Risk

Type I Error ( a

)

Producer Risk

Correct Decision

Power (1b)

ASQ Section 511 5

Normal Distribution

Distribution Plot

Normal, Mean=0, StDev=1

 Typical bell-shaped curve.

0.4

0.3

0.2

0.1

0.0

0.0228

-2 0

X

2

0.0228

 Z-scores determine how many standard deviations a value is from the mean.

April 21, 2010 ASQ Section 511 6

Continuous Data Sample Size n

Z a

Z

 2 b

)

2

S

2

As the effect size decreases, the sample size increases.

 As variability increases, sample size increases.

 Sample size is proportional to risks taken.

April 21, 2010 ASQ Section 511 7

Binomial Distribution

 Binomial Distribution a 

 n x



 p x 

1 p

)

(n

 x)

 where:

• n is the sample size

• x is the number of positives

• p is the probability

• a is the probability of the observing x in a sample of n.

April 21, 2010 ASQ Section 511 8

Binomial Confidence Intervals

 Binomial Distribution a 

 n x



 p x 

1 p

)

(n

 x)

 Solve the equation for p given a

, x and n.

 x=0, n=11 and a

=0.05 (95% confidence).

• p=0.28 (table shows 0.30ucl)

 x=2, n=27 and a

=0.01 (99% confidence).

• p=0.298 (table shows 0.30ucl)

April 21, 2010 ASQ Section 511 9

Poisson Distribution

 Describes the number of times an event occurs in a finite observation space.

 For example, a Poisson distribution can describe the number audit findings.

 The Poisson distribution is defined by one parameter: lambda. This parameter equals the mean and variance. As lambda increases, the

Poisson distribution approaches a normal distribution.

April 21, 2010 ASQ Section 511 10

Hypothesis Testing - Poisson

 P(x) = probability of exactly x occurrences.

 x e

  x !

 is the mean number of occurrences.

April 21, 2010 ASQ Section 511 11

Example of Poisson

 If the average number (

) of audit findings is 5.5.

 What is the probability of a sample with exactly 0 findings?

0.0041 (0.41%)

 What is the probability of having 4 or less findings in a sample

(x=0 + x=1 + x=2 + x=3 + x=4)

0.0041 + 0.0225 + 0.0618 + 0.1133 + 0.1558 = 0.358

(35.8%)

April 21, 2010 ASQ Section 511 12

Poisson Confidence Interval

 The central confidence interval approach can be approximated in two ways:

 95% CI for x=6 would be (2.2,13.1)

1

2

2

0.975;2 x

1

2

2

0.025;2( x

1)

1.965

2 x

2

1.965

 x

1

2

2

April 21, 2010 ASQ Section 511 13

Major Drawback

 What is missing in ALL calculations for the Poisson?

 No reference to sample size.

 Assumes a large population (np>5)

April 21, 2010 ASQ Section 511 14

Comparison

Sample Size

50

100

500

1,000

Poisson

Mean

5

UCL

10.51

0.1

Binomial

19.88 9.94

0.05 10.22 10.22

0.01 2.09 10.45

0.005 1.05 10.47

April 21, 2010 ASQ Section 511 15

N

1

 N

1 was an unpublished report by the

AOAC in 1927.

 It was intended to be a quick rule of thumb for inspection of foods.

 Since it was unpublished, there was not a description of the statistical basis of it.

April 21, 2010 ASQ Section 511 16

N

1

 There is no known statistical justification for the use of the square root of n plus one’ sampling plan.

“Despite the fact that there is no statistical basis for a ‘square root of n plus one’ sampling plan, most firms utilize this approach for incoming raw materials.”

Henson, E., A Pocket Guide to CGMP Sampling, IVT.

April 21, 2010 ASQ Section 511 17

Compare the Plans

ANSI/ASQ Z1.4

Square root N plus one

 Lot Size N=1000

 Sample size n=32

 Acceptance Ac=0

 Rejection Re=1

 AQL=0.160%

 LQ = 6.94%

April 21, 2010

 Lot Size N=1000

 Sample size n=33

 Acceptance Ac=0

 Rejection Re=1

 AQL=0.153%

 LQ = 6.63%

ASQ Section 511 18

Is it a Real Sampling Plan?

 Yes, it meets the Z1.4 definition of a sampling plan.

 It is statistically valid in that it defines the lot size, N, the sample size, n, the accept number,

Ac, and the reject number, Re.

 The Operational Characteristic, OC, curve can be calculated for any square root N plus one plan.

April 21, 2010 ASQ Section 511 19

Sample Size Comparison

 It is very common to use Z1.4 General Level I as the plan for audits.

 The sample sizes for square root N plus one are very close to the sample sizes for Z1.4 GL I.

 Square root N plus one can be used any where that Z1.4 GL I is or could be used.

April 21, 2010 ASQ Section 511 20

Sample Size Comparison

Sqrt(N+1) versus Z1.4

1000

100

10

1

1 10 100 1000

Lot Size

10000 100000 1000000

Sqrt (N+1)

Z1.4

April 21, 2010 ASQ Section 511 21

Is it a Good Plan?

 Like Z1.4 GL I it can be used for audits.

 Any plan is justified by AQL and LQ

 It is easy to use and calculate.

 Works best with an Ac=0.

April 21, 2010 ASQ Section 511 22

Example

Lot Size Sample Size

April 21, 2010

4

10

25

50

100

250

500

1000

10000

Ac=0 Ac=1

11

17

23

6

8

3

4

33

101

ASQ Section 511

AQL LQ AQL LQ

1.69

54 13.50

80

1.27

44 9.78

68

0.85

32

0.64

25

6.30

4.60

51

41

0.46

19

0.30

13

3.30

2.10

0.22

9.5

1.57

31

21

16

0.16

6.7

1.09

11

0.05

2.3

0.35

3.8

23

Using Statistics

 How do you determine when you have too many findings?

 How do you determine the correct sample size for an audit?

 Would a confidence interval approach work?

As long as the observed number is lower than the upper confidence interval, the system is in control.

April 21, 2010 ASQ Section 511 24

Deciding to Audit

 Need to use risk or statistical probability to determine when to audit:

Critical components

Low rank

High Volume suppliers

No third party data available

April 21, 2010 ASQ Section 511 25

Results of an Audit

 The results of an audit can help to establish acceptance controls.

 Better audit results would have less risk, and require smaller sample sizes for incoming inspection.

 Can use AQL or LTPD type of acceptance plans based on audit results.

April 21, 2010 ASQ Section 511 26

Conclusion

 Using the correct sampling strategy helps to assure coverage during an audit.

 Using confidence intervals to determine if a system is in control.

 More compliant systems require larger sample sizes.

April 21, 2010 ASQ Section 511 27

Questions

Steven Walfish steven@statisticaloutsourcingservices.com

301-325-3129 (Phone)

240-559-0989 (Fax)

April 21, 2010 ASQ Section 511 28

Download