Unit 7
PRECISION, ACCURACY, AND
TOLERANCE
MEASUREMENT
 All measurements are approximations
 Degree of precision of a measurement number
depends on number of decimal places used

Number becomes more precise as number of decimal
places increases
 Measurement 2.3 inches is precise to nearest tenth
(0.1) inch
 Measurement 2.34 inches is precise to nearest
hundredth (0.01) inch
2
RANGE OF A MEASUREMENT
 Range of a measurement includes all values
represented by the number
 Range of the measurement 4 inches includes
all numbers equal to or greater than 3.5
inches and less than 4.5 inches
 Range of the measurement 2.00 inches
includes all numbers equal to or greater than
1.995 inches and less than 2.005 inches
3
ADDING AND SUBTRACTING
 Sum or difference cannot be more precise than least precise
measurement number used in computations
 Add 7.26 + 8.0 + 1.253. Round answer to degree of
precision of least precise number
7.26
+ 8.0
1.253
16.513
• Since 8.0 is least precise measurement, round answer to 1 decimal
place
• 16.513 rounded to 1 decimal place = 16.5 Ans
4
SIGNIFICANT DIGITS
 Rules for determining the number of
significant digits in a given measurement:
– All nonzero digits are significant
– Zeros between nonzero digits are significant
– Final zeros in a decimal or mixed decimal are significant
– Zeros used as place holders are not significant unless
identified as such by tagging (putting a bar directly
above it)
 Zeros are the only problem then…..usually if the zero
disappears in scientific notation then it is not significant.
5
SIGNIFICANT DIGITS
 Examples:
•
•
•
•
3.905 has 4 significant digits (all digits are
significant)
0.005 has 1 significant digit (only 5 is
significant)
0.0030 has 2 significant digits (only 3 and last
0 are significant)
32,000 has 2 significant digits (only 3 and 2,
the zeros are considered placeholders)
6
ACCURACY
 Determined by number of significant digits in
a measurement.
 The greater the number of significant digits,
the more accurate the number
• Product or quotient cannot be more accurate
than least accurate measurement used in
computations
7
ACCURACY
 Determined by number of significant digits in a
measurement. The greater the number of significant
digits, the more accurate the number




Number 0.5674 is accurate to 4 significant digits
Number 600,000 is accurate to 1 significant digit
7.3 × 1.28 = 9.344, but since least accurate number is
7.3, answer must be rounded to 2 significant digits, or
9.3 Ans
15.7  3.2 = 4.90625, but since least accurate number
is 3.2, answer must be rounded to 2 significant digits,
or 4.9 Ans
8
ABSOLUTE AND RELATIVE ERROR
 Absolute error = True Value – Measured
Value
or, if measured value is larger:
 Absolute error = Measured Value – True
Value
Absolute Error
Relative Error 
 100
True Value
9
ABSOLUTE AND RELATIVE ERROR
EXAMPLE
• If the true (actual) value of a shaft diameter is 1.605 inches and
the shaft is measured and found to be 1.603 inches, determine
both the absolute and relative error
– Absolute error = True value – measured value
= 1.605 – 1.603 = 0.002 inch Ans
Absolute Error
0.002
Relative Error 
 100 
 100
True Value
1.605
= 0.1246% Ans
10
TOLERANCE
 Basic Dimension – wanted measurement
 Amount of variation permitted for a given length
 Difference between maximum and minimum limits of
a given length
 Find the tolerance given that the maximum permitted
length of a tapered shaft is 143.2 inches and the
minimum permitted length is 142.8 inches
Total Tolerance = maximum limit – minimum limit
= 143.2 inches – 142.8 inches
= 0.4 inch Ans
11
TOLERANCE
 Unilateral
 Tolerance in one direction
 Example: Door, piston, tire to wheel well on your car
 Bilateral
 Tolerance in two directions
 Example: cuts and pilot holes
 Total tolerance refers to the amount of tolerance
allowed.


Unilateral is all in one direction from Basic Dimension
Bilateral is divided (does not have to be evenly divided
always)
12
TOLERANCE
 The basic dimension on a project is 3.75 inches and
you have a bilateral tolerance of ±0.15 inches. What
are your max and min measurement?

3.60 to 3.80 inches are allowable.
 The total tolerance for a job is 0.5 cm. The basic
dimension is 22.45 cm and you are told it is an equal,
bilateral tolerance. What are your max and min
limits?

22.20cm to 22.70cm
13
PRACTICE PROBLEMS
1. Determine the degree of precision and the range for
each of the following measurements:
a. 8.02 mm
b. 4.600 in
c. 3.0 cm
2. Perform the indicated operations. Round your
answers to the degree of precision of the least
precise number
a. 37.691 in + 14.2 in + 3.87 in
b. 2.83 mi + 7.961 mi – 5.7694 mi
c. 15 lb – 7.6 lb + 6.592 lb
14
PRACTICE PROBLEMS (Cont)
3. Determine the number of significant digits for the
following measurements:
a. 0.00476
b. 72.020
c.
14,700
4. Perform the indicated operations. Round your
answers to the same number of significant digits as
the least accurate number
a. 42.15 mi × 0.0234
b. 16.40  0.224 × 0.0027
c. 4.007555  1.050 × 12.763
15
PRACTICE PROBLEMS (Cont)
5. Complete the table below:
Actual or
True Value
Measured
Value
a.
4.983 lb.
4.984 lb.
b.
17 in.
16 in.
c.
16.87 mm
16.84 mm
Absolute
Error
Relative
Error
16
PRACTICE PROBLEMS (Cont)
6. Complete the following table:
Maximum
Limit
Minimum
Limit
a.
4 7/16 in
4 5/16 in
b.
14.83 cm
14.78 cm
c.
5 5/32 mm
5 1/32 mm
Tolerance
17
PRACTICE PROBLEMS (Cont)
7. What is the basic dimension of a washer
that has total tolerance of 0.3 mm with
unilateral tolerance and the max limit is
12.75 mm?
8. What is the basic dimension if you have an
equal bilateral tolerance that has a max limit
of 3.5 inches and a min limit of 3.25 inches?
18
PROBLEM ANSWER KEY
1. a. 0.01 mm; equal to or greater than 8.015 and less than
2.
3.
4.
5.
6.
8.025 mm
b. 0.001 in; equal to or greater than 4.5995 and less than
4.6005 in
c. 0.1 cm; equal to or greater than 2.95 and less than 3.05
cm
a. 55.8 in
b. 5.02 mi
c. 14 lb
a. 3
b. 5
c. 3
a. .986 mi
b. .20
c. 48.71
a. 0.001 lb; 0.02% b. 1 in; 5.882% c. 0.03 mm;
0.178%
a. 1/8 in
b. 0.05 cm
c. 1/8 mm
19
PROBLEM ANSWER KEY
7. 12.45 mm
8. 3.375 inches
20