Data Analyses – Method of Least Squares
After plotting the data for a calibration curve (e.g. a Beer’s Law plot of absorbance vs. concentration), the
method of least squares can be used to calculate a best straight line relationship ( ) to represent
the data. The resulting parameters and error analysis can then be used to estimate, for example, an unknown
analyte concentration from a measured absorbance along with the associated error in that estimate ( ). A
spreadsheet or other data analysis program can be used to do these calculations. Tables 1 and 2 below outline
the calculations and excel functions that are required to complete this analysis
Table 1 is for a calibration set consisting of N data pairs (
, ) with means and .
If there are multiple y values for each x value, don’t average together the y values. Include a separate
calibration point for every (x,y) pair (e.g. if you have five concentrations and each is measured three times,
then your calibration set should have fifteen (x,y) pairs; N=15).
TABLE 1 : REGRESSION ANALYSIS OF CALIBRATION DATA
TERMS
Define:
EQUATIONS
EXCEL FUNCTION
N = number of (x,y) pairs
used for calibration
Then:
Slope
y-int
std.dev. about
regression (sr ≡ sy)
std. deviation of slope
std. deviation of y-int
! "
=DEVSQ(yi : yf)
=COVAR(yi : yf , xi : xf) * N
=COUNT(xi : xf)
=SLOPE(yi : yf , xi : xf)
=DEVSQ(xi : xf)
#
∑ % ∑% (or =LINEST(yi : yf , xi : xf, TRUE, TRUE))
=INTERCEPT(yi : yf , xi : xf) (OR LINEST)
=STEYX(yi : yf , xi : xf)
From LINEST
From LINEST
The parameters obtained from the regression analysis in Table 1 can then be used with the functions
in Table 2 to estimate the error in the concentration of an unknown sample.
TABLE 2 : ESTIMATION OF ERROR IN X FOR UNKNOWN SAMPLE
TERMS
EQUATIONS
Define for unknown
sample c
calculated
concentration of
unknown sample c
M
is the mean of a set of
replicate analyses of c
number of replicates used to
calculate st. dev. of estimated
results for unknown
sample c
1 1 " || ( )
Confidence Interval
C.I. = ± t *sc
Values sr, m, N, Sxx, and are defined in Table 1.
EXCEL FUNCTION
=AVERAGE(yci : ycf)
=(AVERAGE(yci:ycf) − INTERCEPT(yi:yf,
xi xf)) / SLOPE(yi:yf,xi:xf)
=COUNT(yci : ycf)
= sr / abs(m) *
SQRT((1/M + 1/N + (ycAVERAGE(y))^2 / (m^2*Sxx))
Using Student’s t for (N-2) D.of F.
N is from calibration data set above.
To use MS Excel Linest and display all results:
1. Enter equation =LINEST(yi : yf , xi : xf,TRUE,TRUE) in a cell where you want to place the
slope, m
2. Select a 2-column by 5-row set of cells with m in the top-left corner
3. Depress F2 which displays the equation (Mac Press CRTL-u)
4. Depress CRTL-SHIFT-ENTER, which fills in the array (Mac Press CMD-ENTER).
5. Values displayed are as follows:
Slope Std. error in slope Coeff. of determination F statistic Regression sum of squares m
sm
r2
F
ssreg
b
sb
sy
DOF
ssresid
 Intercept
 Std. error in intercept
 Std. error in y
 Degrees of freedom
 Sum of squares of residuals
See Excel Help for more info on these terms. You usually don’t need r2, F, ssreg, or ssresid.