Page 289 - Statistics for Environmental Engineers
P. 289
L1592_frame_C33 Page 295 Tuesday, December 18, 2001 2:51 PM
33
The Method of Least Squares
KEY WORDS confidence interval, critical sum of squares, dependent variable, empirical model,
experimental error, independent variable, joint confidence region, least squares, linear model, linear
least squares, mechanistic model, nonlinear model, nonlinear least squares, normal equation, parameter
estimation, precision, regression, regressor, residual, residual sum of squares.
One of the most common problems in statistics is to fit an equation to some data. The problem might
be as simple as fitting a straight-line calibration curve where the independent variable is the known
concentration of a standard solution and the dependent variable is the observed response of an instrument.
Or it might be to fit an unsteady-state nonlinear model, for example, to describe the addition of oxygen
to wastewater with a particular kind of aeration device where the independent variables are water depth,
air flow rate, mixing intensity, and temperature.
The equation may be an empirical model (simply descriptive) or mechanistic model (based on fun-
damental science). A response variable or dependent variable (y) has been measured at several settings
of one or more independent variables (x), also called input variables, regressors, or predictor variables.
Regression is the process of fitting an equation to the data. Sometimes, regression is called curve fitting
or parameter estimation.
The purpose of this chapter is to explain that certain basic ideas apply to fitting both linear and
nonlinear models. Nonlinear regression is neither conceptually different nor more difficult than linear
regression. Later chapters will provide specific examples of linear and nonlinear regression. Many books
have been written on regression analysis and introductory statistics textbooks explain the method.
Because this information is widely known and readily available, some equations are given in this chapter
without much explanation or derivation. The reader who wants more details should refer to books listed
at the end of the chapter.
Linear and Nonlinear Models
The fitted model may be a simple function with one independent variable, or it may have many
independent variables with higher-order and nonlinear terms, as in the examples given below.
+
+
+
+
+
Linear models η = β 0 β 1 x β 2 x 2 η = β 0 β 1 x 1 β 2 x 2 β 2 x 1 x 2
(
(
θ 1
Nonlinear models η = ----------------------------------- η = exp – θx 1 ) 1 – x 2 ) θ 2
(
–
1 – exp θ 2 x)
To maintain the distinction between linear and nonlinear we use a different symbol to denote the
parameters. In the general linear model, η = f (x, β), x is a vector of independent variables and β are
parameters that will be estimated by regression analysis. The estimated values of the parameters β 1 , β 2 ,…
will be denoted by b 1 , b 2 ,…. Likewise, a general nonlinear model is η = f (x, θ) where θ is a vector of
parameters, the estimates of which are denoted by k 1 , k 2 ,….
The terms linear and nonlinear refer to the parameters in the model and not to the independent
variables. Once the experiment or survey has been completed, the numerical values of the dependent
© 2002 By CRC Press LLC