Page 289 - Statistics for Environmental Engineers
P. 289

L1592_frame_C33  Page 295  Tuesday, December 18, 2001  2:51 PM







                       33




                       The Method of Least Squares






                       KEY WORDS confidence interval, critical sum of squares, dependent variable, empirical model,
                       experimental error, independent variable, joint confidence region, least squares, linear model, linear
                       least squares, mechanistic model, nonlinear model, nonlinear least squares, normal equation, parameter
                       estimation, precision, regression, regressor, residual, residual sum of squares.

                       One of the most common problems in statistics is to fit an equation to some data. The problem might
                       be as simple as fitting a straight-line calibration curve where the independent variable is the known
                       concentration of a standard solution and the dependent variable is the observed response of an instrument.
                       Or it might be to fit an unsteady-state nonlinear model, for example, to describe the addition of oxygen
                       to wastewater with a particular kind of aeration device where the independent variables are water depth,
                       air flow rate, mixing intensity, and temperature.
                        The equation may be an empirical model (simply descriptive) or mechanistic model (based on fun-
                       damental science). A response variable or dependent variable (y) has been measured at several settings
                       of one or more independent variables (x), also called input variables, regressors, or predictor variables.
                       Regression is the process of fitting an equation to the data. Sometimes, regression is called curve fitting
                       or parameter estimation.
                        The purpose of this chapter is to explain that certain basic ideas apply to  fitting both linear and
                       nonlinear models. Nonlinear regression is neither conceptually different nor more difficult than linear
                       regression. Later chapters will provide specific examples of linear and nonlinear regression. Many books
                       have been written on regression analysis and introductory statistics textbooks explain the method.
                       Because this information is widely known and readily available, some equations are given in this chapter
                       without much explanation or derivation. The reader who wants more details should refer to books listed
                       at the end of the chapter.



                       Linear and Nonlinear Models

                       The  fitted model may be a simple function with one independent variable, or it may have many
                       independent variables with higher-order and nonlinear terms, as in the examples given below.
                                                  +
                                                                      +
                                                                                +
                                                       +
                                                                           +
                          Linear models     η =  β 0 β 1 x β 2 x 2  η =  β 0 β 1 x 1 β 2 x 2 β 2 x 1 x 2
                                                                            (
                                                                       (
                                                     θ 1
                          Nonlinear models  η =  -----------------------------------  η =  exp – θx 1 ) 1 –  x 2 ) θ 2
                                                      (
                                                       –
                                                1 –  exp  θ 2 x)
                        To maintain the distinction between linear and nonlinear we use a different symbol to denote the
                       parameters. In the general linear model, η = f (x, β), x is a vector of independent variables and β are
                       parameters that will be estimated by regression analysis. The estimated values of the parameters β 1 , β 2 ,…
                       will be denoted by b 1 , b 2 ,…. Likewise, a general nonlinear model is η = f (x, θ) where θ is a vector of
                       parameters, the estimates of which are denoted by k 1 , k 2 ,….
                        The terms  linear and  nonlinear refer to the parameters in the model and not to the independent
                       variables. Once the experiment or survey has been completed, the numerical values of the dependent
                       © 2002 By CRC Press LLC
   284   285   286   287   288   289   290   291   292   293   294