metode kuadrat terkecil
TRANSCRIPT
METODE KUADRAT TERKECIL(LEAST SQUARE METHOD)
Budi Waluyo
FAKULTAS PERTANIAN
UNIVERSTAS BRAWIJAYA 2009
Metode kuadrat terkecil
• digunakan untuk mendapatkan penaksir koefisien regresi linier
Model regresi sederhana
• Model regresi linier sederhana dinyatakan dengan persamaan : – Y = 0 + 1X + , model umum– Yi = 0 + 1Xi + i , model setiap
pengamatan
• Didapatkan eror, yaitu atau i
– = Y – Ŷ = Y – bo – b1X, atau
– i = Yi – Ŷi = Yi – bo – b1Xi
Graphical - Judgmental Solution
• Titik-titik merah adalah nilai hasil eksperimen, di-notasikan Yi , yang diduga membentuk garis lurus
Garis inilah model yang akan di-taksir, dengan cara menaksir koefisiennya, yaitu b0 dan b1, sehingga terbentuk persamaan b0 + b1 Xi.
Garis tegak lurus sumbu horisontal yang menghubungkan titik eksperimen dengan garis lurus dugaan dinamai error.
Graphical - Judgmental Solution
b1
b0
1
The Least Square Method
nnn
iii
xbbxy
xbbxy
xbbxy
xbbxy
yxy
10
31033
21022
11011
...
...
ˆ
2i
n
1ii )y(yZMin
2i10
n
1ii )xbb(yZMin
Classic Minimization
2i10
n
1ii )xbb(yZMin
We want to minimize this function with respect to b0 and b1
This is a classic optimization problem.
We may remember from high school algebra that to find the minimum value we should get the derivative and set it equal to zero.
The Least Square Method
n10nn
31033
21022
11011
iii
xbbxy
...
...
xbbxy
xbbxy
xbbxy
yxy
Note : Our unknowns are b0 and b1 .
xi and yi are known. They are our data
2i10
n
1ii )xbb(yZ
Find the derivative of Z with respect to b0 and b1 and set them equal to zero
Derivatives
n
1i
2i10i )xbby(Z
n
1ii10i
0
0)xbby)(1(2b
Z
n
1ii10ii
1
0)xbby)(x(2b
Z
b0 and b1
n
xx
n
yxxy
b2
2
1 )(
)(
xbyb 10
Pizza Restaurant ExampleWe collect a set of data from random stores of our Pizza restaurant example
Restaurant Student population Quarterly Sales (1000s) ($1000s)
i xi yi
1 2 582 6 1053 8 884 8 1185 12 1176 16 1377 20 1578 20 1699 22 14910 26 202
ExampleRestaurant i Xi Yi
1 2 582 6 1053 8 884 8 1185 12 1176 16 1377 20 1578 20 1699 22 14910 26 20
Xi Yi
116630704944140421923140338032785252
Xi2
4366464144256400400484676
Total 140 1300 21040 2528
b1
n
xx
n
yxxy
b2
2
1 )(
)(
10)140(
2528
10)1300)(140(
21040
21
b
5568
2840b1
b0
XbbY 10
13010
1300Y
1410
140X
)14(5b130 0
60b0
Estimated Regression Equation
XY 560
Now we can predict.For example, if one of restaurants of this Pizza Chain is close to a campus with 16,000 students. We predict the mean of its quarterly sales is
dollarsthousandY
Y
140
)16(560
• Simple Linear Regression Model
Y = 0 + 1X +
• Simple Linear Regression Equation
E(Y) = 0 + 1X
• Estimated Simple Linear Regression Equation
Ŷ = b0 + b1X
Summary ; The Simple Linear Regression Model
• Least Squares Criterion
min (Yi - Ŷi)2
where
Yi = observed value of the dependent variable
for the i th observation
Ŷi = estimated value of the dependent variable
for the i th observation
Summary ; The Least Square Method
• Slope for the Estimated Regression Equation
• Y -Intercept for the Estimated Regression Equation
Xi = value of independent variable for i th observation
Yi = value of dependent variable for i th observation
X = mean value for independent variable
Y = mean value for dependent variable
n = total number of observations
n
XX
n
YXYX
bi
i
ii
ii
2
2
1 )(
)(
__ __
Summary ; The Least Square Method
XbYb 10
Coefficient of DeterminationQuestion : How well does the estimated regression line fits the data.
Coefficient of determination is a measure for Goodness of Fit.Goodness of Fit of the estimated regression line to the data. Given an observation with values of Yi and Xi. We put Xi in the equation and get . Ŷi = b0 + b1Xi
(Yi – Ŷi) is called residual.
It is the error in using Ŷi to estimate Yi.
SSE = (Yi- Ŷi)2
^̂Yi
SSE : Pictorial Representation
Y = 60+5x
y10 - y10^
SSE Computations
i Xi Yi
1 2 582 6 1053 8 884 8 1885 12 1176 16 1377 20 1578 20 1699 22 14910 26 202
SSE Computations
i Xi Yi Ŷi = 60 + 5Xi 1 2 58 702 6 105 903 8 88 1004 8 188 1005 12 117 1206 16 137 1407 20 157 1608 20 169 1609 22 149 17010 26 202 190
SSE Computations
i Xi Yi Ŷi = 60 + 5xi (Yi - Ŷi ) (Yi- Ŷi
)2
1 2 58 70 -12 1442 6 105 90 15 2253 8 88 100 -12 1444 8 188 100 18 3245 12 117 120 -3 96 16 137 140 -3 97 20 157 160 -3 98 20 169 160 9 819 22 149 170 -21 44110 26 202 190 12 144
SSE Computations
i Xi Yi Ŷi = 60 + 5xi (Yi - Ŷi ) (Yi- Ŷi )2 1 2 58 70 -12 1442 6 105 90 15 2253 8 88 100 -12 1444 8 118 100 18 3245 12 117 120 -3 96 16 137 140 -3 97 20 157 160 -3 98 20 169 160 9 819 22 149 170 -21 44110 26 202 190 12 144
Total SSE = 1530
SSE = 1530 measures the error in using estimated equation to predict sales
SST Computations
Now suppose we want to estimate sales without using the level of advertising. In other words, we want to estimate Y without using X.
= ( yi) / n = 1300/10 = 130yy
This is our estimate for the next value of y.Given an observation with values of yi and xi.
yyIf Y does not depend on X, then b1 = 0. Therefore = b0 + b1x ===> b0 = Here we do not take x into account, we simply use the average of y as our sales forecast.
(yi –y ) is the error in using x to estimate yi.
SST = (yi- y )2
SST : Pictorial Representation
y10 - y
= 130yi
SST Computationsi Xi Yi (Yi - ) (Yi - )2 Y
1 2 58 -72 51842 6 105 -25 6253 8 88 -42 17644 8 188 -12 1445 12 117 -13 1696 16 137 7 497 20 157 27 7298 20 169 39 15219 22 149 19 36110 26 202 72 5184
Total SST = 15730
SST = 15730 measures the error in using mean of y values to predict sales
Y
SSE , SST and SSR
SST : A measure of how well the observations cluster around ySSE : A measure of how well the observations cluster around ŷ
If x did not play any role in vale of y then we shouldSST = SSE
If x plays the full role in vale of y then SSE = 0
SST = SSE + SSR
SSR : Sum of the squares due to regression
SSR is explained portion of SSTSSE is unexplained portion of SST
Coefficient of Determination for Goodness of Fit
SSE = SST - SSR
The largest value for SSE is
SSE = SST
SSE = SST =======> SSR = 0
SSR/SST = 0 =====> the worst fit
SSR/SST = 1 =====> the best fit
Coefficient of Determination for Pizza example
In the Pizza example, SST = 15730SSE = 1530SSR = 15730 - 1530 = 14200
r2 = SSR/SST : Coefficient of Determination
1 r2 0
r2 = 14200/15730 = .9027In other words, 90% of variations in y can be explained by the regression line.
SST Calculations
2)( YYSST
n
YYSST
2
2)(
SST Calculations
n
YYSST
2
2)(
15730)10/)1300((184730 2 SST
Observation Xi Yi Yi^21 2 58 33642 6 105 110253 8 88 77444 8 118 139245 12 117 136896 16 137 187697 20 157 246498 20 169 285619 22 149 22201
10 26 202 408041300 184730
SSR Calculations
Observation X Y XY Y 2̂ X̂ 21 2 58 116 3364 42 6 105 630 11025 363 8 88 704 7744 644 8 118 944 13924 645 12 117 1404 13689 1446 16 137 2192 18769 2567 20 157 3140 24649 4008 20 169 3380 28561 4009 22 149 3278 22201 484
10 26 202 5252 40804 67610 140 1300 21040 184730 2528
n
XX
n
YXXY
SSR2
2
2
1420010/)140(2528
]10/)1300)(140(21040[SSR
2
2
SSR Calculations
9027.r
15730/14200SST/SSRr
2
2
15301420015730SSE
SSRSSTSSE
Reed Auto periodically has a special week-long sale. As part of the advertising campaign Reed runs one or more television commercials during the weekend preceding the sale. Data from a sample of 5 previous sales showing the number of TV ads run and the number of cars sold in each sale are shown below.
Number of TV Ads Number of Cars Sold
1 14
3 24
2 18
1 17
3 27
Example : Reed Auto Sales
We need to calculate X, Y, XY , X2, Y2
Example : Reed Auto SalesnYYSST /)( 22
nXX
nYXXYSSR
/
/)(22
2
X Y XY X2 Y21 14 14 1 1963 24 72 9 5762 18 36 4 3241 17 17 1 2893 27 81 9 729
10 100 220 24 2114
x = 10
y = 100
xy = 220
x2 = 24
y2 = 2114
Example : Reed Auto Sales
114SST
5/)100(2114SST 2
1002024
200220SSR
5/1024
5/)100)(10(220SSR
2
2
2
nYYSST /)( 22 nXX
nYXXYSSR
/
/)(22
2
Example : Read Auto Sales
Alternatively; we could compute SSE and SST and then find SSR = SST -SSE
y x y^2 yhat=10+5x y-yhat (y-yhat) 2̂14 1 196 15 -1 124 3 576 25 -1 118 2 324 20 -2 417 1 289 15 2 427 3 729 25 2 4Sy Sx Sy^2 SSE100 10 2114 14
114
1000.877193
SST = Sy^2-[(Sy)^2]/n
SSR = SST - SSER2=100/114
• Coefficient of Determination
r 2 = SSR/SST = 100/114 = .88
The regression relationship is very strong since
88% of the variation in number of cars sold can be
explained by the linear relationship between the
number of TV ads and the number of cars sold.
Example : Reed Auto Sales
Correlation Coefficient = Sign of b1 times Square Root of the Coefficient of Determination)
The Correlation Coefficient
21xy r)bofsign(r
Correlation coefficient is a measure of the strength of a linear association between two variables. It has a value between -1 and +1
rxy = +1 : two variables are perfectly related through a line with positive slope.
rxy = -1 : two variables are perfectly related through a line with negative slope.
rxy = 0 : two variables are not linearly related.
IN our Pizza example, r2 = .9027 and sign of b1 is positive
The Correlation Coefficient : example
9501.r
9027.r
r)bofsign(r
xy
xy
21xy
There is a strong positive relationship between x and y.
Coefficient of Determination and Correlation Coefficient are both measures of associations between variables.
Correlation Coefficient for linear relationship between two variables.
Coefficient of Determination for linear and nonlinear relationships between two and more variables.
Correlation Coefficient and Coefficient of Determination
Exercise
• Given the following experimental data on rice yield (t/ha), plant height (cm) and tiller number, determine the relationships of these variables with each other using correlation and regression analysis. Obtain a model relating YIELD to the variables PLTHT and TILLER# and interpret results. Test for the significance of the parameter estimates and the regression equation. Evaluate the adequacy of the model obtained.
• SELAMAT BELAJAR