Warum ist

14

Anmerkung: SST = Summe der Quadrate insgesamt, SSE = Summe der quadrierten Fehler und SSR = Regressionssumme der Quadrate. Die Gleichung im Titel wird oft geschrieben als:

i=1n(yiy¯)2=i=1n(yiy^i)2+i=1n(y^iy¯)2

Ziemlich einfache Frage, aber ich suche nach einer intuitiven Erklärung. Intuitiv scheint es mir sinnvoller zu sein, SSTSSE+SSR zu sein. Zum Beispiel suppose Punkt xi y-Wert hat , entspricht yi=5 und y i = 3 , in dem y i der entsprechende Punkt auf der Regressionsgeraden ist. Nehmen Sie außerdem an, dass der mittlere y-Wert für den Datensatz ˉ y = 0 ist . Dann für diesen bestimmten Punkt i, Sy^i=3y^iy¯=0SST=(50)2=52=25 , währendund. Offensichtlich. Würde dieses Ergebnis nicht auf den gesamten Datensatz verallgemeinern? Ich verstehe es nicht.S S RSSE=(53)2=22=4SSR=(30)2=32=99+4<25

Nocken
quelle

Antworten:

15

Addieren und Subtrahieren ergibt Wir müssen also zeigen, dass n i = 1 ist

i=1n(yiy¯)2=i=1n(yiy^i+y^iy¯)2=i=1n(yiy^i)2+2i=1n(yiy^i)(y^iy¯)+i=1n(y^iy¯)2
i=1n(yiy^i)(y^iy¯)=0. Write
i=1n(yiy^i)(y^iy¯)=i=1n(yiy^i)y^iy¯i=1n(yiy^i)
So, (a) the residuals ei=yiy^i need to be orthogonal to the fitted values, i=1n(yiy^i)y^i=0, and (b) the sum of the fitted values needs to be equal to the sum of the dependent variable, i=1nyi=i=1ny^i.

Actually, I think (a) is easier to show in matrix notation for general multiple regression of which the single variable case is a special case:

eXβ^=(yXβ^)Xβ^=(yX(XX)1Xy)Xβ^=y(XX(XX)1XX)β^=y(XX)β^=0
As for (b), the derivative of the OLS criterion function with respect to the constant (so you need one in the regression for this to be true!), aka the normal equation, is
SSRα^=2i(yiα^β^xi)=0,
which can be rearranged to
iyi=nα^+β^ixi
The right hand side of this equation evidently also is i=1ny^i, as y^i=α^+β^xi.
Christoph Hanck
quelle
3

(1) Intuition for why SST=SSR+SSE

When we try to explain the total variation in Y (SST) with one explanatory variable, X, then there are exactly two sources of variability. First, there is the variability captured by X (Sum Square Regression), and second, there is the variability not captured by X (Sum Square Error). Hence, SST=SSR+SSE (exact equality).

(2) Geometric intuition

Please see the first few pictures here (especially the third): https://sites.google.com/site/modernprogramevaluation/variance-and-bias

Some of the total variation in the data (distance from datapoint to Y¯) is captured by the regression line (the distance from the regression line to Y¯) and error (distance from the point to the regression line). There's not room left for SST to be greater than SSE+SSR.

(3) The problem with your illustration

You can't look at SSE and SSR in a pointwise fashion. For a particular point, the residual may be large, so that there is more error than explanatory power from X. However, for other points, the residual will be small, so that the regression line explains a lot of the variability. They will balance out and ultimately SST=SSR+SSE. Of course this is not rigorous, but you can find proofs like the above.

Also notice that regression will not be defined for one point: b1=(XiX¯)(YiY¯)(XiX¯)2, and you can see that the denominator will be zero, making estimation undefined.

Hope this helps.

--Ryan M.

RMurphy
quelle
1

When an intercept is included in linear regression(sum of residuals is zero), SST=SSE+SSR.

prove

SST=i=1n(yiy¯)2=i=1n(yiy^i+y^iy¯)2=i=1n(yiy^i)2+2i=1n(yiy^i)(y^iy¯)+i=1n(y^iy¯)2=SSE+SSR+2i=1n(yiy^i)(y^iy¯)
Just need to prove last part is equal to 0:
i=1n(yiy^i)(y^iy¯)=i=1n(yiβ0β1xi)(β0+β1xiy¯)=(β0y¯)i=1n(yiβ0β1xi)+β1i=1n(yiβ0β1xi)xi
In Least squares regression, the sum of the squares of the errors is minimized.
SSE=i=1n(ei)2=i=1n(yiyi^)2=i=1n(yiβ0β1xi)2
Take the partial derivative of SSE with respect to β0 and setting it to zero.
SSEβ0=i=1n2(yiβ0β1xi)1=0
So
i=1n(yiβ0β1xi)1=0
Take the partial derivative of SSE with respect to β1 and setting it to zero.
SSEβ1=i=1n2(yiβ0β1xi)1xi=0
So
i=1n(yiβ0β1xi)1xi=0
Hence,
i=1n(yiy^i)(y^iy¯)=(β0y¯)i=1n(yiβ0β1xi)+β1i=1n(yiβ0β1xi)xi=0
SST=SSE+SSR+2i=1n(yiy^i)(y^iy¯)=SSE+SSR

DavidCruise
quelle