มีวิธีที่ฉลาด / ชาญฉลาดในการทำความเข้าใจตัวตนถดถอยเชิงเส้นนี้สำหรับหลาย ๆ

ในการถดถอยเชิงเส้นฉันได้พบผลลัพธ์ที่น่ายินดีว่าถ้าเราพอดีกับแบบจำลอง

$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$

แล้วถ้าเราสร้างมาตรฐานและศูนย์ ,และข้อมูล$Y$$X_1$$X_2$

$$R^2 = \mathrm{Cor}(Y,X_1) \beta_1 + \mathrm{Cor}(Y, X_2) \beta_2.$$

สิ่งนี้ทำให้ฉันรู้สึกเหมือนเป็นตัวแปร 2 รุ่นของสำหรับการถดถอยซึ่งเป็นที่ชื่นชอบ$R^2 = \mathrm{Cor}(Y,X)^2$$y=mx+c$

แต่ข้อพิสูจน์เดียวที่ฉันรู้ไม่ได้อยู่ในเชิงสร้างสรรค์หรือลึกซึ้ง (ดูด้านล่าง) และยังมองมันรู้สึกว่าควรเข้าใจได้ง่าย

ตัวอย่างความคิด:

• และพารามิเตอร์ให้เรา 'สัดส่วนของและ$\beta_1$$\beta_2$$X_1$$X_2$ใน$Y$และดังนั้นเราจึงได้สัดส่วนตามความสัมพันธ์ของพวกเขา ...
• $\beta$s มีความสัมพันธ์บางส่วน $R^2$ คือความสัมพันธ์หลายกำลังสอง ... ความสัมพันธ์คูณด้วยความสัมพันธ์บางส่วน ...
• ถ้าเราปรับมุมฉากก่อนจากนั้น $\beta$จะเป็น $\mathrm{Cov}/\mathrm{Var}$... ผลลัพธ์นี้มีความหมายทางเรขาคณิตหรือไม่?

ดูเหมือนว่าไม่มีหัวข้อใดที่จะนำพาฉันไปได้ทุกที่ ทุกคนสามารถให้คำอธิบายที่ชัดเจนเกี่ยวกับวิธีการเข้าใจผลลัพธ์นี้

---

หลักฐานไม่น่าพอใจ

$$\begin{equation} R^2 = \frac{SS_{reg}}{SS_{Tot}} = \frac{SS_{reg}}{N} = \langle(\beta_1 X_1 + \beta_2 X_2)^2\rangle \\= \langle\beta_1^2 X_1^2\rangle + \langle\beta_2^2 X_2^2\rangle + 2\langle\beta_1\beta_2X_1X_2\rangle \end{equation}$$

และ

$$\begin{equation} \mathrm{Cor}(Y,X_1) \beta_1 + \mathrm{Cor}(Y, X_2) \beta_2 = \langle YX_1\rangle\beta_1 + \langle Y X_2\rangle \beta_2\\ =\langle \beta_1 X_1^2 + \beta_2 X_1 X_2\rangle \beta_1 + \langle \beta_1 X_1 X_2 + \beta_2 X_2^2\rangle \beta_2\\ =\langle \beta_1^2 X_1^2\rangle + \langle \beta_2^2 X_2^2 \rangle + 2\langle \beta_1 \beta_2 X_1 X_2\rangle \end{equation}$$

QED

The hat matrix is idempotent.

(This is a linear-algebraic way of stating that OLS is an orthogonal projection of the response vector onto the space spanned by the variables.)

---

Recall that by definition

$$R^2 = \frac{ESS}{TSS}$$

where

$$ESS = (\hat Y)^\prime \hat Y$$

is the sum of squares of the (centered) predicted values and

$$TSS = Y^\prime Y$$

is the sum of squares of the (centered) response values. Standardizing $Y$ beforehand to unit variance also implies

$$TSS = Y^\prime Y = n.$$

Recall, too, that the estimated coefficients are given by

$$\hat\beta = (X^\prime X)^{-} X^\prime Y,$$

whence

$$\hat Y = X \hat \beta = X (X^\prime X)^{-} X^\prime Y = H Y$$

where $H$ is the "hat matrix" effecting the projection of $Y$ onto its least squares fit $\hat Y$. It is symmetric (which is obvious from its very form) and idempotent. Here is a proof of the latter for those unfamiliar with this result. It's just shuffling parentheses around:

$$\eqalign{H^\prime H = H H &=\left( X (X^\prime X)^{-} X^\prime\right)\left(X (X^\prime X)^{-} X^\prime \right) \\ &= X (X^\prime X)^{-} \left(X^\prime X \right) (X^\prime X)^{-} X^\prime \\ &= X (X^\prime X)^{-} X^\prime = H. }$$

Therefore

$$R^2 = \frac{ESS}{TSS} = \frac{1}{n} (\hat Y)^\prime \hat Y = \frac{1}{n}Y^\prime H^\prime H Y = \frac{1}{n}Y^\prime H Y = \left(\frac{1}{n}Y^\prime X\right) \hat \beta.$$

The crucial move in the middle used the idempotence of the hat matrix. The right hand side is your magical formula because $\frac{1}{n}Y^\prime X$ is the (row) vector of correlation coefficients between $Y$ and the columns of $X$.

The following three formulas are well known, they are found in many books on linear regression. It is not difficult to derive them.

$\beta_1= \frac {r_{YX_1}-r_{YX_2}r_{X_1X_2}} {\sqrt{1-r_{X_1X_2}^2}}$

$\beta_2= \frac {r_{YX_2}-r_{YX_1}r_{X_1X_2}} {\sqrt{1-r_{X_1X_2}^2}}$

$R^2= \frac {r_{YX_1}^2+r_{YX_2}^2-2 r_{YX_1}r_{YX_2}r_{X_1X_2}} {\sqrt{1-r_{X_1X_2}^2}}$

If you substitute the two betas into your equation $R^2 = r_{YX_1} \beta_1 + r_{YX_2} \beta_2$, you will get the above formula for R-square.

---

Here is a geometric "insight". Below are two pictures showing regression of $Y$ by $X_1$ and $X_2$. This kind of representation is known as variables-as-vectors in subject space (please read what it is about). The pictures are drawn after all the three variables were centered, and so (1) every vector's length = st. deviation of the respective variable, and (2) angle (its cosine) between every two vectors = correlation between the respective variables.

$\hat{Y}$ is the regression prediction (orthogonal projection of $Y$ onto "plane X"); $e$ is the error term; $cos \angle{Y \hat{Y}}={|\hat Y|}/|Y|$, multiple correlation coefficient.

The left picture depicts skew coordinates of $\hat{Y}$ on variables $X_1$ and $X_2$. We know that such coordinates relate the regression coefficients. Namely, the coordinates are: $b_1|X_1|=b_1\sigma_{X_1}$ and $b_2|X_2|=b_2\sigma_{X_2}$.

And the right picture shows corresponding perpendicular coordinates. We know that such coordinates relate the zero order correlation coefficients (these are cosines of orthogonal projections). If $r_1$ is the correlation between $Y$ and $X_1$ and $r_1^*$ is the correlation between $\hat Y$ and $X_1$ then the coordinate is $r_1|Y|=r_1\sigma_{Y} = r_1^*|\hat{Y}|=r_1^*\sigma_{\hat{Y}}$. Likewise for the other coordinate, $r_2|Y|=r_2\sigma_{Y} = r_2^*|\hat{Y}|=r_2^*\sigma_{\hat{Y}}$.

So far it were general explanations of linear regression vector representation. Now we turn for the task to show how it may lead to $R^2 = r_1 \beta_1 + r_2 \beta_2$.

First of all, recall that in their question @Corone put forward the condition that the expression is true when all the three variables are standardized, that is, not just centered but also scaled to variance 1. Then (i.e. implying $|X_1|=|X_2|=|Y|=1$ to be the "working parts" of the vectors) we have coordinates equal to: $b_1|X_1|=\beta_1$; $b_2|X_2|=\beta_2$; $r_1|Y|=r_1$; $r_2|Y|=r_2$; as well as $R=|\hat Y|/|Y|=|\hat Y|$. Redraw, under these conditions, just the "plane X" of the pictures above:

On the picture, we have a pair of perpendicular coordinates and a pair of skew coordinates, of the same vector $\hat Y$ of length $R$. There exist a general rule to obtain perpendicular coordinates from skew ones (or back): $\bf P = S C$, where $\bf P$ is points X axes matrix of perpendicular ones; $\bf S$ is the same sized matrix of skew ones; and $\bf C$ are the axes X axes symmetric matrix of angles (cosines) between the nonorthogonal axes.

$X_1$ and $X_2$ are the axes in our case, with $r_{12}$ being the cosine between them. So, $r_1 = \beta_1 + \beta_2 r_{12}$ and $r_2 = \beta_1 r_{12} + \beta_2$.

Substitute these $r$s expressed via $\beta$s in the @Corone's statement $R^2 = r_1 \beta_1 + r_2 \beta_2$, and you'll get that $R^2 = \beta_1^2 + \beta_2^2 + 2\beta_1\beta_2r_{12}$, - which is true, because it is exactly how a diagonal of a parallelogram (tinted on the picture) is expressed via its adjacent sides (quantity $\beta_1\beta_2r_{12}$ being the scalar product).

This same thing is true for any number of predictors X. Unfortunately, it is impossible to draw the alike pictures with many predictors.