Regression

Simple Linear Regression

$\large y_i$ $\large x_i$ simple linear regression $\large y$ :

\begin{matrix} {\hat{y}}_{i} = β_{0} + β_{1} x \\ y_{i} = β_{0} + β_{1} x_{i} + ϵ_{i} \\ = {\hat{y}}_{i} + ϵ_{i} \end{matrix}

where,

\begin{matrix} \begin{aligned} β_{1} & = \frac{\sum (x_{i} - \bar{x}) (y_{i} - \bar{y})}{\sum (x_{i} - \bar{x})^{2}} \\ = \frac{n \sum x_{i} y_{i} - \sum x_{i} \sum y_{i}}{n \sum x_{i}^{2} - (\sum x_{i})^{2}} \end{aligned} \\ β_{0} = \bar{y} - β_{1} \bar{x} \end{matrix}

\begin{matrix} Sum of Squares Total (SST) \\ = \sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} \\ Sum of Squares Regression (SSR) \\ = \sum_{i = 1}^{n} (\hat{y_{i}} - \bar{y})^{2} \\ Sum of Squares Error (SSE) \\ = \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2} \\ S S T = S S R + S S E \\ R^{2} = \frac{S S R}{S S T} \end{matrix}

$\large y_i$ $\large k$ $\large x_i$ multiple linear regression $\large y$ :

\begin{matrix} {\hat{y}}_{i} = β_{0} + β_{1} x_{i, 1} + β_{2} x_{i, 2} + . . . + β_{k} x_{i, k} \\ y_{i} = β_{0} + β_{1} x_{i, 1} + β_{2} x_{i, 2} + . . . + β_{k} x_{i, k} + ϵ_{i} \\ = {\hat{y}}_{i} + ϵ_{i} \end{matrix}

\begin{matrix} R^{2} = \frac{S S R}{S S T} \\ {\bar{R}}^{2} = 1 - \frac{(1 - R^{2}) (n - 1)}{n - k - 1} \end{matrix}

$\large \bar{R}^2$ $\large R^2$ .