Lesson 1 of 8 · 14 min

The regression model and least squares

Simple linear regression explains the variation in Y with one variable X by fitting the straight line that makes the sum of squared vertical misses as small as possible.

In short

  • The dependent variable (Y) is the one we want to explain; the independent variable (X) is the one doing the explaining. We say “Y is regressed on X”.
  • Population model: Yi=b0+b1Xi+εiY_i = b_0 + b_1X_i + \varepsilon_i. b0b_0 is the intercept, b1b_1 the slope coefficient, εi\varepsilon_i the error term.
  • Ordinary least squares (OLS) picks b^0\hat b_0 and b^1\hat b_1 to minimise the sum of squares error (SSE), the sum of squared residuals.
  • Slope = covariance of X and Y ÷ variance of X. Intercept = Yˉ−b^1Xˉ\bar Y - \hat b_1\bar X, so the line runs through (Xˉ,Yˉ)(\bar X, \bar Y).
  • The slope and the correlation always have the same sign, because both take the sign of the covariance.

Unlock this lesson free for 7 days

Create a free account to get 7 days of full access — every lesson, video, flashcard, mock and the question bank. No card needed.

Start your 7-day free trialLog in
The regression model and least squares · Simple Linear Regression