Least squares line through (-2, 0), (-1, 1), (0, 1), (1, 3), (2, 5) Step by Step

Step by step

The question

Question: find the least squares line through (−2, 0), (−1, 1), (0, 1), (1, 3) and (2, 5).

(−2, 0), (−1, 1), (0, 1), (1, 3), (2, 5)

We want a line y = b + m x.

These 5 points are not all on one line, so we find the line that fits best.

line: y = b + m x (b: where it starts, m: slope)

1 One equation per point

Each point gives one equation: put its x and y into y = b + m x.

5 points, 5 equations, but only 2 unknowns, b and m.

(−2, 0): b + (−2)m = 0

(−1, 1): b + (−1)m = 1

As a matrix: A has a column of 1s for b and a column of the x values for m.

The right side holds the y values.

No line goes through all 5 points exactly.

[1, −2; 1, −1; 1, 0; 1, 1; 1, 2] × [b; m] = [0; 1; 1; 3; 5]

2 Write the normal equations

The best b and m solve A transpose A, times x, equals A transpose y.

These are called the normal equations.

With them, the leftovers are at a right angle to the columns of A: as small as they can be.

A^TA x = A^Ty

A transpose: the rows of A become its columns.

A^T = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2]

Careful!

A transpose goes in front of A, and in front of y.

Both sides get it.

Careful: Aᵀ in front, on both sides

3 Work out the two sides

A transpose times A.

A^TA = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [1, −2; 1, −1; 1, 0; 1, 1; 1, 2]

A^TA = [1 × 1 + 1 × 1 + 1 × 1 + 1 × 1 + 1 × 1, 1 × (−2) + 1 × (−1) + 1 × 0 + 1 × 1 + 1 × 2; −2 × 1 + (−1) × 1 + 0 × 1 + 1 × 1 + 2 × 1, −2 × (−2) + (−1) × (−1) + 0 × 0 + 1 × 1 + 2 × 2]

The top left counts the points: 5.

A^TA = [5, 0; 0, 10]

A transpose times y.

A^Ty = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [0; 1; 1; 3; 5]

A^Ty = [1 × 0 + 1 × 1 + 1 × 1 + 1 × 3 + 1 × 5; −2 × 0 + (−1) × 1 + 0 × 1 + 1 × 3 + 2 × 5]

A^Ty = [10; 12]

4 Solve for b and m

The two equations as an augmented matrix: the unknowns are b and m.

= [5, 0, 10; 0, 10, 12]

We call the rows R 1 and R 2.

→ [1, 0, 2; 0, 10, 12]

→ [1, 0, 2; 0, 1, 6 over 5]

Row 1 says b = 2.

b = 2

Row 2 says m = 6/5.

m = 65

So the least squares line is y = 6/5 times x + 2.

y = 65x + 2

5 Check

Let's check.

The leftover of a point is its y minus the line's y.

For the best line, the leftovers add to 0, and x times leftover adds to 0.

Each y minus the line's y at that x: 2/5, 1/5, −1, −1/5 and 3/5.

leftovers: 0 − (−25) = 25, 1 − 45 = 15, 1 − 2 = −1, 3 − 165 = −15, 5 − 225 = 35

Added: 0.

added: 25 + 15 + (−1) + (−15) + 35 = 0

x times leftover, added: 0.

Both are 0, so the answer is right!

x × leftover: −2 × 25 + −1 × 15 + 0 × (−1) + 1 × (−15) + 2 × 35

= 0

Let's look back.

First: one equation per point.

Then: write the normal equations.

Then: work out the two sides.

Then: solve for b and m.

Last: check.

Answer