Least squares line through (-2, 4), (-1, 3), (0, 3), (1, 1), (2, 0) Step by Step

Step by step

The question

Question: find the least squares line through (−2, 4), (−1, 3), (0, 3), (1, 1) and (2, 0).

(−2, 4), (−1, 3), (0, 3), (1, 1), (2, 0)

We want a line y = b + m x.

These 5 points are not all on one line, so we find the line that fits best.

line: y = b + m x (b: where it starts, m: slope)

1 One equation per point

Each point gives one equation: put its x and y into y = b + m x.

5 points, 5 equations, but only 2 unknowns, b and m.

(−2, 4): b + (−2)m = 4

(−1, 3): b + (−1)m = 3

As a matrix: A has a column of 1s for b and a column of the x values for m.

The right side holds the y values.

No line goes through all 5 points exactly.

[1, −2; 1, −1; 1, 0; 1, 1; 1, 2] × [b; m] = [4; 3; 3; 1; 0]

2 Write the normal equations

The best b and m solve A transpose A, times x, equals A transpose y.

These are called the normal equations.

With them, the leftovers are at a right angle to the columns of A: as small as they can be.

A^TA x = A^Ty

A transpose: the rows of A become its columns.

A^T = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2]

Careful!

A transpose goes in front of A, and in front of y.

Both sides get it.

Careful: Aᵀ in front, on both sides

3 Work out the two sides

A transpose times A.

A^TA = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [1, −2; 1, −1; 1, 0; 1, 1; 1, 2]

A^TA = [1 × 1 + 1 × 1 + 1 × 1 + 1 × 1 + 1 × 1, 1 × (−2) + 1 × (−1) + 1 × 0 + 1 × 1 + 1 × 2; −2 × 1 + (−1) × 1 + 0 × 1 + 1 × 1 + 2 × 1, −2 × (−2) + (−1) × (−1) + 0 × 0 + 1 × 1 + 2 × 2]

The top left counts the points: 5.

A^TA = [5, 0; 0, 10]

A transpose times y.

A^Ty = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [4; 3; 3; 1; 0]

A^Ty = [1 × 4 + 1 × 3 + 1 × 3 + 1 × 1 + 1 × 0; −2 × 4 + (−1) × 3 + 0 × 3 + 1 × 1 + 2 × 0]

A^Ty = [11; −10]

4 Solve for b and m

The two equations as an augmented matrix: the unknowns are b and m.

= [5, 0, 11; 0, 10, −10]

We call the rows R 1 and R 2.

→ [1, 0, 11 over 5; 0, 10, −10]

→ [1, 0, 11 over 5; 0, 1, −1]

Row 1 says b = 11/5.

b = 115

Row 2 says m = −1.

m = −1

So the least squares line is y = −x + 11/5.

y = −x + 115

5 Check

Let's check.

The leftover of a point is its y minus the line's y.

For the best line, the leftovers add to 0, and x times leftover adds to 0.

Each y minus the line's y at that x: −1/5, −1/5, 4/5, −1/5 and −1/5.

leftovers: 4 − 215 = −15, 3 − 165 = −15, 3 − 115 = 45, 1 − 65 = −15, 0 − 15 = −15

Added: 0.

added: −15 + (−15) + 45 + (−15) + (−15) = 0

x times leftover, added: 0.

Both are 0, so the answer is right!

x × leftover: −2 × (−15) + −1 × (−15) + 0 × 45 + 1 × (−15) + 2 × (−15)

= 0

Let's look back.

First: one equation per point.

Then: write the normal equations.

Then: work out the two sides.

Then: solve for b and m.

Last: check.

Answer