Least squares line through (-2, 1), (-1, 0), (0, 2), (1, 2), (2, 5) Step by Step

Step by step

The question

Question: find the least squares line through (−2, 1), (−1, 0), (0, 2), (1, 2) and (2, 5).

(−2, 1), (−1, 0), (0, 2), (1, 2), (2, 5)

We want a line y = b + m x.

These 5 points are not all on one line, so we find the line that fits best.

line: y = b + m x (b: where it starts, m: slope)

1 One equation per point

Each point gives one equation: put its x and y into y = b + m x.

5 points, 5 equations, but only 2 unknowns, b and m.

(−2, 1): b + (−2)m = 1

(−1, 0): b + (−1)m = 0

As a matrix: A has a column of 1s for b and a column of the x values for m.

The right side holds the y values.

No line goes through all 5 points exactly.

[1, −2; 1, −1; 1, 0; 1, 1; 1, 2] × [b; m] = [1; 0; 2; 2; 5]

2 Write the normal equations

The best b and m solve A transpose A, times x, equals A transpose y.

These are called the normal equations.

With them, the leftovers are at a right angle to the columns of A: as small as they can be.

A^TA x = A^Ty

A transpose: the rows of A become its columns.

A^T = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2]

Careful!

A transpose goes in front of A, and in front of y.

Both sides get it.

Careful: Aᵀ in front, on both sides

3 Work out the two sides

A transpose times A.

A^TA = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [1, −2; 1, −1; 1, 0; 1, 1; 1, 2]

A^TA = [1 × 1 + 1 × 1 + 1 × 1 + 1 × 1 + 1 × 1, 1 × (−2) + 1 × (−1) + 1 × 0 + 1 × 1 + 1 × 2; −2 × 1 + (−1) × 1 + 0 × 1 + 1 × 1 + 2 × 1, −2 × (−2) + (−1) × (−1) + 0 × 0 + 1 × 1 + 2 × 2]

The top left counts the points: 5.

A^TA = [5, 0; 0, 10]

A transpose times y.

A^Ty = [1, 1, 1, 1, 1; −2, −1, 0, 1, 2] × [1; 0; 2; 2; 5]

A^Ty = [1 × 1 + 1 × 0 + 1 × 2 + 1 × 2 + 1 × 5; −2 × 1 + (−1) × 0 + 0 × 2 + 1 × 2 + 2 × 5]

A^Ty = [10; 10]

4 Solve for b and m

The two equations as an augmented matrix: the unknowns are b and m.

= [5, 0, 10; 0, 10, 10]

We call the rows R 1 and R 2.

→ [1, 0, 2; 0, 10, 10]

→ [1, 0, 2; 0, 1, 1]

Row 1 says b = 2.

b = 2

Row 2 says m = 1.

m = 1

So the least squares line is y = x + 2.

y = x + 2

5 Check

Let's check.

The leftover of a point is its y minus the line's y.

For the best line, the leftovers add to 0, and x times leftover adds to 0.

Each y minus the line's y at that x: 1, −1, 0, −1 and 1.

leftovers: 1 − 0 = 1, 0 − 1 = −1, 2 − 2 = 0, 2 − 3 = −1, 5 − 4 = 1

Added: 0.

added: 1 + (−1) + 0 + (−1) + 1 = 0

x times leftover, added: 0.

Both are 0, so the answer is right!

x × leftover: −2 × 1 + −1 × (−1) + 0 × 0 + 1 × (−1) + 2 × 1

= 0

Let's look back.

First: one equation per point.

Then: write the normal equations.

Then: work out the two sides.

Then: solve for b and m.

Last: check.

Answer