Fit a line, then question it
Build a regression baseline and inspect errors on unseen examples.
- Fit slope and intercept
- Measure mean absolute error
- Recognize extrapolation risk
Fit the relationship
Linear regression models a target as an intercept plus a weighted sum of features. With one feature, the least-squares slope is the covariance numerator divided by the feature’s squared-deviation sum. The intercept places the line through the means.
This relationship describes the examples you supplied. A good fit does not establish causation. A feature associated with a target might be a proxy, a consequence, or a coincidence. Look at individual errors and the measurement process.
Evaluate outside the fitting step
Mean absolute error averages absolute misses in the target’s units. It is easy to interpret: an MAE of two minutes means the average absolute timing error was two minutes on that evaluation set.
Predictions far outside the training range are extrapolation. A straight line may produce physically impossible values. Define a supported input range and compare against a simple baseline such as the training-target mean.
A small experiment you can run.
The invented data follows a perfect line so the test error is zero. This is a demonstration of mechanics, not evidence of real-world predictive quality.
xs, ys = [1, 2, 3, 4], [3, 5, 7, 9]
x_mean, y_mean = sum(xs)/len(xs), sum(ys)/len(ys)
denominator = sum((x-x_mean)**2 for x in xs)
if not denominator:
raise ValueError("The feature must vary")
slope = sum((x-x_mean)*(y-y_mean) for x, y in zip(xs, ys)) / denominator
intercept = y_mean - slope*x_mean
test_x, test_y = [5, 6], [11, 13]
predictions = [intercept+slope*x for x in test_x]
mae = sum(abs(a-b) for a, b in zip(predictions, test_y))/len(test_y)
print("Slope:", slope, "Intercept:", intercept, "Test MAE:", mae)
Save the file, open your terminal in that folder, and run python linear-regression-from-scratch.py. Use python3 or py if required by your installation. Setup guide
The original slope is 2, intercept is 1, and test MAE is 0.
Introduce a noisy observation.
- Change the last training target from 9 to 12.
- Recalculate the slope, intercept, and test MAE.
- Compare with a predictor that always returns the training mean.
Compare with a suggested solution
The fitted line tilts toward the unusually high target. Record whether test error improved, rather than treating closer fit to a noisy point as automatic progress. A baseline must also be evaluated on the same test rows.
One idea to take with you.
Make it part of your progress.
Finish the practice and answer the knowledge check to mark this lesson complete.
Go deeper with primary documentation
Optional references for further study. This lesson and its examples were written for Artificials.
scikit-learn: model evaluationscikit-learn: common pitfalls