JIT — Intro to Data Science · Python Lab · Problem 29 of 30

Fitting a line and testing it

A model judged on students it has never seen — and the perfect score that means something has gone badly wrong.

From a line you can see to a line you can use

Problem 27 drew a line of best fit. Problem 28 said the relationship was real. This one fits the line properly, uses it to predict marks for students the line has never seen, and measures how well it did.

Thirty students this time, because the next idea needs enough data to be split in two.

New words — train and test A line fitted to some data will always look good on that data — it was built from it. The only honest question is how it does on students it was not shown.
train_test_split deals the rows into two piles. The model learns from the training pile and is judged on the testing pile. test_size=0.3 keeps 30% back for judging.
random_state=42 fixes which rows go where, so you and a colleague running the same file get the same split. Leave it out and every run deals a different hand — the model appears to get better or worse each time you press run, and nothing has changed but the shuffle.
New words — fit and predict model.fit(X_train, y_train) finds the best slope and intercept from the training pile.
model.predict(X_test) uses them to guess the marks of the held-back students.
X is what you know, y is what you want to work out. Capital X because it may hold several columns; small y because it is always one.
Careful — X needs double brackets df[["hours"]], not df["hours"]. Scikit-learn expects X as a table of one or more columns, even when there is only one — the same single-column DataFrame you met in problem 20.
New word — R squared r2_score(y_test, predicted) asks how much of the variation in the real marks the line accounts for. 1.0 is perfect, 0 is no better than always guessing the average, and it can go negative if the line is worse than that.
On a single predictor it is exactly the r from problem 28, squared: r was 0.95, so R² lands near 0.90. The name is not a coincidence.
Careful — never let the answer into X If the marks column is included in X, the model is handed the very thing it is supposed to work out. R² comes back as exactly 1.000 and the model is worthless — it has learnt to copy, not to predict.
A perfect score is not a triumph. It is nearly always a sign that something you should not know has leaked into what you are allowed to know, and it is one of the most expensive mistakes in this subject because the number looks so good.

What you are building

Slope: 2.283 Intercept: 34.904 R-squared on the test set: 0.901

Each extra hour of study is worth about 2.3 marks, a student who studied nothing would be predicted around 35, and the line accounts for about 90% of the variation among students it had never seen.

Coming later R² judges a model that predicts a number. Problem 30 judges one that predicts a category — pass or fail — where being right 90% of the time can still mean being wrong about everybody who matters.

Build the programme

Your plan

The steps still to place

The whole programme is laid out below. Five pieces are missing. Two of them stop the programme, one produces a suspiciously perfect score, and the rest quietly report a number that is not the one you asked for.

Leave nothing on "choose…". A wrong pick does not always cause an error — sometimes it just prints something you did not expect, which is the harder kind of mistake to spot.

Run and read

Here is the whole programme. There is one file and one split, both fixed, so every run gives the same answer — which is the point of random_state.

Your programme

Output

Nothing has run yet.