JIT — Intro to Data Science · Python Lab · Problem 27 of 30

Marks against hours studied

One dot per student, a line fitted through them — and a second file where the same line means nothing at all.

Does one thing move with the other?

Every chart so far has described one column. This one asks a question about two: as the hours a student puts in go up, do their marks go up with them?

A scatter plot answers that by putting one dot per student — hours along the bottom, marks up the side. You are not reading individual dots. You are looking at the cloud they make.

New word — scatter plt.scatter(df["hours"], df["marks"]). Two columns of the same length, taken pairwise: the first student's hours with the first student's marks, and so on down the file.
The order of the rows does not matter. Shuffle the file and the picture is identical, because each dot carries both of its numbers with it.
Careful — scatter, not plot plt.plot() joins the points with a line in file order. On data that arrives sorted, that line looks meaningful and is not — it is a join-the-dots of whatever order the rows happened to be in. Use plot for something measured over time, where the order is real. Use scatter when you are comparing two measurements of the same thing.
What to look for Direction — does the cloud rise to the right, fall, or lie flat?
Strength — is it a tight band or a shapeless blob? A tight rising band means hours tell you a lot about marks; a blob means they tell you almost nothing.
Oddities — a dot far from the rest is worth asking about, exactly as in problem 24.
New word — line of best fit np.polyfit(x, y, 1) works out the straight line that passes closest to all the points, and hands back two numbers: the slope and the intercept. The line itself is then m * x + c — the same y = mx + c you know from school.
A slope of about 4 means that each extra hour of study goes with about four more marks. It does not mean the hour caused them.
Two files, and the difference is the point study_strong.csv — the same ten students, where hours and marks rise together.
study_none.csv — the same hours, with marks that have nothing to do with them.
Both files fit a line, because a line can always be fitted. On the second one the line comes out almost flat and the dots are scattered nowhere near it. Fitting a line never tells you whether fitting a line was worth doing — your eyes do that, and problem 28 puts a number on it.
Coming later You will judge these two clouds by eye. Problem 28 measures the same thing with a correlation coefficient, and asks whether a relationship this strong could have turned up by chance.

Build the programme

Your plan

The steps still to place

The whole programme is laid out below. Five pieces are missing. None of them will stop the programme, and two of them produce a picture that looks convincing and shows something else entirely.

Leave nothing on "choose…". A wrong pick does not always cause an error — sometimes it just prints something you did not expect, which is the harder kind of mistake to spot.

Run and read

Here is the whole programme. Run it on both files. The second one is the important one — it is what "no relationship" actually looks like, and it is worth recognising on sight.

Your programme

Output

Nothing has run yet.