JIT Intro to Data Science — Python Lab 9: Filling the gaps in a register

The marks that were never entered

Two students missed the exam. The clerk still had to type something into every row, so she typed -999 — a code the department uses to mean "no value here". Files arrive like this constantly. Sometimes the code is -999, sometimes -1, sometimes NA or an empty cell.

Python has no idea any of that is special. To Python, -999 is a number like any other, and it will cheerfully add it to your total.

New word — sentinel value A value chosen to mean "this is missing" rather than to be used as data. It is only a convention between people. Nothing in the file marks it as different, so nothing warns you before it wrecks your arithmetic.

What two sentinels do to an average

Six real marks average about 76. Add two entries of −999 and the average of all eight comes out at -192.625. Nobody scored below zero, so a negative average is obviously wrong — but only if somebody looks. A sentinel of 0 instead of -999 would have dragged the average to 57 and looked entirely believable.

The giveaway is the smallest value Before you calculate anything, look at the smallest and largest values in a column. A minimum of −999 in a column of exam marks tells you immediately that something in there is not a mark. This is the first thing to do with any new dataset, and it takes one line.

New words — count and index marks.count(-999) says how many times that value appears. If it appears none, you get 0.
marks.index(-999) says where the first one sits. If it appears none, this one does not hand back a polite answer — it stops the programme. Two similar-looking methods with very different manners when the thing is absent.

Filling a gap Once you know the position, you overwrite it: marks[first] = fill. That reaches into the list and changes the item where it stands.
Watch the difference between = and == here. One puts a value in. The other merely asks a question, throws the answer away, and leaves your list exactly as it was — without a word of complaint.

What to put in the gap is your decision, not Python's

Python will accept anything. The choice is yours, and each choice says something different:

Fill with	What you are claiming	Effect on the average
0	they sat the exam and scored nothing	drags it down hard
35	treat an absence as a bare pass	drags it down
76	assume they were an average student	leaves it almost unmoved

None of these is the truth. The truth is that you do not know. Filling gaps is a decision you should be able to defend, and the honest fourth option — leaving them out of the calculation entirely — needs tools you do not have yet.

What you are building

As recorded: [78, -999, 65, 80, -999, 54, 88, 92] Entries: 8 Smallest value on file: -999 Missing: 2 Average before anything is done: -192.625 After filling: [78, 76, 65, 80, 76, 54, 88, 92] Gaps left: 0 Average now: 76.125

Coming later Two gaps can be filled by hand. Two hundred cannot. In problem 23 pandas does the whole column in one line, and it can also drop the incomplete rows instead — the same decision, taken at scale.

Build the programme

Your plan

The steps still to place

The whole programme is laid out below. Five pieces are missing. Two of the wrong choices leave your list quietly unchanged or overwrite a mark that was perfectly good, and Python will not mention either.

Leave nothing on "choose…". A wrong pick does not always cause an error — sometimes it just prints something you did not expect, which is the harder kind of mistake to spot.

Run and read

Here is the whole programme. The value used to fill the gaps is yours to choose, and so is the register itself — as long as exactly two entries are the missing code. Try filling with 0, then with 76, and watch the last line.

the register as it arrived (exactly two -999 entries)

the value to put in the gaps

Your programme

Output

Nothing has run yet.