Working with datasets

⏱ 30 min

Hook

You train a classifier and it scores 99% accuracy. You ship it, and in production it is barely better than a coin flip. Nothing was wrong with the model — you measured it on the same rows you trained it on, so it had already seen every answer. A dataset is not something you feed to a model whole; it is something you inspect, summarize, and split before a single prediction is fair to believe.

Concept

Working with a dataset means three habits, in order: load it into the (X, y) shape from the last lesson, describe it so you know what you are holding, and split it so your evaluation is honest.

Loading a tabular file is a job for the standard library's csv module: it hands you each row as a dict, and you coerce the fields into a numeric feature vector exactly as before. Describing means computing summary statistics — the count, minimum, maximum, and mean of each feature — plus the class balance, the count of examples per label. These numbers catch problems no model can fix: a feature that is constant carries no signal, a label that appears twice out of a thousand rows will be ignored by most models, and a wildly out-of-range value is usually a parsing bug.

Splitting is the habit that separates honest results from self-deception. You hold back part of the data as a test set the model never sees during training, so its score on that set estimates how it will do on data it has never met. The one rule that makes this work:

warning

Split before you decide anything from the data. If you compute a statistic — a mean to fill blanks, a scaling range, a choice of features — using the test rows, that information leaks into training, and your test score becomes a fiction. Splitting first is what keeps the test set a fair exam.

A leakage-safe split is also deterministic in a course setting: the same input always produces the same train and test sets, so your results are reproducible and a grader can check them. Selecting every third row for the test set is a simple deterministic rule.

from collections import Counter

labeled = [
    ((1.4, 0.2), "a"), ((4.7, 1.4), "b"), ((1.5, 0.2), "a"),
    ((4.5, 1.5), "b"), ((1.3, 0.3), "a"), ((4.9, 1.5), "b"),
]
print("class balance:", dict(Counter(label for _, label in labeled)))


def split(samples, every=3):
    train, test = [], []
    for i, sample in enumerate(samples):
        (test if i % every == 0 else train).append(sample)
    return train, test


train, test = split(labeled)
print("train:", len(train), "test:", len(test))

class balance: {'a': 3, 'b': 3}
train: 4 test: 2

Habit	What you compute	What it protects you from
Load	(X, y) from raw CSV rows	Strings where numbers are expected
Describe	count, min, max, mean; class balance	Dead features, rare classes, bad parses
Split	disjoint train and test sets	Measuring a model on rows it trained on

🧠 Knowledge check

1. A model scores 99% on the same rows it was trained on but far worse in production. What most likely happened?

a) The model is simply excellent and production data is brokenb) It was evaluated on training data, so the score reflects memorization, not generalizationc) The dataset was too small to train ond) The features were coerced to floats

2. What does "data leakage" mean in the context of a train/test split?

a) The dataset file is corrupted and loses rowsb) Information from the test set influences training — for example a statistic computed over all rows before splittingc) The model runs out of memory on large datad) Labels are stored in a separate file from features

Worked example

Let me load a small dataset from CSV, describe it, and split it — the full sequence on real bytes. Suppose readings.csv holds device readings labeled by machine state, and I want to know what I am holding before I model anything.

import csv
from collections import Counter
from io import StringIO

# Standing in for a file on disk so the example runs as shown:
csv_text = """vibration,temperature,state
0.8,41.0,healthy
3.9,72.0,faulty
0.6,39.5,healthy
4.2,75.0,faulty
0.9,43.0,healthy
3.7,70.0,faulty
"""

rows = list(csv.DictReader(StringIO(csv_text)))
X = [(float(r["vibration"]), float(r["temperature"])) for r in rows]
y = [r["state"] for r in rows]


def summarize(column):
    return {
        "min": min(column),
        "max": max(column),
        "mean": round(sum(column) / len(column), 2),
    }


vibration = [v for v, _ in X]
temperature = [t for _, t in X]
print("vibration: ", summarize(vibration))
print("temperature:", summarize(temperature))
print("class balance:", dict(Counter(y)))

test = [(X[i], y[i]) for i in range(len(X)) if i % 3 == 0]
train = [(X[i], y[i]) for i in range(len(X)) if i % 3 != 0]
print("train:", len(train), "test:", len(test))

vibration:  {'min': 0.6, 'max': 4.2, 'mean': 2.35}
temperature: {'min': 39.5, 'max': 75.0, 'mean': 56.75}
class balance: {'healthy': 3, 'faulty': 3}
train: 4 test: 2

Read what the summary tells me before any model runs. Vibration ranges from 0.6 to 4.2 and temperature from about 40 to 75, so the two features live on very different scales — a fact that will matter the moment I compute distances between points. The class balance is even, three of each, so accuracy will be a meaningful metric rather than one dominated by a majority class. Only after seeing all this do I split, and I split with a fixed rule so the test set is the same every run. Notice the split comes last: every number I printed was for understanding the data, not for making a modeling decision that would leak.

Hands-on

Your turn, with a different dataset. The exercise below hands you a CSV of study records — hours studied and prior score, labeled pass or fail — and asks you to load it into (X, y), print the per-feature min/max/mean and the class balance, then split it with the every-third-row rule. It runs in Python Arena, so there is nothing to install.

▶ Python Arenaai101-summarize-and-split

Open in Python Arena ↗

Success criteria: your summary reports the correct min, max, and mean per feature and the correct count per class, and your split returns disjoint train and test sets whose sizes match the every-third-row rule. The Arena checks these against the hidden dataset automatically.

Recap

• You can load a tabular dataset from CSV into the (X, y) shape using only the standard library.
• You can describe a dataset with per-feature count, min, max, and mean plus the class balance, and read those numbers for trouble before modeling.
• You can split data into disjoint train and test sets with a deterministic rule, and explain why splitting first prevents leakage.

Next up: turning that split into a prediction. You will set a baseline, then beat it with your first real model and measure the difference.

🧠 Knowledge check

1. Why use a deterministic split (for example, every third row to test) in a graded course exercise?

a) It guarantees the model reaches 100% accuracyb) It makes the train and test sets reproducible, so results can be checked and compared run to runc) It is the only way to avoid leakaged) Deterministic splits are always larger than random ones

2. You notice one feature has the same value in every row. What does that tell you?

a) The feature is the most important oneb) The feature is constant and carries no signal a model can learn fromc) The dataset is perfectly balancedd) The label column is missing

📌 Key takeaways

• Load, describe, then split — in that order — every time you meet a dataset.
• Summary stats and class balance catch dead features, rare classes, and parse bugs no model can fix. - Split before you decide anything from the data, and make the split deterministic so results are reproducible.