Distinguish machine learning from classical programming, and define the roles of model, features, training data and predictions

What this dot point is asking

NESA wants you to draw the fundamental distinction between writing rules by hand (classical programming) and having an algorithm learn rules from data (machine learning). You also need to know the standard ML vocabulary: model, features, training data, labels, predictions.

The answer

Classical programming

The developer writes the rules. The program takes inputs and applies the rules to produce outputs.

rules + data --> answers

Example: a thermostat. "If temperature > 25, turn on the AC." The developer decides the rule.

Machine learning

The developer provides examples (data plus the correct answers). The algorithm learns the rules.

data + answers --> model

Then in use:

model + new data --> predictions

Example: an image classifier. The developer collects 100,000 photos labelled "cat" or "dog", and trains a model to predict the label from the pixels.

The standard vocabulary

• Training data: the examples used to train the model. Typically a table where each row is one example.
• Features: the input columns. For email spam, features might be subject_length, contains_url, sender_blacklisted. Features need to be numeric (or one-hot encoded categories) for most algorithms.
• Label (also called target): the answer column for each training example. "spam" or "not spam".
• Model: the trained artefact. Internally, a set of learned parameters that map features to predictions.
• Prediction: the model's output for a new, unseen example.
• Training: the algorithm that fits the model's parameters to the training data.
• Inference: using the trained model to make predictions on new data.

Worked Python

A minimal end-to-end ML workflow with scikit-learn:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Training data: 150 iris flowers, 4 features each, 3 species labels.
data = load_iris()
X = data.data         # features: sepal/petal lengths and widths
y = data.target       # label: species (0, 1, or 2)

# Split into training and test sets.
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Train.
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

# Predict on unseen examples.
predictions = model.predict(X_test)

# Evaluate.
print(f"Accuracy: {accuracy_score(y_test, predictions):.2f}")

The developer wrote no rules about iris species. The model learned the boundaries from the training data.

When to use ML vs classical

ML is the right tool when:

• The rules are complex, change over time, or are hard to articulate (spam, image recognition, machine translation).
• Labelled data is available in volume.
• An approximate answer is acceptable (predictions are probabilistic, not exact).

Classical programming is the right tool when:

• The rules are well-understood and stable (calculating GST, sorting a list, parsing JSON).
• Errors are unacceptable (banking transactions, control systems).
• The dataset is small or unavailable.

ML is a tool, not a default. Most software is still classical because the rules are clear and exact answers are required.

The training/test split

You never evaluate a model on data it has already seen. Standard practice:

• 60-80 percent of the data is the training set.
• 10-20 percent is the validation set, used during development to tune hyperparameters.
• 10-20 percent is the test set, used once at the end to estimate real-world performance.

If the test accuracy is much lower than training accuracy, the model is overfitting - it memorised the training data instead of learning patterns that generalise.