Machine Learning
Request access
Menu

Machine Learning

Glossary

111 machine learning words, explained without math. New here? Start with the basics, then training and checking.

The basics

Machine learning
Teaching a computer to spot a pattern from examples instead of writing the rules by hand. Show it a thousand houses with their prices and it learns how size and location relate to price.
Model
The pattern the computer learned, saved so it can make guesses about new examples. Like a recipe it wrote for itself from watching you cook.
Prediction
The model’s guess for one new example: a price, a yes or no, a category.
AI (artificial intelligence)
The broad idea of computers doing things that seem smart. Machine learning is the most common way it is done today.
Algorithm
The step-by-step method used to learn the pattern. Different algorithms learn in different ways, like different study techniques.
Container
In this workspace, one model project: its data, its goal, its versions and its avatar, kept together.
Version
One saved attempt at a model. Each new try gets a new number, so you can compare and go back.
Model contract
The written rules for a container before it trains: what goes in, what it predicts, how it is scored and what it must beat.

Your data

Dataset
The table of examples the model learns from, usually a spreadsheet saved as CSV.
Row (example)
One example in your data: one house, one patient, one sale.
Feature (input column)
A piece of information the model looks at to make its guess: size, age, price yesterday.
Target
The column with the answer you want the model to predict. In a house dataset, the price.
Label
The known answer for one row in the training data. “Malignant”, “cat”, “$420,000”.
CSV
A plain spreadsheet file: values separated by commas, one row per line. Any spreadsheet app can save one.
Missing values
Empty cells. Some models handle them on their own; others need them filled in first.
Categorical data
Values that are names, not amounts: color, city, product type.
Numeric data
Values that are amounts you can compare: price, age, temperature.
Time series
Data where order in time matters: daily prices, hourly sales, sensor readings.
Imbalanced data
When one answer is far more common than another, like 99 normal payments to 1 fraud. A model can look accurate by always guessing the common answer, so accuracy alone misleads.
Data leakage
When information the model shouldn’t have sneaks into training, like tomorrow’s price or the answer itself. Scores look amazing in testing and fall apart in real use.

Kinds of learning

Supervised learning
Learning from examples that come with the right answer, like flashcards with the answer on the back.
Unsupervised learning
Finding groups or odd cases when there is no answer key. A person has to check whether what it found makes sense.
Classification
Predicting a category: spam or not, which digit, which disease type.
Regression
Predicting a number: a price, a score, how many units will sell.
Clustering
Grouping similar rows together without labels, like sorting customers into types.
Anomaly detection
Spotting rows that don’t look like the rest: an odd transaction, a sensor spike.
Forecasting
Predicting a future value from past values in time order.
State
The situation something is in at one moment: clear, misty or rainy; logged in or out; calm or volatile.
Markov property
The assumption that the next step depends only on where you are now, not on how you got here. Simple, and often good enough.
Transition matrix
A table of chances: each row is where you are now, each column is where you might go next, and each row adds up to 100%.
Regime
A stretch of time when things behave a certain way, like a calm or a volatile market. Models often do better when they know which regime they’re in.
Deep learning
Machine learning with large neural networks, very good at images, sound and text. Usually needs more data and a GPU.

Training and checking

Training
Showing the model examples where the answer is known, so it learns the pattern.
Held-back rows (test set)
Examples hidden from the model while it learns, used only at the end to check it on something new. The most honest score comes from here.
Split
Dividing your rows into a part to learn from and a part to test on.
Random split
Picking test rows at random. Fine when rows don’t depend on each other; wrong for time order or grouped data.
Time split (walk-forward)
Training on the past and testing on what came after, the way the model will be used for real.
Group split
Keeping all rows from one experiment, batch or patient on the same side, so the test shows how it does on a new group.
Embargo (gap)
Skipping some rows between training and test in time-ordered data, so answers that overlap in time can’t leak across.
Cross-validation (folds)
Splitting the training rows into parts and testing on each part in turn. Shows whether results are steady or just lucky.
Fold
One of those parts. With 3 folds, the model is trained and tested 3 times.
Spread (± standard deviation)
How much the score changes from fold to fold. Small spread means the result is steady; big spread means it depends on luck.
Baseline
The simplest possible guess: the average value, or the most common answer. A model has to beat it to be worth using.
Settings (hyperparameters)
Knobs you choose before training: how deep each tree goes, how fast it learns, how many trees.
Tuning
Trying many settings and keeping the ones that score best on the folds. The held-back rows are never used to choose.
Model comparison
Training several kinds of model on the same folds and keeping the one that does best. Every model’s held-back score is still shown.
Seed
A number that fixes the randomness, so running the same thing again gives the same result.
Reproducibility
Being able to rerun a training and get the same model, because the data, settings, code and seed are all recorded.

Scores

Accuracy
Out of 100 new examples, how many it gets right. Easy to read, but misleading when one answer is much more common.
Average error (MAE)
How far off the guesses are, on average, in the data’s own units. An MAE of $29,000 means a typical guess is about $29,000 too high or too low. Lower is better.
RMSE
Like average error, but big misses count extra. Useful when one huge mistake is worse than several small ones.
R² (variation explained)
How much of the difference between values the model explains. 100% is perfect; 0% is no better than always guessing the average.
Precision
When the model says “yes”, how often it is right. High precision means few false alarms.
Recall
Of all the real “yes” cases, how many the model caught. High recall means few missed cases.
F1 score
One number that balances precision and recall, out of 100. Good when both false alarms and missed cases matter.
Confusion matrix
A grid of what the model predicted against what was true. The diagonal is right answers; everything off it shows exactly which answers it mixes up.
False positive (false alarm)
The model said yes, the truth was no. A healthy patient flagged as sick.
False negative (missed case)
The model said no, the truth was yes. A sick patient missed.
Probability
How likely the model thinks something is, from 0% to 100%.
Calibration
Whether the model’s probabilities are honest. If it says 70% for many cases, about 70 out of 100 should come true.
Brier score
Scores probability guesses: 0 is perfect, higher is worse.
Pinball loss
Scores range forecasts (low, middle, high guesses). Lower is better.
Coverage
How often the true value lands inside the predicted range. An 80% range should catch about 80 out of 100.
Hit rate (direction)
How often the model gets the direction right: up or down.

When things go wrong

Overfitting
Doing much better on examples it studied than on new ones, like a student who memorized last year’s test. Shown by a big gap between training-row and held-back scores.
Underfitting
The model is too simple to catch the pattern at all. Scores are poor everywhere, even on its training rows.
Fit check
Comparing the score on rows it trained on, on the folds, and on held-back rows. Close together is healthy; a big drop on new rows means overfitting.
Generalization
How well what the model learned carries over to new data. It’s the whole point.
Drift
When real-world data slowly stops looking like the training data, so a model that was good gets worse over time.
Bias (in data)
When the training examples don’t fairly represent the people or cases the model will be used on, so it does worse for some of them.
Survivorship bias
Training only on what survived, like today’s stocks, and missing the ones that failed. Results look better than reality.

Kinds of models

Decision tree
A flowchart of yes/no questions (“Is income over $50k?”) that ends in a guess. Easy to follow, but one tree alone is weak.
Gradient-boosted trees
Hundreds of small trees, each one fixing the mistakes of the ones before. Usually the most accurate choice for tables of data.
Random forest
Many trees trained on different random slices of the data, then averaged. Sturdy and hard to overfit.
Linear regression
Draws the best straight-line relationship between inputs and a number. Simple, fast, easy to explain, and often hard to beat on small data.
Logistic regression
The yes/no version of linear regression: turns inputs into a probability.
Neural network
Layers of simple math units connected together, loosely inspired by the brain. Powerful for images, sound and text.
Convolutional network (CNN)
A neural network built for images: it looks for edges and shapes anywhere in the picture.
Transformer
The kind of neural network behind modern language models. Very good at text and increasingly at images.
Pretrained model
A model someone already trained on huge amounts of data, which you adapt to your task with far fewer examples.
Fine-tuning
Taking a pretrained model and training it a little more on your own labeled examples.
Embedding
A list of numbers that captures what an image or text means, so similar things get similar numbers.
k-means
A simple clustering method that sorts rows into k groups by how close they are.
Isolation forest
An anomaly detector that finds rows that are easy to separate from the rest.
Markov chain
A model of a sequence that only looks at the current state to guess the next one. Like saying “if it’s raining now, there’s a 66% chance it’s still raining next hour.”
Hidden Markov model (HMM)
A Markov chain where the real state is hidden (like a calm or stormy market) and you only see clues it gives off. It works out which hidden state you were probably in.

Using a model

Inference
Using a trained model to make predictions on new data.
Deployment
Putting a model where apps and people can use it, usually behind an API.
API
A way for one program to ask another for something over the internet, like asking this workspace for a prediction.
API key
A secret password for an app, so the API knows who is asking and what they may do. Keep it private.
Model file (artifact)
The saved, trained model. Its checksum proves it hasn’t been changed.
Checksum
A fingerprint of a file. If even one byte changes, the fingerprint changes.
Worker
A computer that does the training for you and sends the results back to the workspace.
GPU
A graphics chip that does many calculations at once. Makes deep learning much faster.
Monitoring
Watching a model after launch: are predictions still accurate, is the data drifting, is it fast enough.
Latency
How long one prediction takes to come back.

Tools and libraries

Python
The most common programming language for machine learning.
scikit-learn
A free Python library with most classic models (linear, trees, forests, clustering) and tools for splitting and scoring. A great place to start.
XGBoost
A popular free library for gradient-boosted trees, strong on tables of data.
LightGBM
Microsoft’s gradient-boosted trees library, very fast on large tables.
CatBoost
Yandex’s gradient-boosted trees library, especially good with categorical columns.
PyTorch
The most widely used library for building and training neural networks.
TensorFlow / Keras
Google’s neural-network library; Keras is its simpler front end.
Hugging Face
A hub of free pretrained models and datasets, especially for text and images.
pandas
A Python library for working with tables: loading CSVs, cleaning, joining.
NumPy
The Python library for fast math on lists of numbers. Almost every ML tool is built on it.
Jupyter notebook
A document where you write code, run it and see charts step by step. Popular for exploring data.
MLflow
An open-source tool for tracking experiments and storing model versions.
Kaggle
A website with free datasets and machine-learning competitions.
UCI ML Repository
A long-running university collection of public datasets used for testing models.
OpenML
An open online library of datasets and results that tools can download from directly.

See these ideas in action on the test datasets, or read which model fits your data.