Glossary
111 machine learning words, explained without math. New here? Start with the basics, then training and checking.
The basics
- Machine learning
- Teaching a computer to spot a pattern from examples instead of writing the rules by hand. Show it a thousand houses with their prices and it learns how size and location relate to price.
- Model
- The pattern the computer learned, saved so it can make guesses about new examples. Like a recipe it wrote for itself from watching you cook.
- Prediction
- The model’s guess for one new example: a price, a yes or no, a category.
- AI (artificial intelligence)
- The broad idea of computers doing things that seem smart. Machine learning is the most common way it is done today.
- Algorithm
- The step-by-step method used to learn the pattern. Different algorithms learn in different ways, like different study techniques.
- Container
- In this workspace, one model project: its data, its goal, its versions and its avatar, kept together.
- Version
- One saved attempt at a model. Each new try gets a new number, so you can compare and go back.
- Model contract
- The written rules for a container before it trains: what goes in, what it predicts, how it is scored and what it must beat.
Your data
- Dataset
- The table of examples the model learns from, usually a spreadsheet saved as CSV.
- Row (example)
- One example in your data: one house, one patient, one sale.
- Feature (input column)
- A piece of information the model looks at to make its guess: size, age, price yesterday.
- Target
- The column with the answer you want the model to predict. In a house dataset, the price.
- Label
- The known answer for one row in the training data. “Malignant”, “cat”, “$420,000”.
- CSV
- A plain spreadsheet file: values separated by commas, one row per line. Any spreadsheet app can save one.
- Missing values
- Empty cells. Some models handle them on their own; others need them filled in first.
- Categorical data
- Values that are names, not amounts: color, city, product type.
- Numeric data
- Values that are amounts you can compare: price, age, temperature.
- Time series
- Data where order in time matters: daily prices, hourly sales, sensor readings.
- Imbalanced data
- When one answer is far more common than another, like 99 normal payments to 1 fraud. A model can look accurate by always guessing the common answer, so accuracy alone misleads.
- Data leakage
- When information the model shouldn’t have sneaks into training, like tomorrow’s price or the answer itself. Scores look amazing in testing and fall apart in real use.
Kinds of learning
- Supervised learning
- Learning from examples that come with the right answer, like flashcards with the answer on the back.
- Unsupervised learning
- Finding groups or odd cases when there is no answer key. A person has to check whether what it found makes sense.
- Classification
- Predicting a category: spam or not, which digit, which disease type.
- Regression
- Predicting a number: a price, a score, how many units will sell.
- Clustering
- Grouping similar rows together without labels, like sorting customers into types.
- Anomaly detection
- Spotting rows that don’t look like the rest: an odd transaction, a sensor spike.
- Forecasting
- Predicting a future value from past values in time order.
- State
- The situation something is in at one moment: clear, misty or rainy; logged in or out; calm or volatile.
- Markov property
- The assumption that the next step depends only on where you are now, not on how you got here. Simple, and often good enough.
- Transition matrix
- A table of chances: each row is where you are now, each column is where you might go next, and each row adds up to 100%.
- Regime
- A stretch of time when things behave a certain way, like a calm or a volatile market. Models often do better when they know which regime they’re in.
- Deep learning
- Machine learning with large neural networks, very good at images, sound and text. Usually needs more data and a GPU.
Training and checking
- Training
- Showing the model examples where the answer is known, so it learns the pattern.
- Held-back rows (test set)
- Examples hidden from the model while it learns, used only at the end to check it on something new. The most honest score comes from here.
- Split
- Dividing your rows into a part to learn from and a part to test on.
- Random split
- Picking test rows at random. Fine when rows don’t depend on each other; wrong for time order or grouped data.
- Time split (walk-forward)
- Training on the past and testing on what came after, the way the model will be used for real.
- Group split
- Keeping all rows from one experiment, batch or patient on the same side, so the test shows how it does on a new group.
- Embargo (gap)
- Skipping some rows between training and test in time-ordered data, so answers that overlap in time can’t leak across.
- Cross-validation (folds)
- Splitting the training rows into parts and testing on each part in turn. Shows whether results are steady or just lucky.
- Fold
- One of those parts. With 3 folds, the model is trained and tested 3 times.
- Spread (± standard deviation)
- How much the score changes from fold to fold. Small spread means the result is steady; big spread means it depends on luck.
- Baseline
- The simplest possible guess: the average value, or the most common answer. A model has to beat it to be worth using.
- Settings (hyperparameters)
- Knobs you choose before training: how deep each tree goes, how fast it learns, how many trees.
- Tuning
- Trying many settings and keeping the ones that score best on the folds. The held-back rows are never used to choose.
- Model comparison
- Training several kinds of model on the same folds and keeping the one that does best. Every model’s held-back score is still shown.
- Seed
- A number that fixes the randomness, so running the same thing again gives the same result.
- Reproducibility
- Being able to rerun a training and get the same model, because the data, settings, code and seed are all recorded.
Scores
- Accuracy
- Out of 100 new examples, how many it gets right. Easy to read, but misleading when one answer is much more common.
- Average error (MAE)
- How far off the guesses are, on average, in the data’s own units. An MAE of $29,000 means a typical guess is about $29,000 too high or too low. Lower is better.
- RMSE
- Like average error, but big misses count extra. Useful when one huge mistake is worse than several small ones.
- R² (variation explained)
- How much of the difference between values the model explains. 100% is perfect; 0% is no better than always guessing the average.
- Precision
- When the model says “yes”, how often it is right. High precision means few false alarms.
- Recall
- Of all the real “yes” cases, how many the model caught. High recall means few missed cases.
- F1 score
- One number that balances precision and recall, out of 100. Good when both false alarms and missed cases matter.
- Confusion matrix
- A grid of what the model predicted against what was true. The diagonal is right answers; everything off it shows exactly which answers it mixes up.
- False positive (false alarm)
- The model said yes, the truth was no. A healthy patient flagged as sick.
- False negative (missed case)
- The model said no, the truth was yes. A sick patient missed.
- Probability
- How likely the model thinks something is, from 0% to 100%.
- Calibration
- Whether the model’s probabilities are honest. If it says 70% for many cases, about 70 out of 100 should come true.
- Brier score
- Scores probability guesses: 0 is perfect, higher is worse.
- Pinball loss
- Scores range forecasts (low, middle, high guesses). Lower is better.
- Coverage
- How often the true value lands inside the predicted range. An 80% range should catch about 80 out of 100.
- Hit rate (direction)
- How often the model gets the direction right: up or down.
When things go wrong
- Overfitting
- Doing much better on examples it studied than on new ones, like a student who memorized last year’s test. Shown by a big gap between training-row and held-back scores.
- Underfitting
- The model is too simple to catch the pattern at all. Scores are poor everywhere, even on its training rows.
- Fit check
- Comparing the score on rows it trained on, on the folds, and on held-back rows. Close together is healthy; a big drop on new rows means overfitting.
- Generalization
- How well what the model learned carries over to new data. It’s the whole point.
- Drift
- When real-world data slowly stops looking like the training data, so a model that was good gets worse over time.
- Bias (in data)
- When the training examples don’t fairly represent the people or cases the model will be used on, so it does worse for some of them.
- Survivorship bias
- Training only on what survived, like today’s stocks, and missing the ones that failed. Results look better than reality.
Kinds of models
- Decision tree
- A flowchart of yes/no questions (“Is income over $50k?”) that ends in a guess. Easy to follow, but one tree alone is weak.
- Gradient-boosted trees
- Hundreds of small trees, each one fixing the mistakes of the ones before. Usually the most accurate choice for tables of data.
- Random forest
- Many trees trained on different random slices of the data, then averaged. Sturdy and hard to overfit.
- Linear regression
- Draws the best straight-line relationship between inputs and a number. Simple, fast, easy to explain, and often hard to beat on small data.
- Logistic regression
- The yes/no version of linear regression: turns inputs into a probability.
- Neural network
- Layers of simple math units connected together, loosely inspired by the brain. Powerful for images, sound and text.
- Convolutional network (CNN)
- A neural network built for images: it looks for edges and shapes anywhere in the picture.
- Transformer
- The kind of neural network behind modern language models. Very good at text and increasingly at images.
- Pretrained model
- A model someone already trained on huge amounts of data, which you adapt to your task with far fewer examples.
- Fine-tuning
- Taking a pretrained model and training it a little more on your own labeled examples.
- Embedding
- A list of numbers that captures what an image or text means, so similar things get similar numbers.
- k-means
- A simple clustering method that sorts rows into k groups by how close they are.
- Isolation forest
- An anomaly detector that finds rows that are easy to separate from the rest.
- Markov chain
- A model of a sequence that only looks at the current state to guess the next one. Like saying “if it’s raining now, there’s a 66% chance it’s still raining next hour.”
Using a model
- Inference
- Using a trained model to make predictions on new data.
- Deployment
- Putting a model where apps and people can use it, usually behind an API.
- API
- A way for one program to ask another for something over the internet, like asking this workspace for a prediction.
- API key
- A secret password for an app, so the API knows who is asking and what they may do. Keep it private.
- Model file (artifact)
- The saved, trained model. Its checksum proves it hasn’t been changed.
- Checksum
- A fingerprint of a file. If even one byte changes, the fingerprint changes.
- Worker
- A computer that does the training for you and sends the results back to the workspace.
- GPU
- A graphics chip that does many calculations at once. Makes deep learning much faster.
- Monitoring
- Watching a model after launch: are predictions still accurate, is the data drifting, is it fast enough.
- Latency
- How long one prediction takes to come back.
Tools and libraries
- Python
- The most common programming language for machine learning.
- scikit-learn
- A free Python library with most classic models (linear, trees, forests, clustering) and tools for splitting and scoring. A great place to start.
- XGBoost
- A popular free library for gradient-boosted trees, strong on tables of data.
- LightGBM
- Microsoft’s gradient-boosted trees library, very fast on large tables.
- CatBoost
- Yandex’s gradient-boosted trees library, especially good with categorical columns.
- PyTorch
- The most widely used library for building and training neural networks.
- TensorFlow / Keras
- Google’s neural-network library; Keras is its simpler front end.
- Hugging Face
- A hub of free pretrained models and datasets, especially for text and images.
- pandas
- A Python library for working with tables: loading CSVs, cleaning, joining.
- NumPy
- The Python library for fast math on lists of numbers. Almost every ML tool is built on it.
- Jupyter notebook
- A document where you write code, run it and see charts step by step. Popular for exploring data.
- MLflow
- An open-source tool for tracking experiments and storing model versions.
- Kaggle
- A website with free datasets and machine-learning competitions.
- UCI ML Repository
- A long-running university collection of public datasets used for testing models.
- OpenML
- An open online library of datasets and results that tools can download from directly.
No term matches that. Try a shorter word.
See these ideas in action on the test datasets, or read which model fits your data.