Machine Learning
Request access
Menu

Machine Learning

Machine learning models

Which model to use depends on what your data looks like and how many labeled examples you have. Start from your data below, then read up on every model: what it’s best for, where it breaks, and the settings pros tune. Whatever you pick, judge it the same way: on rows it never saw, against a simple baseline. New to the words? See the glossary.

Start from your data

Tables of numbers and categories

Sales, prices, lab measurements, customer records, sensor readings

Available now

Gradient-boosted trees

The most accurate general choice for tabular data. Handles mixed numbers and categories, missing values and odd scales without much preparation, and trains fast on an ordinary CPU.

Watch for Can’t predict outside the range it has seen (a price twice the highest in training). Needs a few hundred labeled rows to beat a simple baseline reliably.

Tested: California housing, 0.29 average error on held-back rows

Tested: Breast cancer, 96.0% f1 (macro) on held-back rows

Time-ordered data

Stock returns, demand per hour, machine readings over time

Available now

Gradient-boosted trees on lagged and rolling features, tested walk-forward

Trees do well once yesterday’s values, moving averages and calendar features are columns. What matters most is the test: train on the past, score on the future, with a gap between them.

Watch for A random split looks great and means nothing: the model peeks at the future. Markets shift, so check every walk-forward fold, not just the average.

Grouped lab or field data

Images or readings from experiments, batches, plates, patients, sites

Available now

Any model, split by group

Rows from the same experiment look alike. Holding out whole groups shows how the model does on a new experiment, which is the question you actually have.

Watch for Splitting row by row lets near-duplicates sit on both sides and inflates the score.

Small or simple images

Tiny scans, icons, images already turned into a few dozen numbers

Available now

Gradient-boosted trees on the pixel values

When an image is only a handful of pixels, each pixel works like a column in a table, and trees learn it well.

Watch for This stops working as images get bigger and the object can move around the frame.

Tested: Digits (8×8 images), 98.1% accuracy on held-back rows

Real photos and microscope images

Plate photos, microscopy, product photos, X-rays

Planned

A convolutional network or pretrained vision model, fine-tuned on your labels

These models understand shapes, edges and textures wherever they appear in the frame. Starting from a model pretrained on millions of images means a few hundred labeled examples per class can be enough.

Watch for Tree models on raw pixels are not recommended here: they don’t see shapes. With very few labels, use pretrained image features plus a small classifier. Real images need a GPU worker to train in reasonable time.

Few rows, or you need to explain every prediction

A pilot study with 150 rows, a decision you must justify line by line

Planned

Linear or logistic regression

Each input gets one weight you can read and question. With little data it often holds up better than bigger models, and it is the baseline any bigger model should beat.

Watch for Misses interactions and curves unless you add them as columns.

A second opinion on tabular data

Checking that a boosted model’s result isn’t a fluke

Planned

Random forest

Many independent trees averaged together: hard to overfit, little tuning, and a sturdy check against a tuned boosted model.

Watch for Usually a little less accurate than well-tuned boosted trees, and the saved model is larger.

Data with no answer key

Grouping customers, spotting unusual transactions or sensor spikes

Planned

Clustering (k-means) or anomaly detection (isolation forest)

Finds structure or outliers without labels, which is often the first step before you know what to predict.

Watch for There is no accuracy without a reference. Results need a person to review a sample, or a small labeled set to score against.

Text

Support tickets, reviews, notes, filings

Not offered yet

Pretrained language-model embeddings plus a small classifier

Turns each text into numbers that capture meaning, then any of the models above can learn from them.

Watch for Not offered in the workspace yet.

Every model

24 models, from the ones running in the workspace today to the ones on the roadmap.

XGBoost

Gradient-boosted trees · Supervised: classification and regression

Available now

Builds hundreds of small decision trees one after another, each fixing the mistakes of the trees before it.

Best for
Tables of numbers and categories: prices, risk scores, churn, demand, lab measurements. Often the most accurate choice for tabular data. In this workspace it runs as EE VZN.
Watch for
Can overfit small data if trees grow too deep. Can’t predict beyond the range it has seen. Not the right tool for raw images or text.
Data it needs
A few hundred rows and up; scales to millions.
Speed
Fast on CPU; GPU optional.
How explainable
Medium: feature importance and SHAP values show what drives each prediction.
Settings pros tune
max_depth, learning_rate, n_estimators, subsample, colsample_bytree, min_child_weight, reg_lambda
Libraries
xgboost (Python, R, Java)
Full potential
Quantile objectives give ranges (q10/q50/q90), ranking objectives order results, and it trains on GPUs for very large data. The default engine for most containers here.
Won our tests on
California housing ($29,400 average miss), Bike sharing demand (43 rentals an hour average miss), Adult census income (F1 80), Brent crude oil, 20 days ahead (10 percentage points average miss), Natural gas (Henry Hub), 20 days ahead (19 percentage points average miss), Diabetes onset (Pima) (F1 64), Blood cell images (81.2% right), Chip production pass/fail (SECOM) (F1 49), Wafer defect images (SEM) (F1 93), Machine failure (milling) (F1 88), Grid demand, 24 hours ahead (Bangladesh) (654 MW average miss), U.S. rail freight, next month (2.10 percentage points average miss), Gulf of Maine sea temperature, 7 days ahead (NOAA OISST) (0.61 °C average miss), Flu-like illness by US region, 2 weeks ahead (CDC ILINet) (0.38 percentage points average miss)

LightGBM

Gradient-boosted trees · Supervised: classification and regression

On capable workers

Microsoft’s take on boosted trees, grown leaf by leaf with binned features for speed.

Best for
Large tables (hundreds of thousands to millions of rows) where training time matters.
Watch for
Can overfit small datasets faster than XGBoost; keep num_leaves modest.
Data it needs
Thousands of rows and up.
Speed
Very fast on CPU.
How explainable
Medium (feature importance, SHAP).
Settings pros tune
num_leaves, learning_rate, n_estimators, min_data_in_leaf, feature_fraction, bagging_fraction
Libraries
lightgbm
Full potential
Native categorical handling and very low memory use make it a strong challenger on big tables. Joins the model comparison on any worker with lightgbm installed.

CatBoost

Gradient-boosted trees · Supervised: classification and regression

On capable workers

Yandex’s boosted trees, built to handle categorical columns well with little preparation.

Best for
Tables with many text categories: product types, regions, customer segments.
Watch for
Slower to train than LightGBM; the library is large.
Data it needs
Hundreds of rows and up.
Speed
Moderate on CPU; good on GPU.
How explainable
Medium (feature importance, SHAP).
Settings pros tune
depth, learning_rate, iterations, l2_leaf_reg
Libraries
catboost
Full potential
Strong defaults mean it often does well with no tuning at all. Joins the model comparison on any worker with catboost installed.

Random forest

Tree ensembles · Supervised: classification and regression

Available now

Hundreds of decision trees, each trained on a random slice of rows and columns, averaged together.

Best for
A sturdy second opinion on tabular data; a quick strong model with almost no tuning.
Watch for
Usually a little behind tuned boosted trees; saved models get large.
Data it needs
Hundreds of rows and up.
Speed
Fast; trains trees in parallel.
How explainable
Medium (feature importance).
Settings pros tune
n_estimators, max_depth, min_samples_leaf, max_features
Libraries
scikit-learn
Full potential
Hard to overfit badly, so it’s a reliable check that a boosted model’s score isn’t a fluke. Runs in every model comparison here.
Won our tests on
Digits (8x8 images) (98.1% right), Forest cover type (82.7% right), Heart disease (Cleveland) (82.0% right), Yeast protein location (60.5% right), DNA splice junctions (96.7% right), FDA recalls: will a recall be Class I (most serious)? (F1 73)

Linear regression (ridge)

Linear models · Supervised: regression

Available now

Fits the best straight-line relationship between the inputs and a number, with a penalty that keeps weights from getting extreme.

Best for
Small datasets, relationships that are mostly straight-line, and any case where you must explain each prediction.
Watch for
Misses curves and interactions unless you add them as columns.
Data it needs
Works with dozens of rows.
Speed
Instant.
How explainable
High: one readable weight per input.
Settings pros tune
alpha (penalty strength)
Libraries
scikit-learn, statsmodels
Full potential
Often beats trees on small, simple data. It won the diabetes test here. Every regression comparison includes it.
Won our tests on
Diabetes progression (51 points average miss)

Logistic regression

Linear models · Supervised: classification

Available now

The yes/no version of linear regression: turns a weighted sum of inputs into a probability.

Best for
Explainable classification, small data, and well-calibrated probabilities.
Watch for
Needs scaled inputs (handled for you) and misses interactions unless added.
Data it needs
Works with dozens of rows.
Speed
Instant.
How explainable
High: each input pushes the odds up or down by a readable amount.
Settings pros tune
C (inverse penalty strength), penalty type
Libraries
scikit-learn, statsmodels
Full potential
The standard in credit scoring and medicine where decisions must be justified. Every classification comparison includes it.
Won our tests on
Breast cancer (F1 96), Wine cultivar (100.0% right), Iris flowers (93.3% right), German credit risk (F1 68), Mouse brain proteins (35.7% right), Parkinson’s from voice (F1 70)

Lasso and elastic net

Linear models · Supervised: regression

Planned

Linear models whose penalty can push useless inputs’ weights to exactly zero.

Best for
Many columns, few rows, and finding which inputs actually matter.
Watch for
Picks one of several look-alike columns somewhat arbitrarily.
Data it needs
Dozens of rows and up.
Speed
Instant.
How explainable
High.
Settings pros tune
alpha, l1_ratio
Libraries
scikit-learn
Full potential
Built-in feature selection for wide datasets like genomics.

Decision tree

Trees · Supervised: classification and regression

Planned

One flowchart of yes/no questions that ends in a guess.

Best for
Explaining a simple rule to people; quick first look at what matters.
Watch for
One tree alone overfits easily and is rarely the most accurate.
Data it needs
Any size.
Speed
Instant.
How explainable
Very high: you can print the whole flowchart.
Settings pros tune
max_depth, min_samples_leaf
Libraries
scikit-learn
Full potential
The building block of every tree ensemble above.

k-nearest neighbors

Instance-based · Supervised: classification and regression

Planned

Predicts by looking at the k most similar rows it has seen and copying their answer.

Best for
Small datasets, recommendation-style “find similar” problems, a simple baseline.
Watch for
Slow on big data; struggles when there are many columns.
Data it needs
Small to medium.
Speed
Training instant; predicting slow on big data.
How explainable
High: “these 5 cases were most similar.”
Settings pros tune
k, distance measure
Libraries
scikit-learn
Full potential
Pairs well with embeddings for image and text similarity search.

Support vector machine

Kernel methods · Supervised: classification and regression

Planned

Finds the boundary that separates classes with the widest possible margin, optionally bending it with a kernel.

Best for
Medium-sized datasets with clear separation; text classification with many columns.
Watch for
Slow beyond tens of thousands of rows; needs scaling and tuning.
Data it needs
Hundreds to tens of thousands of rows.
Speed
Moderate to slow.
How explainable
Low with kernels.
Settings pros tune
C, kernel, gamma
Libraries
scikit-learn
Full potential
Still strong on small, high-dimensional problems.

Naive Bayes

Probabilistic · Supervised: classification

Planned

Uses probability rules, assuming each input affects the answer independently.

Best for
Quick text classification (spam filters), very small data.
Watch for
The independence assumption is usually wrong, so probabilities are poorly calibrated.
Data it needs
Tiny to large.
Speed
Instant.
How explainable
High.
Settings pros tune
smoothing (alpha)
Libraries
scikit-learn
Full potential
A fast baseline for text before trying bigger models.

Neural network (multilayer perceptron)

Neural networks · Supervised: classification and regression

Planned

Layers of simple math units that learn curved, complex relationships.

Best for
Large datasets with complex patterns; combining tabular data with embeddings.
Watch for
On ordinary tables, boosted trees usually win with less effort. Needs scaling and careful tuning.
Data it needs
Thousands of rows and up.
Speed
Moderate; faster with a GPU.
How explainable
Low.
Settings pros tune
layers, units, learning rate, dropout, epochs
Libraries
PyTorch, TensorFlow/Keras, scikit-learn
Full potential
The foundation for every deep-learning model below.

Convolutional neural network (CNN)

Deep learning: images · Supervised: image classification, detection, counting

On capable workers

A neural network that scans an image for edges, textures and shapes anywhere in the frame.

Best for
Photos, microscopy, plate images, X-rays, product pictures. The recommended path for real images.
Watch for
Needs labeled images and a GPU to train in reasonable time. Start from a pretrained model, not from scratch.
Data it needs
A few hundred labeled images per class when starting from a pretrained model.
Speed
Slow on CPU; fast on GPU.
How explainable
Low (heatmaps show where it looked).
Settings pros tune
architecture (ResNet, EfficientNet), learning rate, image size, augmentation, epochs
Libraries
PyTorch, TensorFlow/Keras, timm
Full potential
Fine-tuned CNNs reach expert-level accuracy on many image tasks. Planned for Life Science containers on GPU workers.

Pretrained vision models and embeddings

Deep learning: images · Supervised and unsupervised

Planned

Large models already trained on millions of images (vision transformers, CLIP, DINO) that turn any image into a list of meaningful numbers.

Best for
Few labels: use the image embeddings with a simple classifier, or search for similar images.
Watch for
Big model files; check the license before commercial use.
Data it needs
Dozens of labeled images can be enough with embeddings.
Speed
Needs a GPU to run quickly.
How explainable
Low.
Settings pros tune
which pretrained model, whether to fine-tune or freeze
Libraries
Hugging Face Transformers, timm, open_clip
Full potential
The fastest route from a folder of images to a working model, and the base for the bio foundation-model track in the whitepaper.

Language-model embeddings

Deep learning: text · Supervised and unsupervised

Planned

Turns each piece of text into numbers that capture its meaning, using a pretrained language model.

Best for
Support tickets, reviews, notes, filings: classify, cluster or search them.
Watch for
Text can carry private data; check what you’re allowed to send to a model.
Data it needs
Hundreds of labeled texts with embeddings.
Speed
Moderate; faster with a GPU.
How explainable
Low.
Settings pros tune
which embedding model, then the classifier’s own settings
Libraries
Hugging Face Transformers, sentence-transformers
Full potential
Embeddings plus logistic regression or boosted trees is a strong, cheap text pipeline.

Classical forecasting (ARIMA, exponential smoothing)

Time series · Supervised: forecasting

Planned

Statistical models that forecast a series from its own past: trend, seasonality and recent values.

Best for
A single series with clear seasonality: weekly sales, monthly demand.
Watch for
Struggles with many related series or lots of outside inputs; boosted trees on lag features often win there.
Data it needs
A few seasons of history.
Speed
Fast.
How explainable
High.
Settings pros tune
order (p, d, q), seasonality
Libraries
statsmodels, Prophet, sktime
Full potential
A must-have baseline for any forecast: if a fancy model can’t beat it, keep it simple.

Sequence neural networks (LSTM, temporal transformers)

Deep learning: time series · Supervised: forecasting

Planned

Neural networks that read data step by step in time order and remember what came before.

Best for
Long, high-frequency series and many related series together (sensors, energy).
Watch for
Data-hungry and easy to overfit on markets; always compare against simple baselines walk-forward.
Data it needs
Large.
Speed
Slow without a GPU.
How explainable
Low.
Settings pros tune
window length, layers, hidden size, learning rate
Libraries
PyTorch, Darts, GluonTS
Full potential
Powerful when there is a lot of history; overkill when there isn’t.

Markov chain

Markov models · Sequences: what comes next

Available now

Learns how often each state is followed by each other state, and assumes the next step depends only on the current one. The result is a table of chances called a transition matrix.

Best for
Sequences of a few states: weather types, customer journeys (visit, cart, buy), machine status, market up/down days. Great for “how likely is it to stay or switch?”
Watch for
Its single best guess is often just “stay the same”, so judge it by its probabilities, not by accuracy. It can’t see anything beyond the current state, so it misses changes that other information would predict.
Data it needs
A few hundred transitions per state.
Speed
Instant: it is just counting.
How explainable
Very high: the whole model is one readable table.
Settings pros tune
which states to use, how many past steps to remember (order), smoothing for rare transitions
Libraries
NumPy or pandas (a few lines), pomegranate
Full potential
A strong, honest baseline for any sequence and the building block of hidden Markov models. Runs in the workspace now with the Sequence template: it compares looking back one and two steps and keeps the better one. Tested below on real weather data.

Tested: Next hour’s weather from this hour’s

Bike sharing demand (Washington, D.C., 2011–2012), weather column. Learned from the first 13,902 hours, tested on the last 3,476.

Now ↓ / Next hour →clearmistyrainheavy rain
clear92.2%6.9%1.0%0.0%
misty18.4%72.6%9.0%0.0%
rain7.5%26.6%65.7%0.3%
heavy rain0.0%0.0%100.0%0.0%
  • On average it gave the weather that actually came 74.6%, against 49.7% from overall weather frequencies. Its probability score is about twice as good (Brier 0.2499 vs 0.5184; lower is better).
  • Its single best guess is always “same as now”, so it gets 85.1% right, exactly like guessing no change.
  • When the weather did change (519 times), it gave the new weather only 13.6%, worse than the 37.1% from overall frequencies. It knows weather tends to stay put; to see a change coming it needs more information, like the hour, pressure, or a hidden-state model.

Hidden Markov model (HMM)

Markov models · Sequences: hidden states and regimes

On capable workers

Assumes there is a hidden state you can’t see (like “calm market” or “volatile market”) that switches like a Markov chain, and that what you observe depends on it. It works out which hidden state you were most likely in at each step.

Best for
Market regimes for the Stock Forecast container, gene and protein sequence labeling, speech, activity from sensor readings, spotting when a process changes mode.
Watch for
You choose how many hidden states there are, and it can find states that are hard to name. Results depend on the starting point, so run it several times. Regimes found in the past can shift.
Data it needs
Long sequences: hundreds to thousands of steps.
Speed
Fast for a few states.
How explainable
Medium: each hidden state has readable averages and switch chances.
Settings pros tune
number of hidden states, what each state emits (normal, counts), number of restarts
Libraries
hmmlearn, pomegranate, statsmodels (Markov switching)
Full potential
Regime labels from an HMM make a powerful extra column for a forecasting model: “we are in a volatile regime” changes what the next 20 days look like. Runs in the workspace now with the Market regimes template on workers with hmmlearn: it tries 2, 3 and 4 regimes against one steady regime. On hourly bike-rental data it found quiet nights, a cold season, busy hours and hot busy hours, and fit new hours far better than one steady pattern.

k-means clustering

Unsupervised: clustering · Unsupervised

Planned

Sorts rows into k groups so rows in a group are close to each other.

Best for
Customer segments, grouping products, a first look at unlabeled data.
Watch for
You choose k; assumes round, similar-sized groups. No “accuracy” without a reference.
Data it needs
Any size.
Speed
Fast.
How explainable
Medium: describe each group by its center.
Settings pros tune
k (number of groups)
Libraries
scikit-learn
Full potential
Segments found here can become labels for a supervised model later.

DBSCAN and HDBSCAN

Unsupervised: clustering · Unsupervised

Planned

Finds dense groups of any shape and leaves scattered points as noise.

Best for
Location data, odd-shaped groups, not knowing how many groups exist.
Watch for
Sensitive to its density settings; slower on big data.
Data it needs
Small to medium.
Speed
Moderate.
How explainable
Medium.
Settings pros tune
eps, min_samples
Libraries
scikit-learn, hdbscan
Full potential
Doubles as an outlier finder: the “noise” points are often the interesting ones.

Isolation forest

Unsupervised: anomaly detection · Unsupervised

Planned

Flags rows that are easy to separate from everything else with a few random splits.

Best for
Fraud, sensor spikes, bad lab runs: finding the odd ones out without labels.
Watch for
Needs a person to review flags; tune how many flags you can handle.
Data it needs
Hundreds of rows and up.
Speed
Fast.
How explainable
Medium.
Settings pros tune
contamination (share expected to be odd), n_estimators
Libraries
scikit-learn, PyOD
Full potential
Scored against a small reviewed set, it becomes measurable like any supervised model.

PCA and UMAP

Unsupervised: dimension reduction · Unsupervised

Planned

Squeezes many columns into a few that keep most of the information, often to plot data in 2D.

Best for
Seeing structure in wide data (genomics, embeddings); shrinking inputs before another model.
Watch for
The new columns are blends and hard to name.
Data it needs
Any size.
Speed
Fast (PCA) to moderate (UMAP).
How explainable
Low to medium.
Settings pros tune
number of components; UMAP neighbors and distance
Libraries
scikit-learn, umap-learn
Full potential
The standard first plot for single-cell and other high-dimensional lab data.

Stacking and ensembles

Combining models · Supervised

Planned

Trains several different models and a final model that learns how to combine their guesses.

Best for
Squeezing out the last bit of accuracy once single models plateau.
Watch for
More complex to run and explain; easy to leak data if the combiner sees held-back rows.
Data it needs
Thousands of rows and up.
Speed
Sum of all the models.
How explainable
Low.
Settings pros tune
which base models, how the combiner is trained
Libraries
scikit-learn, AutoGluon
Full potential
Wins many competitions. A natural next step after the model comparison here.

“Planned” means the workspace doesn’t train it yet. Every container is scored the same way regardless of model: held-back rows, a baseline to beat, and folds to show how stable the result is. See the test datasets.