Machine Learning
Request access
Menu

Machine Learning

Docs

How the platform works, in enough detail to check our claims. New to the terms? Keep the glossary open.

Model contracts

A container is one model project. Before it can train, its contract must answer every question below. The form refuses to start a version until all of them are filled in.

Task and learning type
Regression, classification, image classification, forecast, next state in a sequence, or regime detection; supervised (labeled) or unsupervised. The two must agree: a classifier can’t be unsupervised.
Inputs and target
Which columns go in, which column is the answer, and exactly how that answer is computed and when it becomes known.
Units and horizon
What a score is measured in and how far ahead the prediction looks.
Goal
The one data point to get right, the metric, and a target value. Each version is marked as meeting the goal or not.
Baseline and split
The simple guess every model must beat, and how held-back data is chosen so nothing leaks.
Intended use and limits
What decisions it supports and what it must not be used for.

Evaluation method

  1. Hold back first. 20% of rows are set aside before anything is trained. With a time column, it’s the most recent 20%, with an optional gap (embargo) so answers can’t overlap. With a group column (experiment, batch, patient), whole groups are held back.
  2. Encode from training rows only. Which columns are numbers and which categories exist is decided from the training rows; categories only seen in held-back rows become zeros.
  3. Folds. The training rows are split into folds (expanding windows for time order, separate groups, or random fifths). Every model type and every tuning trial is scored on the same folds; results show the mean and spread.
  4. Pick on folds, judge once. The best model type and settings are chosen on the folds. Only then are the held-back rows scored, once. Ties go to the steadier model, then the simpler one.
  5. Always next to a baseline. Average of training rows, most common class, zero return, overall state frequencies, or one steady regime, depending on the task.
  6. Fit check. The score on rows it trained on vs folds vs held-back rows. A big drop on new rows is reported as overfitting.

Every public result on the test datasets page was produced this way, with its seed and engine version.

Model types

Each run compares the types that fit the task: EE VZN (our gradient-boosted trees engine, with optional tuning), random forest and linear or logistic regression for tables; a convolutional network for images on workers with PyTorch; LightGBM and CatBoost on workers that have them; Markov chains (one and two steps back) for sequences; hidden Markov models (2–4 regimes vs one) for regime detection. Details, strengths and limits for each: models.

Workers and data handling

Container packages

On paid plans, any container can be downloaded as a package: its dataset, its contract and split settings, the Worker Kit and an easy runner. Think of it as Docker for ML: the same evaluation runs the same way on any machine.

API

Create keys on the API keys page. Send them as Authorization: Bearer mlk_…. Keys have read and/or write scopes, are stored only as a hash, can be revoked, and are limited to 120 requests a minute. Everything is JSON; errors carry a plain error message. A live list of endpoints is at /api/v1/.

CallScopeWhat it does
GET /api/v1/containers/readYour containers, with goal and latest version
GET /api/v1/containers/?id=…readOne container: contract, versions, latest results
POST /api/v1/datasets/?container=…writeUpload a CSV (field dataset; optional target, time, group, embargo_rows)
POST /api/v1/evaluations/writeStart an evaluation: {"container","dataset","tune":0|20|50}
GET /api/v1/evaluations/?id=…readStatus and results: plain summary, baseline, fit check, comparison, confusion matrix, folds
GET /api/v1/experiments/?container=…readExperiment registry: dataset checksums, module versions, settings, metrics, model checksums, environment
curl -H "Authorization: Bearer mlk_…" -F [email protected] -F target=revenue \
  "https://ml.reversal.org/api/v1/datasets/?container=CONTAINER_ID"

curl -H "Authorization: Bearer mlk_…" \
  -d '{"container":"CONTAINER_ID","dataset":12,"tune":20}' \
  https://ml.reversal.org/api/v1/evaluations/

Versions, approval and lineage

Security

Not built yet

So you know exactly where things stand:

Questions or a use case we didn’t cover? Request early access and tell us.