"Machine learning" gets used as a synonym for "deep learning" so often that people assume every prediction problem needs a neural network, a GPU, and a few thousand dollars of cloud compute. Most of the time it doesn't. One of the most useful tools in the entire field is something you can explain on a napkin: the regression tree.
The napkin version
A regression tree predicts a number by asking a series of yes/no questions and landing on an answer. That's it. That's the whole idea.
Say you're trying to predict how long a delivery will take. A regression tree might look like this:
Is the distance over 10 miles?
├── Yes: Is it during rush hour (4-6pm)?
│ ├── Yes: predict 52 minutes
│ └── No: predict 34 minutes
└── No: Is it during rush hour?
├── Yes: predict 22 minutes
└── No: predict 14 minutesEvery question splits your data into two groups. Every leaf at the bottom is just the average of whatever deliveries ended up there. There's no hidden math happening — you could trace any prediction back to the exact questions that produced it, by hand, in under a minute.
Where the questions come from
You don't write those questions yourself — the algorithm finds them. Given a pile of past deliveries (distance, time of day, actual delivery time), it tries out different splits — "distance over 10 miles," "distance over 8 miles," "before or after 3pm" — and picks whichever split does the best job of grouping similar delivery times together. Then it repeats that process inside each resulting group, splitting again and again, until further splitting stops meaningfully improving the groups (or you tell it to stop, to avoid a tree so deep it's just memorizing individual deliveries).
That's genuinely the whole training process. No gradient descent, no backpropagation, no architecture decisions. Find the best split, repeat.
Why you'd choose this over a neural network
For a lot of real business problems, a regression tree — or its more powerful cousin, a random forest (many trees, averaged together) — is the better choice, not just the simpler one:
- You can explain every prediction. If a loan-pricing model or a delivery-time estimate needs to be justified to a customer or a regulator, "here's the exact chain of splits that produced this number" is a real answer. "The neural network's weights encode it somehow" is not.
- It works fine on small data. Deep learning wants tens of thousands of examples minimum, often far more. A regression tree can produce something genuinely useful from a few hundred rows in a spreadsheet — squarely the amount of data most small businesses actually have.
- It doesn't need feature engineering gymnastics. Trees handle messy, mixed data (some numbers, some categories, some missing values) without much preprocessing. Neural networks are far pickier about their inputs.
- It's fast to build and cheap to run. Training takes seconds on a laptop. There's no ongoing GPU bill.
None of this means trees are always the right tool — they can struggle with very smooth, continuous relationships, and a single tree can overfit badly if you let it grow too deep (this is exactly why random forests exist: average enough imperfect trees together and the mistakes cancel out). But the instinct to reach for the most complicated tool available is usually backwards. Start with the simplest model that could plausibly work, and only add complexity when the simple version actually falls short — a rule that holds up far outside machine learning, too.