How Many Experiments Do You Need for 3-Factor 2-Level DOE in Machine Learning?

Learn how to calculate the exact number of experimental runs for a full factorial Design of Experiments (DOE) in machine learning and why systematic tuning beats random trial-and-error.

Question

You define 3 factors (learning rate, max depth, batch size) and 2 levels for each. How many experiments will DOE require?

A. 6
B. 12
C. 3
D. 8

Answer

D. 8

Explanation

When you set up a full factorial Design of Experiments (DOE) to optimize a machine learning model, you evaluate every possible combination of your chosen parameter levels. The mathematical formula for calculating the total number of experimental runs is straightforward:

Total Experiments = L^k

Where:

• L represents the number of levels (the specific values or settings you want to test).
• k represents the number of factors (the distinct hyperparameters you are tuning).

By plugging your variables into the equation—3 factors (learning rate, max depth, batch size) tested at 2 levels each (such as low vs. high)—you get:

2³ = 8 experiments

Why a 2³rparameters, and each of the 8 corners (vertices) represents a distinct training run.

Instead of changing one variable at a time or guessing randomly, testing all 8 corners gives your engineering team two critical statistical advantages:

1. Main Effects: You can accurately measure how changing an individual parameter—like increasing max depth from 4 to 12—impacts your model’s loss across all other configurations.
2. Interaction Effects: Machine learning hyperparameters rarely operate independently. A high learning rate might cause your model to diverge when paired with a small batch size, yet train smoothly when paired with a large batch size. Evaluating all 8 vertices systematically isolates these hidden two-way and three-way interactions.

When to Scale Beyond Full Factorial

While 8 experiments is computationally trivial for a decision tree or a lightweight regression model, full factorial designs scale exponentially. If you expand your test suite to 10 hyperparameters at 2 levels each, the required runs jump from 8 to 2¹⁰, totaling 1,024 separate model trainings.

When your hyperparameter space reaches that size, AI engineers transition to fractional factorial designs or orthogonal arrays. These statistical layouts select a balanced, highly strategic subset of the corners—allowing you to evaluate 10 parameters in just 16 or 32 runs while still capturing the most impactful main effects and critical interactions.