Every experiment teaches something. The question is how much

Every lab runs experiments. What sets two labs apart is how much each experiment teaches them.

In one lab, a result tells you about a single variable, and once the project ends the knowledge leaves with whoever ran it. In another, the same experiment narrows the next study, adds to a working model of the system, and is still readable five years later by someone who never met the person who ran it. Both labs are busy. Only one of them is getting smarter as it goes.

Four stages separate a lab that repeats itself from one that learns, and each one teaches more than the last.

Stage 01Each experiment teaches about one variable

This is how most of us were trained. If a coating cures too slowly, a usual first move could be to use more catalyst, so you create three batches at three catalyst levels, hold everything else fixed, and measure the effect of the catalyst concentration on the curing time. Once you find a good concentration, you might also try different hardener types or maybe adjust the baking temperature. You only ever change one variable at a time.

It feels rigorous because only one thing changed. That is also its limit. Changing one factor at a time can never show you what happens when two factors either work together or against each other. And there are quite a lot of these interactions happening in coating systems. Just imagine that the catalyst for example is only active at a lower temperature but not at the high temperature. Or works better with hardener A than with hardener B. You will not discover those things by varying only one factor at a time.

More catalyst shortens the curing time, so one factor at a time gives a single falling line. But that only holds at the low temperature. At the high temperature the catalyst barely changes the curing time, and that flat line is the one this approach never shows you.

The additional cost is that the work is rarely written down in a way anyone can reuse. Different types of spreadsheets are used, some things are not written down but only remembered. So the next person who picks up the project starts from whatever notes were left behind, which is rarely enough. Each project begins near zero and the lab never really compounds.

Stage 02Each experiment teaches about the system

The step up is to design the experiment instead of improvising it. You stop changing one factor at a time and vary several at once (catalyst, temperature, hardener) across a set of runs you lay out beforehand. Statisticians have worked out standard designs, factorial designs and their relatives, that arrange the factor levels so a small, balanced set of runs separates each factor’s effect from the others. The same number of panels now tells you which factors matter and how they interact. This is Design of Experiments (DoE).

One factor at a time moves along two edges of the design space, catalyst along the bottom and temperature up the side, so you never test the upper-right region where the two act together. A designed experiment places the same five runs on the corners instead, so it covers the whole space and leaves no gaps.

The output is a model of the system that someone else can read, argue with, and build on. The approach is also systematic. The plan is fixed before any sample is prepared, so it runs the same way whoever carries it out, and the result no longer depends on one person’s taste or memory.

Even so, DoE is still uncommon in most labs. It is not intuitive, and it takes real knowledge and experience to design a good study, run the analysis, and know which design to reach for at each stage of a project.

There is a ceiling here too. The runs are all chosen before the first batch, so the study cannot adapt to results as they come in, and even with the right sequence of designs, when the project ends the next one starts over from scratch. The lab does not yet remember.

Stage 03Each experiment teaches the next one

Stage two still asks you to lay out the whole plan before you begin. The runs are fixed in advance, so the study cannot adapt to the results as they arrive.

Stage three gives up that fixed plan. You start with a small set of runs, look at what comes back, and let those results choose what to run next. Then you repeat. The study becomes a loop, a fast back-and-forth between the experiments you run and what they tell you.

An algorithm picks each next experiment for you. After every round it builds a working estimate of how the system behaves, where the strong formulations probably sit, and how sure it is about each region, then proposes the run worth doing next. Which run that is depends on your goal. When your goal is the best formulation, this is Bayesian optimization, and it picks the run most likely to move you toward it. When your goal is to understand the system rather than to find one best formulation, this is active learning, and it picks the run that teaches you the most about how the system works.

A fixed plan sets every run beforehand, so you cannot adjust it once results start coming in. A guided search runs a few experiments, looks at what comes back, and lets the model pick the next run, stepping toward the best region in smaller and smaller steps as it closes in.

The boundary that remains is the project itself. An adaptive method is only as good as the data it can see, and it usually sees only the current study. If your last six projects live in six separate folders that are not connected, the method has no access to that context.

Stage 04Each experiment teaches future you

The last stage removes that boundary. The next study draws on every result the lab has ever produced, no matter which project it came from or who ran it. Knowledge accumulates between projects instead of evaporating when one ends.

That takes two things. First, the lab needs a place to keep its results that both a person and a machine can read, so a formulation you made three years ago is as available to the next study as one you made last week. That is the data layer. Second, the routine labor of writing protocols, building plans, and summarizing results can move to software that reads that data layer and drafts the work for you to check.

When each project lives in its own folder, its results stay there, and a new study starts from nothing. A shared database that both people and software can read keeps every project’s results in one place, so the next study begins with all of them.

A lab at this stage compounds. Every experiment pays into a store that the next experiment draws on, and the store outlives any single project or person.

OutlookWhere most labs stand and what the future might bring

Most labs sit in stage one or early stage two. Stage three is rare, and I would guess it is usually found in a large company with the people and budget to run adaptive studies. I don’t think a lab at stage four exists today as I write this essay. But I think this is changing with AI becoming more powerful, reshaping how we work. That is what this site is about.