How We Test

Every number on this site comes off a probe in my own yard. This page is the method, written down before the results, so you can check whether I followed it.

The instrument

Five type-K thermocouples per test. Four sit on the cooking grate at 150 mm intervals, starting at the firebox end. The fifth sits inside the firebox.

Grate temperature is the only temperature that matters to food, and it is not what the lid thermometer shows you. On most offset smokers the lid probe sits up in the dome, several inches above the meat and away from the hot end. The gap between what the lid says and what the grate does is one of the things I publish.

Readings are logged every 30 seconds for the length of the cook. Nothing is smoothed or averaged before it goes into the chart. The chart you see is the log.

The cook

Target is 250 °F, held within plus or minus 25 degrees. That window is not arbitrary. It is roughly what a person tending a fire can hold by hand without instrumentation, which means the test measures the smoker rather than my reflexes.

Fires get tended the way anyone tends them. Splits go on when the fire asks for them, not on a fixed schedule, because a fixed schedule would only test the schedule.

What gets counted

The first two hours of every cook are logged but excluded from the stability figures. A cold smoker is unstable no matter how well it is built, and steel takes time to heat-soak. Swing is reported from hour three onward, once the mass of the smoker is doing what it is going to do.

Each rig gets a minimum of eight cooks and forty logged hours before I write about it. One good cook proves nothing. Weather, fuel and my own attention vary more than the equipment does, and eight runs is roughly where that noise starts to average out.

What gets published

• Temperature at each grate position across the whole cook, as a curve.
• Swing at each position from hour three, given as a range rather than an average.
• The difference between the lid thermometer and the grate probe.
• Fuel burned, by weight.
• Time from lighting to a stable target temperature.
• Photographs of the build details that explain the numbers, shot at the same angles on every unit.

What this method cannot tell you

It measures thermal behaviour. It says nothing about how long the paint lasts, whether the welds survive four winters, or how the company handles a warranty claim. Where I have that kind of information it comes from owners rather than from my own bench, and the article says so.

It also cannot tell you anything about equipment I do not own. Some articles compare units I have measured against units I have only analysed from specifications, physics and owner reports. Those are two different things, and the article separates them unit by unit. Analysis is useful. It is not measurement, and I am not going to blur the two to make an article look more authoritative than it is.

About the photographs

The photographs on this site illustrate. They do not document. Every article carries two of them, and they were produced with an image model rather than shot on the bench, so treat them the way you would treat a diagram: they show what a cut looks like, not what a particular cook produced.

Nothing on this site rests on a photograph. No temperature, no time, no conclusion is drawn from an image, and no picture here is presented as evidence of a test. Measurements come from probes, and the charts are drawn from the logs. When first-hand photographs of the bench exist, they will be labelled as such and this note will change with them.

Testing in progress

The bench is being built out now. Which rigs are under test, and how many hours each has logged, is shown on the home page and updated as the logs come in. An article appears only after that rig has cleared eight cooks and forty hours.

If you think the protocol is wrong, say so: . I would rather fix the method early than defend a bad one later.

What we measure, and what we do not claim

Most sites that write about barbecue wood present flavor rankings as if they were findings. We are not going to do that, and it is worth saying plainly why.

We measure relationships rather than species. How moisture content changes pit temperature and recovery time. How the size of a piece changes how long it burns and whether it flames or smoulders. How an ash bed changes the air path under a charcoal fire. How far the lid thermometer is from the actual grate reading, and how much the grate varies end to end. Every one of those is a physical relationship. It is measured on whatever fuel is on the bench, and it holds regardless of which tree the wood came off.

Species properties themselves, meaning density, energy content and the flavor a wood is known for, come from published wood science and from cooking practice. Those are labelled analyzed on every page that uses them. Applying a published density figure to a relationship we measured is legitimate and we do it. Presenting it as something we tested would not be.

• We do not run blind flavor panels between species, and no flavor ranking on this site is presented as a measured result.
• Moisture content is measured by the oven dry method: a sample cut from the centre of the piece, weighed, dried to constant mass, reweighed. That is the reference method resistance meters are calibrated against. A pin meter is the right tool at your woodpile and the wrong tool for a published number.
• Seasoning times depend on climate. Any drying curve we publish comes with the conditions it was recorded in, and the months are not transferable to your yard. The method is. That is why every wood page here tells you to measure rather than to count.
• Eight cooks and 40 hours qualifies a rig, not a wood. It is a threshold for saying we know how a cooker behaves, and it has never been a threshold for having an opinion about a tree.

The effect of all this is a narrower set of claims than most sites make and a firmer one. If we write that a number was measured, there is a log behind it. If we write that a wood tastes a certain way, that is the accumulated judgment of people who cook, ours included, and we will say so on the page rather than dress it up as data.