BW / OBSERVATORY
LOG / 04CASE STUDYM.S. THESIS, DUKE

Detecting what a satellite
can barely see.

The headline result is not the accuracy. It is how much of the accuracy survived moving to a city the model had never seen.

F1
0.933
RECALL
1.000
PIXEL ACC
95.9%
IOU, TRAINED CITY
0.448
IOU, NEW CITY
0.319
§ 01THE PROBLEM

Ulaanbaatar burns coal to survive its winter. Combined heat and power plants run flat out through a season that regularly reaches thirty below, and the resulting air quality is among the worst of any capital city in the world. Knowing when a plant is emitting, and how far that plume travels, matters for anyone modeling who is actually breathing it.

Ground instrumentation is sparse and reporting is inconsistent. Satellites, on the other hand, pass over every plant every day. So the question was narrow and testable: can freely repeating 3 m multispectral imagery reliably tell you that a plume is present, and where it extends?

§ 02WHY IT IS HARD

Three metres per pixel sounds generous until you try to segment smoke. A plume has no edge. It leaves the stack opaque and dissolves into haze over a few hundred metres, and the point where you stop calling it plume is a judgment call rather than a measurement.

Winter makes it worse. The background is snow and industrial haze, which is bright and textured in the same bands the plume is. And there is no public labeled dataset for this task. If you want ground truth, you draw it yourself.

§ 03APPROACH

Two stages, deliberately separated. A ConvNeXt Large classifier answers whether a scene contains a plume at all. A DeepLabV3+ model with a ResNet-101 encoder then answers which pixels belong to it.

Splitting the task means the segmentation model only ever sees scenes that actually contain a plume, so none of its capacity goes into learning to output empty masks. It also produces two failure modes that can be read independently: missing a plume entirely, and getting its extent wrong. Those have very different consequences downstream, and collapsing them into one number hides which is happening.

§ 04THE DATASET WAS THE PROJECT

109 scenes, hand-annotated and spatially QC’d. Built, not inherited. That one sentence covers most of the calendar.

The work was deciding what counts as plume, applying that definition consistently across scenes captured under different sun angles and snow cover, and then verifying every annotation was spatially aligned to its source imagery. Label quality and scene selection moved final performance more than the choice of architecture did, which is not the lesson I expected going in.

§ 05RESULTS, AND WHAT THEY ACTUALLY SAY

Classification reached an F1 of 0.933 with perfect recall. Segmentation reached 95.9% pixel accuracy.

The classification numbers are worth reading together rather than separately. Recall of 1.000 against an F1 of 0.933 implies precision near 0.87. In plain terms: the model never missed a plume, and roughly one in eight scenes it flagged did not contain one.

For a screening tool that is the correct side to err on. A missed plume is an emission event permanently absent from the record. A false positive costs a few seconds of human review.

§ 06THE FINDING

The models were trained on Ulaanbaatar and then evaluated on Bishkek CHP, a plant in a different country that appeared nowhere in training, with no fine-tuning. IoU fell from 0.448 to 0.319, a drop of roughly 29%.

I led with that number rather than the validation result. A segmentation model that reports 95.9% pixel accuracy and stops there invites the reader to assume it will work on their city. It will not, and the useful question is by how much.

Publishing the degradation turns a single-site result into something reusable: a measured expectation for what happens when this class of model crosses a geography, and a starting point for estimating how much local labeling a new site would need before the output could be trusted.

§ 07WHAT I WOULD DO DIFFERENTLY

The two cities are more alike than the general case. Both are continental, both cold-season, both combined heat and power. The 29% drop should be read as a floor for transfer, not a ceiling. Moving to a coastal plant, a different fuel, or a summer scene would almost certainly cost more.

Ground truth for a diffuse object also deserves more than one pair of eyes. With a single annotator, IoU is measured against one person’s judgment of where a plume ends. Multiple annotators and a published agreement score would separate model error from labeling ambiguity, and I suspect some of the gap I attributed to domain shift is really the second thing.

[01]ALL PROJECTS[02]CONTACTDEFENDED 2026.03.18