Project Overview — Post-Disaster Building Damage Assessment on xBD

We benchmark four deep-learning architectures — ResNet-50 U-Net, ViT-B/16 U-Net, Siamese ResNet, and Siamese ViT — on the xBD post-disaster satellite imagery dataset, the largest publicly available change-detection corpus for building damage. We perform a controlled ablation across backbone, loss function,skip-connection topology, and deep supervision, holding all other training hyper-parameters fixed.

Our best configuration — a Siamese ViT with Focal+Dice loss and deeply-supervised U-Net decoder — achieves 0.756 macro-F1 across the four damage classes, a +11.4 pp improvement over the ResNet-50 baseline. We then package the model behind a FastAPI service that emits ATC-20 placard tags (GREEN / YELLOW / RED) suitable for incident-command consumption.

Keywords: change detection · semantic segmentation · vision transformer · Siamese network · class imbalance · ATC-20 · operational ML.

4.1 Research design

We adopt a controlled comparative study: the same data splits, optimizer, learning-rate schedule, and evaluation protocol are reused across every experiment. Only one architectural factor changes at a time (backbone ∈ {ResNet-50, ViT-B/16}, branches ∈ {single, Siamese}, loss ∈ {CE, Focal, Focal+Dice}, deep-supervision ∈ {on, off}). This isolates the causal contribution of each component and feeds directly into the ablation study in §5.

4.2 Methodology workflow

Figure M1 — Methodology workflow. Six stages from raw xBD GeoTIFFs to a reproducible benchmark report: (i) dataset acquisition, (ii) preprocessing & tiling, (iii) augmentation, (iv) model construction, (v) training with multi-loss supervision, (vi) evaluation against the held-out test split and per-class metrics.

4.3 Data preparation

Each xBD scene is a paired (I_pre, I_post) GeoTIFF at ≈0.31 m GSD. We tile every scene to 512 × 512 patches with a 64-pixel overlap to preserve building boundaries, then channel-normalise using ImageNet statistics. Training augmentation comprises random horizontal/vertical flip, 90°-multiple rotations, and brightness jitter (±0.1). Augmentations are applied identically to I_pre and I_post so the change signal is preserved.

4.4 Model architecture

(a) Patch embedding (ViT). An input image x ∈ ℝ^(H×W×3) is split into N = HW/P² non-overlapping patches of size P × P. Each patch is flattened and linearly projected to dimension D:

(b) Multi-head self-attention. For each head h with learned projections W_Q, W_K, W_V:

(c) Residual bottleneck (ResNet). Lets gradients flow through 50+ layers and stabilises training:

(d) Siamese difference. Twin encoders share weights; at each decoder level ℓ we compute an absolute feature difference that becomes the U-Net skip input — this localises changerather than appearance:

(e) U-Net decoder. A symmetric expanding path upsamples each D_ℓ via transposed convolution, concatenates with the next-finer skip, and applies a double-conv block. The final 1×1 convolution produces the per-pixel 5-class logit map (background + 4 damage tiers).

4.5 Loss function

Because 96% of pixels are background (Figure 2), we replace plain cross-entropy with a compound lossthat suppresses easy-pixel gradients (Focal) and rewards region overlap (Dice).

With γ = 2, an easy pixel (p_t = 0.9) contributes 100× less than a hard one (p_t = 0.1).

Multi-scale deep supervision adds an auxiliary L_total at decoder stages 0–3, weighted {0.4, 0.3, 0.2, 0.1}; this stabilises gradients in the rare-class regions.

4.6 Training procedure

4.7 Evaluation protocol

We report macro-averaged F1 across the four damage classes (background excluded) on the official xBD test split — never on training or hold-out. Per-class F1, precision, and recall are tracked alongside. Macro-F1 weights every damage tier equally regardless of frequency, so it penalises a model that silently ignores destroyed.

For operational deployment we additionally measure (i) per-tile inference latency on a T4 GPU and (ii) the placard-level confusion matrix after building-footprint aggregation (Table 3, §6).

5.1 Why an ablation?

Modern damage-assessment networks stack many ideas at once — pretrained transformers, Siamese branches, compound losses, deep supervision. Reporting only the final number tells us nothing about which idea earned the gain. An ablation study is the scientific instrument that fixes that: we start from a baseline, switch on one component at a time, and attribute the change in macro-F1 to exactly that component. Every other knob (optimiser, learning rate, augmentation, epoch count, see Table M1) is frozen, so the only explanation for a delta is the variable we toggled.

5.2 Experimental design

We enumerate eight configurations (A1…A8). A1 is the literature-baseline ResNet-50 U-Net with plain cross-entropy. From there we sequentially activate (a) Focal loss, (b) Dice loss, (c) Siamese twin-encoders, (d) deep supervision, and finally (e) the ViT-B/16 backbone. Each row in Table 2 differs from the previous by a single factor, so theΔ vs. baseline column reads as a causal contribution. Three random seeds were trained per row; the reported macro-F1 is the median (seed-to-seed std-dev was ≤ 0.4 pp, well below every reported delta).

5.3 Results

Figure 3 — Per-class F1 across architectures. The gap between models is largest on the rare-but-critical destroyed and major classes — exactly where ATC-20 placard decisions hinge. ResNet-50 baselines plateau around 0.71; switching to a ViT backbone and Siamese topology pushes destroyed-class F1 from 0.71 to 0.83.

5.4 Discussion

The ordering of contributions is itself the finding. Practitioners with a small compute budget should adopt Focal+Dice before upgrading the backbone — it is the cheapest, largest win and requires zero new parameters. The Siamese branch is the next priority because change-detection on identical scenes is fundamentally a difference task, not an appearance task, and weight-shared twin encoders make that explicit. The ViT backbone only pays off once the input feature stream is already differential and clean; on raw images it under-performs ResNet because the rare-class signal is drowned out by background patches.

Deep supervision contributes a small but consistent +1 pp by injecting gradient at every decoder stage, which keeps the early upsampling blocks from collapsing onto background. The compounding nature of the contributions — every factor remains additive when stacked — is evidence that the four ideas address orthogonal failure modes (gradient imbalance, change locality, long-range context, gradient flow).

5.5 Threats to validity

• Single dataset. All deltas are measured on xBD. The component ranking may shift on disasters with different damage distributions (e.g. wildfire vs. earthquake) — Lecture L6 returns to this.
• Backbone-size confound. ViT-B/16 has ≈86 M parameters vs. ResNet-50's ≈26 M. The +1.5 pp from A5 → A8 partially reflects capacity, not just attention. A size-matched ResNet-152 control is future work.
• Macro-F1 only. We optimise macro-F1; placard-level deployment cares about confusion between YELLOW and RED. Section 6 closes that loop with the placard confusion matrix.

7.1 Expected outcomes

1. A reproducible benchmark of four architectures on the xBD test split with macro-F1 ≥ 0.75 for the top configuration, and a public ablation table (Table 2) attributing each percentage point to a single component.
2. An open-source model artifact — Siamese ViT + Focal/Dice + deep-sup. U-Net — exported to ONNX and wrapped behind a FastAPI /predict endpoint with documented OpenAPI schema (Lectures W1–W3).
3. An ATC-20 placard mapper with jurisdiction-tunable thresholds (Table 3) that converts per-pixel damage probabilities into a GREEN / YELLOW / RED building tag.
4. Operational latency < 600 ms per 1024×1024 tile on a single T4 GPU, enabling city-scale inference in tens of minutes after imagery arrival.
5. A teaching curriculum (this site) that maps every block of the system back to its underlying paper-grade method (ViT, Siamese, focal/dice, U-Net) so a new student can re-derive the system in eight weeks.

7.2 Application domains

7.3 Impact pathway

The downstream value is not raw F1 — it is hours saved in the placard-issuing pipeline. A trained ATC-20 inspector can evaluate ≈30 buildings per day on foot; a city of 100,000 structures therefore needs ~3,300 inspector-days after a major event. By pre-classifying every building into GREEN / YELLOW / RED and routing inspectors only to YELLOW (uncertain) and RED (high-priority) structures, the system reduces the inspection backlog by an estimated 60–80%and shortens the time-to-reopen for safe buildings from weeks to days.

7.4 Risks and ethical considerations

• False-GREEN risk. A missed RED building has life-safety consequences. Deployment must keep a human-in-the-loop for any building flagged with low confidence; the placard head exposes per-class softmax for exactly this triage.
• Geographic bias. xBD over-represents the global North; transfer-learning experiments to local building stock are mandatory before any new jurisdiction adopts the model.
• Imagery access. Commercial post-event imagery is licensed; the system should be paired with public Sentinel-1/2 fallbacks for equitable access.