Damage Assessment · Week 3 · Jun 23 – Jul 18

Baseline, Siamese, ViT + Ablation + ONNX

Steps 6–10. Train three architectures, run a clean ablation study, and export the winning model to ONNX so S4 can plug it into the FastAPI app.

Baseline · CNN encoder + U-Net decoder (S2 · Jun 23 – Jul 4)

python
1!pip install segmentation_models_pytorch wandb --quiet
2import torch, wandb
3import segmentation_models_pytorch as smp
4from dataset import get_loaders
5from metrics import evaluate
6
7device = 'cuda'
8wandb.init(project='ceamls-baseline')
9
10model = smp.Unet(
11    encoder_name='resnet50',
12    encoder_weights='imagenet',
13    in_channels=3,
14    classes=4,
15    activation=None,
16).to(device)
17
18tr, va, te = get_loaders('/content/drive/MyDrive/ceamls-project/data/patches/index.csv',
19                          '/content/drive/MyDrive/ceamls-project/data/patches', batch=16)
20
21W = torch.tensor([0.07, 0.99, 0.65, 14.3]).to(device)   # from Week 2
22criterion = torch.nn.CrossEntropyLoss(weight=W)
23optim     = torch.optim.AdamW(model.parameters(), lr=1e-4, weight_decay=1e-4)
24
25for epoch in range(30):
26    model.train()
27    for batch in tr:
28        logits = model(batch['post'].to(device))
29        loss = criterion(logits, batch['label'].to(device))
30        optim.zero_grad(); loss.backward(); optim.step()
31        wandb.log({'train_loss': loss.item()})
32    val = evaluate(model, va, device)
33    wandb.log({'val_OA': val['OA'], 'val_wF1': val['wF1'], 'epoch': epoch})
34torch.save(model.state_dict(), '/content/drive/MyDrive/ceamls-project/results/baseline.pt')

Plain-English explanation

ResNet-50 encoder (ImageNet pre-trained) + U-Net decoder, 4 damage classes. 30 epochs on T4 ≈ 6 hours.

Why it matters

The baseline establishes the score every advanced model has to beat. Pre-trained ImageNet weights transfer cleanly because satellite RGB still looks like 'natural' imagery to early conv filters.

Expected output

Epoch 30 · train_loss 0.31 · val_OA 0.81 · val_wF1 0.62

Common errors

CUDA out of memory

Drop batch size from 16 to 8, or enable mixed precision: scaler = torch.cuda.amp.GradScaler().

Siamese network · pre/post change detection (S3 · Jun 23 – Jul 11)

python
1import torch, torch.nn as nn
2import segmentation_models_pytorch as smp
3
4class SiameseDamageNet(nn.Module):
5    def __init__(self, num_classes=4):
6        super().__init__()
7        self.shared = smp.encoders.get_encoder('resnet50', weights='imagenet')
8        self.head = nn.Sequential(
9            nn.AdaptiveAvgPool2d(1), nn.Flatten(),
10            nn.Linear(2048, 256), nn.ReLU(),
11            nn.Linear(256, num_classes),
12        )
13    def forward(self, pre, post):
14        f_pre  = self.shared(pre)[-1]     # deepest feature, [B,2048,16,16]
15        f_post = self.shared(post)[-1]
16        diff   = torch.abs(f_pre - f_post)
17        return self.head(diff)
18
19model = SiameseDamageNet().cuda()
20tr, va, te = get_loaders(idx_csv, patch_dir, batch=8, siamese=True)
21# (training loop identical to baseline, but: model(batch['pre'].cuda(), batch['post'].cuda()))

Plain-English explanation

One encoder, two passes, subtract the features, classify the difference.

Why it matters

A single post-only image cannot tell 'this building was always rubble' apart from 'this building was just destroyed'. The pre image supplies the missing baseline.

Quick quiz

Why share the encoder weights between the two branches?

Vision Transformer · fine-tune ViT-B/16 (S3 · Jun 30 – Jul 11)

python
1!pip install transformers --quiet
2from transformers import ViTForImageClassification, ViTImageProcessor
3
4model = ViTForImageClassification.from_pretrained(
5    'google/vit-base-patch16-224',
6    num_labels=4,
7    ignore_mismatched_sizes=True,
8).cuda()
9
10# ViT wants 224×224, not 512×512
11import torch.nn.functional as F
12def to_vit(x):
13    return F.interpolate(x, size=224, mode='bilinear', align_corners=False)
14
15optim = torch.optim.AdamW(model.parameters(), lr=2e-5, weight_decay=0.01)
16for epoch in range(15):
17    for batch in tr:
18        out  = model(pixel_values=to_vit(batch['post'].cuda())).logits
19        loss = criterion(out, batch['label'].cuda())
20        optim.zero_grad(); loss.backward(); optim.step()

Plain-English explanation

ViT splits an image into 16×16 patches, treats them as tokens, and runs self-attention. We fine-tune the Google pre-trained ViT-B/16 on our 4 damage classes.

Why it matters

Transformers reach further than CNNs — every patch can attend to every other patch from layer 1, useful for spatially-separated rubble fields.

Ablation study · 6 runs, one change per row (S2+S3 · Jul 7–11)

python
1# Each row keeps everything else identical and changes ONE thing
2ablation_rows = [
3    ('A-01 baseline',          dict()),                                # reference
4    ('A-02 no pretrain',       dict(encoder_weights=None)),            # remove ImageNet init
5    ('A-03 focal loss',        dict(loss='focal')),                    # CE → focal
6    ('A-04 no augmentation',   dict(aug='off')),                       # remove flips/rot
7    ('A-05 ViT-B/16',          dict(arch='vit')),                      # CNN → transformer
8    ('A-06 Siamese',           dict(arch='siamese')),                  # add pre branch
9]
10# Train each row identically, log to one Google Sheet:
11# row | val_OA | val_wF1 | val_f1_destroyed | params | hours
12# The drop in val_wF1 vs A-01 tells you which component matters most.

Plain-English explanation

An ablation answers 'if I delete this part, how much worse does the model get?'.

Why it matters

Change exactly one thing per run — otherwise you can't attribute the metric change to a single cause.

Expected output

A-01 baseline wF1 0.62 A-02 no pretrain wF1 0.41 (-0.21 ← pretraining matters most) A-03 focal loss wF1 0.66 (+0.04) A-04 no aug wF1 0.58 (-0.04) A-05 ViT-B/16 wF1 0.64 (+0.02) A-06 Siamese wF1 0.71 (+0.09 ← pre branch is the big win)

Quick quiz

Why is A-02 (no pretrain) usually the biggest drop?

Retrain best model on train+val, export ONNX (S2+S3+S5 · Jul 11–18)

python
1# Best model from ablation: Siamese + Focal+Dice + augmentation
2model.load_state_dict(torch.load('best_siamese.pt'))
3model.eval()
4dummy_pre  = torch.randn(1, 3, 224, 224).cuda()
5dummy_post = torch.randn(1, 3, 224, 224).cuda()
6torch.onnx.export(
7    model, (dummy_pre, dummy_post),
8    '/content/drive/MyDrive/ceamls-project/results/model.onnx',
9    input_names=['pre', 'post'],
10    output_names=['logits'],
11    dynamic_axes={'pre':{0:'B'}, 'post':{0:'B'}, 'logits':{0:'B'}},
12    opset_version=17,
13)
14print('Exported model.onnx — hand to S4 for the FastAPI app')

Plain-English explanation

ONNX is the portable model format. The FastAPI app loads model.onnx with onnxruntime — no PyTorch needed in production.

Why it matters

S4's FastAPI service runs on CPU and must start in <2 seconds. PyTorch would pull 800 MB of CUDA libraries; ONNX Runtime is 12 MB and CPU-native.

Expected output

Exported model.onnx (172 MB)

Handoff to S4 on Jul 18

The deliverable for Week 3 is results/model.onnx. S4 picks it up next and wires it into the FastAPI /classify endpoint — see the Full Code page for the complete server.