Damage Assessment · Week 1 · Jun 1–13

Setup + Read GeoTIFF Satellite Data

Steps 1–2 of the CEAMLS guide. Stand up the cloud workspace, download the xBD dataset (51 GB compressed), and read 16-bit GeoTIFF tiles with Rasterio.

Three things to know before you touch the data

(1) Files are GeoTIFF (.tif), 8 spectral channels, 16-bit values 0–65535 — Paint/Photos cannot open them. (2) All training runs in Google Colab, not on your Windows PC. (3) Save everything to /content/drive/MyDrive/… — Colab's local /content/ resets each session.

Create the Google Drive project folder

python
1# Open drive.google.com → New → Folder
2# Name it: ceamls-project
3# Inside it, create subfolders:
4#   data/   notebooks/   results/   patches/

Plain-English explanation

Every student creates the same folder layout in their own Drive so notebooks and paths line up.

Why it matters

Colab notebooks reference '/content/drive/MyDrive/ceamls-project/...'. If the folder layout differs across students, every shared notebook breaks on path errors.

Line by line

ceamls-project/Top-level project root, mounted at /content/drive/MyDrive/ceamls-project in Colab.
data/Holds the extracted xBD GeoTIFF tiles and JSON labels.
notebooks/All .ipynb files and Python modules (dataset.py, metrics.py).
results/Trained model weights, ONNX exports, confusion matrices, plots.

Expected output

ceamls-project/ ├── data/ (will hold ~51 GB extracted) ├── notebooks/ ├── results/ └── patches/

Enable the Colab GPU runtime

python
1# In Colab: Runtime → Change runtime type → T4 GPU → Save
2import torch
3print('CUDA available:', torch.cuda.is_available())
4print('Device:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CPU')

Plain-English explanation

Free Colab gives you an NVIDIA T4 GPU. Training the baseline CNN+UNet on CPU would take days; on T4 it takes hours.

Why it matters

The xBD dataset has 850k patches. Each training epoch needs ~10⁹ FLOPs per patch; only GPU makes that tractable in a summer project.

Expected output

CUDA available: True Device: Tesla T4

Common errors

CUDA available: False

You forgot to switch to GPU runtime. Runtime → Change runtime type → T4 GPU. Re-run the cell.

Register on xBD and download the 51 GB dataset

python
1# 1. Go to https://xview2.org/dataset → Register
2# 2. Download all 7 parts (~7.3 GB each) to your Windows PC
3# 3. Upload all 7 .tar.gz parts to Drive → ceamls-project/data/
4# 4. Then in Colab, combine and extract:
5!cat /content/drive/MyDrive/ceamls-project/data/xview2_*.tar.gz \
6    > /content/drive/MyDrive/ceamls-project/data/combined.tar.gz
7!tar -xzf /content/drive/MyDrive/ceamls-project/data/combined.tar.gz \
8    -C /content/drive/MyDrive/ceamls-project/data/

Plain-English explanation

xBD is the official Department-of-Defense satellite damage dataset: 18,336 pre/post image pairs covering 6 disaster types.

Why it matters

The dataset is split into 7 parts because a single 51 GB upload would time out. cat concatenates them back into one archive before extraction.

Expected output

combined.tar.gz: 51.2 GB Extracted: 36,672 GeoTIFF tiles + 36,672 JSON labels

Common errors

tar: Unexpected EOF in archive

One part downloaded incompletely. Re-download the smallest part and re-run cat.

Install Rasterio and read a GeoTIFF

python
1!pip install rasterio --quiet
2from google.colab import drive
3drive.mount('/content/drive')
4
5import rasterio
6import numpy as np
7
8path = '/content/drive/MyDrive/ceamls-project/data/train/images/hurricane-harvey_00000000_post_disaster.tif'
9with rasterio.open(path) as src:
10    print('Channels:', src.count)
11    print('Size:', src.width, 'x', src.height)
12    print('Dtype:', src.dtypes[0])
13    print('CRS:', src.crs)
14    img = src.read([5, 3, 2])           # bands 5,3,2 ≈ true colour R,G,B
15    img = img.astype(np.float32) / 65535.0
16    img = np.transpose(img, (1, 2, 0))  # (H, W, 3)
17print('Shape ready for display:', img.shape)

Plain-English explanation

Rasterio is the geospatial-aware image reader. It preserves 16-bit precision and the CRS (coordinate reference system) — cv2 throws both away.

Why it matters

xBD stores 8 spectral bands at 16-bit; bands 5/3/2 form the natural-colour RGB triplet. Dividing by 65535 converts to [0,1] floats ready for PyTorch.

Line by line

src.read([5,3,2])Read three specific bands; returns a (3, H, W) array.
/65535.0Max 16-bit value; normalises to [0,1] without clipping.
np.transpose(...)Rasterio uses (C,H,W); matplotlib/PyTorch viewers use (H,W,C).
src.crsCoordinate reference system (e.g. EPSG:32615) — what you'd lose with cv2.imread.

Expected output

Channels: 8 Size: 1024 x 1024 Dtype: uint16 CRS: EPSG:32615 Shape ready for display: (1024, 1024, 3)

Common errors

RasterioIOError: not recognized as a supported file format

File is corrupt. Re-extract the .tar.gz part that contained it.

Quick quiz

Why use Rasterio instead of cv2.imread() for xBD .tif files?

Build the pre/post pair index CSV

python
1import os, glob, pandas as pd
2
3IMG_DIR = '/content/drive/MyDrive/ceamls-project/data/train/images'
4LBL_DIR = '/content/drive/MyDrive/ceamls-project/data/train/labels'
5
6pre_imgs  = sorted(glob.glob(f'{IMG_DIR}/*_pre_disaster.tif'))
7post_imgs = [p.replace('_pre_', '_post_') for p in pre_imgs]
8pre_lbls  = [p.replace(IMG_DIR, LBL_DIR).replace('.tif', '.json') for p in pre_imgs]
9post_lbls = [p.replace('_pre_', '_post_') for p in pre_lbls]
10
11meta = pd.DataFrame({
12    'event': [os.path.basename(p).split('_')[0] for p in pre_imgs],
13    'pre_img':  pre_imgs,
14    'post_img': post_imgs,
15    'pre_lbl':  pre_lbls,
16    'post_lbl': post_lbls,
17})
18meta.to_csv('/content/drive/MyDrive/ceamls-project/data/pairs.csv', index=False)
19print('Pairs found:', len(meta))
20print(meta['event'].value_counts())

Plain-English explanation

One row per (pre, post) image pair — 18,336 in total across 6 disaster events.

Why it matters

S2 (baseline) and S3 (Siamese, ViT) all load data through this CSV. A single source of truth keeps the splits identical across teammates.

Expected output

Pairs found: 18336 hurricane-harvey 4348 hurricane-michael 2950 santa-rosa-wildfire 2562 midwest-flooding 2470 ...