Best practices for fire classification with variable-sized input regions using Prithvi backbone

[Lrogic]

This is a repost of an issue I created earlier. I was told to repost it in discussions instead. Here is the original issue and post: #774

I'm working on an image classification task where the goal is to classify satellite imagery as "burned" or "unburned". I'm using the Prithvi backbone, and I'm running into challenges due to the fact that the input fires vary significantly in spatial size.

Approaches Tried

1. Using a large input size (e.g., 512×512)

• Ensures that large fires fit within the input image.
• Issue: Performs poorly for small fires, since they occupy only a tiny portion of the image and the signal gets lost.

2. Tiling into smaller patches (e.g., 32×32)

• Take a larger fire image and divide it into many smaller tiles of a fixed size (like 32×32) so they can be passed to the model individually.
• Each tile is classified independently as "burned" or "unburned".
• Challenges:
  • Each tile only sees a small part of the fire, so it may lack context to make an informed prediction.
  • Requires aggregation of tile-level predictions to make a final decision for the entire fire:
    • Average probabilities?
    • Majority vote?
    • Use the most confident tile?
  • Lack of global context and extra aggregation logic make this less ideal for a simple binary classification task.

Approach Considered but Not Yet Tried

3. Resizing each fire region to a fixed shape (e.g., 224×224)

• Could use stretching or padding to match Prithvi’s expected input size.
• Concerns:
  • Padding may dominate for small fires, as the smallest can be 100× smaller than the largest.
  • Resizing by stretching/compressing may distort spatial information, reducing the benefit of Prithvi’s pretraining.

---

Question:
Is there a recommended approach for handling classification of objects (like fires) that have highly variable spatial extent when using ViT-based models like Prithvi?

Thanks for any help or guidance!

-------------------------------------------------------------- UPDATE --------------------------------------------------------------
small_fire_signals_224px.zip

@bhansaliraunak
Thanks for replying to my issue earlier, and sorry for the late reply. I've created a dataset with 7 small fires in a bigger space as you requested, and attached it as a zip above.

To clarify the dataset:

• The images are formed from HLS (Harmonized Landsat and Sentinel-2) data.
• Each TIFF file contains 18 bands in total. These bands are grouped into three sets of six, representing three images across time.
• Bands 0–5 correspond to the first image with bands: ['blue', 'green', 'red', 'nir', 'swir1', 'swir2'].
• Bands 6–11 correspond to the second image with the same band order.
• Bands 12–17 correspond to the third image, again with the same bands.
• The three images cover the same geographic area, but each is a median composite for a different time period within the same year:
  • First image: June 1 – July 10
  • Second image: July 10 – August 20
  • Third image: August 20 – September 30

So when you load any one tiff it should be three images across time that look something like this:

Where you can see the blackish burn scar in the second and third images.

Let me know if anyone discovers a good performing method on this type of data. Or if I should format my data in a different way to get better results. Thanks!

[bhansaliraunak]

@Lrogic The good performing method challenge here is spatial scale mismatch + multi-temporal, multiple spectral input in an image level classification setting.
Most pre-trained vision models are designed for 3-channel RGB, so if you simply throw this in a plain vanilla Prithvi backbone without adapting for scale and temporal change, you'll likely get overfitting to fires of a particular size & confusion between small burns & noise.

[Lrogic]

I'm using the prithvi_eo_v2_300 backbone, which to my understanding is pretrained on multi-temporal and multi-spectral inputs. So I think the main issue is probably the spatial scale mismatch. Do you have any suggestions for how I could format the data to fix this problem, or perhaps recommend a different model that might work better?

In your earlier reply, you mentioned Adaptive Cropping followed by contextual padded aggregation. Could you clarify what that means?

Thanks for any help or ideas!

[bhansaliraunak]

@Lrogic
So Adaptive cropping with contextual padded aggregation goes as follow:

1. First calculate the change index across time to localise candidate burn scars (you may compute pre_fire & post_fire NBR indexes assuming pre_fire is the first six bands, and post_fire is the last six bands)
2. Then you threshold the change map & find the burn scars (where you're returning the bounding boxes for the burn scars above a certain threshold)
3. For each candidate region, you crop adaptively, and then you pad to your model's req. input size
4. Then you feed all patches through your model. If patches overlap, you may aggregate by averaging per output localisation. (For better results, you may overlap crops at different scales)

[Lrogic]

Thanks for the explanation! I have a few follow-up questions. When you say “crop adaptively,” do you mean that for smaller fires you crop a small area, and for larger fires you crop a bigger area, essentially just around the burn scar?

In this approach, would the 30 m resolution of the HLS data be preserved, without stretching or compressing the image to fit the model’s required input size? I assume that any adjustment to the input size would be handled by padding. And for very large fires, would you create overlapping patches and aggregate the results afterward?

[bhansaliraunak]

1. Is the 30 m resolution of HLS data preserved without rescaling?
   The idea is to avoid any resampling or resizing that would change the pixel resolution so if your model requires the fixed input size (say 256 x 256 pixels) but your crop is only 150 x 150 pixels, you pad the rest typically with the background values or zeroes (You might need to check this part, as padding with the exact background values isn't always quite straightforward).
   So in a way, you attempt to preserve the 30m resolution.

2. For large fires, do you use overlapping patches and then aggregate?
   When a burn scar is too large to fit into a single input patch, you could split it into overlapping crops followed by aggregating the outputs (For aggregation, you could perform average prediction in overlapping regions or use confidence weighting or something that could lead to better accuracy when aggregating - it's upto you)

Note: Sorry for the bit delay. Appreciate your patience and ready to jump back into the discussion.

[Lrogic]

That makes sense. Unfortunately, I'm no longer going to be working on this project, so I won't be able to implement these ideas. But if I ever return to it I will try them and let you know. Thanks for taking the time to think about this and respond.