Satellite Methane Detection & Agentic Incident Investigation

Problem

Methane leaks at refineries and gas plants are both a safety and an emissions risk. Ground sensors only cover fixed points, satellite imagery is too large to review by hand, and many apparent plumes are actually planned venting or flaring, so raw detections alone would bury operators in false alarms.

Approach

I designed a hybrid on-premise/AWS architecture with three layers: a Spark/Sedona ingestion pipeline for three public satellite sensors, per-sensor segmentation models with physics-based features and emission-rate quantification, and an agentic layer where rules triage each detection before an LLM agent investigates the root cause. The model is deliberately framed as a human-reviewed candidate ranker, not an autonomous decision-maker.

How It Works

1. 1
   IngestAirflow DAGs resolve satellite tiles from each site footprint and pull Sentinel-2, Landsat 8/9, and NASA EMIT scenes into a Hadoop data lake.
2. 2
   Bronze & Silver layersSpark and rasterio decode pixels into partitioned tables; Apache Sedona clips them to each site polygon.
3. 3
   DetectPer-sensor PyTorch segmentation trained on Dataiku GPU nodes on AWS EKS: a DeepLabV3+/ConvNeXt model with SimCLR self-supervised pretraining on Sentinel-2 band ratios, a U-Net for Landsat, and a matched filter plus U-Net for hyperspectral EMIT.
4. 4
   QuantifyIntegrated Mass Enhancement with reanalysis wind data estimates the emission rate in kg/hr, calibrated to about 5% mean error; plume polygons and rates land in Gold tables and GeoTIFFs.
5. 5
   TriageRules check distance to assets, gas-detector alarms within ±60 minutes, impact buffers, and work orders for planned venting, then assign a severity level.
6. 6
   InvestigateFor higher-severity events, a LangGraph Deep Agent on Gemini, running as Databricks Lakeflow Jobs, correlates the detection with sensor, asset, work-order, P&ID, and HSE data, finds the root cause, recommends next-best actions, and writes a report guarded by citation checks.
7. 7
   Deliver & evaluatePlume polygons publish to ArcGIS and alerts to a Delta table; LLM-as-judge scores faithfulness, relevancy, and completeness in Langfuse.

What I Built

• The hybrid on-premise/AWS architecture: Airflow ingestion, PySpark/Apache Sedona medallion pipelines, Dataiku GPU training on AWS EKS, and Databricks agents with Lakeflow Jobs publishing to ArcGIS.
• Ingestion and medallion pipelines for three satellite sensors, including the migration of ingestion from Databricks to Airflow + Hadoop.
• PyTorch segmentation models (U-Net, DeepLabV3+ with ConvNeXt) with SimCLR self-supervised pretraining on 57K tiles, reaching an F1 above 0.7 on real plumes.
• The whole agentic layer: LLM parsing of plant diagrams and procedure PDFs into tables and wiki pages, rule-based triage, the Deep Agent for root cause and next-best actions, and the alert contract.
• An evaluation harness with ground-truth cases and LLM-as-judge thresholds, plus about 150 automated tests.

Impact

The platform goes to production in October 2026. It takes raw satellite pixels to a reviewable incident report, validated on public methane benchmarks and a synthetic end-to-end incident pilot, and filters out planned operations before anyone is alerted. I wrote nearly all of the ML and agent code and most of the data engineering.