STRATA: from atmospheric rollout to energy-asset analysis#

The executable walkthrough now follows a complete study: inspect a native-layout dataset, locate three illustrative Canary assets, extract 24 hours, evaluate paired reference series, calculate explicit production scenarios, vary assumptions and export results.

The default example is synthetic. It exercises the real adapter but does not execute the neural network or demonstrate STRATA forecast skill. A second path consumes a completed upstream rollout without changing the analysis code.

1. Run the complete offline example#

From the Energy Data Insights repository root:

uv sync --locked --all-packages
uv run python examples/analysis/plot_strata_storm_energy.py
uv run python -m energy_analysis.strata --demo --output artifacts/strata-demo

The gallery creates three figures: geographic field snapshots, weather time series, and energy/threshold sensitivity. The CLI writes these study artifacts:

File

Meaning

asset_cells.csv

Requested coordinates, selected cube indices, cell coordinates and distance

weather.csv

Asset/valid-time rows, wind components/speed, model-level label, rain rate and temperature

scenario.csv

Baseline and scenario MWh, alert flags and gross energy value at risk

summary.csv

Per-asset energy totals and threshold-exposure hours

reference_metrics.csv

MAE/RMSE/bias and pair counts, only when a reference is supplied

weather.png

Three-panel time series with its input basis in the title

manifest.json

Data basis, assumptions, versions, source paths and sampled CSV SHA-256 hashes

Use an empty output directory for each run; existing artifacts are never mixed with a new study.

The default synthetic scenario gives the following rounded values:

Illustrative asset

Baseline MWh

Scenario MWh

Energy at risk MWh

Gross value at risk EUR

Tenerife PV

114.919

101.047

13.872

832.32

Gran Canaria wind

324.000

311.175

12.825

769.50

La Palma PV

57.460

57.460

0.000

0.00

La Palma has threshold exposure during zero planned PV production; it therefore has zero energy at risk in this example. An alert is not itself a production loss. These results describe the constructed inputs, not actual plants.

2. Understand the interfaces before substituting data#

Native field

Dimensions

Meaning / conversion

time

time

Simulation initialization, decoded datetime

step

step

Lead timedelta; numeric values require explicit seconds units

lat, lon

face,x,y

Cell centres in degrees; longitude wrapping is handled

T_2m

time,step,face,x,y

Kelvin → °C by subtracting 273.15

U, V

time,step,level,face,x,y

Model-level wind components in m/s

precip_liq_surf_mass_flux

time,step,face,x,y

Water-equivalent m/s → mm/h by multiplying by 3,600,000

level

level

SCREAM model-level label; neither metres nor pressure

The upstream writer omits weather-variable unit metadata. When missing, this adapter applies the documented native SCREAM units. Conflicting declared units are rejected. A generic dataset with these names but other units must be normalized explicitly before using this adapter.

extract_strata_timeseries(..., wind_level=30) selects coordinate label 30. The original sample_strata_output(..., wind_level=0) remains a positional single-cell interface; prefer the new label-based time-series API.

Nearest-cell lookup scans only latitude/longitude in bounded chunks, supports cube seams and longitude wrapping, and records great-circle distance. It loads weather only after selecting a cell and initialization. The coordinate scan still reads the global coordinate arrays once per asset; this is a small portfolio workflow. For repeated or very large studies, retain the mapping and use extract_strata_timeseries with saved GridCell values. Do not build an unbounded in-memory tree over all global cells merely for three sites.

3. Generate genuine STRATA output in its GPU environment#

This integration targets upstream commit febe5219bb39e2da8b96af48c377d888952d5054. That revision fixes the saved-frame timing for --output-steps > 1. Older rollouts can label fields later than their actual forecast lead; do not silently use those outputs as correct time pairs.

The checkpoint, auxiliary meshes and GPU runtime are separate from Energy Data Insights. The supported upstream container is based on NVIDIA PyTorch 26.09; it includes compatible CUDA, DALI, NATTEN and PhysicsNeMo dependencies. The published four-GPU recipe uses 4 × H100 80 GB, batch size 1. A CPU-only installation of this repository cannot run that recipe.

On the provisioned GPU host:

git clone https://github.com/NVlabs/STRATA.git
cd STRATA
git checkout febe5219bb39e2da8b96af48c377d888952d5054
docker build -f docker/Dockerfile -t strata .
mkdir -p strata-data strata-output
docker run --rm -it --gpus all --ipc=host \
  -v "$PWD:/workspace/STRATA" -w /workspace/STRATA strata bash

Inside the container, acquire the checkpoint and generate auxiliary geometry:

hf download nvidia/strata --local-dir /workspace/STRATA/strata-data
export AUX_DATA_ROOT=/workspace/STRATA/strata-data
bash data_prep/scrip_generation/generate_all.sh --output-dir "$AUX_DATA_ROOT"
cp data/scream_vertical_coordinate.nc "$AUX_DATA_ROOT/"
export PYTHONPATH=/workspace/STRATA
export SCREAM_MAIN_ZARR_PATH=hf://datasets/nvidia/STRATA-SCREAM-sdecadal

Remote initial-condition streaming avoids a complete multi-terabyte training store download. It still transfers a global initial state. Keep checkpoint revision/hash, generated mesh hashes, upstream commit, container digest, device configuration and the full run log with the output.

Start with a two-step smoke run (two ten-minute leads):

torchrun --nproc_per_node=4 scripts/ace/run_screamcast_nudged.py \
  --run-name edi_strata_smoke \
  --checkpoint "$AUX_DATA_ROOT/checkpoints/strata_v1.pth" \
  --forecast-model screamcast --correction none \
  --n-steps 2 --output-steps 1 \
  --tile-size 128 --halo-width 16 --halo-adjoint \
  --no-halo-adjoint-cross-face --tile-jitter 4 --tile-jitter-mode cycle16 \
  --sht-omega-lmax 3 --inference-batch-size 1 \
  --output-variables T_2m U V precip_liq_surf_mass_flux \
  --output-levels 30 --initial-time 2020-10-13T00:00:00 \
  /workspace/STRATA/strata-output/strata_smoke.zarr

After confirming a successful completed run, repeat with a new output path and --n-steps 144 --output-steps 6 for 24 hourly saved frames. These arguments reduce saved output, not the model’s internal state or computation. Four float32 channels on six 2048² faces and 24 frames are about 9.7 GB before compression, coordinates and overhead. Keeping every ten-minute frame is about 58 GB. Hourly endpoint fields can miss peaks: use ten-minute output for a precipitation/exposure integration study when resources allow.

This recipe is documented from upstream source; it is not GPU-tested by this PR. Runtime and memory vary with the host. The released ice precipitation channel has known limitations and is not included in this integration.

4. Consume the completed output in Energy Data Insights#

Create an assets.csv with independently verified asset coordinates. This example uses an illustrative site, not a registry record:

asset,latitude,longitude
illustrative_tenerife,28.30,-16.57

Back in the Energy Data Insights repository:

uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
  --input /path/to/strata_24h.zarr --assets assets.csv \
  --wind-level 30 --time-index 0 --max-distance-km 25 \
  --output artifacts/strata-real

This produces weather, mapping, plot and manifest. It produces no energy scenario unless an explicit baseline is supplied. Zarr is an optional reader dependency; this command provisions it without modifying the workspace lock. The reader targets native cubed-sphere output, not pressure-remapped output or raw training stores shaped (time,lev,ncol).

The equivalent Python entry point is:

import pandas as pd
import xarray as xr
from energy_analysis.strata import sample_assets

with xr.open_zarr("/path/to/strata_24h.zarr", chunks=None, consolidated=False) as ds:
    weather, cells = sample_assets(ds, pd.read_csv("assets.csv"), wind_level=30)
print(cells)
print(weather.head())

Confirm the run finished and all intended chunks were written. STRATA preallocates arrays; an interrupted or partial store can contain zeros that pass a finite-value check. The manifest explicitly records that full store integrity has not been verified. Its CSV hashes identify the sampled artifacts, not the checkpoint or entire source store.

5. Compare against an appropriate reference#

STRATA v1 emulates a SCREAM simulation. Its October 2020 timestamps do not reproduce observed October 2020 weather. Use held-out SCREAM evolution, not date-matched ERA5, AEMET or operational forecasts, to evaluate this checkpoint. Upstream identifies October 11 onward as held-out sdecadal initializations; October 13 is used above. Fit thresholds or calibration on other periods.

Prepare reference.csv independently, with one row for every forecast asset and valid time and these columns:

asset,valid_time,wind_model_level,wind_speed_ms,precipitation_mm_h,temperature_c

Reference preparation must attach the raw SCREAM coordinates/time convention, map the same geographic cells using upstream cube ordering, select the same model level, convert units, and sample at the forecast valid times. The raw Hugging Face stores lack coordinates and are not interchangeable with the rollout cube. This PR does not automatically acquire or remap the raw training store. If you already have a verified reference exported in the rollout schema, use sample_assets(reference_dataset, assets, wind_level=30) and export its weather table as the reference CSV. Retain its independent provenance.

uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
  --input /path/to/strata_24h.zarr --assets assets.csv --wind-level 30 \
  --reference reference.csv --output artifacts/strata-validated

compare_reference requires identical unique asset/time pairs and matching level labels; missing values and partial joins fail. It reports MAE, RMSE, signed bias and pair count per asset/variable. The included persistence comparator holds the first saved forecast value constant. It is a descriptive baseline, not a reference-initialization persistence benchmark. The gallery deliberately uses known-offset synthetic reference values to verify arithmetic; those scores are not evidence of STRATA skill. Multiple held-out initializations, uncertainty and appropriate forecast baselines are still needed for scientific evaluation.

6. Add an explicit energy scenario#

Provide baseline.csv with exactly one row per sampled asset/valid time:

asset,valid_time,baseline_mw

baseline_mw is planned mean MW for the preceding saved interval, from an independent production model or dispatch plan. The CLI reorders by exact keys; it rejects missing, duplicate or extra pairs. Use simulation UTC timestamps without a timezone suffix, consistent with weather.csv.

uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
  --input /path/to/strata_24h.zarr --assets assets.csv --wind-level 30 \
  --baseline baseline.csv --wind-threshold-ms 20 --rain-threshold-mm-h 15 \
  --price-eur-mwh 60 --output artifacts/strata-scenario

For the example, wind exceedance reduces power by 15%, rain exceedance by 5%, and both reductions add. The Python energy_scenario API also accepts different reduction fractions. Energy is mean baseline MW × interval hours × multiplier; gross value at risk is the MWh difference × the supplied EUR/MWh price. Negative prices are permitted and can reverse the sign of this gross value measure. This is not profit, avoided cost, technical availability or a probability.

The gallery tests wind thresholds 16/20/24/28 m/s while keeping rain fixed. This is deterministic sensitivity, not an ensemble uncertainty estimate. The right-endpoint threshold mask applies over the preceding output interval; exposure hours are a discretized scenario and cannot establish actual event onset.

Do not treat model-level U/V as 10-m wind, hub-height wind, gusts or a vendor shutdown threshold. PV needs irradiance and thermal/electrical models; wind needs height-consistent wind and a turbine response; rain alone cannot predict flooding, soiling removal or site damage. Those extensions require separate measurements, response models and validation.

Validation and attribution#

Focused local checks:

uv run pytest methods/tests/test_strata_impact.py analysis/tests/test_strata.py
uv run ruff check methods/src/energy_methods/strata_impact.py analysis/src/energy_analysis/strata.py
uv run ty check methods/src/energy_methods/strata_impact.py analysis/src/energy_analysis/strata.py

Tests cover known unit conversions, noncontiguous level labels, multiple initializations, irregular lead spacing, dateline/cube-face selection, invalid fields, exact pairing, hand-calculated MWh/EUR results, exports and CLI execution. The Zarr round-trip test runs when Zarr is installed; use uv run --with 'zarr>=3,<4' pytest analysis/tests/test_strata.py to include it.

The source contract and GPU settings come from NVIDIA’s STRATA repository, especially screamcast/zarr_writer.py and scripts/ace/run_screamcast_nudged.py. Repository-specific scenario equations and fixtures are original illustrative adaptations; they are not methods claimed by the STRATA authors.

Upstream source is Apache-2.0, checkpoint weights use OpenMDW 1.1, and SCREAM data/static fields carry their separate CC BY 4.0 attribution. No weights or SCREAM datasets are redistributed here. The Energy Data Insights project license continues to govern this integration; upstream licenses do not grant additional rights to this repository.