STRATA: from atmospheric rollout to energy-asset analysis#
The executable walkthrough now follows a complete study: inspect a native-layout dataset, locate three illustrative Canary assets, extract 24 hours, evaluate paired reference series, calculate explicit production scenarios, vary assumptions and export results.
The default example is synthetic. It exercises the real adapter but does not execute the neural network or demonstrate STRATA forecast skill. A second path consumes a completed upstream rollout without changing the analysis code.
1. Run the complete offline example#
From the Energy Data Insights repository root:
uv sync --locked --all-packages
uv run python examples/analysis/plot_strata_storm_energy.py
uv run python -m energy_analysis.strata --demo --output artifacts/strata-demo
The gallery creates three figures: geographic field snapshots, weather time series, and energy/threshold sensitivity. The CLI writes these study artifacts:
File |
Meaning |
|---|---|
|
Requested coordinates, selected cube indices, cell coordinates and distance |
|
Asset/valid-time rows, wind components/speed, model-level label, rain rate and temperature |
|
Baseline and scenario MWh, alert flags and gross energy value at risk |
|
Per-asset energy totals and threshold-exposure hours |
|
MAE/RMSE/bias and pair counts, only when a reference is supplied |
|
Three-panel time series with its input basis in the title |
|
Data basis, assumptions, versions, source paths and sampled CSV SHA-256 hashes |
Use an empty output directory for each run; existing artifacts are never mixed with a new study.
The default synthetic scenario gives the following rounded values:
Illustrative asset |
Baseline MWh |
Scenario MWh |
Energy at risk MWh |
Gross value at risk EUR |
|---|---|---|---|---|
Tenerife PV |
114.919 |
101.047 |
13.872 |
832.32 |
Gran Canaria wind |
324.000 |
311.175 |
12.825 |
769.50 |
La Palma PV |
57.460 |
57.460 |
0.000 |
0.00 |
La Palma has threshold exposure during zero planned PV production; it therefore has zero energy at risk in this example. An alert is not itself a production loss. These results describe the constructed inputs, not actual plants.
2. Understand the interfaces before substituting data#
Native field |
Dimensions |
Meaning / conversion |
|---|---|---|
|
|
Simulation initialization, decoded datetime |
|
|
Lead timedelta; numeric values require explicit seconds units |
|
|
Cell centres in degrees; longitude wrapping is handled |
|
|
Kelvin → °C by subtracting 273.15 |
|
|
Model-level wind components in m/s |
|
|
Water-equivalent m/s → mm/h by multiplying by 3,600,000 |
|
|
SCREAM model-level label; neither metres nor pressure |
The upstream writer omits weather-variable unit metadata. When missing, this adapter applies the documented native SCREAM units. Conflicting declared units are rejected. A generic dataset with these names but other units must be normalized explicitly before using this adapter.
extract_strata_timeseries(..., wind_level=30) selects coordinate label 30.
The original sample_strata_output(..., wind_level=0) remains a positional
single-cell interface; prefer the new label-based time-series API.
Nearest-cell lookup scans only latitude/longitude in bounded chunks, supports
cube seams and longitude wrapping, and records great-circle distance. It loads
weather only after selecting a cell and initialization. The coordinate scan
still reads the global coordinate arrays once per asset; this is a small
portfolio workflow. For repeated or very large studies, retain the mapping
and use extract_strata_timeseries with saved GridCell values. Do not build
an unbounded in-memory tree over all global cells merely for three sites.
3. Generate genuine STRATA output in its GPU environment#
This integration targets upstream commit
febe5219bb39e2da8b96af48c377d888952d5054.
That revision fixes the saved-frame timing for --output-steps > 1. Older
rollouts can label fields later than their actual forecast lead; do not silently
use those outputs as correct time pairs.
The checkpoint, auxiliary meshes and GPU runtime are separate from Energy Data Insights. The supported upstream container is based on NVIDIA PyTorch 26.09; it includes compatible CUDA, DALI, NATTEN and PhysicsNeMo dependencies. The published four-GPU recipe uses 4 × H100 80 GB, batch size 1. A CPU-only installation of this repository cannot run that recipe.
On the provisioned GPU host:
git clone https://github.com/NVlabs/STRATA.git
cd STRATA
git checkout febe5219bb39e2da8b96af48c377d888952d5054
docker build -f docker/Dockerfile -t strata .
mkdir -p strata-data strata-output
docker run --rm -it --gpus all --ipc=host \
-v "$PWD:/workspace/STRATA" -w /workspace/STRATA strata bash
Inside the container, acquire the checkpoint and generate auxiliary geometry:
hf download nvidia/strata --local-dir /workspace/STRATA/strata-data
export AUX_DATA_ROOT=/workspace/STRATA/strata-data
bash data_prep/scrip_generation/generate_all.sh --output-dir "$AUX_DATA_ROOT"
cp data/scream_vertical_coordinate.nc "$AUX_DATA_ROOT/"
export PYTHONPATH=/workspace/STRATA
export SCREAM_MAIN_ZARR_PATH=hf://datasets/nvidia/STRATA-SCREAM-sdecadal
Remote initial-condition streaming avoids a complete multi-terabyte training store download. It still transfers a global initial state. Keep checkpoint revision/hash, generated mesh hashes, upstream commit, container digest, device configuration and the full run log with the output.
Start with a two-step smoke run (two ten-minute leads):
torchrun --nproc_per_node=4 scripts/ace/run_screamcast_nudged.py \
--run-name edi_strata_smoke \
--checkpoint "$AUX_DATA_ROOT/checkpoints/strata_v1.pth" \
--forecast-model screamcast --correction none \
--n-steps 2 --output-steps 1 \
--tile-size 128 --halo-width 16 --halo-adjoint \
--no-halo-adjoint-cross-face --tile-jitter 4 --tile-jitter-mode cycle16 \
--sht-omega-lmax 3 --inference-batch-size 1 \
--output-variables T_2m U V precip_liq_surf_mass_flux \
--output-levels 30 --initial-time 2020-10-13T00:00:00 \
/workspace/STRATA/strata-output/strata_smoke.zarr
After confirming a successful completed run, repeat with a new output path and
--n-steps 144 --output-steps 6 for 24 hourly saved frames. These arguments
reduce saved output, not the model’s internal state or computation. Four
float32 channels on six 2048² faces and 24 frames are about 9.7 GB before
compression, coordinates and overhead. Keeping every ten-minute frame is
about 58 GB. Hourly endpoint fields can miss peaks: use ten-minute output for
a precipitation/exposure integration study when resources allow.
This recipe is documented from upstream source; it is not GPU-tested by this PR. Runtime and memory vary with the host. The released ice precipitation channel has known limitations and is not included in this integration.
4. Consume the completed output in Energy Data Insights#
Create an assets.csv with independently verified asset coordinates. This
example uses an illustrative site, not a registry record:
asset,latitude,longitude
illustrative_tenerife,28.30,-16.57
Back in the Energy Data Insights repository:
uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
--input /path/to/strata_24h.zarr --assets assets.csv \
--wind-level 30 --time-index 0 --max-distance-km 25 \
--output artifacts/strata-real
This produces weather, mapping, plot and manifest. It produces no energy
scenario unless an explicit baseline is supplied. Zarr is an optional reader
dependency; this command provisions it without modifying the workspace lock.
The reader targets native cubed-sphere output, not pressure-remapped output or
raw training stores shaped (time,lev,ncol).
The equivalent Python entry point is:
import pandas as pd
import xarray as xr
from energy_analysis.strata import sample_assets
with xr.open_zarr("/path/to/strata_24h.zarr", chunks=None, consolidated=False) as ds:
weather, cells = sample_assets(ds, pd.read_csv("assets.csv"), wind_level=30)
print(cells)
print(weather.head())
Confirm the run finished and all intended chunks were written. STRATA preallocates arrays; an interrupted or partial store can contain zeros that pass a finite-value check. The manifest explicitly records that full store integrity has not been verified. Its CSV hashes identify the sampled artifacts, not the checkpoint or entire source store.
5. Compare against an appropriate reference#
STRATA v1 emulates a SCREAM simulation. Its October 2020 timestamps do not reproduce observed October 2020 weather. Use held-out SCREAM evolution, not date-matched ERA5, AEMET or operational forecasts, to evaluate this checkpoint. Upstream identifies October 11 onward as held-out sdecadal initializations; October 13 is used above. Fit thresholds or calibration on other periods.
Prepare reference.csv independently, with one row for every forecast asset
and valid time and these columns:
asset,valid_time,wind_model_level,wind_speed_ms,precipitation_mm_h,temperature_c
Reference preparation must attach the raw SCREAM coordinates/time convention,
map the same geographic cells using upstream cube ordering, select the same
model level, convert units, and sample at the forecast valid times. The raw
Hugging Face stores lack coordinates and are not interchangeable with the
rollout cube. This PR does not automatically acquire or remap the raw training
store. If you already have a verified reference exported in the rollout schema,
use sample_assets(reference_dataset, assets, wind_level=30) and export its
weather table as the reference CSV. Retain its independent provenance.
uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
--input /path/to/strata_24h.zarr --assets assets.csv --wind-level 30 \
--reference reference.csv --output artifacts/strata-validated
compare_reference requires identical unique asset/time pairs and matching
level labels; missing values and partial joins fail. It reports MAE, RMSE,
signed bias and pair count per asset/variable. The included persistence comparator
holds the first saved forecast value constant. It is a descriptive baseline,
not a reference-initialization persistence benchmark. The gallery deliberately
uses known-offset synthetic reference values to verify arithmetic; those scores
are not evidence of STRATA skill. Multiple held-out initializations, uncertainty
and appropriate forecast baselines are still needed for scientific evaluation.
6. Add an explicit energy scenario#
Provide baseline.csv with exactly one row per sampled asset/valid time:
asset,valid_time,baseline_mw
baseline_mw is planned mean MW for the preceding saved interval, from an
independent production model or dispatch plan. The CLI reorders by exact keys;
it rejects missing, duplicate or extra pairs. Use simulation UTC timestamps
without a timezone suffix, consistent with weather.csv.
uv run --with 'zarr>=3,<4' python -m energy_analysis.strata \
--input /path/to/strata_24h.zarr --assets assets.csv --wind-level 30 \
--baseline baseline.csv --wind-threshold-ms 20 --rain-threshold-mm-h 15 \
--price-eur-mwh 60 --output artifacts/strata-scenario
For the example, wind exceedance reduces power by 15%, rain exceedance by 5%,
and both reductions add. The Python energy_scenario API also accepts different
reduction fractions. Energy is mean baseline MW × interval hours × multiplier;
gross value at risk is the MWh difference × the supplied EUR/MWh price. Negative
prices are permitted and can reverse the sign of this gross value measure.
This is not profit, avoided cost, technical availability or a probability.
The gallery tests wind thresholds 16/20/24/28 m/s while keeping rain fixed. This is deterministic sensitivity, not an ensemble uncertainty estimate. The right-endpoint threshold mask applies over the preceding output interval; exposure hours are a discretized scenario and cannot establish actual event onset.
Do not treat model-level U/V as 10-m wind, hub-height wind, gusts or a vendor shutdown threshold. PV needs irradiance and thermal/electrical models; wind needs height-consistent wind and a turbine response; rain alone cannot predict flooding, soiling removal or site damage. Those extensions require separate measurements, response models and validation.
Validation and attribution#
Focused local checks:
uv run pytest methods/tests/test_strata_impact.py analysis/tests/test_strata.py
uv run ruff check methods/src/energy_methods/strata_impact.py analysis/src/energy_analysis/strata.py
uv run ty check methods/src/energy_methods/strata_impact.py analysis/src/energy_analysis/strata.py
Tests cover known unit conversions, noncontiguous level labels, multiple
initializations, irregular lead spacing, dateline/cube-face selection, invalid
fields, exact pairing, hand-calculated MWh/EUR results, exports and CLI execution.
The Zarr round-trip test runs when Zarr is installed; use
uv run --with 'zarr>=3,<4' pytest analysis/tests/test_strata.py to include it.
The source contract and GPU settings come from NVIDIA’s
STRATA repository, especially
screamcast/zarr_writer.py
and scripts/ace/run_screamcast_nudged.py.
Repository-specific scenario equations and fixtures are original illustrative
adaptations; they are not methods claimed by the STRATA authors.
Upstream source is Apache-2.0, checkpoint weights use OpenMDW 1.1, and SCREAM data/static fields carry their separate CC BY 4.0 attribution. No weights or SCREAM datasets are redistributed here. The Energy Data Insights project license continues to govern this integration; upstream licenses do not grant additional rights to this repository.