Copernicus Climate Data Store API#
This source uses the official cdsapi Python client for direct CDS access.
cds-retrieve accepts a dataset slug and the selection JSON from its download
form. cds-era5 builds sequential monthly requests for ERA5 single-level hourly
data or monthly averaged reanalysis. CDS validates dataset-specific fields,
availability, request limits and permissions.
The adapter saves native files, including any ZIP returned by CDS. It does not extract archives, select nearest grid cells, convert units, shift timestamps or feed the files directly into a scientific method. It complements the separate Open-Meteo and Earthmover adapters; no fallback between sources is automatic.
Install and authenticate#
From the repository root:
uv sync --locked --all-packages --extra cds
For an installed package, use pip install 'energy-data-retrieval[cds]'.
Planning, discovery and verified reuse of completed files do not require the SDK.
Create a CDS account and follow the official setup instructions.
Put your personal access token in CDSAPI_KEY using your environment/secret manager,
or create ~/.cdsapirc outside this repository:
url: https://cds.climate.copernicus.eu/api
key: <PERSONAL-ACCESS-TOKEN>
The current key is a personal access token, not the old UID:API-key format.
The official SDK also supports CDSAPI_RC for a different local configuration
file. This adapter fixes the endpoint to the CDS HTTPS API and enables TLS
verification. It does not use CDSAPI_URL to redirect requests to another service.
ADS/CAMS and CEMS have separate endpoints and are outside this connector’s scope.
Accept each dataset’s terms manually on its CDS download page before retrieving it. The code never accepts terms on your behalf. Credentials do not belong in request JSON, command arguments, notebooks or Git. SDK diagnostic callbacks are suppressed because they can contain credentials or signed download URLs; failures give a local error directing you to check the selection, account and CDS job page.
ERA5 example: Canary Islands#
Inspect the request without credentials, network access or job submission:
uv run energy-retrieve cds-era5 \
--start 2024-02-01 --end 2024-02-29 \
--variables 2m_temperature surface_solar_radiation_downwards \
--area 29 -17 28 -16 --plan-only
Download the same small-area selection:
uv run energy-retrieve cds-era5 \
--start 2024-02-01 --end 2024-02-29 \
--variables 2m_temperature surface_solar_radiation_downwards \
--area 29 -17 28 -16 --output data/cds-era5
--area is north, west, south, east, matching CDS. This differs from the
west/south/east/north --bbox used by some other repository commands. Coordinates
use degrees and longitude −180…180; split areas crossing the dateline explicitly.
Dates are inclusive source labels, and hourly requests include all 24 UTC hours
for each selected day. Actual Gregorian days are generated, including leap days.
The helper performs one request per intersecting month, with no parallel jobs.
Large areas or many variables may still exceed CDS service limits; narrow those
selections or use explicit generic requests.
For monthly means, supply whole calendar months and add --monthly-means:
uv run energy-retrieve cds-era5 \
--start 2024-01-01 --end 2024-12-31 --monthly-means \
--variables 2m_temperature surface_solar_radiation_downwards \
--area 29 -17 28 -16 --data-format netcdf --output data/cds-monthly
This selects reanalysis-era5-single-levels-monthly-means, product
monthly_averaged_reanalysis, with the CDS 00:00 selection. It is not a series
of midnight observations. ERA5-Land has different products and accumulation
semantics; use its exact catalogue request through cds-retrieve instead.
Any CDS catalogue selection#
Use the CDS dataset page’s Show API request code. Save only the request
dictionary as JSON (double quotes; no Python code or credentials), then run:
uv run energy-retrieve cds-retrieve \
--dataset reanalysis-era5-single-levels \
--request data-retrieval/examples/cds-era5-request.json --plan-only
uv run energy-retrieve cds-retrieve \
--dataset reanalysis-era5-single-levels \
--request data-retrieval/examples/cds-era5-request.json --output data/cds-custom
The generic command sends one exact selection; it does not invent field names, split requests, infer calendars or claim all catalogue datasets have been tested. It can carry catalogue-specific pressure-level, ERA5-Land or climate projection requests after you provide their correct dataset IDs and selections. The native dataset page is retained in provenance. Cite that dataset’s specific DOI and documentation in downstream analyses; the CDS access citation alone does not replace them.
Python API#
from pathlib import Path
from energy_data_retrieval.sources.cds import CDSClient, CDSRequest
request = CDSRequest(
"reanalysis-era5-single-levels",
{
"product_type": ["reanalysis"],
"variable": ["2m_temperature"],
"year": ["2024"],
"month": ["02"],
"day": ["01"],
"time": ["12:00"],
"area": [29, -17, 28, -16],
"data_format": "grib",
"download_format": "unarchived",
},
)
artifact = CDSClient(maximum_download_bytes=256 * 1024**2).download(request, Path("data/cds"))
print(artifact.path, artifact.sha256, artifact.reused)
Outputs, limits and restart behaviour#
Each selection lives in <output>/<dataset>/<request-sha256>/, containing the
native download.* file and manifest.json. The manifest includes the exact
selection, endpoint, retrieval time, SDK version, dataset link, reference IDs,
byte count and artifact SHA-256. Signed URLs and SDK response objects are not
stored. The content signature determines the suffix: .grib, classic NetCDF
.nc, HDF5/NetCDF4-container .h5, .zip, .gz, or .bin for other formats.
Signature detection identifies a container; it does not validate scientific
contents. Even download_format=unarchived may yield an archive for multiple files.
Results publish as a complete directory after transfer and hashing. Exceptions
remove temporary files; a hard process kill can leave an unpublished .cds-*
directory for manual inspection. Repeating the same selection reuses the file
only after verifying its request, size and SHA-256. Corrupt or incomplete existing
results fail instead of silently overwriting. Completed months survive a later
failure. Remote jobs are not checkpointed or reattached after interruption;
rerunning an unfinished selection may submit a new CDS job.
--maximum-download-mib defaults to 1024 per result: advertised size is checked
before downloading and actual size afterwards. This is not a custom streaming
quota; the SDK owns transfer. --timeout-seconds defaults to 60 per network
request and --attempts to 3. A network timeout is not a queue deadline:
CDS processing can take much longer. The planner limits selections to 1,200 months
and 64 variables; generic JSON is limited to 1 MiB.
A request hash does not pin CDS’s changing upstream data. ERA5T can be revised. Use a new output root to acquire a fresh vintage while retaining prior evidence. Location discovery lists the two ERA5 products as unverified candidates; neither discovery nor the Open-Meteo sample probe submits CDS jobs or reads CDS credentials.
Scientific interpretation and references#
Native temperature is commonly in kelvin; accumulated radiation is commonly in J/m². Do not relabel radiation as W/m² or divide every product by 3,600: hourly, monthly averaged and ERA5-Land accumulations have different conventions. Inspect the variables, units, dimensions and time bounds using the ERA5 data documentation. For interval-start weather output, explicitly acquire any extra endpoint required by the chosen transformation. No endpoint is automatically added here.
The repository thesis remains the primary reference. Source-specific citations
are maintained in methods/src/energy_methods/references.json:
copernicus_cds_api, hersbach_2020_era5, era5_single_levels
(DOI), and
era5_single_levels_monthly (DOI).
Tests use synthetic SDK responses; no live authenticated download is claimed.