energy_data_retrieval.sources.cmip6.client#

Bounded ESGF STAC discovery with explicit CMIP6 identity and native-file plans.

The STAC index contains datasets; their HTTP assets are native NetCDF files. Year selection chooses intersecting whole files, never crops, regrids, corrects calendars or manufactures an hourly weather product. Saved metadata retains the provider’s citation links, identifiers, licenses, units and ensemble identity.

class energy_data_retrieval.sources.cmip6.client.CMIP6Query(source_id, experiment_id, member_id, table_id, variable_id, grid_label, institution_id=None, activity_id=None, version=None, start_year=None, end_year=None)[source]#

Bases: object

One explicit model/experiment/member/table/variable/grid combination.

An omitted version resolves the index’s latest non-retracted version at discovery time. The plan pins that concrete version. Optional inclusive years select overlapping native files, whose actual coverage may be wider.

Parameters:
  • source_id (str)

  • experiment_id (str)

  • member_id (str)

  • table_id (str)

  • variable_id (str)

  • grid_label (str)

  • institution_id (str | None)

  • activity_id (str | None)

  • version (str | None)

  • start_year (int | None)

  • end_year (int | None)

source_id: str#
experiment_id: str#
member_id: str#
table_id: str#
variable_id: str#
grid_label: str#
institution_id: str | None#
activity_id: str | None#
version: str | None#
start_year: int | None#
end_year: int | None#
filter()[source]#

Return the exact CQL2 JSON filter used by both official STAC indices.

Return type:

dict[str, JSONValue]

class energy_data_retrieval.sources.cmip6.client.CMIP6File(dataset_id, filename, url, size_bytes, sha256, start_label, end_label)[source]#

Bases: object

A pinned native file and the provider checksum required for transfer.

Parameters:
  • dataset_id (str)

  • filename (str)

  • url (str)

  • size_bytes (int)

  • sha256 (str)

  • start_label (str | None)

  • end_label (str | None)

dataset_id: str#
filename: str#
url: str#
size_bytes: int#
sha256: str#
start_label: str | None#
end_label: str | None#
property relative_path: str#
class energy_data_retrieval.sources.cmip6.client.CMIP6Plan(query, index, collection, created_at_utc, datasets, requests, files)[source]#

Bases: object

Portable discovery snapshot, including the complete selected STAC items.

Parameters:
  • query (CMIP6Query)

  • index (str)

  • collection (str)

  • created_at_utc (str)

  • datasets (tuple[dict[str, JSONValue], ...])

  • requests (tuple[RequestRecord, ...])

  • files (tuple[CMIP6File, ...])

query: CMIP6Query#
index: str#
collection: str#
created_at_utc: str#
datasets: tuple[dict[str, JSONValue], ...]#
requests: tuple[RequestRecord, ...]#
files: tuple[CMIP6File, ...]#
property total_size_bytes: int#
property sha256: str#
as_dict()[source]#

Return an independent JSON-safe payload with its SHA-256 fingerprint.

Return type:

dict[str, object]

class energy_data_retrieval.sources.cmip6.client.CMIP6Client(http, index='east', page_size=20, maximum_datasets=100, maximum_files=1000)[source]#

Bases: object

Search one explicitly chosen ESGF index without collecting an unbounded archive.

Parameters:
  • http (HTTPClient)

  • index (str)

  • page_size (int)

  • maximum_datasets (int)

  • maximum_files (int)

http: HTTPClient#
index: str#
page_size: int#
maximum_datasets: int#
maximum_files: int#
search(query)[source]#

Complete a bounded search, rejecting truncation and mismatching facets.

East and west currently spell the collection ID differently. Resolve it from the catalogue, and explicitly retain CMIP6 even if CMIP7 becomes the service default. A completed search covers only this chosen index.

Parameters:

query (CMIP6Query)

Return type:

CMIP6Plan

energy_data_retrieval.sources.cmip6.client.save_plan(plan, path)[source]#

Write a new portable JSON plan; never replace an earlier discovery snapshot.

Parameters:
Return type:

None

energy_data_retrieval.sources.cmip6.client.load_plan(path)[source]#

Validate a bounded JSON snapshot, its fingerprint, metadata and file paths.

Parameters:

path (Path)

Return type:

CMIP6Plan

energy_data_retrieval.sources.cmip6.client.download_plan(plan, output, http, *, maximum_total_bytes=2147483648)[source]#

Download pinned files sequentially with preflight bounds and verified restart.

The supplied HTTP client’s host allowlist and per-file byte limit apply. Every source size and SHA-256 must agree before a native file is published. Existing individual downloads are reused only after full hash verification.

Parameters:
Return type:

dict[str, object]