energy_data_retrieval.sources.cmip6.client#
Bounded ESGF STAC discovery with explicit CMIP6 identity and native-file plans.
The STAC index contains datasets; their HTTP assets are native NetCDF files. Year selection chooses intersecting whole files, never crops, regrids, corrects calendars or manufactures an hourly weather product. Saved metadata retains the provider’s citation links, identifiers, licenses, units and ensemble identity.
- class energy_data_retrieval.sources.cmip6.client.CMIP6Query(source_id, experiment_id, member_id, table_id, variable_id, grid_label, institution_id=None, activity_id=None, version=None, start_year=None, end_year=None)[source]#
Bases:
objectOne explicit model/experiment/member/table/variable/grid combination.
An omitted version resolves the index’s latest non-retracted version at discovery time. The plan pins that concrete version. Optional inclusive years select overlapping native files, whose actual coverage may be wider.
- Parameters:
source_id (str)
experiment_id (str)
member_id (str)
table_id (str)
variable_id (str)
grid_label (str)
institution_id (str | None)
activity_id (str | None)
version (str | None)
start_year (int | None)
end_year (int | None)
- source_id: str#
- experiment_id: str#
- member_id: str#
- table_id: str#
- variable_id: str#
- grid_label: str#
- institution_id: str | None#
- activity_id: str | None#
- version: str | None#
- start_year: int | None#
- end_year: int | None#
- class energy_data_retrieval.sources.cmip6.client.CMIP6File(dataset_id, filename, url, size_bytes, sha256, start_label, end_label)[source]#
Bases:
objectA pinned native file and the provider checksum required for transfer.
- Parameters:
dataset_id (str)
filename (str)
url (str)
size_bytes (int)
sha256 (str)
start_label (str | None)
end_label (str | None)
- dataset_id: str#
- filename: str#
- url: str#
- size_bytes: int#
- sha256: str#
- start_label: str | None#
- end_label: str | None#
- property relative_path: str#
- class energy_data_retrieval.sources.cmip6.client.CMIP6Plan(query, index, collection, created_at_utc, datasets, requests, files)[source]#
Bases:
objectPortable discovery snapshot, including the complete selected STAC items.
- Parameters:
query (CMIP6Query)
index (str)
collection (str)
created_at_utc (str)
datasets (tuple[dict[str, JSONValue], ...])
requests (tuple[RequestRecord, ...])
files (tuple[CMIP6File, ...])
- query: CMIP6Query#
- index: str#
- collection: str#
- created_at_utc: str#
- datasets: tuple[dict[str, JSONValue], ...]#
- requests: tuple[RequestRecord, ...]#
- property total_size_bytes: int#
- property sha256: str#
- class energy_data_retrieval.sources.cmip6.client.CMIP6Client(http, index='east', page_size=20, maximum_datasets=100, maximum_files=1000)[source]#
Bases:
objectSearch one explicitly chosen ESGF index without collecting an unbounded archive.
- Parameters:
http (HTTPClient)
index (str)
page_size (int)
maximum_datasets (int)
maximum_files (int)
- http: HTTPClient#
- index: str#
- page_size: int#
- maximum_datasets: int#
- maximum_files: int#
- search(query)[source]#
Complete a bounded search, rejecting truncation and mismatching facets.
East and west currently spell the collection ID differently. Resolve it from the catalogue, and explicitly retain CMIP6 even if CMIP7 becomes the service default. A completed search covers only this chosen index.
- Parameters:
query (CMIP6Query)
- Return type:
- energy_data_retrieval.sources.cmip6.client.save_plan(plan, path)[source]#
Write a new portable JSON plan; never replace an earlier discovery snapshot.
- Parameters:
plan (CMIP6Plan)
path (Path)
- Return type:
None
- energy_data_retrieval.sources.cmip6.client.load_plan(path)[source]#
Validate a bounded JSON snapshot, its fingerprint, metadata and file paths.
- Parameters:
path (Path)
- Return type:
- energy_data_retrieval.sources.cmip6.client.download_plan(plan, output, http, *, maximum_total_bytes=2147483648)[source]#
Download pinned files sequentially with preflight bounds and verified restart.
The supplied HTTP client’s host allowlist and per-file byte limit apply. Every source size and SHA-256 must agree before a native file is published. Existing individual downloads are reused only after full hash verification.
- Parameters:
plan (CMIP6Plan)
output (Path)
http (HTTPClient)
maximum_total_bytes (int)
- Return type:
dict[str, object]