energy_data_retrieval.storage#
Partition-sized Parquet outputs with verifiable, restart-safe provenance.
- class energy_data_retrieval.storage.Artifact(filename: 'str', sha256: 'str', bytes: 'int', rows: 'int', request_fingerprint: 'str')[source]#
Bases:
object- Parameters:
filename (str)
sha256 (str)
bytes (int)
rows (int)
request_fingerprint (str)
- filename: str#
- sha256: str#
- bytes: int#
- rows: int#
- request_fingerprint: str#
- energy_data_retrieval.storage.fingerprint(metadata)[source]#
Fingerprint only non-secret request semantics and source/time conventions.
- Parameters:
metadata (Mapping[str, JSONValue])
- Return type:
str
- energy_data_retrieval.storage.existing_artifact(path, request_fingerprint)[source]#
- Parameters:
path (Path)
request_fingerprint (str)
- Return type:
Artifact | None
- energy_data_retrieval.storage.write_parquet(frame, path, *, request, provenance)[source]#
Publish a Parquet partition, then its manifest; refuse mismatched overwrites.
A crash between the two atomic publications is detected on restart. The user must inspect that incomplete pair; an unaudited file is never silently reused.
- Parameters:
frame (DataFrame)
path (Path)
request (Mapping[str, JSONValue])
provenance (Mapping[str, JSONValue])
- Return type: