energy_data_retrieval.transport#
Bounded HTTP reads with source allowlists, retries and public-only caching.
- exception energy_data_retrieval.transport.RetrievalError[source]#
Bases:
RuntimeErrorSanitized retrieval failure; never contains a credential-bearing URL.
- class energy_data_retrieval.transport.HTTPConfig(timeout_seconds=45.0, attempts=4, retry_base_seconds=1.0, maximum_retry_seconds=60.0, minimum_interval_seconds=0.25, maximum_response_bytes=67108864, cache_ttl_seconds=3600.0)[source]#
Bases:
objectFinite resource and retry limits, shared by all sources.
- Parameters:
timeout_seconds (float)
attempts (int)
retry_base_seconds (float)
maximum_retry_seconds (float)
minimum_interval_seconds (float)
maximum_response_bytes (int)
cache_ttl_seconds (float)
- timeout_seconds: float#
- attempts: int#
- retry_base_seconds: float#
- maximum_retry_seconds: float#
- minimum_interval_seconds: float#
- maximum_response_bytes: int#
- cache_ttl_seconds: float#
- class energy_data_retrieval.transport.RequestRecord(endpoint, parameters, retrieved_at_utc, sha256, size_bytes, cache_hit=False)[source]#
Bases:
objectNon-secret provenance for the exact stored, sanitized response bytes.
- Parameters:
endpoint (str)
parameters (dict[str, str])
retrieved_at_utc (str)
sha256 (str)
size_bytes (int)
cache_hit (bool)
- endpoint: str#
- parameters: dict[str, str]#
- retrieved_at_utc: str#
- sha256: str#
- size_bytes: int#
- cache_hit: bool#
- class energy_data_retrieval.transport.JSONResponse(value: 'JSONValue', record: 'RequestRecord')[source]#
Bases:
object- Parameters:
value (JSONValue)
record (RequestRecord)
- value: JSONValue#
- record: RequestRecord#
- class energy_data_retrieval.transport.TextResponse(value, record)[source]#
Bases:
objectUTF-8 response with a hash of its stored bytes after credential redaction.
- Parameters:
value (str)
record (RequestRecord)
- value: str#
- record: RequestRecord#
- class energy_data_retrieval.transport.BytesResponse(value, record, content_type, request_id)[source]#
Bases:
objectExact public response bytes, media type and provider request identifier.
- Parameters:
value (bytes)
record (RequestRecord)
content_type (str)
request_id (str | None)
- value: bytes#
- record: RequestRecord#
- content_type: str#
- request_id: str | None#
- energy_data_retrieval.transport.atomic_bytes(path, data)[source]#
Replace one file atomically; failed writes never publish a partial file.
- Parameters:
path (Path)
data (bytes)
- Return type:
None
- energy_data_retrieval.transport.object_value(value)[source]#
- Parameters:
value (JSONValue)
- Return type:
dict[str, JSONValue]
- energy_data_retrieval.transport.array_value(value)[source]#
- Parameters:
value (JSONValue)
- Return type:
list[JSONValue]
- class energy_data_retrieval.transport.HTTPClient(allowed_hosts, *, config=None, cache_directory=None, transport=None, sleep=<built-in function sleep>)[source]#
Bases:
objectBounded HTTPS retrieval with a fixed host allowlist and no redirects.
Authentication must be passed separately through
secretsorsecret_headers. Authenticated responses never use the public cache. Supply an httpx transport for offline tests, rather than a preauthenticated client. This ensures authorization cannot silently bypass cache isolation.- Parameters:
allowed_hosts (frozenset[str])
config (HTTPConfig | None)
cache_directory (Path | None)
transport (httpx.BaseTransport | None)
sleep (Callable[[float], None])
- request_bytes(endpoint, parameters=None, *, method='GET', form=None, files=None)[source]#
Retrieve public binary/text data or submit an idempotent calculation.
POST is for read-only provider calculations, never state changes. Public fields only; no authentication and no cache. List-valued query/form fields retain repeated keys. Uploads and responses obey the configured byte limit. Provenance includes method, form values and upload hashes. Redirects are refused; retries include 529 (PVGIS computation overload).
- Parameters:
endpoint (str)
parameters (Mapping[str, str | list[str]] | None)
method (Literal['GET', 'POST'])
form (Mapping[str, str | list[str]] | None)
files (Mapping[str, tuple[str, bytes, str]] | None)
- Return type:
- download_file(endpoint, path, *, expected_sha256=None, expected_size_bytes=None)[source]#
Stream one public export to disk with verified whole-file restart semantics.
An interrupted transfer is retried from byte zero. HTTP ranges are not assumed safe for changing archives. Reuse requires both an intact file and its matching manifest. Credential-bearing and redirected URLs fail. A supplied source byte count is checked before publishing or reusing a file.
- Parameters:
endpoint (str)
path (Path)
expected_sha256 (str | None)
expected_size_bytes (int | None)
- Return type:
- get_text(endpoint, parameters=None, *, secrets=None, secret_headers=None, use_cache=True)[source]#
Read bounded UTF-8 text, retaining tabs, line endings and physical units.
Credentials must use
secretsorsecret_headers. Authenticated responses bypass the public cache; logs, response text and provenance redact credentials. Public CSV/PSV/TSV bytes remain unchanged. Cache entries are accepted only after source, size and hash verification.- Parameters:
endpoint (str)
parameters (Mapping[str, str] | None)
secrets (Mapping[str, str] | None)
secret_headers (Mapping[str, str] | None)
use_cache (bool)
- Return type:
- get_json(endpoint, parameters=None, *, secrets=None, secret_headers=None, allow_legacy_charset=False, use_cache=True, public_pagination_token=False)[source]#
Fetch JSON, preserving zero/null values and non-secret request provenance.
public_pagination_tokenopts into a provider’s non-authenticationtokencursor, including next links on its first page. Other credential names and explicit secrets remain redacted; authentication tokens cannot use this option. The default treats all token parameters as secrets. Header credentials are never included in request provenance or cache keys.allow_legacy_charsetpermits only a declared Latin-1/Latin-9 fallback when Unicode JSON decoding fails; it never overrides valid Unicode JSON.- Parameters:
endpoint (str)
parameters (Mapping[str, str] | None)
secrets (Mapping[str, str] | None)
secret_headers (Mapping[str, str] | None)
allow_legacy_charset (bool)
use_cache (bool)
public_pagination_token (bool)
- Return type: