energy_data_retrieval.failure_database#

Public failure databases and attributed, versioned offline extracts.

No credentials or live calls are needed to load the bundled snapshots. Live refreshes are bounded and explicit. NRC raw IRIS records, PVROM work orders and commercial databases are not public APIs; these connectors use published data.

energy_data_retrieval.failure_database.source_catalogue()[source]#

Sources, access constraints, population limits, retrieval URLs and hashes.

Return type:

DataFrame

energy_data_retrieval.failure_database.load_failure_database(source_id=None, *, include_flagged=False)[source]#

Load all accessible snapshot records, preserving source-specific units.

Flagged rows are retained on disk but excluded by default. Different populations/modes overlap and must NOT be summed as a system failure rate. The source catalogue explains the scope and excludes license-gated sources.

Parameters:
  • source_id (str | None)

  • include_flagged (bool)

Return type:

DataFrame

energy_data_retrieval.failure_database.parse_nrc_2020(content)[source]#

Extract ALL 300 rows of Table 1, PDF pages 30-38 (printed 6-14).

Preserve inconsistent published units and quarantine their rows. Missing counts remain missing. Gamma beta is a RATE, not a lifetime scale. Source 5th/95th percentiles describe a selected industry distribution, not a new frequentist confidence interval. This is intentionally version-specific.

Parameters:

content (bytes)

Return type:

list[dict[str, Any]]

energy_data_retrieval.failure_database.parse_pvrom_2018(content)[source]#

All Appendix A failure fits (not repair fits), PDF pages 25-26.

Prefer day/hour rows over duplicate year fits, whose parameters sometimes conflict. Lognormal without ‘-n’ has ambiguous parameter semantics and is quarantined. Normal models require explicit conditioning on positive time. Preventive maintenance is retained but excluded from failure calculations.

Parameters:

content (bytes)

Return type:

list[dict[str, Any]]

energy_data_retrieval.failure_database.parse_wind_2016(content)[source]#

Figure 3’s embedded numerical table, 19 subassemblies x 4 repair classes.

Rates are failures/turbine/year; the primary study does not publish counts and exposure per cell, so no confidence interval is invented.

Parameters:

content (bytes)

Return type:

list[dict[str, Any]]

energy_data_retrieval.failure_database.aggregate_backblaze(frame)[source]#

Aggregate daily drive snapshots by model into failures and drive-days.

Exactly one observed drive-day per row. Gaps add no exposure. Reject duplicate keys, missing identifiers and non-binary flags. This estimates an exposure-weighted rate; the first observed date is NOT a drive’s birth date.

Parameters:

frame (DataFrame)

Return type:

DataFrame

energy_data_retrieval.failure_database.parse_backblaze_zip(content, *, max_uncompressed_bytes=1000000000)[source]#

Read the public 2013 archive without extracting files; bounded memory.

Validate each daily file and count only its observed drive-days. Each day’s count is aggregated before reading the next. Repeated flags and post-failure reappearances are audited separately. Each reported flag remains an event, matching the provider’s AFR numerator; these records must not be treated as independent first-failure lifetimes.

Parameters:
  • content (bytes)

  • max_uncompressed_bytes (int)

Return type:

list[dict[str, Any]]

energy_data_retrieval.failure_database.fetch_failure_database(source_id, *, client=None, max_bytes=100000000)[source]#

Fetch a registered public release, verify its hash, then normalize it.

Fixed source URLs, no arbitrary URL fetch or archive extraction. A changed release fails closed for review. Returns records and provenance; callers decide where to save, so live retrieval never overwrites the shipped data.

Parameters:
  • source_id (str)

  • client (Client | None)

  • max_bytes (int)

Return type:

tuple[list[dict[str, Any]], dict[str, str]]

energy_data_retrieval.failure_database.database_coverage()[source]#

Record, population and device counts by source, including quarantined rows.

Return type:

DataFrame