Unplanned power failure during PDU replacement

Incident Report for Faithlife

Resolved

PDU replaced and all core networking devices have PSU, PDU, and PDU bank redundancy.
Posted Oct 27, 2021 - 15:22 PDT

Update

Continuing to monitor systems until PDU replacement completes and full power redundancy is restored.
Posted Oct 27, 2021 - 12:36 PDT

Monitoring

Network devices brought back online.

Monitoring systems and rebalancing power cables.
Posted Oct 27, 2021 - 11:41 PDT

Identified

While replacing a failed Power Distribution Unit (PDU) to restore redundancy our primary network cabinet suffered power loss.
Posted Oct 27, 2021 - 11:35 PDT
This incident affected: Core Services (Accounts API, Licenses API, Files API, OAuth API), Proclaim (Proclaim API, Proclaim Slide Renderer, Proclaim Sync API, Digital Signage), Faithlife.com (Faithlife.com, Community API), Logos Bible Software/Verbum (Logos.com, Verbum.com, App.logos.com, Resources API, Sync, Atlas API, Sign In), and Logos Community, Logos Sermons, Amber.Faithlife.com, Faithlife TV, Faithlife Ebooks.