Activity Log fails to load or return errors for some users

Incident Report for 1Password

Postmortem

Date of Incident: 2025-07-22
Time of Incident (UTC): 10:40 - 06:30
Services Affected: Admin Console Activity log and User Details page
Impact Duration: 43 hrs

Summary

A change to our internal data model that removed an unused type definition led to multiple failures in the Reports system.

Impact on Customers

  • Events API and Reporting: The activity log widget failed to display activities.
  • Admin console: User details page failed to display user details

What Happened?

A software update introduced an API change that caused a mismatch between client and server. The Audit log page in the admin console was still collecting data but was not able to render it.

  • Timeline of Events (UTC):

    • 10:40 : Detected by 1Password personnel and root cause identified
    • 10:42: Issue Identified
    • 13:38 Fix created and testing initiated
    • 06:30: Fix released, Service fully restored
  • Root Cause Analysis: A User State was removed from the 1Password.com server but not the client. This was a breaking API change.

How Was It Resolved?

  • Resolution Steps: The user state in the 1Password client was removed to get the server and client back into parity.
  • Verification of Resolution: 1Password engineering tested the changes and validated that full system functionality was restored.

What We Are Doing to Prevent Future Incidents

  • User State change process: 1Password is investigating how to catch breaking API changes by implementing additional end-to-end tests in the CI pipeline.
  • Audit log architecture change: Engineering is investigating a rework of the audit log page to change how it aggregates data so that it is not reliant on an endpoint. This would mitigate future occurrences that caused this issue.

Next Steps and Communication

  • No action is required from our customers at this time.
  • If you are still experiencing issues, please contact our support team at support@1password.com.

We are committed to providing a reliable and stable service, and we are taking the necessary steps to learn from this event and prevent it from happening again. Thank you for your understanding.

Sincerely,

The 1Password Team

Posted Jul 25, 2025 - 12:04 EDT

Resolved

This incident has been resolved.
Posted Jul 23, 2025 - 15:00 EDT

Update

We are continuing to work on a fix for this issue.
Posted Jul 22, 2025 - 19:22 EDT

Identified

There is an issue affecting the Activity Log, which may fail to load or return errors for some users. Our team is actively working to restore full functionality as soon as possible.
Posted Jul 22, 2025 - 19:21 EDT
This incident affected: Enterprise (Events API and Reporting).