Document Request Link Generation Degradation

Incident Report for Socure

Postmortem

Root Cause Analysis

Incident: Unable to generate DocV links via dashboard

Date Range: Tue, Jun 16, 2026 8:52 am – 10:12 am ET

Impact: Document Verification link generation via Admin Dashboard

Status: Resolved

1. Summary

On Jun 16, 2026, a scheduled deployment introducing stricterOn June 16, 2026, a scheduled deployment introducing stricter access control logic inadvertently broke the Document Verification (DocV) link generation workflow for a subset of client configurations. The issue persisted for approximately 1 hour and 20 minutes after being reported by a client. The incident was mitigated via a full rollback. We are actively overhauling our release validation protocols and implementing shadow-mode deployment capabilities to ensure enterprise permission configurations are thoroughly tested before enforcing hard access restrictions in the future.

2. Timeline

Time (ET)	Event
June 16, 5:16 AM ET	A production deployment was completed, introducing updates to the system's permission logic. This logic contained a defect rendering the DocV request links feature unusable.
8:52 am ET	Clients started reporting Admin Dashboard DocV link generation stuck in loading for some users.
9:04 am–9:24 am ET	Upon determining that the problem stemmed from recent updates to role and permission provisioning within the admin dashboard, the team resolved to execute a revert.
10:00 am–10:08 am ET	The rollback was verified in a staging environment before being deployed to production, followed by thorough regression and sanity testing.
10:10 am ET	The client confirmed that the Document Verification link generation workflow was fully restored.
10:11 am ET	The incident was resolved.

3. Root Cause

Primary Root Cause:
- Stricter permission validation logic was deployed in the morning that was meant to make our permission model more consistent across pages. However, this update caused unintended disruptions for specific role setups, preventing authorized users from accessing the document request link generation process.

4. Resolution

To restore functionality, the team implemented a rollback of the stricter permission enforcement logic. Following the deployment, we performed validation of the admin dashboard's link generation flow and verified that the service was operating correctly.

5. Corrective and Preventive Actions

Action	Description	ETA / Status
Revisit stricter access control with a safer rollout approach	Reintroduce the permission enforcement in shadow mode first using a safer rollout approach (e.g., staged rollout and additional validation) to prevent unintended impact to existing customer role configurations. Communicate with impacted customers to guide them in updating their permission settings	07/03
Add monitoring for errors/issues in document link generation	Add monitoring for errors in document link generation to be notified of and address issues in the future proactively.	07/03
Expand validation of access control changes to cover all user configurations	Update our release validation process so access-control and permission changes are tested against a broader range of role setups (including environment-scoped scenarios) prior to production release.	07/03

6. Lessons Learned

Permission/environment enforcement needs testing across representative customer role setups. Dark-launch/shadow-mode logging to measure who would be blocked, communicating to customers, and establishing a rollout plan is a safer path than hard enforcement.

7. Next Steps & Ongoing Commitment

We are committed to Corrective and Preventive Actions noted above.

Posted Jun 19, 2026 - 14:34 EDT

Resolved

The issue affecting Document Request link generation has been resolved, and functionality has been fully restored.

Our engineering team has completed the necessary changes and validated the fix. Users should now be able to generate Document Request links normally.

We apologize for the inconvenience and appreciate your patience while we worked to resolve the issue.

We are currently completing our post-incident review and will publish a Root Cause Analysis (RCA) within the next few days once that review is complete.

Posted Jun 16, 2026 - 10:14 EDT

Update

We are continuing to work on a fix for the issue affecting Document Request link generation. The revert process is taking longer than initially anticipated.

Our team remains actively engaged and is working through the revert. Once it is complete, we will immediately begin validation to confirm that functionality has been fully restored.

We apologize for the continued inconvenience and appreciate your patience while we work toward resolution. We will provide another update as soon as recovery has been confirmed.

Posted Jun 16, 2026 - 09:52 EDT

Identified

We have identified the issue causing Document Request link generation to remain in a loading state for some users.

Our investigation determined that the degradation was caused by a recent change that introduced an additional permission requirement for this action. We are currently reverting the related change to restore normal functionality.

We expect recovery within approximately 15 minutes and will provide another update once the functionality has been fully restored.

Posted Jun 16, 2026 - 09:32 EDT

Investigating

We are investigating an issue where some users may be unable to generate Document Request links from the dashboard. Affected users may see the link generation process continue loading instead of returning a generated document request link.

Initial investigation indicates this may be related to role provisioning or permission configuration impacting a subset of accounts. Our team is working to confirm the required permissions and restore functionality for impacted users.

We will provide another update as soon as more information is available.

Posted Jun 16, 2026 - 09:20 EDT

This incident affected: Socure ID+ Platform (Predictive DocV).