[Resolved] Alert: NA11 | Intermittent Latency PROD & UAT, June 15, 2021

[ROOT CAUSE ANALYSIS]

Description  Intermittent Failures Accessing CLM NA11 Services
Timeframe 18:24 UTC June 15, 2021 to 14:57 UTC June 17, 2021
Impact Customers in CLM NA11 experienced intermittent failures accessing the system.
Cause Upstream ISP provider experienced multiple DDoS events, impacting network operation.
Resolution ISP provider implemented additional defenses to mitigate current and future DDoS events. ISP will continue to monitor for future DDoS events and mitigate them.
Next Steps CLM will re-evaluate their upstream provider’s services. Additionally, CLM will implement new services and systems that will improve reliability, stability, and security.


[RESOLVED June 23, 2021, 09:45 CDT] The DocuSign CLM Technology team has declared this incident resolved.  The last detected latency related to this incident occurred on 6/17/2021.  The DocuSign Technology team will provide an official Root Cause Analysis of this incident within 48 hours.  Thank you for your patience and we apologize for the inconvenience.

[UPDATED June 22, 2021, 09:45 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the day until resolution is declared.

[UPDATED June 21, 2021, 10:00 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the day until resolution is declared.

[UPDATED June 18, 2021, 17:20 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the weekend until resolution is declared.

[UPDATED June 18, 2021, 08:10 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the day until resolution is declared.

[UPDATED June 17, 2021, 08:30 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the day until resolution is declared.

Please check the status of trust.springcm.com for updates regarding this issue.

[UPDATED June 16, 2021, 16:30 CDT ] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the evening until resolution is declared.

Please check the status of trust.springcm.com for updates regarding this issue.

[UPDATED June 16, 2021 13:20 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident until resolution is declared.

Please check the status of trust.springcm.com for updates regarding this issue.

[UPDATED June 16, 2021 06:40 CDT] The DocuSign CLM Technology team has seen some improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the morning until resolution is declared.

Please check the status of trust.springcm.com for updates regarding this issue.

[UPDATED June 15, 2021 20:15 CDT] The DocuSign CLM Technology team has seen an improvement in the performance issue but cannot declare resolved until our 3rd party vendor has given clearance.  The team will continue to monitor the incident through the night until resolution is declared.

Please check the status of trust.springcm.com for updates regarding this issue.

[UPDATED June 15, 2021 19:29 CT] The DocuSign CLM Technology team has isolated the performance issue and is working with our 3rd party vendor on which the incident it occurring.  During this time customers with accounts on our NA11 Production and UAT instances may experience intermittent latency issues as well as the potential for encountering page errors while navigating the site.  

Please check the status of trust.springcm.com frequently for updates regarding this issue.

[POSTED June 15, 2021 15:18 CT] The DocuSign CLM Technology team is working to isolate a performance degradation issue, on the application tier of the NA11 instance.

During this time, customers may experience intermittent latency and page errors when navigating through the site. 

Please check the status of trust.springcm.com frequently for updates regarding this issue.