We have continued our monitoring activity around this event. There has been no significant reoccurrence of this issue. At this time, we are marking this as closed and resolved. However, as we have not had a confirmation from Salesforce that the issues have been completely resolved we are monitoring further and will reopen this event if needed.
Posted May 24, 2022 - 16:44 UTC
Update
Salesforce has not confirmed this issue has been resolved, but at this time, we are seeing fewer instances of jobs failing due to the issue reported. We have also attempted to mitigate jobs that do not succeed due to this issue and are stuck in progress indefinitely. The result does not eliminate the problem, but it will prevent the stuck jobs from blocking your release.
We will continue to monitor the situation for another 24 hours for any new developments.
Posted May 23, 2022 - 18:20 UTC
Update
Salesforce continues to investigate the root cause of the issue impacting committing jobs. We still can not provide an estimated time of when the Salesforce team will resolve the matter. We continue to work with them directly to assist them in their investigations.
You may also be impacted by a known issue found below relating to Salesforce Sandboxes.
Thank you again for your understanding while we work with Salesforce to assist in resolving this issue.
Posted May 20, 2022 - 18:37 UTC
Update
We are continuing to monitor for any further issues.
Posted May 20, 2022 - 18:29 UTC
Update
We continue to work with Salesforce, which has acknowledged that this issue affects several partners like Copado and individual Salesforce customers. While they are working to resolve the issue, we are also attempting to mitigate the problem as best we can. However, any efforts on our part will not entirely correct the issue until Salesforce resolves the problem on their end.
As more information is available, we will continue to share it.
Posted May 19, 2022 - 17:19 UTC
Update
We continue to work with Salesforce on this issue as we still have some clients stating they are having trouble committing jobs. Salesforce has confirmed that the issue is related to performance degradation on their platforms. We will continue to work with the Salesforce team to resolve the issue.
We apologize for the inconvenience this may be causing and appreciate your patience.
Posted May 18, 2022 - 18:56 UTC
Monitoring
While the rate of failure has decreased, we continue to see performance related issues reported by customers and via our log monitoring. We are updating this incident to Monitoring status but will note the continued degraded performance for now.
Posted May 17, 2022 - 20:53 UTC
Investigating
We are seeing multiple examples of customers' Copado performance being impacted by Salesforce performance degradation. While Salesforce has indicated that their latest incident is resolved, we are seeing that the degradation continues.
We are engaged with our internal infrastructure and engineering teams, as well as with Salesforce, to help remediate this issue as much as possible from our end. We will share updates here as they become available.
Posted May 17, 2022 - 17:57 UTC
This incident affected: North America (CICD, Copado Robotic Testing) and EMEA (CICD).