tag:status.umbraco.io,2005:/historyUmbraco Cloud Status - Incident History2024-03-28T13:51:18+01:00Umbraco Cloudtag:status.umbraco.io,2005:Incident/203763162024-04-02T08:00:00+02:002024-03-27T13:28:34+01:00Autoupgrade for Umbraco Forms 13.1.1<p><strong>THIS IS A SCHEDULED EVENT Apr <var data-var='date'>2</var>, <var data-var='time'>08:00</var> - <var data-var='time'>17:00</var> CEST</strong></p><p><small>Mar <var data-var='date'>27</var>, <var data-var='time'>13:28</var> CET</small><br><strong>Scheduled</strong> - Umbraco Forms 13.1.1 has been released and will be automatically upgraded on projects running version 13.1.x.<br /><br />Release notes can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-forms/release-notes</p>tag:status.umbraco.io,2005:Incident/203865322024-03-28T11:39:31+01:002024-03-28T11:39:31+01:00Umbraco Cloud Websites - Small subset of customers reporting 522 errors<p><small>Mar <var data-var='date'>28</var>, <var data-var='time'>11:39</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>28</var>, <var data-var='time'>10:33</var> CET</small><br><strong>Identified</strong> - Small subset of customers encountered 522 errors when visiting websites hosted on Umbraco Cloud.</p>tag:status.umbraco.io,2005:Incident/203202692024-03-27T00:00:02+01:002024-03-27T00:00:05+01:00Autoupgrade for Umbraco CMS 10.8.5, 12.3.9 and 13.2.2, Umbraco Forms 13.1.0, Umbraco Deploy 10.4.0, 12.2.0 and 13.1.0, Umbraco Storage Providers 10.0.1, 12.0.1 and 13.0.1<p><small>Mar <var data-var='date'>27</var>, <var data-var='time'>00:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>26</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>15:47</var> CET</small><br><strong>Scheduled</strong> - Umbraco CMS 10.8.5, 12.3.9, 13.2.2 are released and will be automatically upgraded on projects running version 10.8.*, 12.3.* and 13.2.* respectively.<br /><br />Also, Umbraco Forms 13.1.0 is released and will be automatically upgraded on projects running version 13.*, and have automatic minor upgrades enabled.<br /><br />In addition, Umbraco Deploy and dependent packages 10.4.0, 12.2.0, 13.1.0 are released and will be automatically upgraded on projects running version 10.*, 12.*, 13.* respectively, and have automatic minor upgrades enabled.<br /><br />In addition, Umbraco Deploy Contrib 10.2.0, 12.1.0 and 13.1.0 are released and will be automatically upgraded on projects running version 10.*, 12.*, 13.* respectively, and have automatic minor upgrades enabled.<br /><br />In addition, Umbraco Deploy Forms 13.0.1 is released and will be automatically upgraded on projects running version 13.0.0.<br /><br />Finally, Umbraco StorageProviders and dependent packages 10.0.1, 12.0.1 and 13.0.1 are released and will be automatically upgraded on projects running version 10.0.0 12.0.0 and 13.0.0 respectively.<br /><br />Release notes can be found at docs.umbraco.com:<br />CMS: <br />- https://our.umbraco.com/download/releases/1085<br />- https://our.umbraco.com/download/releases/1239<br />- https://our.umbraco.com/download/releases/1322<br />Forms: https://docs.umbraco.com/umbraco-forms/release-notes<br />Deploy: https://docs.umbraco.com/umbraco-deploy/release-notes<br />Storage Provider: <br />- https://github.com/umbraco/Umbraco.StorageProviders/releases/tag/release-10.0.1<br />- https://github.com/umbraco/Umbraco.StorageProviders/releases/tag/release-12.0.1<br />- https://github.com/umbraco/Umbraco.StorageProviders/releases/tag/release-13.0.1</p>tag:status.umbraco.io,2005:Incident/203180692024-03-21T14:00:04+01:002024-03-21T14:00:04+01:00Deployments between environments failing<p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>14:00</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>11:45</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>11:09</var> CET</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>10:53</var> CET</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.umbraco.io,2005:Incident/202440292024-03-19T13:15:27+01:002024-03-19T13:15:28+01:00Autoupgrade for Umbraco Storage Providers 10.0.1, 12.0.1, 13.0.1<p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>13:15</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance is completed.<br /><br />No projects have been automatically upgraded on Umbraco during the maintenance, due to technical difficulties. Umbraco.StorageProviders and dependent packages 10.0.1, 12.0.1, 13.0.1 will be rescheduled for auto-upgrades at a later date.<br /><br />We are sorry for the inconvenience.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>14</var>, <var data-var='time'>15:13</var> CET</small><br><strong>Scheduled</strong> - Umbraco.StorageProviders and dependent packages 10.0.1, 12.0.1, 13.0.1 are released and will be automatically upgraded on projects running version 10.0.0, 12.0.0 and 13.0.0 respectively.<br /><br />Release can be found at GitHub<br />https://github.com/umbraco/Umbraco.StorageProviders/releases/tag/release-13.0.1<br /><br />Expected upgrade window: March 19th 2024, 8:00-17:00 CET</p>tag:status.umbraco.io,2005:Incident/202328052024-03-13T17:11:12+01:002024-03-13T17:11:12+01:00Login functionality for new projects' backoffice experiencing issues<p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>17:11</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>16:02</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>13:59</var> CET</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>12:14</var> CET</small><br><strong>Investigating</strong> - We've received reports of users unable to login to the backoffice of newly created projects. We are currently investigating this issue.</p>tag:status.umbraco.io,2005:Incident/201855052024-03-12T16:00:56+01:002024-03-12T16:00:57+01:00Autoupgrade for Umbraco CMS 13.2.0<p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>16:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>15:47</var> CET</small><br><strong>Scheduled</strong> - Umbraco CMS 13.2.0 has been released and will be automatically upgraded on projects running version 13.x and have automatic minor upgrades enabled.<br /><br />Release notes for the CMS can be found at our.umbraco.com:<br />https://our.umbraco.com/download/releases/1320</p>tag:status.umbraco.io,2005:Incident/201645042024-03-06T09:47:19+01:002024-03-06T09:47:19+01:00GraphQL intermittent timeouts<p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>09:47</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>17:08</var> CET</small><br><strong>Update</strong> - The fix is having a positive effect on the GraphQL service.<br />We are continuing to monitor the fix throughout the afternoon.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>13:14</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>10:58</var> CET</small><br><strong>Identified</strong> - The issue has been identified.<br />A fix is being worked on.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>09:57</var> CET</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>09:57</var> CET</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.umbraco.io,2005:Incident/200375532024-02-21T07:30:00+01:002024-02-21T12:31:50+01:00Heartcore API Unavailable<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>07:30</var> CET</small><br><strong>Resolved</strong> - We've identified an incident from 07:30 AM to 8:30 AM GMT+1 . Heartcore APIs were unavailable during the period. We have investigated and resolved the issue.</p>tag:status.umbraco.io,2005:Incident/199826722024-02-20T17:00:56+01:002024-02-20T17:00:58+01:00Autoupgrade for Umbraco Forms 8.13.14, 10.5.4, 12.2.3, 13.0.2, and Umbraco Deploy 4.9.4, 10.3.4, 12.1.4 and 13.0.4<p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>17:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>20</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>11:46</var> CET</small><br><strong>Scheduled</strong> - Umbraco Forms 8.13.14, 10.5.5, 12.2.3, 13.0.2 are released and will be automatically upgraded on projects running version 8.13.X, 10.5.X, 12.1.X, 13.0.X respectively.<br /><br />Also, Umbraco Deploy 4.9.4, 10.3.4, 12.1.4 and 13.0.4 are released and will be automatically upgraded on projects running version 4.9.X, 10.3.X, 12.2.X and 13.0.X respectively.<br /><br />Release notes for both products can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-forms/release-notes<br />https://docs.umbraco.com/umbraco-deploy/release-notes</p>tag:status.umbraco.io,2005:Incident/200192872024-02-19T16:45:16+01:002024-02-19T16:45:18+01:00Heartcore API Availability<p><small>Feb <var data-var='date'>19</var>, <var data-var='time'>16:45</var> CET</small><br><strong>Resolved</strong> - The Heartcore APIs are continuing to operate as normal. This incident is now resolved.</p><p><small>Feb <var data-var='date'>19</var>, <var data-var='time'>11:14</var> CET</small><br><strong>Monitoring</strong> - We have identified and resolved the cause of the issue. Heartcore APIs should be available again and we are continuing to monitor.</p><p><small>Feb <var data-var='date'>19</var>, <var data-var='time'>11:02</var> CET</small><br><strong>Investigating</strong> - We have received preliminary reports of the Heartcore APIs being unresponsive. We are investigating the issue now.</p>tag:status.umbraco.io,2005:Incident/199750672024-02-13T21:54:45+01:002024-02-13T21:54:45+01:00Degraded Performance For Project & Environment Creation (EU West)<p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>21:54</var> CET</small><br><strong>Resolved</strong> - Our supplier was able to identify the root cause of the issue and implement a fix. <br /><br />All Umbraco Cloud services should now function as normal. <br />This issue has been resolved.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>19:11</var> CET</small><br><strong>Update</strong> - We are still seeing degraded performance with project/environment creation in the EU West region. <br />Our supplier has identified a potential mitigation technique and is seeing improvements. <br /><br />We will continue to monitor and update the situation here as information becomes available.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>14:41</var> CET</small><br><strong>Identified</strong> - We have identified an issue from one of our suppliers that could impact the ability to create new projects or environments in the West Europe region. <br />We are currently in contact and will update this message as further information comes available.</p>tag:status.umbraco.io,2005:Incident/199381272024-02-08T14:44:25+01:002024-02-08T14:44:25+01:00Umbraco Cloud Portal operations intermittently failing<p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>14:44</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>12:15</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>11:38</var> CET</small><br><strong>Identified</strong> - Umbraco Cloud operations intermittently fail.<br /><br />You might experience errors during operations such as:<br />- Project creation<br />- Environment addition<br />- Change project plan<br />- CI/CD API operations<br /><br />We have identified the issue and are working on a fix.</p>tag:status.umbraco.io,2005:Incident/198921452024-02-05T09:06:27+01:002024-02-05T09:06:27+01:00UmbracoID 2FA email delivery delayed<p><small>Feb <var data-var='date'> 5</var>, <var data-var='time'>09:06</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 4</var>, <var data-var='time'>17:31</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and the authentication emails are being delivered again; we are monitoring the affected components.</p><p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>15:25</var> CET</small><br><strong>Identified</strong> - We have identified the root cause of the problem and we are in contact with the relevant service providers.<br />It appears that the Outlook email provider in particular is affected by this problem.<br />For urgent requests, please note you can reset the Multi-Factor Authentication by reaching out to Umbraco Support.</p><p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>11:46</var> CET</small><br><strong>Investigating</strong> - Two-factor authentication emails for UmbracoID may take more than usual to arrive. We are currently investigating this issue. If you are affected by the issue, please note you can reset the Multi-Factor Authentication by reaching out to Umbraco Support.</p>tag:status.umbraco.io,2005:Incident/198563292024-01-30T08:59:07+01:002024-01-30T08:59:07+01:00Heartcore webhook delivery delayed<p><small>Jan <var data-var='date'>30</var>, <var data-var='time'>08:59</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>15:34</var> CET</small><br><strong>Update</strong> - A fix has been applied and pending webhooks should have all been delivered. For the next 6-8 hours there may be brief intermittent delays as poisoned messages that caused the issue are periodically retried and handled by the patch.</p><p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>11:48</var> CET</small><br><strong>Monitoring</strong> - We are aware of an issue that is currently causing delivery of webhooks from Umbraco Heartcore sites to be significantly delayed.<br /><br />We have created and deployed a fix for this issue, and are monitoring as the system works through and delivers the backlog of delayed webhooks.</p>tag:status.umbraco.io,2005:Incident/197360152024-01-17T11:29:20+01:002024-01-17T11:29:25+01:00Cloud Portal degraded performance for UK-Region<p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>11:29</var> CET</small><br><strong>Resolved</strong> - The root cause has been identified and a fix has been deployed and confirmed working. <br />The root cause was due to an expired certificate for internal services used in the Umbraco Cloud Portal.</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>10:59</var> CET</small><br><strong>Investigating</strong> - We are aware of degraded performance for the Cloud Portal in the UK-region. <br />We are investigating to find the root cause and will update you in an hour (12:00 CET)</p>tag:status.umbraco.io,2005:Incident/196747682024-01-16T17:00:36+01:002024-01-16T17:00:37+01:00Autoupgrade for Umbraco Forms 8.13.13, 10.5.3, 12.2.2, 13.0.1, and Umbraco Deploy 4.9.3, 10.3.3, 12.1.3 and 13.0.3<p><small>Jan <var data-var='date'>16</var>, <var data-var='time'>17:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>16</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>07:28</var> CET</small><br><strong>Scheduled</strong> - Umbraco Forms 8.13.13, 10.5.3, 12.2.2, 13.0.1 are released and will be automatically upgraded on projects running version 8.13.X, 10.5.X, 12.2.X, 13.0.X respectively.<br /><br />Also, Umbraco Deploy 4.9.3, 10.3.3, 12.1.3 and 13.0.3 are released and will be automatically upgraded on projects running version 4.9.X, 10.3.X, 12.1.X and 13.0.X respectively.<br /><br />Release notes for both products can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-forms/release-notes<br />https://docs.umbraco.com/umbraco-deploy/release-notes<br /><br />Expected upgrade window: January 16th 2024, 8:00-17:00 CET</p>tag:status.umbraco.io,2005:Incident/195990652024-01-09T17:00:17+01:002024-01-11T16:01:23+01:00Autoupgrade for Umbraco CMS 8.18.12, 10.8.3, 12.3.6, 13.0.3, and Umbraco Deploy 4.9.2, 10.3.2, 12.1.2 and 13.0.2<p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>17:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>07:39</var> CET</small><br><strong>Scheduled</strong> - Umbraco CMS 8.18.12, 10.8.3, 12.3.6 and 13.0.3 are released and will be automatically upgraded on projects running version 8.18.X, 10.8.X, 12.3.X, 13.0.X respectively.<br /><br />Also, Umbraco Deploy 4.9.2, 10.3.2, 12.1.2 and 13.0.2 are released and will be automatically upgraded on projects running version 4.9.X, 10.3.X, 12.1.X and 13.0.X respectively.<br /><br />Release notes for the CMS can be found at our.umbraco.com:<br />https://our.umbraco.com/download/releases/81812 <br />https://our.umbraco.com/download/releases/1083 <br />https://our.umbraco.com/download/releases/1236 <br />https://our.umbraco.com/download/releases/1303<br /><br />Release notes for Umbraco Deploy can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-deploy/release-notes<br /><br />Expected upgrade window: January 9th 2024, 8:00-17:00 CET</p>tag:status.umbraco.io,2005:Incident/194575242023-12-19T09:00:57+01:002023-12-19T09:01:06+01:00Autoupgrade for Umbraco Forms 12.2.1<p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>09:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>00:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>17:08</var> CET</small><br><strong>Scheduled</strong> - Umbraco Forms 12.2.1 is released and will be automatically upgraded on projects running version 12.2.X respectively.<br /><br />Release notes can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-forms/release-notes<br /><br />Expected upgrade window: December 19th 2023, 8:00-17:00 CET</p>tag:status.umbraco.io,2005:Incident/194134462023-12-13T08:49:09+01:002023-12-13T08:49:10+01:00Issues with CMS Versions 8, 10, 12 Following Security Patch on Umbraco Cloud [Status: Investigating]<p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>08:49</var> CET</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>22:39</var> CET</small><br><strong>Update</strong> - A patch containing a fix has been rolled out for the affected environments.<br />We will continue to monitor the issue until further notice.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>19:28</var> CET</small><br><strong>Monitoring</strong> - We are pleased to announce that the auto-upgrade process with the new patches for the Umbraco Cloud issues has begun. This update is aimed at addressing the problems identified in CMS versions 8.18.10, 10.8.1, and 12.3.4.<br /><br />Impact:<br />The affected users, particularly those logging into a backoffice running Umbraco versions 8.18.10, 10.8.1, or 12.3.4, may have experienced functionality disruptions or other anomalies in the following scenarios:<br />-Environments with a multilingual setup.<br />- Environments including a rich text editor and using custom style sheets (CSS) on one or more content nodes.<br /><br />Current Actions:<br />Patch Creation: New patch versions for Umbraco 8, 10, and 12 have been developed. These include versions 8.18.11, 10.8.2, and 12.3.5, respectively.<br />NuGet Accessibility: The patch versions are now accessible on NuGet.<br /><br />Auto-Upgrade in Progress: Cloud projects running the affected patch versions have begun the auto-upgrade process to these new patches on Tuesday evening.<br /><br />Monitoring and Next Steps:<br />Our team is closely monitoring the auto-upgrade process to ensure smooth and effective deployment.<br />We anticipate that upgrading to these new patch versions will resolve the known issues on Umbraco Cloud.<br />A further update will be provided once all cloud projects with the affected versions have been upgraded with the new patch versions.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>17:08</var> CET</small><br><strong>Update</strong> - Our ongoing investigation into the issues following the recent security patch on Umbraco Cloud has led us to the conclusion that a patch is required for all versions that were affected by the security update. This includes CMS versions 8.18.10, 10.8.1, and 12.3.4.<br /><br />Impact:<br />Users logging into a backoffice running either 8.18.10, 10.8.1, or 12.3.4 and with either of these setups:<br />- a multilingual setup<br />- including a rich text editor and using custom style sheets (CSS) on one or more content nodes<br />are likely to experience functionality disruptions or other anomalies. These issues have been linked to the recent security patch deployment.<br /><br />Current Actions:<br />We are currently creating a new patch version for Umbraco 8, Umbraco 10, and Umbraco 12.<br />The patch versions, 8.18.11, 10.8.2, or 12.3.5 will be accessible in NuGet.<br />Cloud projects running these patch versions will be auto-upgraded with these patches later this Tuesday evening.<br /><br />We expect an upgrade to the new patch versions will resolve any known issue on Umbraco Cloud.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>15:58</var> CET</small><br><strong>Identified</strong> - Status CET 15:40:<br />Our ongoing investigation into the issues following the recent security patch on Umbraco Cloud has led us to the conclusion that a patch is required for all versions that were affected by the security update. This includes CMS versions 8.18.10, 10.8.1, and 12.3.4.<br /><br />Current Impact:<br />Users may experience functionality disruptions or other anomalies when logging into the backoffices of mentioned CMS versions. These issues have been linked to the recent security patch deployment.<br /><br />Current Actions:<br />We are currently creating a new patch version for Umbraco 8, Umbraco 10, and Umbraco 12.<br />When these are created, they will be deployed to the affected cloud environments, which we expect to resolve the issue</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>14:22</var> CET</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.umbraco.io,2005:Incident/192430132023-12-07T08:23:47+01:002023-12-07T08:23:47+01:00Heartcore Content Delivery Platform - Database Maintenance<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>08:23</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>08:17</var> CET</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>08:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>29</var>, <var data-var='time'>16:41</var> CET</small><br><strong>Scheduled</strong> - At this time we need to perform scheduled maintenance on the primary Heartcore database.<br /><br />During the maintenance window content authors will still have full access to the Heartcore back-office and will be able to perform all regular functions. However, the GraphQL and REST APIs will become unavailable for a brief period.<br /><br />Additionally, we will pause data ingestion into the primary database before we take it offline. This means that changes in the back-office and triggered webhooks will not take effect until a short time after we bring the primary database back up. We will pause ingestion immediately at commencement of the maintenance window, and expect to start it again approximately 20 minutes later.<br /><br />We expect the GraphQL and REST APIs to go down shortly after commencement of maintenance, during which time most queries will not succeed for a period of approximately 10 minutes.</p>tag:status.umbraco.io,2005:Incident/190106142023-11-05T23:42:43+01:002023-11-05T23:42:43+01:00Umbraco Cloud Portal Incident - Project/Hostnames/Environment operations failing<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>23:42</var> CET</small><br><strong>Resolved</strong> - The incident has been resolved. Cloudflare's incident has been marked as resolved and we have not seen any further issues in the Cloud Portal.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>14:29</var> CET</small><br><strong>Update</strong> - Cloudflare has now marked the issue as resolved. <br />https://www.cloudflarestatus.com/incidents/hm7491k53ppg<br /><br />We are now seeing restored service to the Umbraco Cloud Portal. <br />We will continue to monitor stability.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>12:17</var> CET</small><br><strong>Monitoring</strong> - We are continuing to monitor the Coudflare incident, and remain in contact with our suppliers. <br /><br />The current status of Umbraco Cloud Portal Services: <br />-Project Creation: Available (Degraded Performance)<br />-Environment Creation: Available (Degraded Performance) <br />-Change Plan: Available (Degraded Performance)<br />-Custom Hostnames Addition: (Degraded Performance)</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>03:10</var> CET</small><br><strong>Update</strong> - We are continuing to monitor the Coudflare incident, and remain in contact with our suppliers. <br /><br />The addition of custom hostnames is continuing to be affected.<br /><br />The current status of Umbraco Cloud Portal Services: <br />-Project Creation: Available (Degraded Performance)<br />-Environment Creation: Available (Degraded Performance) <br />-Change Plan: Available (Degraded Performance)<br />-Custom Hostnames Addition: Currently Unavailable</p><p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>20:13</var> CET</small><br><strong>Update</strong> - We are continuing to monitor the Cloudflare incident for any updates to the situation. <br />Currently some services have been restored, however you may still see degraded performance in these areas. <br /><br />Umbraco Cloud Portal Services: <br />-Project Creation: Available (Degraded Performance)<br />-Environment Creation: Available (Degraded Performance) <br />-Change Plan: Available (Degraded Performance)<br />-Custom Hostnames Addition: Currently Unavailable <br /><br />We will continue to update this incident as soon as possible.</p><p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>17:21</var> CET</small><br><strong>Update</strong> - We are continuing to monitor the ongoing Cloudflare issue, and are in touch with our suppliers about this issue.<br /><br />We will continue to update this incident as soon as possible.</p><p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>15:21</var> CET</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>14:45</var> CET</small><br><strong>Identified</strong> - We have identified an issue negatively impacting major Umbraco Cloud Portal services:<br /><br />- Project Creation: Currently unavailable<br />- Creating environment: Currently unavailable<br />- Change plan: Currently unavailable<br />- Custom Hostnames addition: Currently unavailable<br /><br />Please refrain from initiating new projects or environments, change plans or custom hostname additions during this period.<br /><br />This is due to the Cloudflare API currently experiencing issues - https://www.cloudflarestatus.com/incidents/hm7491k53ppg<br /><br />We will update this incident as soon as possible.</p>tag:status.umbraco.io,2005:Incident/189764892023-10-30T19:00:00+01:002023-10-31T09:01:17+01:00Umbraco Cloud Websites & Portal unavailable<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>19:00</var> CET</small><br><strong>Resolved</strong> - On October 30, 2023, Umbraco Cloud experienced a service disruption due to an incident at our network service provider, Cloudflare. The incident began at 20:03 UTC, impacting the availability and functionality of various services. The Umbraco Cloud Portal was down during this period, and also a limited number of cloud projects.<br /><br />Timeline:<br />- 20:03 UTC: Cloudflare reports an investigation into availability issues impacting several services.<br />- 20:05 UTC: First update from Cloudflare, the investigation continues.<br />- 20:23 UTC: Second update from Cloudflare, the investigation continues.<br />- 20:33 UTC: The issue is identified, and a fix is being implemented by Cloudflare.<br />- 20:34 UTC: A fix has been implemented, and monitoring begins to ensure stability.<br />- 20:35 UTC: Cloudflare reports the incident as resolved.<br /><br />Impact:<br /><br />The incident affected the availability of the Umbraco Cloud Portal, hindering access to certain cloud projects. Authentication processes, as well as loading functionalities, were particularly impacted, causing inconvenience to our users.<br /><br />Resolution and Mitigation:<br />Upon identifying the root cause, Cloudflare implemented a fix at 20:34 UTC, which restored the services to their normal functionality.<br />Our engineering team remained in close contact with Cloudflare throughout the incident, ensuring that necessary steps were taken to mitigate the impact on Umbraco Cloud services.<br />Alternative routing solutions were explored to minimize service disruption.</p>tag:status.umbraco.io,2005:Incident/188533652023-10-24T17:00:32+02:002023-10-24T17:00:33+02:00Autoupgrade for Umbraco Deploy 10.2.6, 11.1.6 and 12.0.4<p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>17:00</var> CEST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>24</var>, <var data-var='time'>08:00</var> CEST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>14:57</var> CEST</small><br><strong>Scheduled</strong> - Umbraco Deploy 10.2.6, 11.1.6 and 12.0.4 are released and will be automatically upgraded on projects running version 10.2.X, 11.1.X and 12.0.X respectively.<br /><br />Release notes can be found at docs.umbraco.com:<br />https://docs.umbraco.com/umbraco-deploy/release-notes<br /><br />Expected upgrade window: October 24th 2023, 8:00-17:00 CET</p>tag:status.umbraco.io,2005:Incident/188535142023-10-20T11:55:27+02:002023-10-20T11:55:27+02:00A subset of requests to Umbraco Cloud Websites experience intermittent failures - Cloudflare errors with status codes 525<p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>11:55</var> CEST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>09:19</var> CEST</small><br><strong>Monitoring</strong> - We have seen a reduction in these issues since 02:03 CEST, and continue to monitor the situation.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>22:47</var> CEST</small><br><strong>Update</strong> - We are continuing to monitor the issue and discuss potential resolutions with our 3rd party suppliers.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>16:48</var> CEST</small><br><strong>Identified</strong> - We have identified the root cause of the issue. We continue being in contact with our suppliers to fix the issue.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>15:08</var> CEST</small><br><strong>Investigating</strong> - We are currently investigating the issue. We've contacted our 3rd party provider Cloudflare and we're working towards resolution.</p>