Incidents | Codemagic CI/CD Incidents reported on status page for Codemagic CI/CD https://codemagic.betteruptime.com/ https://d1lppblt9t2x15.cloudfront.net/logos/868ba487599abb6682f8dcf72edbab20.png Incidents | Codemagic CI/CD https://codemagic.betteruptime.com/ en Scheduled maintenance affecting macOS build machine availability https://codemagic.betteruptime.com/incident/446735 Sat, 16 Nov 2024 14:00:15 -0000 https://codemagic.betteruptime.com/incident/446735#678a34d23e09b91eb874dacf9920a6826cfb3d52e7e94762f47005e89be21328 Maintenance completed Scheduled maintenance affecting macOS build machine availability https://codemagic.betteruptime.com/incident/446735 Sat, 16 Nov 2024 00:00:15 -0000 https://codemagic.betteruptime.com/incident/446735#75f0c9ed67bd8996b1dee77db96fd681736848eb3150778b69019e1cae255b23 One of our data centers is performing emergency maintenance due to a manufacturer recall of a critical component in their Uninterruptible Power Supply (UPS) systems. During the maintenance window, we will be operating with fewer macOS build machines than normally due to which there may be longer build queues. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 10:00:00 -0000 https://codemagic.betteruptime.com/incident/454360#318f119cfd7905fd84e1cb31844b3df96fe73726d4ec4368055615a6656bfc41 The issue has been resolved. All Xcode 16.1 machines are available again. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 10:00:00 -0000 https://codemagic.betteruptime.com/incident/454360#318f119cfd7905fd84e1cb31844b3df96fe73726d4ec4368055615a6656bfc41 The issue has been resolved. All Xcode 16.1 machines are available again. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 10:00:00 -0000 https://codemagic.betteruptime.com/incident/454360#318f119cfd7905fd84e1cb31844b3df96fe73726d4ec4368055615a6656bfc41 The issue has been resolved. All Xcode 16.1 machines are available again. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 03:17:00 -0000 https://codemagic.betteruptime.com/incident/454360#09ec66fe3b8cf2b973abc716832135ec6f662d8776b118e9694ea66de534e88f Builds stay queued when using Xcode 16.1. We are working to restore the availability of the machines. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 03:17:00 -0000 https://codemagic.betteruptime.com/incident/454360#09ec66fe3b8cf2b973abc716832135ec6f662d8776b118e9694ea66de534e88f Builds stay queued when using Xcode 16.1. We are working to restore the availability of the machines. Xcode 16.1 build machines are unavailable https://codemagic.betteruptime.com/incident/454360 Fri, 01 Nov 2024 03:17:00 -0000 https://codemagic.betteruptime.com/incident/454360#09ec66fe3b8cf2b973abc716832135ec6f662d8776b118e9694ea66de534e88f Builds stay queued when using Xcode 16.1. We are working to restore the availability of the machines. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 06:10:00 -0000 https://codemagic.betteruptime.com/incident/426331#79f1009e64c7cf25c6a15512930acf9fc8619f4e6e37ed3b8e77b3348d7432e4 The incident has been resolved. All builds have started, and we will continue monitoring for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 06:10:00 -0000 https://codemagic.betteruptime.com/incident/426331#79f1009e64c7cf25c6a15512930acf9fc8619f4e6e37ed3b8e77b3348d7432e4 The incident has been resolved. All builds have started, and we will continue monitoring for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 06:10:00 -0000 https://codemagic.betteruptime.com/incident/426331#79f1009e64c7cf25c6a15512930acf9fc8619f4e6e37ed3b8e77b3348d7432e4 The incident has been resolved. All builds have started, and we will continue monitoring for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 00:00:00 -0000 https://codemagic.betteruptime.com/incident/426331#2a3ee33f49cb4ba1bcd6c312c8e8a640bf8f7c8c046590bf54f3d3d334c1dccd We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 00:00:00 -0000 https://codemagic.betteruptime.com/incident/426331#2a3ee33f49cb4ba1bcd6c312c8e8a640bf8f7c8c046590bf54f3d3d334c1dccd We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/426331 Mon, 09 Sep 2024 00:00:00 -0000 https://codemagic.betteruptime.com/incident/426331#2a3ee33f49cb4ba1bcd6c312c8e8a640bf8f7c8c046590bf54f3d3d334c1dccd We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 07:20:00 -0000 https://codemagic.betteruptime.com/incident/425548#2d6cbda14293a7b442799988e8befd19306fb9fb945312dfa038d864a2fc6b56 All builds have now started, and we will continue to monitor for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 07:20:00 -0000 https://codemagic.betteruptime.com/incident/425548#2d6cbda14293a7b442799988e8befd19306fb9fb945312dfa038d864a2fc6b56 All builds have now started, and we will continue to monitor for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 07:20:00 -0000 https://codemagic.betteruptime.com/incident/425548#2d6cbda14293a7b442799988e8befd19306fb9fb945312dfa038d864a2fc6b56 All builds have now started, and we will continue to monitor for any further issues. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 00:30:00 -0000 https://codemagic.betteruptime.com/incident/425548#b147d4a9b4fa7fa5a0d47cf2a6bdf5c5f11b1f4f9a5e4b911d39e4ddcf53c989 We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 00:30:00 -0000 https://codemagic.betteruptime.com/incident/425548#b147d4a9b4fa7fa5a0d47cf2a6bdf5c5f11b1f4f9a5e4b911d39e4ddcf53c989 We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sun, 08 Sep 2024 00:30:00 -0000 https://codemagic.betteruptime.com/incident/425548#b147d4a9b4fa7fa5a0d47cf2a6bdf5c5f11b1f4f9a5e4b911d39e4ddcf53c989 We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 08:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#aa76e74d8ad4a99f53c16dbbd148ffd083762ebae519454725411c54beb33a35 This incident has been resolved. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 08:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#aa76e74d8ad4a99f53c16dbbd148ffd083762ebae519454725411c54beb33a35 This incident has been resolved. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 08:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#aa76e74d8ad4a99f53c16dbbd148ffd083762ebae519454725411c54beb33a35 This incident has been resolved. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 01:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#8e239814ad3dc5af784c421c630a8ee5da5104e94a2012dc67351e11d46528ee We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 01:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#8e239814ad3dc5af784c421c630a8ee5da5104e94a2012dc67351e11d46528ee We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Builds stuck in queued status https://codemagic.betteruptime.com/incident/425548 Sat, 07 Sep 2024 01:00:00 -0000 https://codemagic.betteruptime.com/incident/425548#8e239814ad3dc5af784c421c630a8ee5da5104e94a2012dc67351e11d46528ee We are experiencing an issue where some macOS builds are remaining in a queued state for an extended period. Web app is unavailable https://codemagic.betteruptime.com/incident/425318 Fri, 06 Sep 2024 19:45:00 -0000 https://codemagic.betteruptime.com/incident/425318#9c25735a51454fd2c1dbf02bcbe9c4a652d2ca5a6a42c5b185293f1ed34f80cf We have identified the cause of the issue and managed to resolve it. We will continue to monitor the situation to make sure everything is fixed. Web app is unavailable https://codemagic.betteruptime.com/incident/425318 Fri, 06 Sep 2024 18:47:00 -0000 https://codemagic.betteruptime.com/incident/425318#edced6d9143e9abf8a8350c101f07c4e22689ec1a61bd83abeb506dbbd2e9292 There is an ongoing incident that is causing issues with authentication. We are looking into it and working hard to restore the service as soon as possible. An update will be provided as soon as there is more information available. Sorry for the inconvenience! Builds on Windows machines remain queued https://codemagic.betteruptime.com/incident/379283 Tue, 04 Jun 2024 12:30:00 -0000 https://codemagic.betteruptime.com/incident/379283#1d2b06215fc6d7a20edf74401c943cb66b7e6c8a2ebbfb69485b6594e21dac9c We have identified the root cause and fixed the issue. All builds are expected to start soon. Builds on Windows machines remain queued https://codemagic.betteruptime.com/incident/379283 Tue, 04 Jun 2024 09:10:00 -0000 https://codemagic.betteruptime.com/incident/379283#75ba5f169f1651226e4eaeb8fc36e1b9806a3159ebc3561b90eba898385c3ddf We are investigating an issue with running builds on Windows machines. Build dashboards aren't working https://codemagic.betteruptime.com/incident/375969 Tue, 28 May 2024 18:15:00 -0000 https://codemagic.betteruptime.com/incident/375969#5972de45be868b19e145d4f8c4385e2066e080b3d888e8857591c6be7f88d85f This incident has been resolved Build dashboards aren't working https://codemagic.betteruptime.com/incident/375969 Tue, 28 May 2024 15:15:00 -0000 https://codemagic.betteruptime.com/incident/375969#0da660ece048937dc788c84e204b5d99a04940cb590b7cdc3d8f28d632cc7890 We are investigating an issue with Google App Engine affecting the use of Build dashboards. Windows Server 2019 brownout https://codemagic.betteruptime.com/incident/355827 Wed, 24 Apr 2024 15:00:00 -0000 https://codemagic.betteruptime.com/incident/355827#d8dc1cf19240540e4c2019b76edd74eb8245fe197e17ad40293ca1b03185ac8f Maintenance completed Windows Server 2019 brownout https://codemagic.betteruptime.com/incident/355827 Wed, 24 Apr 2024 09:00:00 -0000 https://codemagic.betteruptime.com/incident/355827#3128f3eba8c9be1a02f0765e1cb975ec328baeb8cff0062dce7932e751689a55 We are conducting brownouts for the Windows Server 2019 build image and it will be removed from Codemagic on May 1, 2024. Read more here: https://github.com/orgs/codemagic-ci-cd/discussions/2474 Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 14:55:00 -0000 https://codemagic.betteruptime.com/incident/344594#94adc575fdd2a68ae10d5d3c92bec72a61a7ef82e61f3ab9523d3dd0a660c9e5 We were able to resolve the issue and will continue monitoring. Please note that it may take longer than usual for builds to start. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 14:55:00 -0000 https://codemagic.betteruptime.com/incident/344594#94adc575fdd2a68ae10d5d3c92bec72a61a7ef82e61f3ab9523d3dd0a660c9e5 We were able to resolve the issue and will continue monitoring. Please note that it may take longer than usual for builds to start. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 14:45:00 -0000 https://codemagic.betteruptime.com/incident/344594#972f91a6471d4f9824f39fc1ff54107ca2f8c868b1b0b8254f30397da0a4f761 Unfortunately, the issue has returned. We are working actively on resolving it. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 14:45:00 -0000 https://codemagic.betteruptime.com/incident/344594#972f91a6471d4f9824f39fc1ff54107ca2f8c868b1b0b8254f30397da0a4f761 Unfortunately, the issue has returned. We are working actively on resolving it. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 13:43:00 -0000 https://codemagic.betteruptime.com/incident/344594#f4d7235f5fc3d57fb2e2f6c070e8194d6f86bff837ed2693845cafb65b229986 We have identified the root cause and deployed a fix. All new builds are expected to start soon. We apologize for any inconvenience caused. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 13:43:00 -0000 https://codemagic.betteruptime.com/incident/344594#f4d7235f5fc3d57fb2e2f6c070e8194d6f86bff837ed2693845cafb65b229986 We have identified the root cause and deployed a fix. All new builds are expected to start soon. We apologize for any inconvenience caused. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 13:33:00 -0000 https://codemagic.betteruptime.com/incident/344594#114772a115c3556726f1bdf344eeb88aac49cb9c13f87d79aadeb862c5709bfe New builds started at this time fail with "Something Went Wrong". Builds that are already running are not affected. Unable to start new builds https://codemagic.betteruptime.com/incident/344594 Fri, 22 Mar 2024 13:33:00 -0000 https://codemagic.betteruptime.com/incident/344594#114772a115c3556726f1bdf344eeb88aac49cb9c13f87d79aadeb862c5709bfe New builds started at this time fail with "Something Went Wrong". Builds that are already running are not affected. Elevated error rate https://codemagic.betteruptime.com/incident/339315 Thu, 07 Mar 2024 12:37:00 -0000 https://codemagic.betteruptime.com/incident/339315#0fb89feadebd5d44202f616c7e9603cecd9d24be056573edbc663d6713805911 We have identified the root cause of the issue and deployed a fix. All new builds are expected to run as usual. Thank you for your patience. Elevated error rate https://codemagic.betteruptime.com/incident/339315 Thu, 07 Mar 2024 12:25:00 -0000 https://codemagic.betteruptime.com/incident/339315#c0849fb40b9a79b8ada6a42e2325760b983331e14f41ef07e0f7d7c698944cae We are seeing an increased number of builds fail with the "Something Went Wrong" error during the cleanup phase.