Intermittent issues in Business Central and Scanning

This is the Preliminary PIR of Microsoft.

What happened?

Between 07:50 UTC and 16:00 UTC on 09 October 2025, Microsoft services and Azure customers leveraging Azure Front Door (AFD) may have experienced increased latency and/or timeouts primarily across Europe and Africa. This included the ability to access the Azure Portal as well as other management portals across Microsoft.

Peak failure rates for AFD reached approximately 17% in Africa and 6% in Europe. Availability was restored by 12:50 UTC, though some customers continued to experience elevated latency. Latency returned to normal baseline levels by 16:00 UTC, at which point the incident was mitigated.

What do we know so far?

AFD routes traffic using globally distributed edge sites and supports various Microsoft services. One of the trigger conditions for this incident was a software defect in the latest version of the AFD control plane. When any AFD ‘create’ or ‘update’ command is issued, metadata is also created so that it can interact with the data plane.

While onboarding a subset of customer workloads to a newer version of the AFD control plane, our system observed failures due to erroneous metadata being created. These errors were triggered by a specific customer sequence of operations. Our automated protection system stopped the propagation of this metadata to the data plane, which protected customers from impact. In addition, as the newer control plane was running in tandem with the existing control plane, we disabled the new control plane from taking any requests.

Following this, we initiated a cleanup of the affected metadata. The operation triggered a latent bug in the data plane that crashed infrastructure resources, resulting in a disruption to a significant number of edge sites across Europe and Africa. As part of our AFD mechanisms to manage traffic, load was automatically distributed to the remaining edge sites. This increased traffic on the remaining healthy edge sites resulted in high utilization, thereby increasing latency and timeouts.

How did we respond?

07:50 UTC on 09 October 2025 – Customer impact began and increased gradually over the next 90 minutes. 08:13 UTC on 09 October 2025 – We detected significant resource availability loss across multiple AFD edge sites. We began investigating. 09:04 UTC on 09 October 2025 – We identified the latent bug that impacted the AFD data plane. 09:08 UTC on 09 October 2025 – Automated restarts of our infrastructure start to occur, and manual intervention begins for resources that did not recover automatically. Other critical services, such as the Azure Portal, performed failover operations to migrate traffic from the affected endpoints, helping restore service availability. 12:50 UTC on 09 October 2025 – Availability for AFD fully recovered however a subset of customers may still have experienced elevated latency. 16:00 UTC on 09 October 2025 – After continuous monitoring of latency improvement, we declared the issue as mitigated after receiving confirmation of recovery. How are we making incidents like this less likely or less impactful?

We have fixed the control plane defect which generated the erroneous tenant metadata which led to the data plane crash. (Completed) We are fixing the latent bug in the data plane, to prevent recurrence. (Estimated completion: October 2025) We are improving data plane recovery time following resource exhaustion from such regressions. (Estimated completion: March 2026) This is our Preliminary PIR to share what we know so far.

After our internal retrospective is completed (generally within 14 days) we will publish a Final PIR with additional details.

Microsoft reports that Azure Front Door (AFD) availability has largely stabilized. Most customers should now experience normal access, though slightly higher response times may still occur while remaining resources are being restored.

We expect full resolution over the next few hours.

We are observing an increase in connection issues to 3PL Dynamics reported over the past 30 minutes. Our team is actively investigating the cause and has contacted Microsoft to share the current findings and request further analysis.

We will provide an update as soon as more information becomes available or within the next 60 minutes.

Microsoft reports that recovery of the Azure Front Door service is nearly complete. Approximately 96% of affected resources have been restored, and work continues to bring the remaining 4% of customers fully back online.

As this incident impacted authentication and connectivity to various Microsoft cloud services, some Boltrics users may still experience brief connection delays during the final recovery phase.

Microsoft is continuing to monitor the situation closely. The next update will be provided within 60 minutes or sooner if full recovery is confirmed.

Microsoft continues to restore affected Azure Front Door instances following a capacity loss across Europe, the Middle East, and Africa. Service availability is steadily improving as additional infrastructure nodes come back online. Boltrics systems depending on Azure services may still experience intermittent connection issues during this recovery phase.

Microsoft is expecting that instances will be online in the next 90 minutes.

Microsoft has confirmed a wider service degradation affecting authentication and access to multiple cloud services. This may explain the intermittent connection problems observed in Business Central.

Our team continues to monitor the situation closely and will provide further updates as more information becomes available.

Several customers are reporting temporary issues when using Business Central. For most users, these appear to be short-lived connection hiccups. We suspect the problem may be related to Microsoft’s Front Door service and are currently investigating further.

Possible Workaround: Try starting Business Central in a private browser session. If scanners experience issues, close all active sessions and allow the scanners to establish a new connection.

We will provide further updates as our investigation progresses.