Episode #301
Introduction
In episode 301 of our SAP on Azure video podcast we dive into Azure Failure Prediction and Detection and how Microsoft is proactively protecting mission-critical SAP workloads.
We’re joined by Andrew, Ronit, and Ayberk from Microsoft to discuss how Azure uses machine learning and advanced telemetry to predict hardware issues and minimize downtime—even before customers notice. Whether your SAP systems demand always-on reliability or you’re curious about the latest in cloud infrastructure resilience, this episode is packed with insights you won’t want to miss.
Find all the links mentioned here: https://www.saponazurepodcast.de/episode301
Reach out to us for any feedback / questions:
- Goran Condric: https://www.linkedin.com/in/gorancondric/
- Holger Bruchelt: https://www.linkedin.com/in/holger-bruchelt/
#Microsoft #SAP #Azure #SAPonAzure #AzureInfrastructure #CloudReliability #HighAvailability #LiveMigration #EnterpriseCloud #AzureMonitor
Summary created by AI
- Azure Failure Prediction And Detection:
- Goran, Holger, Ayberk, Andrew, and Ronit explained how Azure Failure Prediction and Detection (Afpd) shifts infrastructure reliability from reactive break-fix operations to proactive identification, mitigation, and remediation of hardware risks affecting customer workloads.
- Resiliency Challenge: Ayberk explained that hardware failures—including disk wear, memory errors, power-supply degradation, and network-component failures—are inevitable at Azure scale. The objective is not to eliminate every failure, but to reduce the likelihood that infrastructure problems affect mission-critical workloads, particularly large coordinated AI and business workloads.
- Closed-Loop Platform: Afpd combines fleet-wide telemetry from hardware, firmware, operating systems, and applications with expert-defined rules, machine-learning models, analytics, automation, repair, and continuous learning. Deterministic rules are used where failure patterns are understood; machine learning is used for complex patterns involving factors such as workload characteristics, firmware versions, environmental conditions, and operating history.
- Detection And Prediction: Andrew clarified that Afpd supports both prediction and detection. Prediction identifies elevated risk before a failure, while detection identifies problems that are already occurring; neither guarantees that every failure can be predicted or that the exact failure time is known.
- Remediation Process: After mitigation, affected infrastructure may be removed from production for diagnosis and repair. It can return to service only after applicable health and validation checks. Andrew emphasized that Afpd complements, rather than replaces, application-level high availability, disaster recovery, monitoring, and operational planning.
- Workload Mitigation And Customer Actions:
- Andrew and Goran described how Azure responds when a host is considered potentially unhealthy: eligible VMs may be live migrated automatically, while customers may need to redeploy in cases where live migration is unavailable and can control the timing of that interruption.
- Platform Mitigation: When a potential infrastructure issue is identified, Azure may stop placing new workloads on the affected server and may move eligible workloads to other infrastructure. For eligible VMs, Azure can attempt live migration from the affected physical host to another host without requiring a VM reboot, although eligibility depends on the circumstances and zero interruption is not guaranteed.
- Redeployment Requirement: If a VM cannot be moved automatically, Azure sends a notification asking the customer to redeploy it to another host, potentially with a deadline. Redeployment stops the VM, moves it to a healthy host, and starts it again, so the customer should use the notification deadline to schedule an orderly shutdown and minimize impact to SAP components.
- Mitigation Limitations: Andrew corrected the interpretation that marking a node for mitigation guarantees no customer impact. The intended effect is to reduce the likelihood of impact, while infrastructure failures and interruptions can still occur.
- Infrastructure Repair: Once workloads have been moved or otherwise mitigated, Azure can take the affected infrastructure out of production for diagnosis and remediation, then return it to service after health and validation checks.
- Resource Health Notifications:
- Ronit demonstrated how customers can inspect Afpd events for an individual VM in Azure Resource Health, including the detected hardware condition, Azure’s planned live migration, the available redeployment option, and any applicable timing information.
- VM-Level Visibility: Resource Health provides a VM-specific view of the event, its current state, and available actions. In Ronit’s demonstration, the test VM was marked Degraded, and the message explained that the physical host potentially had degraded memory that could result in VM crashes.
- Available Choices: The notification stated that Azure would attempt to safely live migrate the VM. The customer could either wait for a completion notification or redeploy the VM to another host before the specified date. The date indicates when impact may occur; it does not mean Azure will necessarily redeploy the VM at that exact time.
- Event History: Resource Health also retains previous health events in Health History, making it the starting point for investigating what happened to a VM, what the platform is doing, and which customer actions remain available.
- Operational Awareness: Goran highlighted that customers may overlook these notifications, recounting an instance in which a production HANA VM was marked for redeployment before a financial quarter-end close. The discussion emphasized incorporating Afpd visibility into operational awareness for business-critical SAP environments.
- Fleet Monitoring With Resource Graph:
- Ronit explained that Azure Resource Graph complements Resource Health by allowing customers to query degraded VMs across an entire subscription or environment instead of investigating each VM individually.
- Scale Monitoring: Resource Health is suited to investigating one VM, whereas Azure Resource Graph supports customers operating environments with hundreds or thousands of VMs. Ronit demonstrated a query that identifies resources currently marked as degraded.
- Query Results: The query returns the annotation name, summary, and supporting metadata that can help explain the event. Customers can adapt the documented query examples to their own environments and use the results to build dashboards and monitor workloads at scale.
- Event Grid Integration:
- Ronit showed how customers can route Afpd notifications into existing monitoring, alerting, and automation systems through Event Grid and process the resulting JSON payload with services such as Event Hubs, Azure Functions, or webhooks.
- Event Subscription: Event Grid allows customers to subscribe to Afpd-related events and connect them to downstream processing. Ronit demonstrated an Event Grid system topic configured to send events to an event hub, where notifications were received as JSON payloads.
- Automation Options: Customers can connect Event Grid to an Azure Function to process an event and trigger a workflow, or use webhooks to integrate with existing monitoring platforms. Possible actions include routing an annotation, generating an alert, or initiating an automated response appropriate to the workload.
- Consistent Payloads: The payload includes the annotation name, summary, target-resource information, and supporting data. Ronit stated that the notification information is intended to remain consistent across Resource Health, Azure Resource Graph, and Event Grid so customers can select the interface that best fits their operations.
- Three Consumption Paths: Ronit summarized the options as Resource Health for individual-VM investigation, Resource Graph for querying multiple VMs at scale, and Event Grid for integration with monitoring, alerting, and automation workflows.
- 0:00 Intro
- 0:44 Why SAP workload resilience on Azure matters
- 1:18 Meet the Azure Compute experts
- 2:35 Cloud-scale hardware failures and the core challenge
- 4:22 Why AI workloads raise the reliability bar
- 5:18 What AFPD is (and what it is not)
- 6:18 Rules + machine learning for failure prediction
- 7:18 From prediction to mitigation and remediation
- 9:35 End-to-end AFPD flow
- 10:48 Live migration vs redeploy decision path
- 12:08 Redeploy behavior and customer-controlled timing
- 14:05 Resource Health demo for single VM visibility
- 16:08 Fleet-wide monitoring with Azure Resource Graph
- 18:00 Event Grid integration and automation workflows
- 20:02 Wrap-up and key takeaways
