StepFly: AI Agent Framework for IT Incident Troubleshooting

What is StepFly

StepFly is an end-to-end agentic framework that automates troubleshooting guide execution for IT incident management. The system addresses the manual, slow, and error-prone nature of traditional troubleshooting guide execution in large-scale IT systems.

Traditional troubleshooting guides require manual execution by site reliability engineers, leading to delays and human errors during critical incidents. Agentic AI enhances incident response speed while also providing a more specific and in-depth post-incident analysis to prevent the same errors from recurring in the future [3].

The framework leverages large language models to interpret unstructured troubleshooting documentation and convert it into automated workflows. AI agents possess several key attributes, including complex goal structures, natural language interfaces, the capacity to act independently of user supervision, and the integration of software tools or planning systems [6].

What is new vs previous approaches

StepFly introduces specialized capabilities that existing LLM-based incident management solutions lack.

How does StepFly work

StepFly operates through a three-stage workflow that transforms manual troubleshooting guides into automated execution systems.

1. Guide Quality Improvement Stage: TSG Mentor tool assists site reliability engineers in identifying and fixing quality issues in existing troubleshooting guides before automation begins.
2. Offline Preprocessing Stage: Large language models extract structured execution directed acyclic graphs (DAGs) from unstructured troubleshooting guides and create dedicated Query Preparation Plugins for data-intensive operations.
3. Online Execution Stage: DAG-guided scheduler-executor framework with memory system ensures correct workflow execution and supports parallel processing of independent troubleshooting steps.

The system maintains workflow integrity through its memory system while enabling parallel execution of independent troubleshooting steps. Agentic engineering operates at a higher level of abstraction: it’s a control plane that orchestrates cross-team workflows, maintains long-term memory across agents, and manages state and traceability across the full software delivery lifecycle [7].

Benchmarks and evidence

StepFly demonstrates superior performance across multiple metrics compared to baseline approaches.

Who should care

Builders

Software engineers and DevOps professionals can integrate StepFly’s open-source framework into existing incident management workflows. According to Anthropic, a provider of large language models (LLMs), AI agents are most commonly deployed in software engineering, accounting for roughly half of use cases, followed by back-office automation, marketing, sales, finance, and data analysis [2].

Enterprise

Large-scale IT organizations can reduce incident response times and human errors through automated troubleshooting guide execution. AI agents can detect issues like CPU spikes or failed processes, analyze root causes using real-time and historical data, and apply fixes automatically [5].

End users

Site reliability engineers and IT operations teams benefit from reduced manual workload and faster incident resolution times during critical system outages.

Investors

The framework represents Microsoft’s investment in agentic AI for enterprise operations, potentially reducing operational costs and improving system reliability across cloud infrastructure.

How to use StepFly today

StepFly is available as an open-source framework with implementation guidance and sample data.

1. Access the repository: Visit https://github.com/microsoft/StepFly to download the framework code and documentation.
2. Review sample data: Examine provided troubleshooting guide examples to understand the expected input format and structure.
3. Install dependencies: Set up required Python packages and large language model access according to the repository documentation.
4. Prepare troubleshooting guides: Use TSG Mentor tool to improve existing guide quality before automation implementation.
5. Configure execution environment: Set up the DAG-guided scheduler-executor framework with appropriate memory system configuration.
6. Test with sample incidents: Run the framework against provided sample incidents to validate installation and configuration.

StepFly vs competitors

StepFly competes with other AI-powered incident management and troubleshooting automation solutions.

Risks, limits, and myths

• Quality dependency: StepFly’s effectiveness depends on the quality of input troubleshooting guides, requiring initial manual review and improvement.
• LLM limitations: The framework inherits potential biases and errors from underlying large language models used for guide interpretation.
• Complex incident handling: Highly complex or novel incidents may require human intervention beyond automated troubleshooting capabilities.
• Integration complexity: Organizations need existing monitoring and telemetry systems to provide data for automated troubleshooting execution.
• Myth – Complete automation: StepFly augments rather than replaces human site reliability engineers, requiring oversight for critical incidents.
• Myth – Universal applicability: Not all troubleshooting guides are suitable for automation, particularly those requiring subjective judgment or manual hardware intervention.