Cloud Opsbench

Root-Cause Analysis Diagnostic AI Agents

Snapshot

Cloud-OpsBench is a 2026 research benchmark , not yet a formal industry standard: it makes agentic root-cause analysis in Kubernetes-style cloud systems reproducible by replaying verified incidents from deterministic state snapshots rather than relying on unstable live-cluster experiments. [odj6eh] [gqs05q] Its significance is methodological—turning cloud operations from a loosely comparable demonstration problem into a process-aware evaluation task.
Created by Yilun Wang, Guangba Yu, Haiyu Huang, Yujie Huang, Zirui Wang, Pengfei Chen, and Michael R. Lyu (2026) · Maintained by LLM4Ops / research community · Type: community
ℹ️
“A reproducible benchmark for agentic root cause analysis in cloud systems.” [odj6eh]
Cloud-OpsBench evaluates agents that investigate incidents interactively, select diagnostic tools and targets, consume returned evidence, and progressively establish a diagnosis. [odj6eh] Its current release describes 754 fault cases across 57 fault types , spanning application services and Kubernetes platform layers on two Microservice workloads. [odj6eh] [gqs05q] Because the project is a paper, dataset, and open-source benchmark repository rather than a standards-body publication, its present authority is community , with stewardship likely to be determined by the authors and the public repository rather than a formal working group. [odj6eh] [gqs05q]

The Question this Spec Answers

Cloud-OpsBench addresses a weakness in existing cloud-operations evaluation: many Benchmarks test whether a model can classify an incident from fixed telemetry, but do not adequately test whether an agent can conduct a multi-step investigation, choose tools, acquire evidence, and reach a diagnosis. [odj6eh] The project frames the task as interactive and evidence-grounded cloud root-cause analysis , rather than static prediction. [odj6eh]
Before Cloud-OpsBench, researchers commonly faced a trade-off between live-cluster realism and experimental control. Live fault injection could preserve operational realism but introduced runtime noise, nondeterminism, infrastructure requirements, and inconsistent observations; static datasets improved repeatability but constrained the agent to pre-collected evidence. [odj6eh] [gqs05q] Related research is identified as using historical telemetry or other benchmark designs, while Cloud-OpsBench’s distinguishing move is to capture verified incidents as replayable snapshots. [byww9r] [iku5sc]
The benchmark makes it possible to compare agents against the same incident evidence while allowing them to choose diagnostic actions independently. That enables evaluation of not only whether an agent identifies the faulty component and fault type, but also whether it acquires milestone evidence along the way. [gqs05q] In practical terms, it makes tractable the repeatable testing of tool-using SRE agents, cloud troubleshooting copilots, and training or reinforcement-learning environments without requiring every evaluator to operate a live Kubernetes testbed. [gqs05q] [q84ojq]

Identity & Status

  • Full name: Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems . [odj6eh]
  • Common abbreviation: Cloud-OpsBench. [odj6eh] [gqs05q]
  • Type: Benchmark and evaluation infrastructure for agentic cloud root-cause analysis; operationally, it is a dataset-backed behavior and evaluation specification rather than a network protocol or data interchange standard. [odj6eh] [gqs05q]
  • Authority type: Community. The available evidence identifies an academic paper and an open-source GitHub repository, not a formal standards body, foundation, de-jure working group, or multi-vendor consortium. [odj6eh] [gqs05q]
  • Created by: Yilun Wang, Guangba Yu, Haiyu Huang, Yujie Huang, Zirui Wang, Pengfei Chen, and Michael R. Lyu. [odj6eh] [a9rczc]
  • Created year: 2026; the paper was publicly listed on arXiv in February 2026. [a9rczc] [2ry3vd]
  • Original publisher: The authors’ research group, with the work publicly distributed through arXiv and GitHub. [odj6eh] [gqs05q] [2ry3vd]
  • Maintained by: The LLM4Ops GitHub project and its author/community contributors; the available evidence does not identify a formal foundation or named successor steward. [gqs05q]
  • Current version: The paper is identified as arXiv version 2, and the repository was updated as recently as October 5, 2026. [odj6eh] [gqs05q]
  • Lifecycle stage: Active research benchmark / early community project. It is not presented as a completed formal standard. [odj6eh] [gqs05q]
  • License: The supplied search results do not establish the license of the paper text, dataset, or repository. This should be verified directly in the repository before treating the benchmark as procurement- or redistribution-ready. [gqs05q]
  • Patent terms: No patent grant or patent policy is identified in the available sources.
  • Canonical URL: The research paper is hosted at arXiv, while the implementation and artifacts are hosted in the LLM4Ops/Cloud-OpsBench GitHub repository. [odj6eh] [gqs05q]

Governance classification

Cloud-OpsBench should not be treated as de-jure , Industry Consortium , or vendor-led-open . No formal standards procedure, foundation membership model, or vendor-led governance structure is visible in the available evidence. [odj6eh] [gqs05q] Its practical authority currently comes from author credibility, reproducibility of the artifacts, citation and reuse by later research, and the degree to which external implementations adopt its snapshot and process-evaluation model. [byww9r] [f2t7oj] [6fv5pb]

Why It Matters

What it unlocks

  • Matched evaluation of interactive investigations: Cloud-OpsBench replays the same incident observations for different agents while allowing agents to select tools and targets independently, separating the incident-generation process from agent evaluation. [odj6eh]
  • Deterministic experimentation: Immutable snapshots and cached tool outputs allow repeated runs without requiring a live cluster and reduce the runtime variation that normally complicates cloud-operations research. [gqs05q]
  • Process-level scoring: The benchmark evaluates milestone coverage in addition to final diagnosis, making it possible to distinguish an agent that reached the right answer through relevant evidence from one that guessed correctly. [odj6eh] [gqs05q]
  • Full-stack fault coverage: The current artifact set spans application and Kubernetes platform failures, including admission, scheduling, startup, runtime, routing, performance, and infrastructure faults. [gqs05q]
  • A shared substrate for training and evaluation: The benchmark has been described as usable not only as an evaluation environment but also as a data engine, sandbox, and reinforcement-learning environment for diagnostic agents. [q84ojq]

What impact it has had

  • A reproducibility reference point: Later benchmark work explicitly describes Cloud-OpsBench as introducing a “state-snapshot paradigm” that captures failures under controlled conditions and replays them through deterministic mocked interfaces. [byww9r]
  • Influence on benchmark comparisons: Subsequent research places Cloud-OpsBench alongside OpenRCA, ORCA-bench, and Incident-Arena as part of the emerging ecosystem for evaluating operational agents. [6fv5pb] [iku5sc]
  • Open-source availability: The project publishes benchmark artifacts, case metadata, logs, alerts, tool caches, and evaluation scripts through GitHub, allowing researchers to run cases without deploying the original Kubernetes workloads. [gqs05q]
  • Early ecosystem reuse: A Hugging Face dataset entry describes a duplicate of the Cloud-OpsBench dataset, indicating that the artifacts have begun to propagate beyond the original repository. [17x5wm]
  • Visible but early adoption: The GitHub topic listing showed 39 stars as of the returned search result, a modest signal of developer attention rather than evidence of broad production adoption. [o3brwo]
There is not yet evidence in the available results of major cloud vendors, enterprise procurement bodies, or production SRE organizations formally adopting Cloud-OpsBench as a governance requirement. The strongest current impact signal is research reuse and conceptual influence, not institutional standardization. [byww9r] [f2t7oj] [6fv5pb] [iku5sc]

Position in the Ecosystem Stack

What it depends on

  • Kubernetes and cloud-native operational behavior: The benchmark targets Kubernetes-based cloud systems and covers faults at both application-service and platform layers. [odj6eh] [gqs05q]
  • Microservice workloads: Its cases are instantiated over two microservice systems, which provide the service topology and operational context in which faults are injected and observed. [odj6eh]
  • Diagnostic interfaces: Agents interact through standardized diagnostic tools and targets, including Kubernetes-style commands, logs, alerts, connectivity probes, and related cached observations. [odj6eh] [gqs05q] [q84ojq]
  • State snapshots and serialized artifacts: Each case packages system state, alerts, logs, metadata, and tool outputs so that the incident can be replayed deterministically. [gqs05q]
  • Evidence-graph annotations: The benchmark uses milestone-based evidence graphs to represent diagnostic facts and their dependencies independently of an agent’s exact command sequence. [odj6eh]

What depends on it

  • Later operational-agent benchmarks: ORCA-bench cites Cloud-OpsBench as a benchmark that combines source-code access with reproducible snapshots. [iku5sc]
  • Incident-Arena: Incident-Arena lists Cloud-OpsBench as a related benchmark for Kubernetes RCA based on replayed state snapshots. [6fv5pb]
  • GraphMind: GraphMind cites Cloud-OpsBench as prior work in operational traces, workflow automation, and agentic operations research. [vd37bh]
  • Agentic-bootstrap research: Related work describes Cloud-OpsBench as a reusable environment for trajectories involving thoughts, actions, and observations. [q84ojq]
  • Hugging Face dataset mirrors: The dataset has been copied or mirrored in a public Hugging Face repository, extending access beyond the original GitHub distribution. [17x5wm]

Companion specs

Cloud-OpsBench operates alongside Kubernetes diagnostic conventions, container and microservice observability practices, log and alert schemas, and agent trajectory formats. It is also conceptually adjacent to OpenRCA, ORCA-bench, and Incident-Arena, which address overlapping questions around operational-agent evaluation but use different data and interaction designs. [byww9r] [6fv5pb] [iku5sc] Unlike a protocol stack, its companions are primarily datasets, evaluation harnesses, and operational tooling rather than formally negotiated interoperability standards.

Strategic positioning

Cloud-OpsBench colonizes the evaluation layer between cloud observability and autonomous operations : whoever controls the benchmark’s definitions of valid evidence, diagnostic milestones, and representative faults can influence what vendors optimize for when claiming that an operational agent is reliable. [odj6eh] [gqs05q]

Lineage

Predecessors

  • Live fault-injection testbeds: These preserve runtime realism but introduce infrastructure cost and nondeterminism; Cloud-OpsBench retains verified fault construction while moving evaluation to replayable snapshots. [odj6eh]
  • Historical telemetry benchmarks such as OpenRCA: These provide real or recorded operational evidence but do not necessarily provide Cloud-OpsBench’s matched, interactive, snapshot-backed tool environment. [iku5sc]
  • Static classification benchmarks: These can measure diagnosis from fixed inputs but generally underrepresent adaptive investigation and process evidence, which Cloud-OpsBench makes central. [odj6eh] [gqs05q]
  • Kubernetes troubleshooting datasets: These established the value of platform-layer incidents but are extended here toward multi-step, tool-using agent evaluation. [odj6eh] [gqs05q]

Parallel efforts

  • OpenRCA: Focuses on historical telemetry for root-cause analysis and is identified as a related benchmark in later work. [iku5sc]
  • ORCA-bench: Evaluates whether language-model agents are ready for on-call work and compares Cloud-OpsBench with other operational benchmarks. [iku5sc]
  • Incident-Arena: Emphasizes incident-response evaluation and places Cloud-OpsBench in a comparative table of agentic reliability benchmarks. [6fv5pb]
  • GraphMind: Explores operational traces and self-evolving workflow automation, citing Cloud-OpsBench as a related source of operational-agent research context. [vd37bh]

Likely successors

No formal successor to Cloud-OpsBench is visible in the available evidence. The likely direction is not immediate replacement but expansion toward broader cloud providers, richer production traces, longer-horizon incidents, safety-aware remediation, and evaluation beyond diagnosis to action planning. Incident-Arena and ORCA-bench are the clearest adjacent efforts that could absorb or supersede parts of its role if they achieve wider adoption. [6fv5pb] [iku5sc]

Governance & Stewardship

Editors, authors, and sponsoring partners

The named authors are Yilun Wang, Guangba Yu, Haiyu Huang, Yujie Huang, Zirui Wang, Pengfei Chen, and Michael R. Lyu . [odj6eh] [a9rczc] [2ry3vd] The available search results do not assign formal editor, chair, or maintainer roles among them, and no sponsoring foundation or standards consortium is identified. [odj6eh] [gqs05q]
The project is associated with the LLM4Ops GitHub organization or project namespace, which publishes the benchmark implementation and artifacts. [gqs05q] The evidence does not show a formal governance charter, contributor council, release committee, or multi-vendor steering body.

Where decisions get made

The visible decision surfaces are the public GitHub repository and the associated research-paper publication process. [odj6eh] [gqs05q] The returned results do not identify a dedicated mailing list, standards working group, or public meeting archive.

Pace

The research paper appeared in February 2026, the paper has at least a version 2 publication, and the repository showed an update in October 2026. [odj6eh] [gqs05q] [a9rczc] [2ry3vd] This suggests active early-stage iteration, but there is not enough history to infer a stable release cadence or a normal interval between major versions.

Versioning policy

The available evidence does not identify a formal semantic-versioning policy. The paper uses arXiv revisioning, while the repository appears to evolve through artifact and code updates rather than a standards-style numbered release train. [odj6eh] [gqs05q]

Stewardship transitions

No stewardship transition is documented in the available sources. The creators and visible project stewards remain closely associated with the same research project and repository. [odj6eh] [gqs05q]

Political fault lines

The central political fault line is methodological: whether cloud-operations agents should be judged in deterministic replay environments, live clusters, historical telemetry, or broader end-to-end on-call simulations. [odj6eh] [byww9r] [6fv5pb] [iku5sc] A second fault line concerns what counts as success—final diagnostic accuracy alone versus evidence acquisition, process quality, and eventually safe remediation. [odj6eh] [gqs05q] The available results do not expose a specific recorded vote, fork threat, or named governance dispute.

Adoption — by Tier

Because Cloud-OpsBench is a young research benchmark rather than a mature platform standard, the three tiers below describe visible adopters, reusers, and adjacent implementations—not a large commercial conformance market.

Incumbents

  • LLM4Ops / Cloud-OpsBench — the reference repository containing the benchmark cases, snapshot artifacts, replay tooling, and evaluation metrics. [gqs05q]
  • Cloud-OpsBench arXiv paper — the authoritative research description of the benchmark’s design, fault construction, snapshot paradigm, and evidence-graph evaluation. [odj6eh]
  • Hugging Face cloud-ops-bench-dataset — a public dataset mirror or duplicate of the released artifacts. [17x5wm]
  • GitHub AIOps topic entry — an ecosystem discovery surface listing the project and showing early developer attention. [o3brwo]

LLM4Ops / Cloud-OpsBench

Steward: The LLM4Ops project and the Cloud-OpsBench authors maintain the visible reference implementation and artifact repository. [gqs05q]
Coverage of the spec: The repository covers 754 cases, 57 fault types, two microservice systems, deterministic snapshots, cached tool outputs, logs, alerts, metadata, and component-, fault-, joint-, and milestone-oriented evaluation. [gqs05q]
Adoption signal: The repository is publicly available, was updated in October 2026, and was listed with 39 GitHub stars in the returned search results. [gqs05q] [o3brwo]
Why it matters: It is the reference distribution channel: the benchmark’s definitions of cases, evidence, replay, and metrics are operationalized here rather than only described in the paper. [odj6eh] [gqs05q]

Cloud-OpsBench arXiv paper

Steward: The paper is authored by Wang, Yu, Huang, Huang, Wang, Chen, and Lyu. [odj6eh] [a9rczc]
Coverage of the spec: It defines the benchmark’s interactive diagnostic model, runtime-verified fault pipeline, snapshot-backed interfaces, and milestone-based evidence graphs. [odj6eh]
Adoption signal: Later research explicitly cites the benchmark’s snapshot paradigm and includes it in comparative surveys of operational-agent benchmarks. [byww9r] [6fv5pb] [iku5sc]
Why it matters: The paper supplies the conceptual authority that allows the repository to function as more than a dataset: it defines the evaluation philosophy that later work is beginning to compare against. [odj6eh] [byww9r]

Hugging Face cloud-ops-bench-dataset

Steward: The returned page identifies the dataset as a duplicate of another public Cloud-OpsBench dataset, but does not establish the identity or formal relationship of the uploader to the original authors. [17x5wm]
Coverage of the spec: It mirrors the benchmark dataset rather than presenting a distinct evaluation protocol. [17x5wm]
Adoption signal: Its presence indicates that the artifacts are being redistributed through a machine-learning dataset hub. [17x5wm]
Why it matters: Dataset mirrors reduce friction for researchers who consume benchmarks through model-training and evaluation platforms rather than GitHub repositories. [17x5wm]

Challengers

  • OpenRCA — a related root-cause-analysis benchmark based on historical telemetry and a different evidence model. [byww9r] [iku5sc]
  • ORCA-bench — an on-call readiness benchmark that evaluates operational language-model agents and compares against Cloud-OpsBench. [iku5sc]
  • Incident-Arena — an incident-response benchmark positioning itself alongside Cloud-OpsBench in a broader reliability evaluation landscape. [6fv5pb]
  • GraphMind — a workflow-automation research effort that uses Cloud-OpsBench as related operational-agent research context. [vd37bh]

OpenRCA

Steward: OpenRCA is a separate research effort; the returned evidence does not identify its full author and governance details. [byww9r] [iku5sc]
Coverage of the spec: It is associated with historical telemetry rather than Cloud-OpsBench’s deterministic snapshot-backed replay design. [iku5sc]
Adoption signal: Later benchmark literature treats OpenRCA as one of the principal comparison points for cloud root-cause analysis. [iku5sc]
Why it matters: OpenRCA represents the competing bet that historical operational evidence is more valuable than fully controlled synthetic replay for measuring agent competence. [iku5sc]

ORCA-bench

Steward: ORCA-bench is maintained by the authors of the on-call readiness study; the returned result does not provide a complete governance roster. [iku5sc]
Coverage of the spec: It addresses operational readiness and on-call behavior, with Cloud-OpsBench cited as a related benchmark that adds source-code access and reproducible snapshots. [iku5sc]
Adoption signal: Its publication and explicit comparative treatment of Cloud-OpsBench show that the benchmark is participating in an active research comparison rather than operating in isolation. [iku5sc]
Why it matters: ORCA-bench pressures Cloud-OpsBench’s scope from “reproducible RCA” toward broader on-call realism and end-to-end operational competence. [iku5sc]

Incident-Arena

Steward: Incident-Arena is an independent research project; the returned result does not identify a formal standards steward. [6fv5pb]
Coverage of the spec: It compares Cloud-OpsBench’s replayed state snapshots with other incident-response benchmark properties, including whether environments are interactive and whether they support other dimensions of reliability evaluation. [6fv5pb]
Adoption signal: Its comparative table treats Cloud-OpsBench as a recognized benchmark category member. [6fv5pb]
Why it matters: Incident-Arena could become a challenger by broadening evaluation from diagnosis toward the “last nine” of operational reliability. [6fv5pb]

Innovators

  • GraphMind — explores self-evolving workflow automation and operational traces in relation to Cloud-OpsBench. [vd37bh]
  • Agentic bootstrap research — treats the benchmark as a source of reusable trajectories and a training environment. [q84ojq]
  • Cloud-OpsBench dataset mirrors — experiment with alternative distribution and reuse of the benchmark artifacts. [17x5wm]
  • AIOps community projects — provide the early discovery and derivative-project surface around the benchmark. [o3brwo]

GraphMind

Steward: GraphMind is an independent research effort that cites Cloud-OpsBench as related work. [vd37bh]
Coverage of the spec: It focuses on turning operational traces into self-evolving workflow automation rather than simply reproducing Cloud-OpsBench’s benchmark protocol. [vd37bh]
Adoption signal: Its citation demonstrates conceptual reuse in research concerned with operational agents and workflow learning. [vd37bh]
Why it matters: GraphMind represents a likely next wave: moving from measuring diagnosis to learning reusable operational procedures from traces. [vd37bh]

Agentic bootstrap research

Steward: The available page is secondary and does not establish a formal maintainer relationship with Cloud-OpsBench. [q84ojq]
Coverage of the spec: It describes Cloud-OpsBench as supporting thoughts-actions-observations trajectories, mocked diagnostic tools, supervised fine-tuning data, and reinforcement-learning environments. [q84ojq]
Adoption signal: The benchmark is being interpreted as more than a static test set—as a reusable environment for agent training and bootstrapping. [q84ojq]
Why it matters: This is strategically important because benchmarks that also supply training trajectories can shape model behavior, not merely measure it. [q84ojq]

Hugging Face cloud-ops-bench-dataset

Steward: The uploader is not identified as an official Cloud-OpsBench maintainer in the returned result. [17x5wm]
Coverage of the spec: The page presents a duplicate dataset rather than a new benchmark implementation. [17x5wm]
Adoption signal: Public mirroring creates a path for experimentation by model developers who use dataset hubs as their primary workflow. [17x5wm]
Why it matters: Distribution outside the originating repository can accelerate derivative work while also creating governance and version-control questions for the benchmark’s authors. [17x5wm]

Notable Holdouts

No named cloud vendor or major AIOps provider is shown in the available results as having explicitly declined Cloud-OpsBench. The more meaningful “holdouts” are methodological: benchmarks centered on historical telemetry, broader on-call simulation, or incident-response evaluation rather than deterministic replay. [6fv5pb] [iku5sc] The available evidence is insufficient to attribute an explicit anti-Cloud-OpsBench position to those projects, so they should be treated as competing approaches rather than confirmed opponents.

Critique & Open Disputes

The available search results do not identify two to four named public critics who have directly attacked Cloud-OpsBench. That absence is consistent with the project’s recent 2026 release and early research status, but it means a full critic map requires direct review of repository issues, paper reviews, and subsequent papers beyond the returned results.
  • ORCA-bench authors ( paper ) — their comparative framing implies that reproducible snapshots alone do not exhaust the requirements for evaluating whether agents are ready for real on-call work. [iku5sc]
  • Incident-Arena authors ( paper ) — their benchmark comparison places Cloud-OpsBench within a broader reliability landscape, implying pressure to evaluate dimensions beyond its replayed Kubernetes RCA setting. [6fv5pb]
  • OpenRCA researchers ( related benchmark discussion ) — their historical-telemetry approach represents a competing argument that operational evaluation should remain grounded in recorded incidents rather than primarily controlled synthetic snapshots. [byww9r] [iku5sc]
The editors’ own stated limitations are not clearly enumerated in the returned snippets, although the benchmark’s defined scope is bounded around two microservice workloads, Kubernetes layers, and 57 fault types. [odj6eh] [gqs05q] The principal visible limitation is therefore scope: deterministic replay improves comparability but may not capture every source of live-cluster complexity, organizational context, or remediation risk.
The main archived fault line is between controlled reproducibility and operational realism , with adjacent benchmarks emphasizing historical telemetry or broader on-call behavior. [byww9r] [6fv5pb] [iku5sc] A second fault line concerns whether the final diagnosis, the evidence-acquisition process, or eventual safe action should be the primary evaluation target. [odj6eh] [gqs05q]

Frontier & Open Questions

  • How far should the benchmark expand beyond Kubernetes and its two current microservice workloads? The Cloud-OpsBench authors and repository maintainers are the most likely drivers, while adjacent benchmarks create pressure for broader infrastructure coverage. [odj6eh] [gqs05q] [iku5sc]
  • Should deterministic snapshots be supplemented by live or partially live environments? The resolution will likely be shaped by comparisons with OpenRCA, ORCA-bench, and Incident-Arena, which represent stronger bets on historical or operational realism. [byww9r] [6fv5pb] [iku5sc]
  • How should benchmark scores balance component accuracy, fault-type accuracy, joint diagnosis, and milestone coverage? The current repository exposes all four dimensions, but future research will determine whether process quality should outweigh final-answer correctness. [gqs05q]
  • Should Cloud-OpsBench evaluate remediation and safety, rather than diagnosis alone? Incident-Arena’s broader reliability framing points toward this extension, while Cloud-OpsBench currently centers on interactive evidence-grounded RCA. [odj6eh] [6fv5pb]
  • Who will govern derivative datasets and mirrors? The existence of a duplicate Hugging Face dataset raises practical questions about canonical versions, provenance, release synchronization, and whether unofficial mirrors may diverge from the reference artifacts. [17x5wm]
  • Will Cloud-OpsBench become a benchmark standard or remain an influential research artifact? Its trajectory will depend on external implementations, citation growth, broader production validation, and whether a formal community governance structure emerges around the LLM4Ops repository. [gqs05q] [byww9r] [o3brwo]

Media, Voices, and Coverage

Editor & Maintainer Voices

  • Yilun Wang, Guangba Yu, Haiyu Huang, Yujie Huang, Zirui Wang, Pengfei Chen, and Michael R. Lyu — arXiv paper — the primary authors’ definitive account of the benchmark’s design and research motivation. [odj6eh]
  • LLM4Ops — GitHub repository — the primary implementation, dataset, replay, and evaluation surface. [gqs05q]
  • Haiyu Huang — personal homepage — author publication page confirming the paper’s authorship and February 2026 availability. [a9rczc]

Implementer Coverage

  • Cloud-OpsBench repository — GitHub — documents the case format, snapshot artifacts, tool cache, metrics, and execution model. [gqs05q]
  • Hugging Face dataset mirror — Hugging Face — shows redistribution of the dataset into a model-development ecosystem. [17x5wm]
  • GitHub AIOps topic — GitHub Topics — provides a rough public-interest signal through the project’s listing and star count. [o3brwo]

Critic Coverage

  • ORCA-bench authors — ORCA-bench — useful for understanding the argument that operational-agent evaluation should reflect on-call readiness, not only reproducible RCA. [iku5sc]
  • Incident-Arena authors — Incident-Arena — useful for comparing Cloud-OpsBench’s snapshot model with broader reliability and incident-response dimensions. [6fv5pb]
  • OpenRCA-related research — evaluation discussion — useful for contrasting snapshot replay with historical telemetry. [byww9r]

Conferences & Working Group Forums

  • arXiv Software Engineering listing — arXiv — records the paper’s placement in the software-engineering research stream. [2ry3vd]
  • GitHub project issues and pull requests — LLM4Ops / Cloud-OpsBench — the most likely public venue for implementation decisions, corrections, and future extensions, although the returned search result does not enumerate specific disputes. [gqs05q]

Adjacent Specs and Standards

  • Kubernetes — the platform context whose scheduling, admission, startup, routing, runtime, and infrastructure behavior supplies many benchmark fault classes. [odj6eh] [gqs05q]
  • OpenRCA — a competing or complementary historical-telemetry approach to root-cause-analysis evaluation. [iku5sc]
  • ORCA-bench — an adjacent benchmark focused on operational-agent and on-call readiness. [iku5sc]
  • Incident-Arena — a broader incident-response and reliability-evaluation effort that compares itself with Cloud-OpsBench. [6fv5pb]
  • GraphMind — a related research direction connecting operational traces with self-evolving workflow automation. [vd37bh]
  • AIOps benchmarks — the wider family of datasets and evaluation systems for automated IT operations, within which Cloud-OpsBench is positioned. [o3brwo]
  • Agent trajectory formats — the thoughts-actions-observations structures used by related work to turn benchmark interactions into training or reinforcement-learning data. [q84ojq]
  • Cloud observability conventions — logs, alerts, metrics, connectivity probes, and Kubernetes diagnostic interfaces that Cloud-OpsBench captures and replays. [odj6eh] [gqs05q]

Sources