# Competitor Baselines

This folder contains optional competitor baselines used by the **competitive
benchmark comparison** report.

When present, `python -m computer_use.benchmark.ci` loads all `*.json` files in
this directory and includes them in the leaderboard written to:

- `benchmark_results_ci/competitive.json`
- `benchmark_results_ci/competitive.md`

## Supported JSON Formats

### 1) Explicit competitor + metrics (recommended)

```json
{
  "competitor": {
    "competitor_id": "model_x",
    "name": "Model X",
    "link": "https://example.com"
  },
  "metrics": {
    "success_rate": 0.83,
    "average_steps": 12.4,
    "average_recovery_score": 0.76,
    "category_metrics": {
      "FILE_MANAGEMENT": { "rate": 0.92, "recovery_avg": 0.8 },
      "WEB_BROWSING": { "rate": 0.78, "recovery_avg": 0.7 }
    }
  }
}
```

### 2) A saved benchmark run results file

You can also drop in a full `*_results.json` produced by the benchmark runner
(top-level `run.metrics` is used).

### 3) Raw metrics dict (fallback)

If the JSON object looks like a metrics payload (e.g. contains `success_rate` /
`average_steps`), it will be treated as metrics directly.
