# ptAI API — User Guide

This API lets you run adversarial-ML attacks against your own model and
dataset. Every attack the web console offers is available as a
single HTTP endpoint — upload your assets, pick the algorithm, get the
result.

* Base URL: `https://api.pt4ai.k3ylabs.dev` on the hosted instance, or
  `https://<host>:8000` on your own deployment. The examples below use
  `https://127.0.0.1:8000`; replace it with yours.
* Interactive docs: `GET /docs` — the easiest place to try a request
  from the browser.
* Machine-readable spec: `GET /openapi.json`
* Postman: `GET /postman-import` (import page) or `GET /postman` (raw collection JSON).

---

## 1. What you can do

| Goal | Endpoint |
|---|---|
| Fool the model at inference time (gradient access) | `POST /api/attack/evasion/whitebox` |
| Fool the model with only prediction access | `POST /api/attack/evasion/blackbox` |
| Transfer adversarial inputs from a surrogate | `POST /api/attack/evasion/greybox` |
| Test whether the model leaks training-set membership | `POST /api/attack/inference` |
| Steal a copy of the model from its predictions | `POST /api/attack/extraction` |
| Inject malicious training samples (backdoor it) | `POST /api/attack/poisoning` |
| Run a full robustness sweep with a verdict | `POST /api/attack/pentest` |
| **Run any of the above from a single config file** | `POST /api/attack` |
| Fool an **object detector** (Faster R-CNN / YOLOv5) — boxes vanish, appear or change class | `POST /api/attack/object-detection` — see §14 |

All attack endpoints work with **structured** (tabular CSV) and **image**
(built-in MNIST/CIFAR-10 or your own class-per-folder ZIP) data. Models can be
Keras, PyTorch, scikit-learn, XGBoost, LightGBM, CatBoost or ONNX (see §3).

### The unified endpoint — `POST /api/attack`

One convenience endpoint runs **any** attack from a single **full-config JSON**
instead of separate form fields. The config carries `data_type`, `family`,
`scenario`/`attack`, options, and `params`; you still attach the data files.

```bash
# config.json describes the whole request:
# {"data_type":"structured","family":"evasion","scenario":"white","attack":"pgd",
#  "columns_to_drop":"flow_id,src_ip,dst_ip,src_port,dst_port","label_col":"label",
#  "params":{"eps":0.1,"eps_step":0.01,"max_iter":40}}
curl -k -X POST https://127.0.0.1:8000/api/attack \
  -H 'X-API-Key: your-key-here' \
  -F config_file=@config.json \
  -F dataset=@data.csv -F model=@model.h5 \
  -F scaler=@scaler.joblib -F encoder=@encoder.joblib \
  -o result.zip
```

Download ready-made configs (one per attack) from **`/templates`** — the
*full-config* set. The granular endpoints below still work exactly as before;
`/api/attack` is just a one-file shortcut.

---

## 2. How a request works

Every attack endpoint is a `multipart/form-data` POST. The form fields,
top to bottom in Swagger UI, are:

1. **`data_type`** — `structured` or `image` (pick this first; it
   decides which file slots you need).
2. **Attack dropdown** — algorithm to run. Different endpoints offer
   different choices; only valid algorithms appear.
3. **`params`** — a JSON object string with the algorithm's parameters.
   Don't put an `attack` key inside `params`; that's the dropdown's job.
4. **`params_file`** *(optional)* — a `.json` file with the same
   structure; values override the inline `params` on key collision.
5. **File slots** — `dataset`, `model`, `scaler`, `encoder`
   (structured) or `dataset_zip` (image).
6. **Image-only fields** — `source` (`mnist` / `cifar10` / `zip`),
   `image_size`, `color_mode`, `limit`.

**Every attack returns a ZIP by default** — a single archive with all
results and artifacts. Append `?format=json` to any attack URL if you'd
rather get the JSON envelope inline.

---

## 3. What to attach

### Structured data

| Slot | Required | Format |
|---|---|---|
| `dataset` | yes | `.csv` |
| `model` | yes | Keras `.h5` / `.keras`, or another supported format (below) |
| `scaler` | yes for Keras, optional otherwise | `.joblib` / `.pkl` |
| `encoder` | yes for Keras, optional otherwise | `.joblib` / `.pkl` |
| `columns_to_drop` | no | comma-separated column names (default: `flow_id,src_ip,dst_ip,src_port,dst_port`) |
| `label_col` | no | ground-truth column (default: `label`) |
| `label_binary_col` | no | optional binary label column |

### Image data

| Slot | Required | Notes |
|---|---|---|
| `source` | yes | `mnist` · `cifar10` · `zip` |
| `dataset_zip` | only when `source=zip` | class-per-folder ZIP |
| `model` | usually yes | Keras `.h5` / `.keras` or another supported format (optional for extraction) |
| `image_size` | when `source=zip` | `WxH`, e.g. `28x28` |
| `color_mode` | when `source=zip` | `rgb` · `grayscale` |
| `limit` | no | max samples to keep (default 200) |

### Other model formats

Besides Keras, `model` accepts PyTorch (`.pt` TorchScript, `.pth`, or `.safetensors` plus an
`architecture` file), scikit-learn (`.joblib` / `.pkl` / `.skops`), XGBoost (`.json` / `.ubj`),
LightGBM (`.txt`), CatBoost (`.cbm`) and ONNX (`.onnx`). These files are opened only inside an
isolated sandbox on the server. Models without gradients (trees, ONNX) are attacked white-box
through a surrogate model (a transfer attack); the result says so. If an attack can't run on the
uploaded model type, the API answers `422` with the reason.

---

## 4. Evasion — three sub-endpoints

Pick the sub-endpoint that matches the attacker's access model. Each
exposes only the algorithms valid for that scenario.

| Sub-endpoint | Available algorithms |
|---|---|
| `/api/attack/evasion/whitebox` | `fgsm` · `pgd` · `bim` · `jsma` · `cw` · `deepfool` · `apgd` |
| `/api/attack/evasion/blackbox` | `zoo` · `boundary` · `hsj` |
| `/api/attack/evasion/greybox` | same gradient-based set as white-box (uses a surrogate) |

`deepfool` and `apgd` are image-only.

### Parameters per algorithm

White-box / grey-box:

| Algorithm | Common parameters (defaults shown) |
|---|---|
| `fgsm` | `eps=0.1` · `eps_step` · `num_random_init=0` · `batch_size=32` · `targeted=false` · `y_target` |
| `pgd` | `eps=0.1` · `eps_step=0.01` · `max_iter=40` · `targeted=false` · `y_target` |
| `bim` | `eps=0.1` · `eps_step=0.01` · `max_iter=10` · `targeted=false` · `y_target` |
| `jsma` | `theta=1.0` · `gamma=0.1` · `targeted=true` · `y_target` |
| `cw` | `confidence=0.0` · `learning_rate=0.001` · `binary_search_steps=10` · `max_iter=20` · `initial_const=0.9` · `batch_size=32` |
| `deepfool` *(image)* | `max_iter=50` · `epsilon=1e-6` · `nb_grads=10` |
| `apgd` *(image)* | `eps=0.03` · `max_iter=100` |

Black-box:

| Algorithm | Common parameters (defaults shown) |
|---|---|
| `zoo` | `confidence=0.5` · `learning_rate=0.01` · `max_iter=20` · `binary_search_steps=10` · `initial_const=1.0` · `nb_parallel=20` · `batch_size=1` |
| `boundary` | `max_iter=1000` · `epsilon=0.01` · `step_adapt=0.667` · `init_size=100` · `batch_size=64` |
| `hsj` | `max_iter=25` · `max_eval=10000` · `init_eval=100` |

### curl — white-box PGD on structured data

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/evasion/whitebox \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' \
  -F 'attack=pgd' \
  -F 'params={"eps":0.1,"eps_step":0.01,"max_iter":40}' \
  -F 'dataset=@./data.csv' \
  -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' \
  -F 'encoder=@./encoder.joblib' \
  -o result.zip
```

### curl — black-box HopSkipJump on MNIST

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/evasion/blackbox \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=image' \
  -F 'attack=hsj' \
  -F 'params={"max_iter":25,"max_eval":5000}' \
  -F 'source=mnist' \
  -F 'limit=200' \
  -F 'model=@./mnist_cnn.h5' \
  -o result.zip
```

### curl — grey-box FGSM (gradients on surrogate)

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/evasion/greybox \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' \
  -F 'attack=fgsm' \
  -F 'params={"eps":0.1}' \
  -F 'dataset=@./data.csv' \
  -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' \
  -F 'encoder=@./encoder.joblib' \
  -o result.zip
```

### What you get back

A **ZIP** (`ptai_evasion_<attack>_<id>.zip`) containing `results.json` (the full
envelope below), `metrics.json`, and confusion-matrix CSVs. Add `?format=json` to the URL
to get this envelope inline instead of the ZIP:

```json
{
  "metrics": {
    "clean_accuracy": 0.92,
    "adversarial_accuracy": 0.41,
    "accuracy_drop": 0.51,
    "n_samples": 1000,
    "perturbation_linf": 0.10,
    "perturbation_l2_mean": 1.4
  },
  "artifacts": {
    "confusion_clean":       [[...]],
    "confusion_adversarial": [[...]]
  }
}
```

---

## 5. Membership Inference

Tests whether the model leaks information about its training set.
Rule-Based MIA (loss threshold) and Black-Box MIA (meta-classifier) run
in parallel.

| Field | Choices / default | What it means |
|---|---|---|
| `attack_model_type` | `rf` (default) · `gb` · `lr` | Meta-classifier for Black-Box MIA |
| `split_ratio` *(in `params`)* | `0.7` | Fraction of samples treated as 'members' |
| `max_samples` *(in `params`)* | `1000` | Cap per split (members / non-members) |
| `seed` *(in `params`)* | `42` | RNG seed |

### curl

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/inference \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' \
  -F 'attack_model_type=rf' \
  -F 'params={"split_ratio":0.7,"max_samples":1000}' \
  -F 'dataset=@./data.csv' \
  -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' \
  -F 'encoder=@./encoder.joblib' \
  -o result.zip
```

### What you get back

A **ZIP** with `results.json` + `metrics.json`. Both MIA variants report `accuracy`,
`precision`, `recall`, `f1`, `auc`, `advantage`. **High advantage = the model leaks
membership.** (Add `?format=json` for the inline envelope.)

---

## 6. Model Extraction

CopycatCNN — query the victim model, train a stolen ("thieved") model
on its responses, measure how closely it matches.

| Field *(in `params`)* | Default | What it means |
|---|---|---|
| `nb_stolen` | `500` | Query budget against the victim |
| `nb_epochs` | `5` | Training epochs for the stolen model |
| `batch_size` | `64` | |
| `use_probability` | `true` | `true` = soft labels · `false` = hard labels |

### curl

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/extraction \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=image' \
  -F 'params={"nb_stolen":500,"nb_epochs":5,"use_probability":true}' \
  -F 'source=cifar10' \
  -F 'limit=2000' \
  -F 'model=@./cifar_cnn.h5' \
  -o result.zip
```

### What you get back

A **ZIP** with `results.json` + `metrics.json` reporting `fidelity` (agreement with
victim), `victim_accuracy`, `stolen_accuracy`, plus per-class fidelity and a query-budget
curve. (Add `?format=json` for the inline envelope.)

---

## 7. Poisoning

Inject a small fraction of malicious training samples, retrain a clone
of your model, and measure how badly it's been backdoored.

| `attack` value | Method |
|---|---|
| `backdoor` | BadNets — visible trigger (Gu 2017) |
| `clean_label` | Adversarial perturbation + visible trigger (Turner 2018) |
| `hidden_trigger` | Invisible trigger via feature-space optimisation (Saha 2019) |

### Common parameters (all variants)

| Field | Default |
|---|---|
| `target_class` | `0` |
| `poison_fraction` | `0.10` |
| `nb_epochs` | `8` |
| `batch_size` | `64` |
| `seed` | `42` |

### Trigger placement

Structured data:

| Field | Default | Notes |
|---|---|---|
| `trigger_features` | `-1` | Comma-separated indices; negatives allowed (`-1` = last column) |
| `trigger_value` | `1.0` | Value written to trigger features |

Image data:

| Field | Default | Choices |
|---|---|---|
| `patch_size` | `4` | 1–256 px side length |
| `patch_value` | `1.0` | Pixel value |
| `patch_position` | `bottom-right` | `top-left` · `top-right` · `bottom-left` · `bottom-right` · `center` |

### Variant-specific extras

`clean_label` adds: `pp_poison` (0.33) · `eps` (0.3) · `eps_step` (0.1) ·
`max_iter` (100)

`hidden_trigger` adds: `source_class` · `feature_layer` (-2) ·
`learning_rate` (0.01) · `max_iter` (200) · `htb_batch_size` (32)

### curl — image backdoor

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/poisoning \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=image' \
  -F 'attack=backdoor' \
  -F 'params={"target_class":0,"patch_size":4,"patch_position":"bottom-right","poison_fraction":0.1}' \
  -F 'source=mnist' \
  -F 'limit=1000' \
  -F 'model=@./mnist_cnn.h5' \
  -o result.zip
```

### curl — structured clean-label

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/poisoning \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' \
  -F 'attack=clean_label' \
  -F 'params={"target_class":0,"trigger_features":"-1","trigger_value":1.0,"poison_fraction":0.1,"pp_poison":0.33,"eps":0.3,"eps_step":0.1,"max_iter":100}' \
  -F 'dataset=@./data.csv' \
  -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' \
  -F 'encoder=@./encoder.joblib' \
  -o result.zip
```

### What you get back

A **ZIP** with `results.json` + `metrics.json`, confusion/target-confidence CSVs, and
(for structured data) the full **`poisoned_train.csv`**. Headline metrics: training
samples, poisoned samples, clean accuracy, **Attack Success Rate (ASR)**, plus per-class
CDA / ASR and label distribution before vs. after. (Add `?format=json` for the inline
envelope; the poisoned CSV is then base64 inside `artifacts`.)

---

## 8. Adversarial Penetration Test

Full robustness sweep across FGSM / PGD / BIM at ε ∈ [0.00, 1.00]. Emits a
pass/fail robustness assessment (clean vs. adversarial accuracy per ε).
**Structured data only.**

| Field | Default | What it means |
|---|---|---|
| `use_fgsm` | `true` | Include FGSM in the sweep |
| `use_pgd`  | `true` | Include PGD in the sweep |
| `use_bim`  | `true` | Include BIM in the sweep |
| `eps_step` *(in `params`)* | `0.05` | Spacing of the ε-grid |
| `clean_threshold` *(in `params`)* | none | Minimum acceptable clean accuracy for the verdict |
| `adv_threshold` *(in `params`)* | none | Minimum acceptable adversarial accuracy at the worst ε |

At least one algorithm must be selected.

### curl

```bash
curl -k -X POST https://127.0.0.1:8000/api/attack/pentest \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' \
  -F 'use_fgsm=true' -F 'use_pgd=true' -F 'use_bim=true' \
  -F 'params={"eps_step":0.05}' \
  -F 'dataset=@./data.csv' \
  -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' \
  -F 'encoder=@./encoder.joblib' \
  -o pentest_result.zip
```

### What you get back

A **ZIP** with `results.json` + `metrics.json`. Metrics include: clean accuracy;
per-attack and overall **minimum / average adversarial accuracy**; an accuracy-vs-ε
curve per attack; and robustness flags (`is_robust`, `is_robust_min`, `is_robust_avg`).
(Add `?format=json` for the inline envelope.)

---

## 9. Async execution — long-running attacks

Some attacks are slow — jsma/hsj can run 15+ minutes, and cw/zoo/boundary/
pentest/inference (rf/gb) commonly take several minutes. Every attack
endpoint now runs the attack as a **background job**:

* If it finishes within `PTAI_SYNC_WAIT_SECONDS` (default 25s), you get
  **exactly the same response as before** — the ZIP or JSON envelope,
  inline, no changes needed to existing clients/scripts.
* If it's still running after that window (or you passed `?async=true`),
  you instead get **`202 Accepted`**:
  ```json
  {
    "job_id": "9f5e2c...",
    "status": "queued",
    "status_url": "/api/jobs/9f5e2c...",
    "result_url": "/api/jobs/9f5e2c.../result?format=zip"
  }
  ```

Then:

| Action | Request |
|---|---|
| Poll status | `GET /api/jobs/{job_id}` → `queued` · `running` · `done` · `error` · `timeout` · `cancelled` |
| Fetch the result once `done` | `GET /api/jobs/{job_id}/result?format=zip\|json` — same envelope/ZIP as a synchronous response |
| Cancel | `DELETE /api/jobs/{job_id}` — immediate if still `queued`; if already `running`, the job is marked `cancelled` right away but the underlying computation (a synchronous ART/TensorFlow call in a worker thread) can't be force-killed and finishes in the background rather than stopping instantly |

```bash
# submit, then poll until done
curl -s -k -X POST https://127.0.0.1:8000/api/attack/evasion/blackbox \
  -H 'X-API-Key: your-key-here' \
  -F 'data_type=structured' -F 'attack=hsj' \
  -F 'dataset=@./data.csv' -F 'model=@./model.h5' \
  -F 'scaler=@./scaler.joblib' -F 'encoder=@./encoder.joblib' > submit.json
job_id=$(python3 -c "import json;print(json.load(open('submit.json'))['job_id'])")
until [ "$(curl -s -k https://127.0.0.1:8000/api/jobs/$job_id -H 'X-API-Key: your-key-here' | python3 -c "import json,sys;print(json.load(sys.stdin)['status'])")" = "done" ]; do sleep 5; done
curl -s -k https://127.0.0.1:8000/api/jobs/$job_id/result -H 'X-API-Key: your-key-here' -o result.zip
```

A closed client connection (Postman tab closed, curl killed, proxy
timeout) while polling has **no effect** on the job or on any other
request — job/worker-slot accounting is fully decoupled from any single
HTTP connection, which is also what stops a burst of abandoned requests
from starving every other caller.

---

## 10. Response format

Every attack endpoint returns a **ZIP by default** (all results + artifacts as files).
Append **`?format=json`** to get this envelope inline instead:

```json
{
  "run_id": "0a1b2c3d4e5f...",
  "data_type": "structured",
  "family": "evasion",
  "attack": "pgd",
  "scenario": "white",
  "metrics":   { "...": "..." },
  "plots":     { "...": "base64-encoded PNG" },
  "artifacts": { "...": "..." }
}
```

* `run_id` — one-off identifier for the request (no server-side state).
* `scenario` is only set for evasion (`white` / `black` / `grey`).
* `metrics` carries the headline numbers (accuracy, ASR, fidelity, …) —
  evasion also adds `*_tpr`/`*_fpr`/`*_f1_score`/`*_auc_score` and
  `confidence_curve_clean`/`confidence_curve_adversarial` (a downsampled,
  sorted array for a client-side confidence-shift chart).
* `artifacts` carries supporting data (confusion matrices, poisoned
  CSVs, …).
* `plots` carries base64-encoded PNG sample images where applicable (image
  evasion: `sample_<i>_clean`/`sample_<i>_adversarial`, up to 6 pairs; image
  poisoning: `preview_clean`/`preview_trigger`) — empty for attacks that
  don't produce one. Exact keys present vary per attack family/data type;
  don't assume a fixed key set, check for presence.

The ZIP (default) unpacks this same content into files — always `manifest.json`,
`results.json`, `metrics.json`, plus whatever applies to the attack (confusion CSVs,
`poisoned_train.csv`, `plots/*.png`).

---

## 11. Errors

| Status | When |
|---|---|
| `202` | Attack didn't finish within the sync-wait window — see §9, poll `GET /api/jobs/{job_id}` |
| `400` | Missing required file slot · `params.attack` conflicts with the dropdown · invalid `data_type` · `params` JSON parse error (including `Infinity`/`NaN`, which are rejected) · pentest with no algorithm selected · poisoning `target_class`/`source_class` outside the model's class range |
| `404` | Unknown/expired `job_id` |
| `409` | `GET /api/jobs/{job_id}/result` called while the job is still `queued`/`running`, or after it was `cancelled` |
| `413` | Upload too large · dataset / image batch exceeds server limits · requested `image_size` too large · model has too many parameters |
| `422` | `params` JSON is wrong for the chosen algorithm — the response carries the exact field path (e.g. `params.fgsm.eps: Input should be less than or equal to 10`) |
| `500` | Unexpected server error |
| `504` | Job exceeded the server's wall-clock timeout (`GET /api/jobs/{job_id}` reports `status: "timeout"`) |

Note: attacks no longer get rejected outright when the server is at
`PTAI_MAX_CONCURRENT_ATTACKS` capacity — they queue as a background job
(`status: "queued"`) instead, so ordinary bursts don't produce a hard
failure the way a `429` used to.

The 413 / 504 caps are configurable by the operator of the deployment; if
you hit one, ask them to raise it or reduce your input size.

---

## 12. Authentication

Standard deployments require an API key. Send it on every attack request:

```bash
curl -H 'X-API-Key: your-key-here' ...
```

If you get a `401` or `403`, you need a key from the operator. Public
endpoints (`/`, `/healthz`, `/docs`, `/openapi.json`, `/guide`, `/demo`,
`/postman`, `/postman-import`, `/templates` (and its downloads), `/readme`)
never require a key. Every
`/api/attack/*` and `/api/jobs/*` endpoint requires it.

In Swagger (`/docs`), click **Authorize** (top right) and paste your key
once — every "Try it out" call then includes it automatically. In
Postman, the collection's own **Authorization** tab is pre-wired for an
API key — set the `apiKey` collection variable once and every request
inherits it.

### Browser login (separate, optional, deployment-dependent)

Some deployments also put a **username/password login gate** in front of the
*human-browsable* documentation pages — `/docs`, `/redoc`, `/openapi.json`,
`/guide`, `/postman-import`, `/templates`. The landing page (`/`) and the demo
(`/demo`) always stay public. If you're redirected to `/login` when
opening one of these in a browser, ask the operator for the browser-login
credentials (separate from your `X-API-Key`). A signed session cookie keeps you
in for 12 hours; use the "Logout" link on the landing page to end it early.

This only ever applies to pages you *browse*. Every `/api/*` endpoint, `/healthz`,
the raw Postman download (`/postman`), individual template downloads
(`/templates/config/{name}`, `/templates/params/{name}`), and `/readme` stay
reachable with no session — only the `X-API-Key` header applies to those, exactly
as described above.

---

## 13. Other endpoints

| Path | Purpose |
|---|---|
| `GET /` | Landing page with links to Swagger, the web console, Postman and this guide |
| `GET /healthz` | `{"ok": true}` liveness check |
| `GET /api/auth/whoami` | Check an API key: `{"ok": true}` with a valid `X-API-Key`, `401` otherwise |
| `GET /docs` | Swagger UI — easiest way to try a request |
| `GET /openapi.json` | Raw OpenAPI 3 schema (import into Postman / Insomnia / Bruno) |
| `GET /postman` | Download Postman collection JSON (if shipped) |
| `GET /templates` | **Attacks JSON Templates** page — explanation + per-attack downloads |
| `GET /templates/config.zip` · `GET /templates/params.zip` | Download all full-config / params-only templates as a ZIP |
| `GET /templates/config/{name}` · `GET /templates/params/{name}` | Download one template |
| `GET /guide` | This guide, rendered as HTML |
| `GET /readme` | Download this file as Markdown |
| `GET /demo` | Interactive demo with pre-computed attack results on built-in models |
| `GET /postman-import` | Postman import page |
| `GET /login` · `POST /login` · `GET /logout` | Browser login gate (if the operator enabled it) — see **§12** |
| `GET /api/jobs/{job_id}` · `GET /api/jobs/{job_id}/result` · `DELETE /api/jobs/{job_id}` | Poll / fetch / cancel a background attack job — see §9 |
| `GET /api/object-detection/catalog` · `GET /api/object-detection/images/{id}` | Object-detection models, attacks, compatibility and built-in sample images — see §14 |

---

## 14. Object detection

Evasion attacks from IBM ART against **object detectors**. Unlike the families
above you don't upload a model: you pick one of the **built-in detectors** and
attack a built-in **COCO sample** (12 openly licensed images with ground-truth
boxes) and/or **your own JPEG/PNG images**.

| `model` | Detector | Input · pixel range |
|---|---|---|
| `frcnn_mobilenet_v3_320` | Faster R-CNN MobileNetV3-Large 320 FPN (torchvision) — fastest | 320² · [0, 1] |
| `frcnn_resnet50_fpn` | Faster R-CNN ResNet-50 FPN (torchvision) — slowest | 512² · [0, 1] |
| `yolov5s` | YOLOv5s v7.0 (ultralytics) | 416² · [0, 255] |

| `attack` | ART class | Kind | Main `params` (defaults) |
|---|---|---|---|
| `fgsm` | FastGradientMethod | full image, L∞ | `eps` 8 |
| `pgd` | ProjectedGradientDescent | full image, L∞ | `eps` 8 · `eps_step` 2 · `max_iter` 10 |
| `dpatch` | DPatch | universal patch (top-left) | `patch_size` 64 · `learning_rate` 8 · `max_iter` 40 |
| `robust_dpatch` | RobustDPatch | patch + random crop/brightness/rotation | `patch_size` 64 · `patch_x`/`patch_y` 16 · `max_iter` 30 · `sample_size` 2 · `brightness` 0.2 · `crop` 8 · `rotations` false |
| `adv_patch` | AdversarialPatchPyTorch | patch (Adam, rotation) | `patch_size` 64 · `patch_x`/`patch_y` 16 · `learning_rate` 8 · `max_iter` 40 · `rotation_max` 10 |

**Pixel units:** `eps`, `eps_step` and `learning_rate` are always in **0–255
pixel units**, whatever the model — `eps: 8` means 8/255 on the Faster R-CNN
models and 8 on YOLOv5. Out-of-range values are clamped server-side (the
catalog lists every range).

### Request (multipart form)

| Field | Default | Notes |
|---|---|---|
| `model` | `frcnn_mobilenet_v3_320` | dropdown |
| `attack` | `pgd` | dropdown |
| `params` / `params_file` | `{}` | as in §2 — omitted keys use the defaults above |
| `images` | *(empty)* | comma-separated sample ids from the catalog, e.g. `000000206487.jpg,000000118209.jpg`; empty **and** no uploads → the first 4 samples |
| `files` | — | your own JPEG/PNG images (≤ 10 MB each; at most 8 images per run in total) |
| `label_source` | `ground_truth` | `clean_predictions` = attack/measure against the model's own clean detections (forced for uploads, which have no ground truth) |
| `score_threshold` | `0.5` | detections below it don't count for recall / precision / vanished boxes |
| `seed` | `0` | reproducibility |

```bash
# 1. what can I run?
curl -k https://127.0.0.1:8000/api/object-detection/catalog -H 'X-API-Key: your-key-here'

# 2. PGD on YOLOv5s against two COCO samples → ZIP
curl -k -X POST https://127.0.0.1:8000/api/attack/object-detection \
  -H 'X-API-Key: your-key-here' \
  -F model=yolov5s -F attack=pgd \
  -F 'params={"eps":8,"eps_step":2,"max_iter":10}' \
  -F images=000000206487.jpg,000000118209.jpg \
  -o od_result.zip

# 3. your own images, JSON envelope
curl -k -X POST 'https://127.0.0.1:8000/api/attack/object-detection?format=json' \
  -H 'X-API-Key: your-key-here' \
  -F model=frcnn_mobilenet_v3_320 -F attack=fgsm \
  -F files=@street.jpg -F files=@parking.png
```

Jobs work exactly as in §9: fast runs (seconds) return inline, slower ones
(ResNet-50, patch attacks with many iterations, several images) return `202` —
poll `GET /api/jobs/{job_id}`. `DELETE /api/jobs/{job_id}` also stops the
detector run at its next checkpoint (the current image / patch step finishes).

### What you get back

**ZIP (default)** — `ptai_object_detection_<attack>_<run>.zip`:

| File | Content |
|---|---|
| `result.json` | the full report (below) |
| `img_<i>_clean.png` · `img_<i>_adv.png` | the exact model inputs before / after the attack (letterboxed S×S, lossless) |
| `img_<i>_pert.png` | amplified perturbation heatmap |
| `img_<i>_clean_boxes.jpg` · `img_<i>_adv_boxes.jpg` | annotated copies with detections |
| `patch.png` | the learned patch (patch attacks) |
| `datasets/adversarial_images.npz` | `x` float32 NHWC in [0,1], `image_ids` |
| `datasets/benign_and_adversarial.npz` | clean then adversarial `x`, `is_adversarial`, `image_ids` |
| `datasets/predictions_clean.json` · `predictions_adversarial.json` | detections in COCO results format |

**`?format=json`** — the usual envelope with `family: "object_detection"`:

* `metrics.summary` — clean vs adversarial `map50`, `map` (mAP@[.5:.95]),
  `recall`, `precision`, `fp_per_image`, `detections`; `vanished` /
  `vanished_pct` (confident clean detections with no same-class IoU ≥ 0.5 match
  after the attack), `new_detections`, `label_changes`, and separate
  **`executed`** (ran within constraints) vs **`impact`** (detector measurably
  worse) flags plus a `verdict` sentence.
* `metrics.budget` (full-image attacks: ε and the *measured* max L∞) or
  `metrics.patch` (size, area %, location); `metrics.perturbation` (L∞/L2 in
  0–255 units, % pixels changed); `metrics.timings` (attack vs inference
  seconds); `metrics.per_class` (AP@50 clean vs adversarial).
* `artifacts.images[]` — per image: ground truth and clean / adversarial boxes
  (`[x1,y1,x2,y2]` in pixels of the letterboxed S×S model input, S =
  `metrics.model.input_size`), with category and score.

### Compatibility

Every model × attack cell has an evidence state taken from ptAI's compatibility
mapping (`documented example`, `API candidate`, …) and is marked
`validated: true` once ptAI's own smoke test passed for the installed versions —
see `cells` in the catalog. A cell that isn't runnable returns `422` with the
reason. *Overload* is listed in the matrix but not runnable yet.

Errors follow §11; in addition:

| Status | When |
|---|---|
| `413` | An uploaded image is larger than the per-file limit (10 MB) |
| `422` | More than 8 images in one run, or the model × attack cell isn't runnable |
| `429` | The detector service's queue is full — retry after a run finishes |
| `502` | The detector service failed (a run ended in error, or the API couldn't authenticate to it — operator configuration) |
| `503` | Object detection isn't enabled on this deployment, or its service is unreachable |