# Arkheia — Verify It Yourself *A take-away self-test plan for the technical evaluator. The point of this document: don't take a vendor's word, and don't take a vendor's number — run the signal on your own prompts. Same facts and maturity labels as the diligence page at https://synesis.arkheia.ai/site/diligence/.* Verify the way the product is sold: **from the capability down.** The question that matters is "does it do useful AI work I can trust?" — so test a workflow, watch the detection signal gate it, and check the governance path leaves receipts. The detection layer is the moat underneath; the demo lets you probe it directly. We publish **no** precision / recall / accuracy / effect-size figures. That is intentional: numbers you cannot reproduce on your own traffic are not evidence. Here is how to produce your own. **The shape to aim for — a live customer POC.** A current evaluation runs the way you can run yours: an AI support-triage agent on a support agent's **own laptop**, against a **real Freshdesk queue and knowledge base**, on **GPT-5.4**, with **Arkheia detection** scoring each invocation and output written as **private internal triage notes** with a human in the loop. It is a **POC / pilot, not a production deployment** — exactly the posture you'd start in. The single observed result there (a ~3-hour triage task dropping to ~2 minutes) is *their* datapoint on *their* queue; the point of the steps below is to produce *your own*. --- ## Step 1 — Try the live demo (now) — **live** 1. Open **https://synesis.arkheia.ai/demo/**. 2. Run your own prompts — including ones you *know* are good and ones you *know* are bad/fabricated. 3. Observe, for each invocation: - the **risk verdict** (e.g. LOW / MEDIUM / HIGH), - the **confidence**, - the **detection id** (a reference you can cite). 4. Deliberately hit a model with **no profile** and confirm you get an **evidence-limited LOW** — i.e. *"couldn't assess,"* not a pass. Seeing the system admit the limit is part of the test. The browser calls a **public Arkheia backend proxy**, not the model vendor directly. You can also ask for the **earthworks movement-control walkthrough** (available on request) to see a governed decision gate (approve / block / hold for review with a decision receipt) on a real-world workflow: earthworks movement control + handwritten-docket ↔ invoice reconciliation. *(demo-grade)* --- ## Step 2 — Run it in your own stack (in your stack) — **self-serve** Both surfaces ship from one public repo: **https://github.com/arkheiaai/arkheia-mcp**. Self-serve is the primary route. - **MCP Trust Server** — one-line install, ~60-second installer (needs Node 18+, Python 3.10+, and an API key). Detection / verification, memory, and audit as MCP tools for your agent or IDE: ``` npx @arkheia/mcp-server ``` - **API / Enterprise Proxy (self-host)** — clone the repo and bring up the in-path proxy, then point your SDK base URL at the local proxy so inference flows through and is scored **in your own environment** (secure transit — prompts and data don't leave). The proxy is a thin shim; detection logic runs server-side. Fail-open: a detection outage degrades the signal, never blocks inference. ``` git clone https://github.com/arkheiaai/arkheia-mcp cd arkheia-mcp docker compose up ``` **Free tier: 1,500 detections / month, no credit card.** Mint a free key at **https://app.arkheia.ai/signup**. For higher-tier or managed evaluation keys, contact Arkheia — the public download above is the primary, self-serve route. No internal endpoints, tokens, or auth flows are published — none are public. For maximum data isolation, the self-hosted proxy keeps the in-path components on your own infrastructure so prompts and data never cross a shared backend. --- ## Step 3 — Measure it on your own prompts With access to your own traffic, the things worth measuring: - **Separation on labelled cases.** Build a small set of known-good and known-bad (fabricated / off-baseline) invocations for *your* models, and see how the verdicts and confidences separate them. This is the number that matters — and it is *yours*, not ours. - **Coverage map.** Which of your models have profiles (real verdicts) vs return evidence-limited LOW. This tells you where the signal is informative today. - **Workflow on your own data.** Point a governed agent at a real workflow of yours (a triage queue, a reconciliation, a CRM action) and judge the *work*, not just the verdict — is the output useful, and does the detection signal + gate sit on the path before the work lands? - **Earned-autonomy path.** Confirm authority is staged and earned, not granted: an agent starts at critique-only / private-note and only moves to bounded → supervised → full auto → workflow ownership as trust is demonstrated, with the gate on the path at each step. Verify you can hold it at a lower stage. - **Gate behaviour.** Exercise allow / block / require-approval / kill-switch / rollback and confirm each leaves a **tamper-evident decision receipt**. - **Fail-open behaviour.** Simulate a detection outage and confirm the API proxy still lets inference flow (degrades, does not block). - **Data posture.** Confirm the content-minimising / zero-retention behaviour against your own data-handling requirements; in the self-hosted pattern, confirm nothing crosses a shared backend. --- ## What "good" looks like for you You don't need our metrics. You need: clear separation between your good and bad cases, an honest coverage map (with evidence-limited LOW shown where it applies), a gate that acts and leaves receipts, and a data path that matches your environment. If you can reproduce those, the claims on the diligence page hold up under your own test — which is the only test that counts. --- ## Reachable today - Live detection demo: https://synesis.arkheia.ai/demo/ - Diligence pack: https://synesis.arkheia.ai/site/diligence/ - Home: https://synesis.arkheia.ai/site/ - Public repo (MCP server + self-host proxy): https://github.com/arkheiaai/arkheia-mcp - Free key (1,500 detections/month, no card): https://app.arkheia.ai/signup