What to measure in a pilot

OpsCodex is a young product, built inside Mews Partners — a consultancy that has advised aerospace, automotive and energy manufacturers on PLM and process governance since 1992, with around 350 consultants across seven offices. The product is young; the process expertise behind it is not. That still leaves a real procurement question, and the honest way to handle it is not a louder pitch — it is telling you exactly how to test the claims. Below are the seven questions a careful buyer asks, what we can answer today, and what only a pilot on your own processes can settle.

1. How accurately does it read your real SOPs?

This is the question that decides everything else, and it is the one we will not answer with a number. Extraction quality depends on how your documents are written — a structured SOP with numbered steps and named roles behaves nothing like a slide deck of boxes and arrows. Any vendor quoting a single accuracy figure is quoting it against their own corpus, not yours.

How to measure it: pick five documents that are genuinely representative, including one you consider badly written. Have a process owner who knows the real process mark up the drafts. Count what matters separately: roles, decisions (gateways), exception paths and loops. Exception paths and loops are where document-to-model conversion usually degrades — the happy path is the easy part.

What we can tell you: AI output is a draft, never an approved model. The Process Advisor checks each draft against BPMN conformance and naming rules and offers one-click fixes, so structural errors surface before review rather than during it. The remaining question — semantic correctness — is exactly what the mandatory human review exists for.

2. How much modelling time does it actually save?

Our own test: on average about 70 percent less time to a model ready for review. How much you save on the whole path to approval only a measurement with your own processes can show.

How to measure it: take the same five processes and model two of them by hand, the way you do today, with the clock running. Then run all five through OpsCodex and time the path from ingest to approved. Measure the whole path, not the drafting step — the draft is fast in any AI tool; what varies between tools is how much rework the draft causes and how long approval takes once the reviewers are named.

3. Do the exports open cleanly in your target tool?

OpsCodex models in BPMN 2.0 and exports standards-conformant BPMN XML as specified by the OMG. The intent is that approved processes stay portable: any BPMN-conformant tool should read what OpsCodex writes.

How to measure it: export the approved models from your pilot and open them in whatever you actually use — Camunda, Signavio, ARIS, a bpmn.io-based viewer. Check that the diagram layout survives, not just the semantics. Layout information travels in the BPMN DI section, and tools differ in how faithfully they honour it. Do this in the pilot rather than after the rollout.

4. Is the audit trail tamper-proof, or just read-only in the UI?

A fair question, and the two are very different. In OpsCodex the append-only guarantee is enforced in the database, not in application code: a trigger on the audit table rejects every UPDATE and DELETE, and the database role the application runs under has those rights revoked. The application cannot rewrite its own history even if it is compromised.

The boundary, stated plainly: the trigger covers row-level modification and deletion. It does not cover TRUNCATE, and no application-level control survives someone with superuser access to your own database server. The full description is on the security page.

How to verify it: in your pilot instance, connect as the application's own database user and try to update or delete an audit row. It should fail. That is a five-minute check and worth doing yourself rather than believing this page.

5. Does it really run in your environment?

OpsCodex is deployed single-tenant and on-premise, in one of two ways: Docker Compose on a single host, or a Helm chart on your Kubernetes platform. What each needs:

Host for Docker Compose: Linux (tested on Debian/Ubuntu), 2 vCPUs and 4 GB RAM minimum — 4 vCPUs and 8 GB recommended once AI features are in regular use. 20 GB free disk for database, uploads and documentation content.
Runtime: Docker 24+ with Compose v2. A single host is enough for a pilot — no cluster to stand up, no managed services to procure.
Kubernetes: Kubernetes 1.25 or later, Helm 3.8 or later. The chart installs into one namespace with no cluster-wide rights, runs every container as a non-root user on a read-only root filesystem, and is tested with Pod Security restricted enforced. PostgreSQL and S3 come from your platform or run in the same namespace. The details are on the security page.
Network: Reachable on your LAN. No inbound internet access is required. The only outbound dependency is AI inference — and that goes to the provider you choose, with your key, or to a private endpoint you host yourself.

The Compose numbers are deliberately small: the less a pilot host asks for, the sooner your infrastructure team can say yes. What happens after the pilot is the fairer question:

How many people one installation carries: Measured, not estimated. In a load test of the Kubernetes chart at its default resource limits, 700 people working at the same time — active in the same minutes, not accounts; 70 % reading the portal, 25 % modelling, 5 % exporting — stayed within every threshold fixed before the first run: 95 % of interactive calls under half a second, pages under two seconds, exports under 15 seconds, fewer than 1 % errors. The first limit reached was the backend's CPU; memory and the database never came close. Reading documents in with AI was not part of the test, because generation runs on your model service, not inside OpsCodex.
Backup and restore: With Docker Compose, a backup script writes a timestamped snapshot of the database, the uploads store and the configuration, with sizes and SHA-256 sums recorded in a manifest. Daily at minimum, stored off the deployment host. The restore procedure is written out step by step in the on-premise runbook — ask for it during the pilot and rehearse it then, rather than reading it for the first time during an incident. On Kubernetes, backups are part of your platform's routine: a snapshot of the database and the object storage, taken together.
Upgrades and rollback: Versions are pinned rather than floating. With Docker Compose a backup is taken automatically before every update — the update flow depends on that backup reporting success, so it will not proceed on top of a failed one. On Kubernetes, helm upgrade first runs the database migration as a Job while the old version keeps serving; if the migration fails, nothing else changes. Every release's notes state whether it can be rolled back without restoring the backup. That is what makes a rollback an actual procedure rather than a hope.
Scaling out and failover: On Kubernetes the web tier scales horizontally. The backend runs as exactly one instance on both platforms: an update replaces it, which interrupts the API for a few seconds — 5 to 12 in our measurements — while the user interface keeps being served. Failover for the database and the object storage comes from your platform's services. If your availability requirement goes beyond that, raise it before the pilot rather than after it — it changes the shape of the deployment conversation.

6. What does it cost in total?

There is no published price list. Here is what the cost is made of: four blocks, two of which you can size yourself today:

Infrastructure — you can calculate this: Your own hardware, VM or Kubernetes platform, to the specification above. No per-seat infrastructure cost, no external hosting.
AI usage — you control and see it directly: OpsCodex uses your provider account and your key. Token cost is billed to you by Anthropic, OpenAI or Google directly — not marked up through a licence — so you can cap it, monitor it and switch provider without asking us. If you self-host a model, the cost is your own compute.
Licence — on request: Quoted per engagement. We would rather scope it against your actual deployment than publish a number that fits nobody.
Introduction — on request: Depends on how many processes you start with and how much of the governance model already exists in your organisation.

7. What happens if the vendor disappears?

The standard question for a small supplier, and it deserves a direct answer rather than reassurance. Two of the three usual fears do not apply here, for structural reasons rather than promises:

The system keeps running: OpsCodex runs on your hardware, inside your perimeter. A vendor failure does not switch anything off — unlike a SaaS product, where it would be an immediate outage.
Your process knowledge leaves with you: Models export as BPMN 2.0 XML, an open OMG standard readable by any conformant tool. Documentation and audit records are exportable at any time. For a governance tool this matters more than access to the source code.
Continued maintenance: This is the one the first two do not cover. Talk to us about it as part of contracting rather than treating the answer above as complete.

If a question you need answered is not on this list, ask it: [email protected]. Questions that turn out to be general get added to this page.