What to measure in a pilot
OpsCodex is a young product, built inside Mews Partners — a consultancy that has advised aerospace, automotive and energy manufacturers on PLM and process governance since 1992, with around 350 consultants across seven offices. The product is young; the process expertise behind it is not. That still leaves a real procurement question, and the honest way to handle it is not a louder pitch — it is telling you exactly how to test the claims. Below are the seven questions a careful buyer asks, what we can answer today, and what only a pilot on your own processes can settle.
1. How accurately does it read your real SOPs?
This is the question that decides everything else, and it is the one we will not answer with a number. Extraction quality depends on how your documents are written — a structured SOP with numbered steps and named roles behaves nothing like a slide deck of boxes and arrows. Any vendor quoting a single accuracy figure is quoting it against their own corpus, not yours.
How to measure it: pick five documents that are genuinely representative, including one you consider badly written. Have a process owner who knows the real process mark up the drafts. Count what matters separately: roles, decisions (gateways), exception paths and loops. Exception paths and loops are where document-to-model conversion usually degrades — the happy path is the easy part.
What we can tell you: AI output is a draft, never an approved model. The Process Advisor checks each draft against BPMN conformance and naming rules and offers one-click fixes, so structural errors surface before review rather than during it. The remaining question — semantic correctness — is exactly what the mandatory human review exists for.
2. How much modelling time does it actually save?
We publish no time-saving figure, because we have no measurement from your environment and a number from ours would not transfer.
How to measure it: take the same five processes and model two of them by hand, the way you do today, with the clock running. Then run all five through OpsCodex and time the path from ingest to approved. Measure the whole path, not the drafting step — the draft is fast in any AI tool; what varies between tools is how much rework the draft causes and how long approval takes once the reviewers are named.
3. Do the exports open cleanly in your target tool?
OpsCodex models in BPMN 2.0 and exports standards-conformant BPMN XML as specified by the OMG. The intent is that approved processes stay portable: any BPMN-conformant tool should read what OpsCodex writes.
How to measure it: export the approved models from your pilot and open them in whatever you actually use — Camunda, Signavio, ARIS, a bpmn.io-based viewer. Check that the diagram layout survives, not just the semantics. Layout information travels in the BPMN DI section, and tools differ in how faithfully they honour it. Do this in the pilot rather than after the rollout.
4. Is the audit trail tamper-proof, or just read-only in the UI?
A fair question, and the two are very different. In OpsCodex the
append-only guarantee is enforced in the database, not in application code:
a trigger on the audit table rejects every UPDATE and DELETE, and
the database role the application runs under has those rights revoked. The application
cannot rewrite its own history even if it is compromised.
The boundary, stated plainly: the trigger covers row-level modification
and deletion. It does not cover TRUNCATE, and no application-level control
survives someone with superuser access to your own database server. The full description is
on the security page.
How to verify it: in your pilot instance, connect as the application's own database user and try to update or delete an audit row. It should fail. That is a five-minute check and worth doing yourself rather than believing this page.
5. Does it really run in your environment?
OpsCodex is deployed single-tenant and on-premise. The documented minimum for a pilot host:
- Host
- Linux (tested on Debian/Ubuntu), 2 vCPUs and 4 GB RAM minimum — 4 vCPUs and 8 GB recommended once AI features are in regular use. 20 GB free disk for database, uploads and documentation content.
- Runtime
- Docker 24+ with Compose v2. A single host is enough for a pilot; no Kubernetes cluster, no managed services.
- Network
- Reachable on your LAN. No inbound internet access is required. The only outbound dependency is AI inference — and that goes to the provider you choose, with your key, or to a private endpoint you host yourself.
- Identity
- SSO against your own identity provider over SAML/OIDC, with MFA enforced before a request reaches the application. Test this with your real IdP during the pilot, not with local accounts — it is where on-premise rollouts usually stall.
Those numbers are deliberately small. The point of on-premise is that you provision the host, so the less it asks for, the sooner your infrastructure team can say yes — no cluster to stand up, no managed services to procure. What happens after the pilot is the fairer question, and the honest answer is that the operating discipline is documented while the scale-out story is not:
- Backup and restore
- A backup script writes a timestamped snapshot of the database, the uploads store and the configuration, with sizes and SHA-256 sums recorded in a manifest. Daily at minimum, stored off the deployment host. The restore procedure is written out step by step in the on-premise runbook — ask for it during the pilot and rehearse it then, rather than reading it for the first time during an incident.
- Upgrades and rollback
- Versions are pinned rather than floating, and a backup is taken automatically before every update — the update flow depends on that backup reporting success, so it will not proceed on top of a failed one. That is what makes a rollback an actual procedure rather than a hope.
- Multi-node and failover
- Not something we offer today. What is documented and supported is a single host. If your availability requirement goes beyond that, raise it before the pilot rather than after it — it changes the shape of the deployment conversation.
6. What does it cost in total?
There is no published price list, and rather than invent one we will describe what the cost is made of. Four blocks, two of which you can size yourself today:
- Infrastructure — you can calculate this
- Your own hardware or VM, to the specification above. No per-seat infrastructure cost, no external hosting.
- AI usage — you control and see it directly
- OpsCodex uses your provider account and your key. Token cost is billed to you by Anthropic, OpenAI or Google directly — not marked up through a licence — so you can cap it, monitor it and switch provider without asking us. If you self-host a model, the cost is your own compute.
- Licence — on request
- Quoted per engagement. We would rather scope it against your actual deployment than publish a number that fits nobody.
- Introduction — on request
- Depends on how many processes you start with and how much of the governance model already exists in your organisation.
7. What happens if the vendor disappears?
The standard question for a small supplier, and it deserves a direct answer rather than reassurance. Two of the three usual fears do not apply here, for structural reasons rather than promises:
- The system keeps running
- OpsCodex runs on your hardware, inside your perimeter. A vendor failure does not switch anything off — unlike a SaaS product, where it would be an immediate outage.
- Your process knowledge leaves with you
- Models export as BPMN 2.0 XML, an open OMG standard readable by any conformant tool. Documentation and audit records are exportable at any time. For a governance tool this matters more than access to the source code.
- Continued maintenance
- This is the one the first two do not cover. Talk to us about it as part of contracting rather than treating the answer above as complete.
If a question you need answered is not on this list, ask it: [email protected]. Questions that turn out to be general get added to this page.