Running OmniRoam Tests
This guide covers the full lifecycle of a roaming test: creating a book, running it, watching the cases execute, customising which cases run, re-running a single case, and scheduling routine unattended runs.
- The Test Lifecycle
- Creating a Test Book
- Running a Book
- How Automated Cases Are Driven
- Customising a Run
- Re-running a Single Case
- Routine and Scheduled Runs
- Reading the Results
The Test Lifecycle
Creating a Test Book
The Roaming Test Books screen lists every book with its type, the home and visited networks, the subscriber identity, the tester, and the current status.
A test book is created from the Roaming Test Books screen. When creating a book you choose:
| Field | Purpose |
|---|---|
| Book type | Which GSMA book to run — IR.38, IR.48, or VoLTE. This selects the case set. |
| Roaming partner / PLMNs | The home (PLMN a) and visited (PLMN b) networks. Used to populate the document header and to pick the correct throughput threshold. |
| SIM | The subscriber SIM to test with. Its IMSI and MSISDN are read and used throughout the run and in the final document. |
| Tester details | Name, phone, and email of the person responsible — written into the completed workbook. |
Once created, the book lists every case from the chosen GSMA book, each starting in a "not yet performed" state.
Running a Book
Running a book performs each in-scope case in turn. Before the test cases run, OmniRoam readies the test device on the network — presenting the SIM and confirming the device has attached and has service. This is a prerequisite gate: if the device cannot get service, the dependent cases are reported as blocked rather than failed, because there is no point running a data or voice test with no bearer.
As part of readying the device, OmniRoam reads the subscriber identity off the modem and fills the book automatically: the IMSI is read from the SIM, the home network (HPLMN) is derived from it, and the visited network (VPLMN) is read from the network the device is currently camped on. These populate the book's header — and therefore the completed document — with what was actually tested, rather than relying on values typed in by hand.
While a book runs, the screen shows each step as it happens, with the current step highlighted and a live pass/fail indication per case.
How Automated Cases Are Driven
Each automated case drives the test device, observes what the network does, and compares the outcome against the GSMA expected result. A case only passes on positive evidence — a step that cannot be confirmed is recorded as a failure with the reason, never a silent pass.
Data sessions and throughput
For data cases, OmniRoam establishes a packet data session, runs traffic to measure the achieved download/upload rate and latency, then disconnects so the network writes a clean charging record for the session. The data volumes and timings are read back from the serving gateway's charging records, and — for the IR.38 speed case — the measured rate is asserted against the throughput floor for the home/visited distance.
Voice calls
OmniRoam places a call from the device to a lab answering service, confirms the call reaches the active (answered) state, optionally holds it to confirm stability, then clears it. The verdict records the time to connect and whether the call stayed up.
SMS
OmniRoam sends a message with a unique body to a lab answering service and waits for the reply. Delivery of the reply confirms mobile-originated and mobile-terminated SMS end-to-end. A missing reply is a clear failure with the reason, not a crash.
Supplementary services (call barring and forwarding)
Supplementary services are provisioned over the air from the test device itself — exactly as a subscriber would using the standard service control codes — and then exercised:
For Call Forwarding on No Reply, OmniRoam registers the forwarding to a target number with a ring timer, then interrogates the service and confirms the network stored the forward-to number and timer that were registered.
Cancel Location (IR.38 3.1.2)
Some cases are driven from the network side rather than the device. For LTE Cancel Location, OmniRoam asks the home subscriber server to cancel the subscriber's location on the visited network, and records a pass when the request is accepted — confirming the home network can withdraw a roaming registration.
IMS registration (VoLTE)
For VoLTE, OmniRoam first confirms the device is registered to IMS. This gates the IMS-dependent cases: if registration is not confirmed, the VoLTE call and SMS-over-IMS cases are reported as blocked.
Customising a Run
A run does not have to cover the whole book. Using the test picker, the operator selects exactly which cases to run. This is useful for:
- Re-validating only the cases that previously failed.
- Running just the voice cases, or just the data cases, during fault-finding.
- Skipping cases that depend on an arrangement not yet in place (for example, a CAMEL agreement).
Cases left out of the scope keep their existing verdict and are untouched.
Re-running a Single Case
Every case has a re-run control. Selecting it runs just that one case again, against the same device and SIM, and updates only that case's verdict. This is the fastest way to confirm a fix — for example, re-running the throughput case after a transport change — without repeating the whole book.
Routine and Scheduled Runs
OmniRoam can run tests unattended on a schedule so a roaming relationship is continuously validated rather than checked once.
- Routine checks run a compact set of health checks against a device — data, speed, voice, and SMS — and report a combined pass/fail. These are the same drives used by the full books, packaged as a quick recurring confidence check.
- Schedules run a chosen routine on a recurring interval automatically. Each scheduled run records its own result, building a history over time so a regression (for example, throughput dropping below the floor, or SMS delivery failing) is caught and visible without anyone running a test by hand.
Setting Up a Schedule
Schedules are managed on the Schedules screen under Roaming Tests. A schedule defines:
| Setting | Purpose |
|---|---|
| Name | Identifies the schedule (and labels its metrics and history). |
| Modem(s) | The test device(s) the routine runs against. |
| Checks | Which checks the routine performs — data, speedtest, voice, SMS. |
| Interval | How often the routine runs (for example, every hour). |
Once saved, the schedule runs automatically at its interval with no further interaction. The Routine Test screen shows the run history — each run's combined pass/fail, the measured speeds, and the per-check detail — and a failing run surfaces the same evidence as an interactive run.
Prometheus Monitoring
Scheduled routine results are exported as Prometheus metrics, so roaming
health can be alerted on and dashboarded alongside the rest of the network — the
same way the scheduled tests on test devices are monitored. The exposition is
served at /api/ir38/routine/metrics (reads the latest scheduled run per
schedule and modem):
| Metric | Type | Meaning |
|---|---|---|
omniweb_ir38_routine_pass | gauge | Latest scheduled routine result (1 = pass, 0 = fail) |
omniweb_ir38_routine_check_pass | gauge | Per-check result of the latest run (label check: data/speedtest/voice/sms) |
omniweb_ir38_routine_download_mbps | gauge | Download rate measured on the latest run |
omniweb_ir38_routine_upload_mbps | gauge | Upload rate measured on the latest run |
omniweb_ir38_routine_latency_ms | gauge | Latency measured on the latest run |
omniweb_ir38_routine_last_run_timestamp_seconds | gauge | Unix time of the latest scheduled run |
Every series carries schedule, serial, modem, and imsi labels (plus
check on the per-check metric), so you can alert per schedule, per device, or
per home subscriber. The endpoint needs no login (scrapers can't carry one) and
can be gated with a bearer token when required.
Example alert: roaming voice has been failing on a device for 15 minutes:
omniweb_ir38_routine_check_pass{check="voice"} == 0
Scheduled results accumulate into a history that can be reviewed at any time, and a failing run surfaces the same evidence as an interactive run.
Reading the Results
Each case shows its verdict — PASS, FAIL, or NOT PERFORMED — together with the evidence behind it:
- PASS — the GSMA expected result was confirmed, with the supporting figures.
- FAIL — the expected result was not met; the reason is recorded (for example, "throughput below floor", "no echo reply", or the network release cause for a call that did not connect).
- NOT PERFORMED — the case was not run, or is a manual case awaiting a tester.
When the results look right, generate the completed document — see Completed Test-Book Documents.