Skip to main content

Metrics Reference

This document describes all Prometheus metrics exposed by OmniUPF on the /api/metrics endpoint (the HTTP API runs on api_port, default 8080). Note the /api prefix - every API route is served under it.

All metrics are declared in lib/upf_ex/prometheus_metrics.ex. Metric names, types, and labels below are taken directly from that source; if the code changes, update this file to match.

Metric Categories​

  1. PFCP metrics - control-plane message, association, session, and error counters
  2. Session lifecycle metrics - session create/delete/modify counters and establishment latency
  3. Packet buffering metrics - buffer state and throughput for idle-UE downlink buffering
  4. Downlink Data Report (DLDR) metrics - PFCP Session Report Request notifications and FAR index tracking
  5. GTP-U metrics - echo responses and Error Indication counters
  6. URR metrics - usage-report counters and active URR timers
  7. eBPF map metrics - dataplane map entry gauges
  8. Dataplane throughput - per-direction (N3/N6) counters, on-demand via /api/v1/xdp_stats only (not a scrape metric — see #57)
  9. Resource / routing metrics - TEID/FAR-ID pool and UE route gauges
  10. Walled Garden metrics - captive-portal interception, forwarding, and DNS spoofing counters

PFCP metrics​

Control-plane protocol counters between the UPF and its PFCP peers (SMF / PGW-C / SGW-C).

Metric NameTypeLabelsDescription
upf_pfcp_associationsGaugenoneCurrently active PFCP associations (all peers)
upf_pfcp_sessionsGaugenoneCurrently active PFCP sessions (all peers)
upf_pfcp_peer_restarts_totalCounterpeerPeer restart detections (Recovery Timestamp change)
upf_pfcp_duplicate_requests_totalCounternoneDuplicate PFCP requests detected and suppressed
upf_orphaned_sessions_deleted_totalCounternoneSessions deleted as a result of a peer restart
upf_pfcp_messages_rx_totalCountertype, peerPFCP messages received per message type and peer
upf_pfcp_messages_tx_totalCountertype, peerPFCP messages sent per message type and peer
upf_pfcp_errors_totalCountertypePFCP request rejections, labelled by the reject cause (e.g. session_context_not_found, mandatory_ie_missing)

Session lifecycle metrics​

Metric NameTypeLabelsDescription
upf_sessions_created_totalCounternonePFCP sessions established
upf_sessions_deleted_totalCounternonePFCP sessions deleted
upf_sessions_modified_totalCounternonePFCP sessions modified
upf_session_establishment_duration_microsecondsHistogramnoneSession establishment processing time (microseconds). Buckets: 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000

Packet buffering metrics​

The UPF buffers downlink packets when a UE is idle, holding them until the UE is paged and transitions to connected. Packets arrive from the eBPF dataplane on the buffer UDP port (default 22152).

Metric NameTypeLabelsDescription
upf_buffer_packets_received_totalCounternonePackets received on the buffer port from the dataplane
upf_buffer_packets_buffered_totalCounternonePackets successfully added to the buffer
upf_buffer_packets_flushed_totalCounternonePackets flushed from the buffer
upf_buffer_packets_dropped_totalCounterreasonPackets dropped from / not admitted to the buffer
upf_buffer_packets_currentGaugenonePackets currently held in the buffer
upf_buffer_flush_operations_totalCounternoneSuccessful buffer flush operations (per FAR flush)
upf_buffer_flush_packets_sent_totalCounternonePackets sent during flush operations
upf_buffer_flush_errors_totalCounterreasonFailed buffer flush operations

upf_buffer_packets_dropped_total reason values (from buffer-add drops and buffer-listener errors):

  • global_limit - total buffer capacity (buffer_max_total) reached
  • far_limit - per-FAR buffer limit (buffer_max_per_far) reached
  • expired - TTL (buffer_ttl_ms) exceeded before flush
  • read_error - error reading from the buffer socket
  • too_small - packet too small for a GTP header
  • invalid_gtp_type - non-G-PDU GTP message type
  • unknown_teid - no PDR/FAR found for the TEID
  • not_buffering_far - FAR does not have a BUFF action

upf_buffer_flush_errors_total reason values:

  • far_lookup_failed - could not look up FAR info from the eBPF map
  • no_forw_action - FAR does not have a FORW action set
  • connection_failed - failed to open a UDP socket for flushing

Metrics for PFCP Session Report Request notifications sent to the control plane when downlink packets are buffered. These notifications trigger the SMF/PGW-C to page the UE.

Metric NameTypeLabelsDescription
upf_dldr_sent_totalCounterpeerDownlink Data Reports sent, labelled by SMF/PGW-C address
upf_dldr_errors_totalCounternoneDLDR encoding / send errors
upf_dldr_skipped_totalCounternoneDLDRs skipped because the FAR was already notified
upf_far_index_sizeGaugenoneFARs registered in the FarIndex for DLDR notification

GTP-U metrics​

GTP-U path-management and Error Indication counters. Error Indications are exchanged when a peer receives packets for an unknown TEID (typically after a restart).

Metric NameTypeLabelsDescription
upf_gtpu_echo_responses_tx_totalCounternoneGTP-U Echo Responses sent
upf_gtpu_error_indications_sent_totalCounternoneGTP-U Error Indications sent (for unknown incoming TEIDs)
upf_gtpu_error_indications_rx_totalCounternoneGTP-U Error Indications received from peers
upf_error_indication_sessions_deleted_totalCounternoneSessions deleted as a result of a received Error Indication

When Error Indications are sent: the UPF receives a GTP-U packet for a TEID that does not exist (e.g. after a restart, or the session was already deleted), and informs the sender to stop.

When Error Indications are received: a downstream peer does not recognise a TEID the UPF forwarded to; the UPF deletes the affected session to stop forwarding to a dead tunnel.

URR (Usage Reporting Rule) metrics​

Metric NameTypeLabelsDescription
upf_urr_reports_sent_totalCountertriggerURR usage reports sent, labelled by reporting trigger
upf_urr_report_errors_totalCounternoneURR report encoding / send errors
upf_urr_active_timersGaugenoneActive URR timers (periodic / time-threshold / quota-validity)

trigger reflects the URR reporting trigger that fired - e.g. periodic (PERIO), volume_threshold (VOLTH), time_threshold (TIMTH), quota_validity (QUVTI).

Per-URR byte counters are not exported as Prometheus metrics (to avoid high cardinality). Read individual URR statistics via the REST API at /api/v1/urr_map.

eBPF map metrics​

Gauges for dataplane map utilisation, refreshed on each scrape.

Metric NameTypeLabelsDescription
upf_ebpf_far_entriesGaugenoneFAR entries in the eBPF map
upf_ebpf_pdr_uplink_entriesGaugenoneUplink PDR entries in the eBPF map (PDRs with a TEID)
upf_ebpf_pdr_downlink_entriesGaugenoneDownlink PDR entries in the eBPF map (PDRs keyed by UE IP)

upf_ebpf_far_entries mirrors the allocated-FAR-id pool (upf_far_ids_allocated), since the far_map holds one entry per allocated FAR id. The PDR gauges are aggregated from live session state on each scrape (cost is O(total sessions)).

Dataplane throughput (on-demand only — not a Prometheus scrape metric)​

Per-direction packet and byte counters for user-plane traffic (N3 = GTP-U side towards the gNB/eNB; N6 = SGi side towards the data network) come from the eBPF upf_ext_stat percpu map.

Removed from the Prometheus scrape (issue #57). These were previously exported as upf_n3_* / upf_n6_* gauges, populated on every /api/metrics scrape by a bpf_map_lookup_elem on the upf_ext_stat map — the same per-CPU map the XDP datapath writes on every packet. That per-scrape read was proven to trigger a virtio_net generic-XDP TX-ring wedge (with the scrape blocked, the exact breaking load ran 30/30 cycles with zero NETDEV WATCHDOG). To keep the metrics path from ever touching the datapath, these gauges are no longer collected or exported.

How to read live N3/N6 counters now: GET /api/v1/xdp_stats returns the same fields as JSON, and only touches eBPF when you explicitly call it (not on a timer, not on a Prometheus scrape). Fields: rx_n3 / tx_n3 / rx_n6 / tx_n6 (packets) and rx_n3_bytes / tx_n3_bytes / rx_n6_bytes / tx_n6_bytes.

Durable per-direction throughput in Prometheus should instead come from the substrate fix that lets the datapath be observed safely (see #57), or from an external counter (e.g. the host/tap byte counters).

Resource / routing metrics​

Metric NameTypeLabelsDescription
upf_routes_totalGaugenoneUE IP routes currently tracked by the route manager
upf_teid_allocatedGaugenoneTEIDs currently allocated from the pool
upf_far_ids_allocatedGaugenoneFAR IDs currently allocated from the pool

Walled Garden metrics​

Counters for the out-of-credit / captive-portal walled garden. See the Walled Garden Guide for behaviour.

Metric NameTypeLabelsDescription
upf_walled_garden_active_redirectsGaugenoneActive walled-garden redirect sessions
upf_walled_garden_packets_intercepted_totalCounternonePackets intercepted by the walled garden
upf_walled_garden_packets_dropped_totalCounternonePackets dropped by the walled garden
upf_walled_garden_packets_forwarded_totalCounterdst_ipWhitelisted packets forwarded, per destination IP
upf_walled_garden_bytes_forwarded_totalCounterdst_ipWhitelisted bytes forwarded, per destination IP
upf_walled_garden_dns_spoofed_totalCounterdomainDNS queries spoofed to the portal, per domain
upf_walled_garden_dns_forwarded_totalCounterdomainDNS queries forwarded (whitelisted domains), per domain

Cardinality note. The dst_ip and domain labels are unbounded. Keep the whitelist tight, and aggregate away these labels in recording rules if you scrape a busy walled garden.

Using Prometheus Metrics​

Accessing Metrics​

Metrics are exposed on the /api/metrics endpoint of the HTTP API server, which listens on api_port (default 8080). Note the /api prefix - every API route is served under it:

# View raw metrics
curl http://localhost:8080/api/metrics

# Example output
upf_pfcp_sessions 42
upf_pfcp_associations 2
upf_dldr_sent_total{peer="10.100.50.241"} 17

Prometheus Configuration​

Add the OmniUPF target to your prometheus.yml:

scrape_configs:
  - job_name: 'omniupf'
    metrics_path: /api/metrics
    static_configs:
      - targets: ['localhost:8080']

Grafana Dashboards​

Useful panels to build from these metrics:

  • Session and association counts (upf_pfcp_sessions, upf_pfcp_associations)
  • Per-direction N3/N6 throughput is available on demand via /api/v1/xdp_stats (removed from the scrape — see #57)
  • Buffer pressure (upf_buffer_packets_current, drop rate by reason)
  • DLDR / paging activity (rate(upf_dldr_sent_total[5m]))
  • Error Indication and peer-restart rates for tunnel-health alerting
Last updated on