OmniUPF Architecture Guide

Overview
eBPF Technology Foundation
XDP Datapath
Packet Processing Pipeline
eBPF Map Architecture
Buffering Mechanism
QoS Enforcement
Performance Characteristics
Scalability and Tuning

Overview

OmniUPF leverages eBPF (extended Berkeley Packet Filter) and XDP (eXpress Data Path) to achieve carrier-grade performance for 5G/LTE packet processing. The control plane is implemented in Elixir/OTP using GenServer-based PFCP session management, while the data plane runs eBPF programs directly in the Linux kernel. This separation eliminates the overhead of userspace packet processing and achieves multi-gigabit throughput with microsecond latency.

The eBPF object file (ipentrypoint_bpf.o) is compiled from C source (in the ebpf/xdp/ directory) and loaded at runtime by the Elixir control plane via NIF bindings to libbpf.

Architecture Layers

Key Design Principles

Zero-Copy Processing:

Packets processed entirely in kernel space
No data copying between kernel and userspace
Direct packet manipulation using XDP

Lock-Free Data Structures:

eBPF maps use per-CPU hash tables
Atomic operations for concurrent access
No mutex/spinlock overhead

Hardware Offload Ready:

XDP offload mode supports SmartNIC execution
Compatible with network cards supporting XDP
Fallback to driver-native or generic modes

eBPF Technology Foundation

What is eBPF?

eBPF (extended Berkeley Packet Filter) is a revolutionary Linux kernel technology that allows safe, sandboxed programs to run in kernel space without changing kernel source code or loading kernel modules.

Key Features:

Safety: eBPF verifier ensures programs cannot crash the kernel
Performance: Runs at native kernel speed (no interpretation overhead)
Flexibility: Can be updated at runtime without kernel restart
Observability: Built-in tracing and statistics

eBPF Program Lifecycle

eBPF Maps

eBPF maps are kernel data structures shared between eBPF programs and userspace.

Map Types Used in OmniUPF:

Map Type	Description	Use Case
`BPF_MAP_TYPE_HASH`	Hash table with key-value pairs	PDR lookup by TEID or UE IP
`BPF_MAP_TYPE_ARRAY`	Array indexed by integer	QER, FAR, URR lookup by ID
`BPF_MAP_TYPE_PERCPU_HASH`	Per-CPU hash table (lock-free)	High-performance PDR lookups
`BPF_MAP_TYPE_LRU_HASH`	LRU (Least Recently Used) hash	Automatic eviction of old entries

Map Operations:

Lookup: O(1) hash lookup (sub-microsecond)
Update: Atomic updates from userspace
Delete: Immediate removal of entries
Iterate: Batch operations for map dumps

XDP Datapath

XDP Overview

XDP (eXpress Data Path) is a Linux kernel hook that allows eBPF programs to process packets at the earliest possible point-right after the network driver receives them, before the kernel networking stack.

XDP Attach Modes

OmniUPF supports three XDP attach modes, each with different performance and compatibility characteristics.

1. XDP Offload Mode

Hardware Execution (Best Performance):

eBPF program runs directly on SmartNIC hardware
Packet processing in NIC without touching CPU
Achieves 100 Gbps+ throughput
Requires compatible SmartNIC (Netronome, Mellanox ConnectX-6)

Configuration (in runtime.exs):

xdp_attach_mode = "offload"

Limitations:

Requires expensive SmartNIC hardware
Limited eBPF program complexity
Not all eBPF features supported in hardware

2. XDP Native Mode (Default for Production)

Driver-Level Execution (High Performance):

eBPF program runs in network driver context
Packets processed before SKB (socket buffer) allocation
Achieves 10-40 Gbps per core
Requires driver with XDP support (most modern drivers)

Configuration (in runtime.exs):

xdp_attach_mode = "native"

Advantages:

Very high performance (multi-million pps)
Wide hardware compatibility
Full eBPF feature set

Supported Drivers:

Intel: i40e, ice, ixgbe, igb
Mellanox: mlx4, mlx5
Broadcom: bnxt
Amazon: ena
Most 10G+ network cards

3. XDP Generic Mode

Software Emulation (Compatibility):

eBPF program runs after kernel allocates SKB
Software emulation of XDP behavior
Works on any network interface
Useful for testing and development

Configuration (in runtime.exs):

xdp_attach_mode = "generic"

Use Cases:

Development and testing
Virtualized environments (VMs without SR-IOV)
Older network hardware
Loopback interface testing

Performance: 1-5 Gbps (significantly slower than native/offload)

XDP Return Codes

eBPF programs return XDP action codes to tell the kernel what to do with packets:

Return Code	Meaning	Use in OmniUPF
`XDP_PASS`	Send packet to kernel network stack	Buffering (local delivery), ICMP, unknown traffic
`XDP_DROP`	Drop packet immediately	Invalid packets, rate limiting, policy drops
`XDP_TX`	Transmit packet back out same interface	Not currently used
`XDP_REDIRECT`	Send packet to different interface	Main forwarding path (N3 ↔ N6)
`XDP_ABORTED`	Processing error, drop packet and log	eBPF program errors

Packet Processing Pipeline

Program Structure

OmniUPF uses eBPF tail calls to create a modular packet processing pipeline.

Tail Calls:

Allow eBPF programs to call other eBPF programs
Reuses same stack frame (bounded stack depth)
Enables modular pipeline design
Maximum 33 tail call depth

Uplink Packet Processing

Downlink Packet Processing

eBPF Map Architecture

Map Memory Layout

Map Sizing

OmniUPF automatically calculates map sizes based on max_sessions configuration:

PDR Maps = 2 × max_sessions  (uplink + downlink)
FAR Maps = 2 × max_sessions  (uplink + downlink)
QER Maps = 1 × max_sessions  (shared per session)
URR Maps = 3 × max_sessions  (multiple URRs per session)

Example (max_sessions = 65,535):

PDR maps: 131,070 entries each
FAR map: 131,070 entries
QER map: 65,535 entries
URR map: 131,070 entries

Total Memory:

PDR maps: 3 × 131,070 × 212 B = ~83 MB
FAR map:  131,070 × 20 B = ~2.6 MB
QER map:  65,535 × 36 B = ~2.3 MB
URR map:  131,070 × 20 B = ~2.6 MB
Total: ~91 MB kernel memory

Buffering Mechanism

Buffering Overview

OmniUPF implements packet buffering for handover scenarios by encapsulating packets in GTP-U and sending them to a userspace process via UDP socket.

Buffering Architecture

Buffer Encapsulation Details

When buffering is enabled (FAR action bit 2 set), the eBPF program:

Calculates Original Packet Size:

orig_packet_len = ntohs(ip->tot_len);  // From IP header

Expands Packet Header:

// Add space for: Outer IP + UDP + GTP-U
gtp_encap_size = sizeof(struct iphdr) + sizeof(struct udphdr) + sizeof(struct gtpuhdr);
bpf_xdp_adjust_head(ctx, -gtp_encap_size);

Builds Outer IP Header:

ip->saddr = original_sender_ip;  // Preserve source to avoid martian filtering
ip->daddr = local_upf_ip;        // Local IP where userspace listener binds
ip->protocol = IPPROTO_UDP;
ip->ttl = 64;

Builds UDP Header:

udp->source = htons(22152);  // BUFFER_UDP_PORT
udp->dest = htons(22152);
udp->len = htons(sizeof(udphdr) + sizeof(gtpuhdr) + orig_packet_len);

Builds GTP-U Header:

gtp->version = 1;
gtp->message_type = GTPU_G_PDU;
gtp->teid = htonl(far_id | (direction << 24));  // Encode FAR ID and direction
gtp->message_length = htons(orig_packet_len);

Returns XDP_PASS:
- Kernel delivers packet to local UDP socket on port 22152
- Userspace buffer manager receives and stores packet

Buffer Flush Operation

When handover completes, SMF updates FAR to clear BUFFER flag. Buffered packets are replayed:

Buffer Management Parameters

Parameter	Default	Description
Max Per FAR	10,000 packets	Maximum packets buffered per FAR
Max Total	100,000 packets	Maximum total buffered packets
Packet TTL	30 seconds	Time before buffered packets expire
Buffer Port	22152	UDP port for buffer delivery
Buffer Cleanup Interval	60 seconds	How often to check for expired packets

QoS Enforcement

Rate Limiting Algorithm

OmniUPF implements a sliding window rate limiter for QoS enforcement.

Sliding Window Implementation

Algorithm (from qer.h):

static __always_inline enum xdp_action limit_rate_sliding_window(
    const __u64 packet_size,
    volatile __u64 *window_start,
    const __u64 rate)
{
    static const __u64 NSEC_PER_SEC = 1000000000ULL;
    static const __u64 window_size = 5000000ULL;  // 5ms window

    // Rate = 0 means unlimited
    if (rate == 0)
        return XDP_PASS;

    // Calculate transmission time for this packet
    __u64 tx_time = packet_size * 8 * (NSEC_PER_SEC / rate);
    __u64 now = bpf_ktime_get_ns();

    // Check if we're ahead of window (packet would transmit in the future)
    __u64 start = *window_start;
    if (start + tx_time > now)
        return XDP_DROP;  // Rate limit exceeded

    // If window has passed, reset it
    if (start + window_size < now) {
        *window_start = now - window_size + tx_time;
        return XDP_PASS;
    }

    // Update window to account for this packet
    *window_start = start + tx_time;
    return XDP_PASS;
}

Key Parameters:

Window Size: 5ms (5,000,000 nanoseconds)
Per-Direction: Separate windows for uplink and downlink
Atomic Updates: Uses volatile pointers for concurrent access
MBR = 0: Treated as unlimited bandwidth

QoS Example Calculation

Scenario: MBR = 100 Mbps, Packet Size = 1500 bytes

Transmission Time:

tx_time = 1500 bytes × 8 bits/byte × (1,000,000,000 ns/sec ÷ 100,000,000 bps)
tx_time = 1500 × 8 × 10 = 120,000 ns = 120 μs

Rate Check:
- If last packet transmitted at t=0, next packet can transmit at t=120μs
- If packet arrives at t=100μs, it's dropped (too early)
- If packet arrives at t=150μs, it's forwarded (window advanced)

Maximum Packet Rate:

Max PPS = (100 Mbps ÷ 8) ÷ 1500 bytes = 8,333 packets/second
Inter-packet gap = 120 μs

Performance Characteristics

Throughput

Configuration	Throughput	Packets/Second	Latency
XDP Offload (SmartNIC)	100 Gbps	148 Mpps	< 1 μs
XDP Native (10G NIC, single core)	10 Gbps	8 Mpps	2-5 μs
XDP Native (10G NIC, 4 cores)	40 Gbps	32 Mpps	2-5 μs
XDP Generic	1-5 Gbps	0.8-4 Mpps	50-100 μs

Latency Breakdown

Total Packet Processing Latency (XDP Native):

Stage	Latency	Cumulative
NIC RX	0.5 μs	0.5 μs
XDP Hook Invocation	0.1 μs	0.6 μs
PDR Lookup (Hash)	0.3 μs	0.9 μs
QER Rate Check	0.1 μs	1.0 μs
FAR Processing	0.5 μs	1.5 μs
URR Update	0.2 μs	1.7 μs
GTP-U Encap/Decap	0.8 μs	2.5 μs
XDP_REDIRECT	0.5 μs	3.0 μs
NIC TX	0.5 μs	3.5 μs

Total: ~3.5 μs per packet (XDP Native, 10G NIC)

CPU Utilization

Per-Core Processing Capacity:

Single core: 8-10 Mpps (XDP Native)
With hyper-threading: 12-15 Mpps
Multi-core scaling: Near-linear up to 8 cores

CPU Usage by Packet Rate:

CPU % ≈ (Packet Rate / 10,000,000) × 100% per core

Example: 2 Mpps traffic uses ~20% of one core

Memory Bandwidth

eBPF Map Access:

Hash lookup: ~100 ns (cache hit)
Hash lookup: ~300 ns (cache miss)
Array lookup: ~50 ns (always cache hit)

Memory Bandwidth Required:

Bandwidth = Packet Rate × (Avg Packet Size + Map Lookups × 64 bytes)

Example: 10 Mpps × (1500 B + 3 lookups × 64 B) ≈ 160 Gbps memory bandwidth

Scalability and Tuning

Horizontal Scaling

Multiple UPF Instances:

Session Distribution:

SMF distributes sessions across UPF instances
Each UPF handles subset of UE sessions
No inter-UPF communication needed (stateless)

Vertical Scaling

CPU Tuning:

Enable CPU affinity for XDP processing
Use RSS (Receive Side Scaling) to distribute RX queues
Pin eBPF programs to specific cores

NIC Tuning:

Increase RX ring buffer size
Enable multi-queue NICs (RSS)
Use flow director for traffic steering

Kernel Tuning:

# Increase locked memory limit for eBPF maps
ulimit -l unlimited

# Disable IRQ balance for XDP cores
systemctl stop irqbalance

# Set CPU governor to performance
cpupower frequency-set -g performance

# Increase network buffer sizes
sysctl -w net.core.rmem_max=134217728
sysctl -w net.core.wmem_max=134217728

Capacity Planning

Formula:

Required CPU Cores = (Expected PPS ÷ 10,000,000) × 1.5  (50% headroom)
Required Memory = (Max Sessions × 212 B × 3) + 100 MB (eBPF maps + overhead)
Required Network = (Peak Throughput × 2) + 10 Gbps (headroom)

Example (1 million sessions, 20 Gbps peak):

CPU: (20 Gbps ÷ 10 Gbps per core) × 1.5 = 3-4 cores
Memory: (1M × 212 B × 3) + 100 MB ≈ 750 MB
Network: (20 Gbps × 2) + 10 Gbps = 50 Gbps interfaces

UPF Operations Guide - General UPF operations and deployment
Rules Management Guide - PDR, FAR, QER, URR details
Monitoring Guide - Performance monitoring and metrics
Web UI Operations Guide - Control panel usage
Troubleshooting Guide - Common issues and diagnostics
Walled Garden Guide - Out-of-credit redirect using the BUFF mechanism for userspace packet interception

Table of Contents​

Overview​

Architecture Layers​

Key Design Principles​

eBPF Technology Foundation​

What is eBPF?​

eBPF Program Lifecycle​

eBPF Maps​

XDP Datapath​

XDP Overview​

XDP Attach Modes​

1. XDP Offload Mode​

2. XDP Native Mode (Default for Production)​

3. XDP Generic Mode​

XDP Return Codes​

Packet Processing Pipeline​

Program Structure​

Uplink Packet Processing​

Downlink Packet Processing​

eBPF Map Architecture​

Map Memory Layout​

Map Sizing​

Buffering Mechanism​

Buffering Overview​

Buffering Architecture​

Buffer Encapsulation Details​

Buffer Flush Operation​

Buffer Management Parameters​

QoS Enforcement​

Rate Limiting Algorithm​

Sliding Window Implementation​

QoS Example Calculation​

Performance Characteristics​

Throughput​

Latency Breakdown​

CPU Utilization​

Memory Bandwidth​

Scalability and Tuning​

Horizontal Scaling​

Vertical Scaling​

Capacity Planning​

Related Documentation​

Table of Contents

Overview

Architecture Layers

Key Design Principles

eBPF Technology Foundation

What is eBPF?

eBPF Program Lifecycle

eBPF Maps

XDP Datapath

XDP Overview

XDP Attach Modes

1. XDP Offload Mode

2. XDP Native Mode (Default for Production)

3. XDP Generic Mode

XDP Return Codes

Packet Processing Pipeline

Program Structure

Uplink Packet Processing

Downlink Packet Processing

eBPF Map Architecture

Map Memory Layout

Map Sizing

Buffering Mechanism

Buffering Overview

Buffering Architecture

Buffer Encapsulation Details

Buffer Flush Operation

Buffer Management Parameters

QoS Enforcement

Rate Limiting Algorithm

Sliding Window Implementation

QoS Example Calculation

Performance Characteristics

Throughput

Latency Breakdown

CPU Utilization

Memory Bandwidth

Scalability and Tuning

Horizontal Scaling

Vertical Scaling

Capacity Planning

Related Documentation