Troubleshooting Guide

Problem Resolution for RAN Monitor

Common issues, diagnostic procedures, and solutions

Overview
Device Connection Issues
Data Collection Problems
Web UI Issues
Database Problems
Performance Issues
Alarm Problems
Diagnostic Tools
Getting Help

Overview

This guide helps you diagnose and resolve common issues with RAN Monitor. Each section provides symptoms, diagnosis steps, and solutions.

Troubleshooting Approach

1. Identify the Symptom

What is not working as expected?
When did the problem start?
What changed recently?

2. Gather Information

Check application logs
Review device status in Web UI
Check database connectivity
Review recent configuration changes

3. Diagnose the Root Cause

Use diagnostic tools
Review error messages
Test individual components
Isolate the problem

4. Implement Solution

Apply fix based on diagnosis
Verify solution resolves issue
Monitor for recurrence
Document findings

Before You Start

Check the Basics:

Is RAN Monitor running? (ps aux | grep ran_monitor)
Are required services running? (MySQL, InfluxDB)
Is network connectivity working?
Have there been recent changes?

Device Connection Issues

Problem: Device Not Registering

Symptoms:

Device shows "Not Registered" in Web UI
Red (failed) status in Base Stations page
No metrics are being collected from device
Error messages in application logs

Diagnostic Steps:

1. Verify Network Connectivity

# Test basic connectivity
ping <device-ip>

# Test management port
telnet <device-ip> 8080

Expected: Successful ping and telnet connection If Fails: Network issue - check routes, firewall, device status

2. Check Configuration

In Web UI → Base Stations → Click device → Review configuration:

Is IP address correct?
Is port correct (typically 8080)?
Are credentials configured?

In config/runtime.exs:

%{
  address: "10.7.15.66",  # Correct IP?
  name: "Site-A-BS1",
  port: "8080",           # Correct port?
  web_username: "admin",  # Correct username?
  web_password: "password" # Correct password?
}

3. Check Application Logs

Web UI → Application Logs → Filter for device name

Look for:

[error] Authentication failed → Incorrect credentials
[error] Connection refused → Port/firewall issue
[error] Timeout → Network connectivity problem
[error] Certificate error → Manager key/certificate issue

Solutions:

Network Issue:

Verify device is powered on and operational
Check network routes between RAN Monitor and device
Verify firewall allows:
- RAN Monitor → Device port 8080
- Device → RAN Monitor port 9076 (webhooks)
Test from RAN Monitor server directly

Incorrect Credentials:

Verify credentials work directly on device WebLM interface
Update credentials in config/runtime.exs
Restart RAN Monitor
Monitor logs for successful registration

Port/Firewall Issue:

Verify correct port in configuration
Check firewall rules on both sides
Test port accessibility: telnet <device-ip> 8080
Review device-side security settings

Manager Key/Certificate Issue:

Verify files exist:
- priv/external/nokia/ne.key.pem
- priv/external/nokia/ne.cert.der
Check file permissions (should be readable)
Verify files are valid Nokia manager credentials
Contact Nokia support if keys are invalid

Problem: Session Keeps Expiring

Symptoms:

Device repeatedly disconnects and reconnects
"Session expired" messages in logs
Intermittent red/green status in Web UI
Gaps in metric collection

Diagnostic Steps:

1. Check Session Information

Web UI → Base Stations → Click device → Session Lifecycle:

What is session expiry time?
Is keep-alive working?
How often is session expiring?

2. Check Keep-Alive Interval

In config/runtime.exs:

nokia: %{
  ne3s: %{
    reregister_interval: 30  # Should be 30-60 seconds
  }
}

3. Check Network Stability

Are there intermittent network issues?
Check for packet loss: ping <device-ip> -c 100
Review network logs for flapping interfaces

4. Check Clock Synchronization

# On RAN Monitor server
date

# On device (if accessible)
# Verify time is synchronized

Solutions:

Keep-Alive Interval Too Long:

Reduce reregister_interval to 30 seconds
Restart RAN Monitor
Monitor session stability

Network Instability:

Work with network team to diagnose
Check for intermittent connectivity
Review switch/router logs
Consider redundant network paths

Clock Synchronization:

Configure NTP on both RAN Monitor and devices
Verify clocks are synchronized
Check for large time differences

Device-Side Issue:

Check device logs for errors
Verify device management interface is stable
Consider device reboot if software issue suspected

Problem: Metrics Not Appearing

Symptoms:

Device shows as "Associated" (green) in Web UI
But no metrics appearing in InfluxDB
No data in Grafana dashboards
InfluxDB Status page shows zero or low counts

Diagnostic Steps:

1. Verify Device is Associated

Web UI → Base Stations:

Device status is green?
Last contact timestamp is recent?
Session is active?

2. Check InfluxDB Connection

Web UI → InfluxDB Status:

Connection status green?
Can RAN Monitor write to InfluxDB?

Test connectivity:

# From RAN Monitor server
curl http://<influxdb-host>:8086/ping

3. Check Application Logs

Look for:

[error] InfluxDB write failed → Connection or permission issue
[error] Failed to collect metrics → Device communication issue
[info] Metrics collected: 0 → Device not returning data

4. Check InfluxDB Directly

Query InfluxDB for recent data:

# InfluxDB 1.x
influx -database 'nokia-monitor' -execute '
  SELECT COUNT(*) FROM PerformanceMetrics
  WHERE basebandName='''Site-A-BS1'''
  AND time > now() - 1h
'

# InfluxDB 2.x
influx query 'from(bucket:"nokia-monitor")
  |> range(start: -1h)
  |> filter(fn: (r) => r.basebandName == "Site-A-BS1")
  |> filter(fn: (r) => r._measurement == "PerformanceMetrics")
  |> count()'

Solutions:

InfluxDB Connection Issue:

Verify InfluxDB is running
Check config/runtime.exs for correct:
- Host address
- Port (8086)
- Database/bucket name
- Credentials/API token
Test connectivity from RAN Monitor server
Verify firewall allows port 8086
Restart RAN Monitor after fixing configuration

InfluxDB Permission Issue:

Verify credentials have write permission to bucket/database
Check InfluxDB logs for authentication errors
Recreate API token with proper permissions
Update config/runtime.exs with new token
Restart RAN Monitor

InfluxDB Storage Full:

Check disk space: df -h
Review retention policies
Clean old data or expand storage
See Data Retention Policy Guide

Device Not Returning Data:

Check device is configured to send metrics
Verify webhook URL is correct in device configuration
Check device logs for errors
Verify RAN Monitor webhook receiver is running (port 9076)

Data Collection Problems

Problem: Gaps in Historical Data

Symptoms:

Grafana dashboards show gaps in time series
Missing data points for certain time periods
InfluxDB queries return incomplete results

Diagnostic Steps:

1. Check Application Uptime

Were there service interruptions during gap period?

# Check system logs for restarts
journalctl -u ran_monitor --since "2025-12-29" --until "2025-12-30"

2. Check Device Connectivity History

Web UI → Base Stations → Device → Review "Last Contact" history

Was device connected during gap period?
Are there connectivity issues?

3. Check InfluxDB Availability

Were there InfluxDB outages during gap period?

Check InfluxDB logs
Review monitoring/alerting history

Solutions:

RAN Monitor Downtime:

Data gap is normal during service interruption
Historical data cannot be backfilled
Document incident and restore service

Device Disconnection:

Investigate why device disconnected
Fix connectivity issue
Data gap normal during disconnection
Future data will resume collection

InfluxDB Outage:

Metrics were likely collected but not stored
Check RAN Monitor logs for write failures
Restore InfluxDB service
Data gap cannot be recovered

Prevention:

Implement monitoring for RAN Monitor uptime
Set up alerts for extended disconnections
Monitor InfluxDB health
Consider HA/redundancy for critical systems

Web UI Issues

Problem: Cannot Access Web UI

Symptoms:

Browser cannot connect to https://<ran-monitor-ip>:9443
Connection timeout or refused
SSL certificate errors

Diagnostic Steps:

1. Verify Web UI is Running

Check application logs:

[info] Running ControlPanelWeb.Endpoint with cowboy

Check process:

ps aux | grep control_panel
netstat -tulpn | grep 9443

2. Test Connectivity

From another machine:

telnet <ran-monitor-ip> 9443

From RAN Monitor server itself:

curl -k https://localhost:9443

3. Check Firewall

# Check if port is open
sudo iptables -L -n | grep 9443

# Or
sudo firewall-cmd --list-ports

Solutions:

Port Not Open:

Add firewall rule:

sudo firewall-cmd --add-port=9443/tcp --permanent
sudo firewall-cmd --reload

Test access again

Web UI Not Started:

Check config/runtime.exs for web endpoint configuration
Verify SSL certificate files exist
Check application logs for startup errors
Restart RAN Monitor

SSL Certificate Issues:

Verify certificate files exist and are readable:

ls -l priv/cert/omnitouch.pem
ls -l priv/cert/omnitouch.crt

Check certificate validity:

openssl x509 -in priv/cert/omnitouch.crt -text -noout

Regenerate if expired or missing
Restart RAN Monitor

Wrong Port:

Check config/runtime.exs for configured port
Use correct port in browser
Or set CONTROL_PANEL_HTTPS_PORT environment variable

Problem: Web UI Loads But Shows No Data

Symptoms:

Web UI is accessible
Pages load but show empty lists or zero counts
Dashboard shows no devices

Diagnostic Steps:

1. Check Device Configuration

Is anything configured in config/runtime.exs?

airscales: [
  # Should have at least one device
]

2. Check Database Connection

Are devices stored in MySQL?

mysql -u ran_monitor_user -p ran_monitor -e "SELECT * FROM airscales;"

3. Check Application Logs

Look for database connection errors or query failures.

Solutions:

No Devices Configured:

Add devices to config/runtime.exs
Restart RAN Monitor
Devices should appear in Web UI

Database Connection Issue:

Verify MySQL is running
Check connection configuration in config/runtime.exs
Test database connection
Restart RAN Monitor

Database Problems

Problem: MySQL Connection Errors

Symptoms:

Application logs show database connection errors
Web UI shows errors loading pages
"Database connection timeout" messages

Diagnostic Steps:

1. Verify MySQL is Running

systemctl status mysql
# or
systemctl status mariadb

2. Test Connection

From RAN Monitor server:

mysql -h <mysql-host> -u <username> -p <database>

3. Check Configuration

In config/runtime.exs:

config :ran_monitor, RanMonitor.Repo,
  username: "ran_monitor_user",
  password: "password",
  hostname: "localhost",
  database: "ran_monitor",
  pool_size: 10

Solutions:

MySQL Not Running:

Start MySQL service:
```
systemctl start mysql
```
Verify it starts correctly
RAN Monitor will reconnect automatically

Connection Configuration Error:

Verify hostname, username, password, database name
Test connection manually
Update config/runtime.exs if incorrect
Restart RAN Monitor

Network Issue:

Check network connectivity to MySQL server
Verify firewall allows port 3306
Check MySQL bind address (should allow remote connections if needed)

Too Many Connections:

Check MySQL max_connections setting
Reduce pool_size in configuration if needed
Restart RAN Monitor

Performance Issues

Problem: High CPU or Memory Usage

Symptoms:

RAN Monitor using excessive CPU or RAM
System becomes slow or unresponsive
Database connections timing out
Response time degraded

Diagnostic Steps:

1. Check Resource Usage

# CPU and memory
top -p $(pgrep -f ran_monitor)

# Detailed process info
ps aux | grep ran_monitor

2. Check Number of Monitored Devices

How many devices are configured?

More devices = more resources needed
Check if device count recently increased

3. Check Collection Intervals

Are polling intervals very frequent?

More frequent = higher CPU/network usage
Default is 10 seconds for metrics

4. Check Database Pool Size

In config/runtime.exs:

pool_size: 10  # May need adjustment

Solutions:

Too Many Devices for Resources:

Monitor resource usage trends
Increase server resources (CPU/RAM)
Or reduce number of monitored devices
Consider scaling horizontally (multiple instances)

Database Pool Too Large:

Reduce pool_size in configuration
Rule of thumb: 2 connections per device + 5 for Web UI
Restart RAN Monitor
Monitor resource usage

Memory Leak:

Monitor memory usage over time
If continuously increasing, may be memory leak
Restart RAN Monitor as temporary fix
Report issue with logs and metrics

InfluxDB Write Performance:

Check InfluxDB resource usage
Verify InfluxDB isn't bottleneck
Consider separate InfluxDB server
Review retention policies to reduce data volume

Problem: Slow Web UI Response

Symptoms:

Web UI takes long time to load pages
Dashboard is sluggish
Timeouts when viewing device details

Diagnostic Steps:

1. Check Server Resources

Is RAN Monitor server overloaded?

top
free -h
df -h

2. Check Database Performance

Are database queries slow?

# MySQL slow query log
mysql -u root -p -e "SHOW VARIABLES LIKE 'slow_query_log%';"

3. Check Network Latency

Is there high latency to database or clients?

Solutions:

Server Resource Issue:

Reduce load on server
Increase server resources
Move databases to separate servers

Database Performance:

Optimize MySQL configuration
Add indexes if needed (tables should have them)
Increase database server resources

Network Latency:

Investigate network path
Consider moving components closer
Use local database if possible

Alarm Problems

Problem: Alarms Not Appearing

Symptoms:

Known faults not showing in Alarms page
Alarm count is zero when faults exist
Delayed alarm notifications

Diagnostic Steps:

1. Check Device is Sending Alarms

Verify in device management interface that alarms are configured to be sent.

2. Check Webhook Receiver

Is webhook endpoint running?

netstat -tulpn | grep 9076

Look for:

tcp  0  0.0.0.0:9076  0.0.0.0:*  LISTEN

3. Check Webhook Configuration

In device configuration, verify webhook URL points to RAN Monitor:

http://<ran-monitor-ip>:9076/webhook

4. Check Application Logs

Look for webhook receiver errors or alarm parsing failures.

5. Check InfluxDB

Are alarms being written?

influx -database 'nokia-monitor' -execute '
  SELECT COUNT(*) FROM Alarms WHERE time > now() - 1h
'

Solutions:

Webhook Receiver Not Running:

Check config/runtime.exs for webhook endpoint configuration
Verify port 9076 is configured
Restart RAN Monitor
Verify port is listening

Device Not Sending:

Configure device to send alarm notifications
Verify webhook URL in device configuration
Test alarm generation on device

Firewall Blocking:

Verify device can reach RAN Monitor port 9076
Add firewall rule if needed
Test connectivity: telnet <ran-monitor-ip> 9076 from device network

InfluxDB Write Failure:

Check InfluxDB connection
Verify write permissions
Check InfluxDB storage capacity
Review application logs for write errors

Diagnostic Tools

Application Logs

Access via Web UI:

Navigate to Application Logs page
Filter by log level
Search for keywords
Pause to review specific errors

Access via Command Line:

If running as systemd service:

journalctl -u ran_monitor -f

If running via mix:

Logs appear in console output

Log Levels:

Emergency/Alert/Critical - System-critical issues
Error - Errors that need attention
Warning - Potential issues
Info - Normal operational messages
Debug - Detailed diagnostic information

Useful Search Terms:

Device name (e.g., "Site-A-BS1")
"error" or "failed"
"InfluxDB" or "MySQL"
"registration" or "session"

InfluxDB Queries

Query for recent metrics:

influx -database 'nokia-monitor' -execute '
  SELECT * FROM PerformanceMetrics
  WHERE basebandName='''Site-A-BS1'''
  AND time > now() - 5m
  LIMIT 10
'

Count metrics by device:

influx -database 'nokia-monitor' -execute '
  SELECT COUNT(*) FROM PerformanceMetrics
  GROUP BY basebandName
'

Query for alarms:

influx -database 'nokia-monitor' -execute '
  SELECT * FROM Alarms
  WHERE time > now() - 1h
'

MySQL Queries

Check configured devices:

SELECT name, address, port, registration_status
FROM airscales;

Check for errors in database:

mysql -u ran_monitor_user -p ran_monitor -e "SHOW PROCESSLIST;"

Network Diagnostics

Test connectivity:

# Basic connectivity
ping <device-ip>

# Port accessibility
telnet <device-ip> 8080
nc -zv <device-ip> 8080

# Trace route
traceroute <device-ip>

Check firewall:

# List rules
sudo iptables -L -n -v

# Check specific port
sudo iptables -L -n | grep 8080

Getting Help

Before Contacting Support

Gather the following information:

1. Problem Description

What is not working?
When did it start?
What changed recently?

2. Error Messages

Copy exact error messages from logs
Include timestamps
Note frequency of errors

3. System Information

RAN Monitor version
Operating system and version
Database versions (MySQL, InfluxDB)
Number of monitored devices

4. Diagnostic Results

Results from diagnostic steps above
Relevant log excerpts
Configuration (sanitize passwords)

5. Impact

How many devices affected?
Is this blocking operations?
What is the business impact?

Documentation Resources

Web UI Guide - Control panel reference
Common Operations Guide - Routine tasks
Runtime Configuration Guide - Configuration details
Grafana Integration Guide - Analytics setup
Alarm Management Guide - Alarm handling
Data Retention Policy Guide - Data management
Operations Guide - Complete overview

Self-Service Resources

Check Logs First:

Application Logs page in Web UI
System logs: journalctl -u ran_monitor
Database logs

Review Recent Changes:

Configuration file modifications
Device additions/removals
Network changes
Software updates

Test Basic Functionality:

Can you access Web UI?
Are devices showing as connected?
Is InfluxDB accessible?
Are metrics flowing?

Escalation

If you cannot resolve the issue:

Document all diagnostic steps taken
Gather information listed above
Contact Omnitouch support with details
Be prepared to provide:
- Configuration files (sanitized)
- Log excerpts
- Screenshots if relevant
- Steps to reproduce

Operations Guide - Complete operational overview
Web UI Guide - Control panel user guide
Common Operations Guide - Day-to-day tasks
Alarm Management Guide - Alarm handling procedures
Runtime Configuration Guide - Configuration reference
Grafana Integration Guide - Analytics and dashboards
Data Retention Policy Guide - Data lifecycle management

Table of Contents​

Overview​

Troubleshooting Approach​

Before You Start​

Device Connection Issues​

Problem: Device Not Registering​

Problem: Session Keeps Expiring​

Problem: Metrics Not Appearing​

Data Collection Problems​

Problem: Gaps in Historical Data​

Web UI Issues​

Problem: Cannot Access Web UI​

Problem: Web UI Loads But Shows No Data​

Database Problems​

Problem: MySQL Connection Errors​

Performance Issues​

Problem: High CPU or Memory Usage​

Problem: Slow Web UI Response​

Alarm Problems​

Problem: Alarms Not Appearing​

Diagnostic Tools​

Application Logs​

InfluxDB Queries​

MySQL Queries​

Network Diagnostics​

Getting Help​

Before Contacting Support​

Documentation Resources​

Self-Service Resources​

Escalation​

Related Documentation​

Table of Contents

Overview

Troubleshooting Approach

Before You Start

Device Connection Issues

Problem: Device Not Registering

Problem: Session Keeps Expiring

Problem: Metrics Not Appearing

Data Collection Problems

Problem: Gaps in Historical Data

Web UI Issues

Problem: Cannot Access Web UI

Problem: Web UI Loads But Shows No Data

Database Problems

Problem: MySQL Connection Errors

Performance Issues

Problem: High CPU or Memory Usage

Problem: Slow Web UI Response

Alarm Problems

Problem: Alarms Not Appearing

Diagnostic Tools

Application Logs

InfluxDB Queries

MySQL Queries

Network Diagnostics

Getting Help

Before Contacting Support

Documentation Resources

Self-Service Resources

Escalation

Related Documentation