Common Operations Guide

Day-to-Day RAN Monitor Management Tasks

Practical guide for routine operational tasks and device management

---

Table of Contents

1. Overview
2. Adding a New Base Station
3. Removing a Base Station
4. Updating Device Credentials
5. Adjusting Collection Intervals
6. Managing Device Configuration
7. Monitoring System Health
8. Data Management
9. Backup and Recovery
10. System Maintenance

---

Overview

This guide covers routine operational tasks for managing RAN Monitor in day-to-day operations. These procedures are designed for NOC teams, network administrators, and operations staff.

Prerequisites

• RAN Monitor is installed and running
• You have access to the configuration files
• You can restart the RAN Monitor application
• You understand your network topology

For initial setup, see the Getting Started Guide.

---

Adding a New Base Station

When deploying new Nokia AirScale base stations, follow these steps to add them to monitoring.

Step 1: Verify Network Connectivity

Before adding the device to configuration, ensure network connectivity:

# Test basic connectivity
ping <base-station-ip>

# Verify management port is accessible
telnet <base-station-ip> 8080

Expected Result: Successful ping responses and telnet connection

If Fails:

• Check network routes
• Verify firewall rules allow connectivity
• Confirm base station is powered on and operational

Step 2: Gather Device Information

Collect the following information:

Information	Example	Where to Find
IP Address	10.7.15.67	Network documentation or device label
Port	8080	Typically 8080 for Nokia AirScale
Device Name	Site-B-Tower-1	Use site naming convention
Username	admin	From base station provisioning
Password	password123	From base station provisioning

Device Naming Best Practices:

• Use descriptive, consistent naming
• Include site identifier
• Include equipment type if multiple at same site
• Examples: NYC-SiteA-BS1, LAX-Tower-Main, CHI-Indoor-DAS

Step 3: Check for Unconfigured Devices

Before manually adding, check if the device has already attempted to connect:

1. Open Web UI: https://<ran-monitor-ip>:9443
2. Navigate to Unconfigured eNodeBs page
3. Look for your device's IP address or Agent ID
4. Note the Agent ID if found

This helps verify the device can reach RAN Monitor.

Step 4: Add Device to Configuration

Edit config/runtime.exs and add the new device to the airscales list:

config :ran_monitor,
  nokia: %{
    ne3s: %{
      # ... existing ne3s configuration ...
    },
    airscales: [
      # Existing devices
      %{
        address: "10.7.15.66",
        name: "Site-A-BS1",
        port: "8080",
        web_username: "admin",
        web_password: "password1"
      },

      # New device
      %{
        address: "10.7.15.67",          # IP address of new base station
        name: "Site-B-Tower-1",          # Descriptive name
        port: "8080",                    # Management port
        web_username: "admin",           # WebLM username
        web_password: "password123"      # WebLM password
      }
    ]
  }

Important: Ensure proper Elixir syntax - commas, indentation, and map structure must be correct.

Step 5: Validate Configuration

Before restarting, validate the configuration syntax:

elixir -c config/runtime.exs

Expected Output: No errors

If Errors:

• Check for missing commas
• Verify all opening braces { and brackets [ are closed
• Ensure strings are properly quoted
• Check indentation is consistent

Step 6: Restart RAN Monitor

Restart the application to load the new configuration:

# If running in development
# Press Ctrl+C twice, then:
mix phx.server

# If running as a service
systemctl restart ran_monitor

# If running via release
/path/to/ran_monitor/bin/ran_monitor restart

Step 7: Verify Device Registration

After restart, verify the device is now monitored:

1. Check Application Logs:

   [info] Attempting registration with device: Site-B-Tower-1
   [info] Successfully registered with Site-B-Tower-1

2. Check Web UI:

   • Navigate to Base Stations page
   • Find your new device in the list
   • Status should be "Associated" (green)
   • Registration state should be "Registered"

3. Check Device Details:

   • Click on the device
   • Verify session information shows active session
   • Confirm "Last Contact" timestamp is recent

4. Check InfluxDB Status:

   • Navigate to InfluxDB Status page
   • Verify total record count is increasing
   • Measurement counts should grow as data is collected

Step 8: Configure Data Retention (Optional)

If this device requires different retention than the global default:

1. Navigate to Data Retention page
2. Find your new device in the list
3. Update the "Retention Period" field (in hours)
4. System saves automatically

For more details, see the Data Retention Policy Guide.

Step 9: Add to Grafana Dashboards

Update Grafana dashboards to include the new device:

1. Edit dashboard template variables
2. Add device name to dropdown selectors
3. Create device-specific dashboard if needed

For more details, see the Grafana Integration Guide.

---

Removing a Base Station

When decommissioning a base station, follow these steps to remove it from monitoring.

Step 1: Decide on Data Handling

Before removing, decide what to do with historical data:

Option A: Preserve Data

• Disable monitoring but keep historical data
• Useful for decommissioned but potentially returning equipment

Option B: Remove Data

• Delete all historical data for the device
• Frees up InfluxDB storage
• Irreversible - data cannot be recovered

Step 2: Disable Device (Preserve Data)

To stop monitoring but keep historical data:

Edit config/runtime.exs and locate the device in the airscales list. Comment it out or remove it:

airscales: [
  %{
    address: "10.7.15.66",
    name: "Site-A-BS1",
    port: "8080",
    web_username: "admin",
    web_password: "password1"
  },

  # Decommissioned device - commented out to preserve data
  # %{
  #   address: "10.7.15.67",
  #   name: "Site-B-Tower-1",
  #   port: "8080",
  #   web_username: "admin",
  #   web_password: "password123"
  # }
]

Step 3: Remove Data (Optional)

To delete all historical data for the device:

1. Navigate to Data Retention page in Web UI
2. Find the device in the list
3. Click Clear All Data button
4. Confirm the action

Warning: This is permanent and cannot be undone.

Step 4: Restart RAN Monitor

Restart the application to apply configuration changes:

systemctl restart ran_monitor
# or
mix phx.server

Step 5: Verify Removal

After restart:

1. Check Base Stations Page:

   • Device should no longer appear in active list
   • If data was preserved, device may still show in historical queries

2. Check Application Logs:

   • No registration attempts for removed device
   • No errors about missing device

3. Check InfluxDB:

   • If data was deleted, record counts should be lower
   • Device should not appear in new metrics

Step 6: Update Grafana Dashboards

Remove device from Grafana configurations:

1. Edit dashboard template variables
2. Remove device name from dropdowns
3. Delete device-specific dashboards if they exist

---

Updating Device Credentials

When base station passwords are changed, update RAN Monitor configuration.

Step 1: Note Current Status

Before making changes:

1. Verify device is currently connected (green status)
2. Note current session information
3. Take screenshot or record current state for comparison

Step 2: Update Configuration

Edit config/runtime.exs and update the credentials:

airscales: [
  %{
    address: "10.7.15.66",
    name: "Site-A-BS1",
    port: "8080",
    web_username: "admin",
    web_password: "new_password_here"  # Updated password
  }
]

Step 3: Restart RAN Monitor

Apply the configuration change:

systemctl restart ran_monitor

Step 4: Verify Reconnection

After restart:

1. Check Application Logs:

   [info] Attempting registration with device: Site-A-BS1
   [info] Successfully registered with Site-A-BS1

2. Check Web UI:

   • Device status should be "Associated" (green)
   • New session should be established
   • "Last Contact" should be recent

If Device Fails to Connect:

• Verify new credentials are correct
• Check for typos in password
• Confirm credentials work directly on base station
• Review application logs for authentication errors

---

Adjusting Collection Intervals

Change how frequently RAN Monitor collects data from base stations.

Understanding Collection Intervals

RAN Monitor collects three types of data at different intervals:

Data Type	Default Interval	Configurable	Impact of Shorter Interval
Performance Metrics	10 seconds	Yes	More granular data, higher network/storage usage
Alarms	10 seconds	Yes	Faster alarm detection, more queries
Configuration	60 seconds	Yes	More frequent config snapshots, more storage
Health Checks	30 seconds	Yes	More responsive to connectivity issues

When to Adjust Intervals

Shorten Intervals (More Frequent):

• Troubleshooting active issues
• High-value critical infrastructure
• SLA monitoring with tight requirements
• Capacity testing and analysis

Lengthen Intervals (Less Frequent):

• Reduce network traffic
• Reduce InfluxDB storage usage
• Lower-priority test environments
• Bandwidth-constrained links

Step 1: Modify Configuration

Collection intervals are configured in the application code, not runtime.exs. To change them, you'll need to modify the source code and recompile.

Example locations (may vary by version):

• Performance metrics: lib/ran_monitor/nokia/airscale/manager.ex
• Alarms: lib/ran_monitor/nokia/airscale/manager.ex
• Configuration: lib/ran_monitor/nokia/airscale/manager.ex

Note: Contact Omnitouch support for assistance with collection interval changes, as this requires source code modifications.

Step 2: Consider Impact

Before changing intervals:

Network Impact:

• Calculate: devices � counters � interval = queries per second
• Shorter intervals = more network traffic
• Ensure network can handle increased load

Storage Impact:

• Calculate: data points per day � retention period = total storage
• Example: 10s interval = 8,640 measurements/day per counter
• Ensure InfluxDB has sufficient disk space

System Performance:

• More frequent polling = higher CPU usage on RAN Monitor
• Monitor system resources after changes

Step 3: Monitor After Changes

After adjusting intervals:

1. Watch Application Logs for any errors
2. Monitor System Resources:
   • CPU usage on RAN Monitor server
   • Network bandwidth utilization
   • InfluxDB disk I/O and storage growth
3. Verify Data Quality:
   • Check InfluxDB for expected measurement frequency
   • Verify no gaps in data
4. Adjust if Needed:
   • Revert if system is overloaded
   • Fine-tune based on observed performance

---

Managing Device Configuration

How to safely manage base station configurations through RAN Monitor.

Configuration Workflow

For detailed configuration management procedures, see the Web UI Guide - Configuration Management.

Quick Reference:

1. Download current configuration (backup)
2. Modify configuration offline
3. Upload new configuration � get Plan ID
4. Validate configuration using Plan ID
5. Activate if validation succeeds
6. Verify changes took effect

Best Practices

Always Download First:

• Keep backup of current configuration
• Enables rollback if needed
• Documents configuration before change

Validate Before Activating:

• Never activate without validating
• Validation catches syntax errors
• Prevents service interruptions

Schedule Changes Appropriately:

• Use maintenance windows when possible
• Avoid peak traffic hours
• Have rollback plan ready

Document Changes:

• Record what was changed and why
• Note Plan ID for tracking
• Document verification results
• Update change management system

Monitor After Changes:

• Watch for alarms
• Verify metrics normalize
• Check device stability for 15-30 minutes
• Be prepared to rollback if issues occur

---

Monitoring System Health

Routine checks to ensure RAN Monitor is operating correctly.

Daily Health Check

Perform these checks at the start of each shift:

1. Access Web UI Dashboard

https://<ran-monitor-ip>:9443

2. Review System Status

• All devices showing green (associated) status?
• Any red (failed) devices requiring investigation?
• Alarm count reasonable for time of day?

3. Check Alarm Summary

• Any critical alarms active?
• Alarm rate trending up or down?
• Any repeat alarms indicating systemic issues?

4. Verify Data Collection

• Navigate to InfluxDB Status page
• Measurement counts increasing?
• Last update timestamp recent?

5. Review Recent Logs

• Navigate to Application Logs page
• Filter for "Error" level
• Any recurring errors?

For detailed health check procedures, see the Web UI Guide - Web UI Workflows.

Example: Grafana Monitoring Dashboard

Comprehensive monitoring dashboard showing S1 connections status by LNMME, operational state, data transferred, UEs connected, average PRB usage, performance monitoring metrics, and geographic coverage map. This dashboard provides at-a-glance visibility into network health and device status.

Weekly System Review

More thorough check performed weekly:

1. Review Alarm Trends

• Use Grafana to analyze alarm rate over past week
• Identify any alarm storms or patterns
• Correlate with maintenance or changes

2. Check Storage Growth

• InfluxDB disk usage trending
• MySQL database size
• Application log file sizes

3. Review Device Connectivity

• Any devices with frequent disconnections?
• Session timeout issues?
• Pattern of connectivity problems?

4. System Resource Utilization

• CPU usage on RAN Monitor server
• Memory usage trends
• Network bandwidth consumption

5. Configuration Changes

• Review all configuration changes made
• Verify changes were documented
• Correlate with any issues

Time Required: 30-45 minutes

---

Data Management

Managing Data Retention

See the Data Retention Policy Guide for complete details.

Quick Reference:

View Current Retention:

• Navigate to Data Retention page
• Check global default and per-device settings

Adjust Retention Period:

• Update retention hours for specific device
• Or modify global default in config/config.exs

Clean Old Data:

• Manual cleanup: Click "Clean Old Data" button
• Automatic cleanup runs hourly

Storage Planning:

• Monitor InfluxDB disk usage weekly
• Adjust retention based on available storage
• Balance retention period with storage capacity

Exporting Data

Export Device Configurations:

1. Navigate to device detail page
2. Click "Download Configuration"
3. Save XML file to safe location
4. Label with device name and date

Export Metrics (via InfluxDB):

# Export data for specific device
influx -database 'nokia-monitor' -execute '
  SELECT * FROM PerformanceMetrics
  WHERE basebandName='''Site-A-BS1'''
  AND time > now() - 7d
' -format csv > export.csv

Export via Grafana:

• Open dashboard
• Select time range
• Click "Share" � "Export" � "CSV"

---

Backup and Recovery

Regular Backups

What to Backup:

1. Configuration Files

# Backup runtime configuration
cp config/runtime.exs backups/runtime.exs.$(date +%Y%m%d)

# Backup entire config directory
tar -czf backups/config-$(date +%Y%m%d).tar.gz config/

2. Device Configurations

• Download configurations from all devices via Web UI
• Store in version control or backup location
• Perform weekly or before/after changes

3. MySQL Database

# Backup session state database
mysqldump -u ran_monitor_user -p ran_monitor > backups/ran_monitor-$(date +%Y%m%d).sql

4. InfluxDB Data

# InfluxDB 1.x backup
influxd backup -portable -database nokia-monitor /backups/influx-$(date +%Y%m%d)

# InfluxDB 2.x backup
influx backup /backups/influx-$(date +%Y%m%d)

5. SSL Certificates

cp priv/cert/* backups/certificates-$(date +%Y%m%d)/

Backup Schedule

Daily:

• MySQL database (session state)
• Configuration files if changed

Weekly:

• InfluxDB data (or per retention policy)
• Device configurations from all base stations

Before Changes:

• Configuration files
• Device configurations
• Database snapshot

Recovery Process

Recover from Configuration Error:

1. Stop RAN Monitor

   systemctl stop ran_monitor

2. Restore configuration file

   cp backups/runtime.exs.20251230 config/runtime.exs

3. Validate configuration

   elixir -c config/runtime.exs

4. Restart RAN Monitor

   systemctl start ran_monitor

5. Verify devices reconnect

Recover from Database Loss:

1. Stop RAN Monitor

   systemctl stop ran_monitor

2. Restore MySQL database

   mysql -u ran_monitor_user -p ran_monitor < backups/ran_monitor-20251230.sql

3. Restart RAN Monitor

   systemctl start ran_monitor

4. Devices will re-register automatically

5. New metrics collection begins

6. Historical data remains in InfluxDB

Recover from Complete System Loss:

1. Reinstall RAN Monitor on new server
2. Restore configuration files
3. Restore MySQL database
4. Restore InfluxDB data
5. Restore SSL certificates
6. Start RAN Monitor
7. Verify all devices reconnect
8. Verify historical data is accessible

---

System Maintenance

Routine Maintenance Tasks

Monthly:

1. Review Logs

• Archive old application logs
• Check for recurring errors or warnings
• Clean up log files to free disk space

2. Update Documentation

• Document any configuration changes
• Update network diagrams if topology changed
• Review and update operational procedures

3. Security Updates

• Apply OS security patches
• Update database software if needed
• Rotate passwords per security policy

4. Performance Review

• Analyze system resource trends
• Identify any performance degradation
• Plan capacity upgrades if needed

Quarterly:

1. Disaster Recovery Test

• Test backup restoration process
• Verify recovery procedures work
• Update disaster recovery documentation

2. Security Audit

• Review access logs
• Verify user permissions
• Update SSL certificates if expiring soon

3. Capacity Planning

• Review storage growth trends
• Forecast future capacity needs
• Plan hardware upgrades if needed

Annually:

1. SSL Certificate Renewal

• Replace expiring SSL certificates
• Test new certificates before expiry

2. Password Rotation

• Update all base station credentials
• Update database passwords
• Update API tokens

3. System Upgrade

• Plan RAN Monitor version upgrade
• Test in staging environment
• Schedule production upgrade

Maintenance Windows

Planning a Maintenance Window:

1. Schedule During Low Traffic:

   • Nights or weekends typically best
   • Avoid busy hours identified in Grafana

2. Notify Stakeholders:

   • Inform operations team
   • Update status page
   • Set expectations for downtime

3. Prepare Rollback Plan:

   • Backup current state
   • Document rollback steps
   • Have previous version ready if upgrading

4. Perform Maintenance:

   • Follow documented procedures
   • Monitor progress closely
   • Document any deviations

5. Verify System Health:

   • All devices reconnect
   • Metrics flowing normally
   • No errors in logs
   • Run health check procedures

6. Document Results:

   • Record what was done
   • Note any issues encountered
   • Update procedures if needed

---

Related Documentation

• Getting Started Guide - Initial setup procedures
• Web UI Guide - Control panel user guide
• Runtime Configuration Guide - Configuration reference
• AirScale Configuration - Base station setup
• Firmware Management - Software package repository and updates
• Grafana Integration Guide - Analytics and dashboards
• Alarm Management Guide - Alarm handling procedures
• Data Retention Policy Guide - Data lifecycle management
• Troubleshooting Guide - Problem resolution
• Operations Guide - Complete operational overview