Initial Checks
Role | Check | Comments |
---|---|---|
VPN | Connectivity | Make sure the VPN is running OK, and that all systems are accessible by SSH and RDP. |
Web Application | Connectivity | Connect via the IP address of the load balancer (if redundant, connect via the shared IP address). Login as sysadmin, switch to a user with admin rights, and check a few views:
|
Test Call | Functionality | Make a test call. Check that it is received by the system and that audio is heard. If possible:
|
Component and Role Checks
Role | Check | Comments |
---|---|---|
ALL LINUX | Uptime and Errors | Use: uptime to check for how long the system has been up. If it has been restarted, then check the following for why: less /var/log/messages You may need to go back to an older file than the current log file to see why. |
ALL LINUX | Disk Space | Use: df -h to make sure the disk is not full. Particularly on the database systems make sure this space will not run out soon. Note: on some systems, /var/lib/mysql may be mounted separately. Use: mount to verify how it is mounted or cat /etc/fstab |
ALL LINUX | CPU Load | Use both of the following to identify CPU load problems: top sar (historical) sar -u 1 (real-time) |
ALL LINUX | Time Sync | Use the following commands to make sure the time sync is OK: ntpq -p (older systems) chronyc sources (newer systems) timedatectl |
ALL LINUX | File System writable | Sometimes, if a file system error has occurred, linux will either:
Try writing to a file: cat << EOFF > /root/test.txt |
ALL LINUX | Expected processes running? | ps -ef | less |
WINDOWS | Time Sync | Check the ntp status using the monitor application. |
WINDOWS | CPU Load | Check the task manager to make sure no process is running very high CPU. Note: on a loaded system 50% or more CPU is perfectly fine, as long as it is either:
|
DB | DB Master-Master DB Slaves | Make sure replication is running:
Make sure MySQL is not crashing: less /var/log/mysqld.log |
LB | Redundant LB | Check the status of the pcs cluster with: pcs status Make sure one of the load balancers has the shared IP address, and that all resources are started. |
LB | haproxy Stats and Control Page | Open haproxy (on both load balancers in a redundant system) using the following url: http://<load-balancer>:7777 Make sure the expected resources are present and all up:
|
STORE | Redundant Store | Check the status of DRBD with: drbdadm status jtelshared Make sure it is primary/primary (for older installations) or primary secondary (for newer installations). Check the status of the pcs cluster with: pcs status Make sure all of the resources are started:
|
STORE | Non-Redundant Store | Check the status of the samba service: smbstatus ps -ef | grep samba Make sure it is running. |
STORE | Space | Check the available space on the store with df -h |
TEL | 8-Server started | startup.cmd batch file window present? giHal.exe batch file window present and running? giAcu.exe (SIP Registrations) present and running (if required)? 8Server.exe present and running? |
TEL | 8-Server health | Check errors / warnings for excessive error messages, such as ODBC errors or .r5 files not found errors |
TEL | PBX Connector | Telephony connector started and running? Messages being received from the PBX? |
TEL | Platform UDP Listener | Check the Platform UDP listener is started Does the cluster contain all expected members? For example: |
TEL | Disk Space | Use the windows explorer to make sure the disk is not full. |
WEB | Webservers | Check each webserver individually using http://<server>:8080/CarrierPortal/sysadmin/login Make sure it is possible to login. Make sure the logo is visible (if not, this indicates a file share mount error to /home/jtel/shared) |
WEB | Stats Counters | To check the general health of the web application and database, the following stats pages can be used: http(s)://<load-balancer>/CarrierPortal/pages/common/Stats.xhtml Cache counters (average should be below 1 ms): Page counters (average should be below 300 ms): |
CHAT | Application running | Make sure the chat application is running: ps -ef | grep jtel-clientmessenger |
REST | Application running | Make sure the chat application is running: ps -ef | grep jrest |