Under Construction
This site is currently under construction. Contained Information is subject to change.
Introduction
A jtel ACD consists of a minimum 2 of virtual machines and may grow to sizes of 30 or more, in concurrence with for example redundancy or performance requirements. Based on a systems architecture, there are certain dependencies between the services, so a clean startup and shutdown is required and only done in a certain order to prevent problems during and after startup. The following page describes the various possible scenarios for proper shutdown and startup procedures, depending on the systems architecture.
On Premise
Caution for on premise jtel ACD systems:
Shutting down and starting the virtual machines of your jtel ACD back is not part of the jtel service contract and may incur service fees as a result. For further information, or to book a service appointment, please contact us at service@jtel.de
Page Layout
This page is split into three rows, each named after a different type of sizing. The sizing variants are Small, Medium and Large
General
The Hostnames of an ACDs jtel virtual machines may not be in concurrence with the aliases displayed below.
| Alias | Signifies | Shutdown Priority | Startup Priority |
|---|---|---|---|
| acd-dbm | Database Master | Third | Third |
| acd-dbm1 | First Database Master | Third | Third |
| acd-dbm2 | Second Database Master | Third | Third |
| acd-dbs | Database Slave | Second | Fourth |
| acd-dbs1 | First Database Slave | Second | Fourth |
| acd-dbs2 | Second Database Slave | Second | Fourth |
| acd-dbr | Reporting Database | Second | Fourth |
| acd-lb | The Load Balancer | Fourth | Second |
| acd-lb1 | First Load Balancer | Fourth | Second |
| acd-lb2 | Second Load Balancer | Fourth | Second |
| acd-store | The File Storage | Last | First |
acd-tel1 | The Telephony Machine(s) Numbered from 1 ... N | First | Last |
acd-jb1 | The Webserver Machine(s) Numbered from 1 ... N | First | Last |
Dependencies
All services of the platform are dependent on two central entities:
- The Storage(s)
- The Database(s)
To boot the solution, the storage should always be booted first. If this is external to the solution, continue with the next step. Otherwise, the computer (or in case of redundancy - both computers) is started on which the storage is located. This is usually the load balancer, but for larger systems a separate storage machine could have been implemented. Afterwards, the master database is booted. As soon as its is finished, the slave can be started up. Finally, the telephony servers and web servers can be started up in any order, even simultaneously.
Small V1
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-store/lb/dbm/dbs/dbr/jb | Last | First |
Small V2
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-dbm/dbs/dbr | Second | Second |
| acd-store/lb/jb | Last | First |
Small V3
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd-dbm/dbs/dbr | Second | Second |
| acd-store/lb/ | Last | First |
Start Up
Order
A non-redundant solution is assumed in the following. The operation of a redundant solution requires extra schooling.
This results in the following starting order:
- Storage (Separates Storage acd-store or Load-Balancer acd-lb).
Wait until Storage is up. - With separate storage, now start the load balancer acd-lb.
There is no need to wait here. - acd-dbm - Database Master .
Wait until DB-Master is up. - acd-dbs - Database Slave- .
Wait until DB slave is up. - Boot acd-jb1 ... acd-jb(x) aswell as acd-tel1 ... acd-tel(x)
Checks
- Log on to the Web via the load balancer.
- Log on OK?
If Not, check /home/jtel/jboss-(version)/standalone/log/server.log - ESSENTIAL: Logo visible in the web application?
If not, the storage is not up. Check with mount on the web servers. If necessary restore mount /home/jtel/shared Mount.
- Log on OK?
- Call System.
- Calls are put through?
If not, check the telephony server.
Look for red messages in the telephony server and act accordingly. - Calls are signalled in the Agent Home or Mini Client?
If not, check the Hazelcast cluster (PlatformListener for telephony and web server).
- Calls are put through?
Shutdown
Order
Shutting down is done in the reverse order:
- Shutdown All Webservers acd-jb1 ... acd-jb(x) and Telephony servers acd-tel1 ... acd-tel(x)
Finish step or make sure that the computers are really shutting down before continuing. - Shut down database slave acd-dbs.
- Shut down database master acd-dbm.
- Shut down load balancer acd-lb.
- For separate storage, shut down acd-store.
It is not absolutely necessary to wait between the individual steps, just step 1 should be completed before continuing.
Medium V1
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd-dbs/dbr | Second | Second |
| acd-dbm | Third | Third |
| acd-store/lb/ | Last | First |
Medium V2
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd-dbs/dbr | Second | Second |
| acd-dbm | Third | Third |
| acd-lb | Fourth | Fourth |
| acd-store | Last | First |
Shutdown/Startup
Shutting down redundant clusters is easier than starting them back up.
Some components must be checked directly after starting up, before work can continue.
Database:
- MySQL Replication between all replication partners
Load Balancing:
- pcs cluster status
Storage:
- pcs cluster status
- drbd status
Large V1 - Redundant Databases
When shutting down a redundant database architecture, the inactive machines are shutdown first. Therefore, before a shutdown procedure can begin, the active machines must be known. The following table assumes that acd-dbm2 is currently the active master database.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Second |
| acd-dbs2/dbr2 | Second | Second |
| acd-dbm1 | Third | Third |
| acd-dbm2 | Third | Third |
| acd-lb | Fourth | Fourth |
| acd-store | Last | First |
Large V2 - Redundant Databases + Load Balancing
Please view above for database shutdown.
When shutting down a redundant load balancing architecture, the inactive machines are shutdown first. Therefore, before a shutdown procedure can begin, the active machines must be known. The following table assumes that acd-lb2 is the active load balancer.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Seventh |
| acd-dbs2/dbr2 | Third | Sixth |
| acd-dbm1 | Fourth | Fifth |
| acd-dbm2 | Fifth | Fourth |
| acd-lb1 | Sixth | Third |
| acd-lb2 | Seventh | Second |
| acd-store | Last | First |
Large V3 - Redundant Databases + Load Balancing + Storage
Please view above for database and load balancer shutdown.
When shutting down a redundant storage architecture, the inactive machine is shutdown first. Therefore, before a shutdown procedure can begin, the active machine must be known. The following table assumes that acd-store2 is the active storage.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Eighth |
| acd-dbs2/dbr2 | Third | Seventh |
| acd-dbm1 | Fourth | Sixth |
| acd-dbm2 | Fifth | Fifth |
| acd-lb1 | Sixth | Fourth |
| acd-lb2 | Seventh | Third |
| acd-store1 | Eighth | Second |
| acd-store2 | Last | First |
Startup and Shutdown - Redundant components
If the entire system is shut down, some components may not be immediately available or manual intervention may be necessary.
Storage (Redundant) - CentOS
This checklist is not viable for installations that use the new DRBD construct of Primary/Secondary. It is viable only for constructs that use Primary/Primary
Shut down (only one)
Always shut down one after the other.
With:
pcs status
determine which node is Primary.
On the Secondary Node:
drbdadm secondary jtelshared shutdown now
Then shut down the Primary Node.
shutdown now
Boot up (only one)
After booting the Secondary Node, check the status. It should be Secondary and UpToDate
pcs status
Check Sync status with :
cat /proc/drbd
After the sync is done, if necessary, and is ready:
drbdadm primary jtelshared
Check status with: It should be Primary/Primary + UpToDate/UpToDate
cat /proc/drbd
Shut down (both)
With:
pcs status
determine which node is active (the node on which Samba and the virtual IP is running). This is the Primary Node.
Set the secondary Node to secondary with:
drbdadm secondary jtelshared shutdown now
On the Secondary Node first, then Primary:
shutdown now
Start up (both)
Boot the Primary Node first
Check DRBD sync Status and pcs cluster status
Check with:
pcs status cat /proc/drbd
The Primary Node should be primary, and Secondary Node should be secondary. Disc should be UpToDate
Then on Secondary Node
drbdadm primary jtelshared
Check with:
cat /proc/drbd
Both nodes should be set to Primary/Primary + UpToDate/UpToDate
Then check if everything works a final time:
pcs status cat /proc/drbd