| Sv translation | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||
Introduction A jtel ACD consists of a minimum 2 of virtual machines and may grow to sizes of 30 or more, in concurrence with for example redundancy or performance requirements. Based on a systems architecture, there are certain dependencies between the services, so a clean startup and shutdown is required and only done in a certain order to prevent problems during and after startup. The following page describes the various possible scenarios for proper shutdown and startup procedures, depending on the systems architecture.
Glossary | |||||||||||
| Alias | Signifies |
| Info |
|---|
The Hostnames of an ACDs jtel virtual machines may not be in concurrence with the aliases displayed below. |
| Alias | Signifies | Shutdown Priority | Startup Priority |
|---|---|---|---|
| acd-dbm | Database Master | Third | Third |
| acd-dbm1 | First Database Master | Third | Third |
| acd-dbm2 | Second Database Master | Third | Third |
| acd-dbs | Database Slave | Second | Fourth |
| acd-dbs1 | First Database Slave | Second | Fourth |
| acd-dbs2 | Second Database Slave | Second | Fourth |
| acd-dbr | Reporting Database | Second | Fourth |
| acd-lb | The Load Balancer |
| Fourth | Second | ||
| acd-lb1 | First Load Balancer | Fourth | Second |
| acd-lb2 | Second Load Balancer | Fourth | Second |
| acd-store | The File Storage | Last | First |
acd-tel1 | The Telephony Machine(s) Numbered from 1 ... N | First | Last |
acd-jb1 | The Webserver Machine(s) Numbered from 1 ... N |
Simple V1 (One Linux Machine contains all components)
| First |
| Last |
Simple V2 (Two Linux Machines)
Dependencies
All services of the platform are dependent on two central entities:
- The Storage(s)
- The Database(s)
To boot the solution, the storage should always be booted first. If this is external to the solution, continue with the next step. Otherwise, the computer (or in case of redundancy - both computers) is started on which the storage is located. This is usually the load balancer, but for larger systems a separate storage machine could have been implemented. Afterwards, the master database is booted. As soon as its is finished, the slave can be started up. Finally, the telephony servers and web servers can be started up in any order, even simultaneously.
Small V1
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-store/lb/dbm/dbs/dbr/jb | Last | First |
Small V2
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-dbm/dbs/dbr | Second | Second |
| acd-store/lb/jb | Last | First |
Small V3
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd-dbm/dbs/dbr | Second | Second |
| acd-store/lb/ | Last | First |
Start Up
Order
A non-redundant solution is assumed in the following. The operation of a redundant solution requires extra schooling.
This results in the following starting order:
- Storage (Separates Storage acd-store or Load-Balancer acd-lb).
Wait until Storage is up. - With separate storage, now start the load balancer acd-lb.
There is no need to wait here. - acd-dbm - Database Master .
Wait until DB-Master is up. - acd-dbs - Database Slave- .
Wait until DB slave is up. - Boot acd-jb1 ... acd-jb(x) aswell as acd-tel1 ... acd-tel(x)
Checks
- Log on to the Web via the load balancer.
- Log on OK?
If Not, check /home/jtel/jboss-(version)/standalone/log/server.log - ESSENTIAL: Logo visible in the web application?
If not, the storage is not up. Check with mount on the web servers. If necessary restore mount /home/jtel/shared Mount.
- Log on OK?
- Call System.
- Calls are put through?
If not, check the telephony server.
Look for red messages in the telephony server and act accordingly. - Calls are signalled in the Agent Home or Mini Client?
If not, check the Hazelcast cluster (PlatformListener for telephony and web server).
- Calls are put through?
Shutdown
Order
Shutting down is done in the reverse order:
- Shutdown All Webservers acd-jb1 ... acd-jb(x) and Telephony servers acd-tel1 ... acd-tel(x)
Finish step or make sure that the computers are really shutting down before continuing. - Shut down database slave acd-dbs.
- Shut down database master acd-dbm.
- Shut down load balancer acd-lb.
- For separate storage, shut down acd-store.
It is not absolutely necessary to wait between the individual steps, just step 1 should be completed before continuing.
Medium V1
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd-dbs/dbr | Second | Second |
| acd-dbm | Third | Third |
| acd-store/lb/ | Last | First |
Medium V2
| Machine | Shutdown | Startup |
|---|---|---|
| acd-tel | First | Last |
| acd-jbN | First | Last |
| acd- |
| dbs/dbr | Second | Second |
| acd- |
| dbm |
| Third |
Simple V3
| Third | ||
| acd-lb | Fourth | Fourth |
| acd-store |
| Last | First |
Master-Slave
Redundant
Start Up
Dependencies
All services of the platform are dependent on two central entities:
- The Storage
- The Database(s)
To boot the solution, the storage should always be booted first. If this is external to the solution, continue with the next step. Otherwise, the computer (or in case of redundancy - both computers) is started on which the storage is located. This is usually the load balancer, but for larger systems a separate storage machine could have been implemented.
Afterwards, the master database is booted. As soon as its is finished, the slave can be started up.
Finally, the telephony servers and web servers can be started up in any order, even simultaneously.
Order
A non-redundant solution is assumed in the following. The operation of a redundant solution requires extra schooling.
This results in the following starting order:
- Storage (Separates Storage acd-store or Load-Balancer acd-lb).
Wait until Storage is up. - With separate storage, now start the load balancer acd-lb.
There is no need to wait here. - acd-dbm - Database Master .
Wait until DB-Master is up. - acd-dbs - Database Slave- .
Wait until DB slave is up. - Boot acd-jb1 ... acd-jb(x) aswell as acd-tel1 ... acd-tel(x)
Checks
- Log on to the Web via the load balancer.
- Log on OK?
If Not, check /home/jtel/jboss-(version)/standalone/log/server.log - ESSENTIAL: Logo visible in the web application?
If not, the storage is not up. Check with mount on the web servers. If necessary restore mount /home/jtel/shared Mount.
- Log on OK?
- Call System.
- Calls are put through?
If not, check the telephony server.
Look for red messages in the telephony server and act accordingly. - Calls are signalled in the Agent Home or Mini Client?
If not, check the Hazelcast cluster (PlatformListener for telephony and web server).
- Calls are put through?
Shutdown
Order
Shutting down is done in the reverse order:
- Shutdown All Webservers acd-jb1 ... acd-jb(x) and Telephony servers acd-tel1 ... acd-tel(x)
Finish step or make sure that the computers are really shutting down before continuing. - Shut down database slave acd-dbs.
- Shut down database master acd-dbm.
- Shut down load balancer acd-lb.
- For separate storage, shut down acd-store.
Shutdown/Startup
Shutting down redundant clusters is easier than starting them back up.
Some components must be checked directly after starting up, before work can continue.
Database:
- MySQL Replication between all replication partners
Load Balancing:
- pcs cluster status
Storage:
- pcs cluster status
- drbd status
Large V1 - Redundant Databases
When shutting down a redundant database architecture, the inactive machines are shutdown first. Therefore, before a shutdown procedure can begin, the active machines must be known. The following table assumes that acd-dbm2 is currently the active master database.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Second |
| acd-dbs2/dbr2 | Second | Second |
| acd-dbm1 | Third | Third |
| acd-dbm2 | Third | Third |
| acd-lb | Fourth | Fourth |
| acd-store | Last | First |
Large V2 - Redundant Databases + Load Balancing
Please view above for database shutdown.
When shutting down a redundant load balancing architecture, the inactive machines are shutdown first. Therefore, before a shutdown procedure can begin, the active machines must be known. The following table assumes that acd-lb2 is the active load balancer.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Seventh |
| acd-dbs2/dbr2 | Third | Sixth |
| acd-dbm1 | Fourth | Fifth |
| acd-dbm2 | Fifth | Fourth |
| acd-lb1 | Sixth | Third |
| acd-lb2 | Seventh | Second |
| acd-store | Last | First |
Large V3 - Redundant Databases + Load Balancing + Storage
Please view above for database and load balancer shutdown.
When shutting down a redundant storage architecture, the inactive machine is shutdown first. Therefore, before a shutdown procedure can begin, the active machine must be known. The following table assumes that acd-store2 is the active storage.
| Machine | Shutdown | Startup |
|---|---|---|
| acd-telN | First | Last |
| acd-jbN | First | Last |
| acd-dbs1/dbr1 | Second | Eighth |
| acd-dbs2/dbr2 | Third | Seventh |
| acd-dbm1 | Fourth | Sixth |
| acd-dbm2 | Fifth | Fifth |
| acd-lb1 | Sixth | Fourth |
| acd-lb2 | Seventh | Third |
| acd-store1 | Eighth | Second |
| acd-store2 | Last | First |
Startup and Shutdown - Redundant components
If the entire system is shut down, some components may not be immediately available or manual intervention may be necessary.
Storage (Redundant) - CentOS
| Warning |
|---|
This checklist is not viable for installations that use the new DRBD construct of Primary/Secondary. It is viable only for constructs that use Primary/Primary |
Shut down (only one)
Always shut down one after the other.
With:
| Translations Ignore | ||
|---|---|---|
|
determine which node is Primary.
On the Secondary Node:
| Translations Ignore | ||
|---|---|---|
|
Then shut down the Primary Node.
| Translations Ignore | ||
|---|---|---|
|
Boot up (only one)
After booting the Secondary Node, check the status. It should be Secondary and UpToDate
| Translations Ignore | ||
|---|---|---|
|
Check Sync status with :
| Translations Ignore | ||
|---|---|---|
|
After the sync is done, if necessary, and is ready:
| Translations Ignore | ||
|---|---|---|
|
Check status with: It should be Primary/Primary + UpToDate/UpToDate
| Translations Ignore | ||
|---|---|---|
|
Shut down (both)
With:
| Translations Ignore | ||
|---|---|---|
|
determine which node is active (the node on which Samba and the virtual IP is running). This is the Primary Node.
Set the secondary Node to secondary with:
| Translations Ignore | ||
|---|---|---|
|
On the Secondary Node first, then Primary:
| Translations Ignore | ||
|---|---|---|
|
Start up (both)
Boot the Primary Node first
Check DRBD sync Status and pcs cluster status
Check with:
| Translations Ignore | ||
|---|---|---|
|
The Primary Node should be primary, and Secondary Node should be secondary. Disc should be UpToDate
Then on Secondary Node
| Translations Ignore | ||
|---|---|---|
|
Check with: | Translations Ignore | ||
|---|---|---|
|
Both nodes should be set to Primary/Primary + UpToDate/UpToDate
Then check if everything works a final time:
| Translations Ignore | ||
|---|---|---|
|
| Sv translation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Hoch- und RunterfahrenAlle Systeme sind so gestaltet, dass Sie hochfahren und alle Dienste selbstständig starten. Dennoch ergeben sich gewisse Abhängigkeiten zwischen den Diensten, sodass ein sauberes Hoch- und Herunterfahren am besten geschieht, wenn eine gewisse Reihenfolge betrachtet wird. HochfahrenAbhängigkeitenAlle Dienste der Plattform sind von zwei zentrale Entitäten abhängig:
Für das Hochfahren der Lösung, sollte immer zuerst das Storage hochgefahren werden. Wenn dies Extern zur Lösung ist, dann fährt man mit den nächsten Schritt fort. Ansonsten, wird der Rechner (oder bei Redundanz - beide Rechner) gestartet auf den das Storage ist. Dies ist in der Regel der Load-Balancer, bei größere Systeme kann jedoch eine separate Storage-Maschine implementiert worden sein. Anschließend, wird die Master Datenbank hochgefahren. Sobald dieser oben ist, kann der Slave hochgefahren werden. Zum Schluss können die Telefonieserver und Webserver in einer beliebigen Reihenfolge, auch gleichzeitig, hochgefahren werden. ReihenfolgeIm Folgenden wird von einer nicht redundante Lösung ausgegangen. Der Betrieb einer redundanten Lösung bedarf eine gesonderte Schulung. Somit ergibt sich folgende Startreihenfolge:
Prüfungen
HerunterfahrenReihenfolgeDas Herunterfahren geschieht in der umgekehrten Reihenfolge:
Zwischen den einzelnen Schritten ist es nicht zwingend notwendig zu warten, ledeglich Schritt 1 sollte insgesamt abgeschlossen sein bevor man fortfährt. Hoch- und Runterfahren - Redundante KomponetenWenn das gesamte System heruntergefahren wird, kann es sein, dass manche Komponenten nicht sofort verfügbar sind, bzw. dass ein manueller Eingriff notwendig ist. Storage (Redundant)Runterfahren (nur einer)Immer eins nacheinander herunterfahren. Mit:
feststellen welcher Node aktiv ist. Auf den anderen node:
Dann diesen Node herunterfahren. Hochfahren (nur einer)Wurde nur ein Node neu gestartet, dann kann der Betrieb wieder aufgenommen werden indem man auf den Secondary folgendes nach dem Neustart eingibt:
Sync status prüfen mit:
Nachdem der Sync ggf. erfolgt ist, und fertig ist:
Runterfahren (beide)Mit:
feststellen welcher Node aktiv ist (der Node auf den das Samba sowie die virtuelle IP läuft). Dies ist der Primary Node. Mit:
den Cluster auf inaktiv setzen. Auf auf den Secondary zuerst, dann Primary:
Hochfahren (beide)Beide nodes booten. Auf beide Nodes (Primary zuerst):
Prüfen mit:
Beide Nodes sollten auf Secondary stehen, aber kein Sync sollte erfolgen. Dann auf beide Nodes (Primary zuerst):
Prüfen mit:
Beide Nodes sollten auf Primary stehen, aber kein Sync sollte erfolgen. Dann auf den Primary:
Dann prüfen ob alles läuft:
|
...