Requirement

The system must first be updated to a version with MySQL 8.x (Release 3.15 to 3.17).

Preparation for the update

With every update there is a golden rule that must be strictly followed regardless of the customer's wishes:

Determine Authoritative Share 

In some installations, the file share is on a system provided by the customer.

In this case, the repositories are checked out locally to the load balancer on the non-authoritative share, but the files are later copied to the "authoritative" file share.

After changes should be checked for the Authoritative File-Share.

Authoritative File-Share

This is /home/jtel/shared

Non-Authoritative File-Share

This is /srv/jtel/shared

Determine if Authoritative

If /home/jtel/shared is mounted on the load balancer from a third-party system, it is usually not authoritative.

You can easily determine this by creating a file on /srv/jtel/shared. If this does not appear in /home/jtel/shared, then /srv/jtel/shared is not authoritative.

Data backup

A data backup should unconditionally be made. The MINIMUM is that the database is backed up and the data directory is backed up, but this should only be a stopgap solution.

Better is a snapshot of all virtual machines involved, in cold state (shut down).

There is a risk that if a snapshot is taken during operation (especially of the database machines), it may not be consistent.

Manual backup

Dienste Stoppen

service jboss stop

service wildfly stop

service jtel-listener stop

service jtel-clientmessenger stop

Backup database (on DB master)

Enter the corresponding password at <password>:

su jtel
cd
mkdir -p /home/jtel/shared/Backup
mysqldump -u root -p<password> --add-drop-database --add-drop-table --events --routines --triggers --databases JTELWeb JTELStats JTELStats2 JTELLog | gzip > /home/jtel/shared/Backup/backup-$(hostname -s)-database-$(date +%F-%H%M%S).sql.gz

Backup Data Directory

This can be done via the Windows Explorer or a corresponding cp command on the Linux.

Remember GIT status (Authoritative Share)

From the load balancer, check the GIT status:

cd /home/jtel/shared/JTEL
git status
git rev-parse HEAD
(save output)
 
cd /home/jtel/shared/JTELCarrierPortal
git status
git rev-parse HEAD
(save output)
 
# IF A custom directory exists
 
cd /home/jtel/shared/Customer_Directory
git status
git rev-parse HEAD
(save output)

Snapshot Backup

This is done via the VMWare console, or Hyper-V console. This is usually done by the customer.

Confirmation

Restore

Snapshots

The snapshots are all reactivated and booted. There should be normal operation with the old software.

Manual Restoration

With all services stopped, restore the database. Enter the corresponding password at <password>:

cp /home/jtel/shared/Backup/backup-acd-dbm-database-2016-06-30-293211.sql.gz .
gunzip backup-acd-dbm-database-2016-06-30-293211.sql.gz
mysql -u root -p<password> -e"backup-acd-dbm-database-2016-06-30-293211.sql"

In parallel, the old data directory can be restored.

Further steps may be necessary (restore GIT repositories, restore 8 servers, ...) depending on which components were affected by the update.

The updating of the system

It should be mentioned again: all jtel services of the system must be shut down. 

After that the update can be started. The steps are performed in the order documented here

Updating the software on the STORE

This step must be performed as user "jtel".

Check if changes / patches are available

Before updating the software, you should check whether files in the respective directory have been changed.

Here is an example based on the software directory JTELCarrierPortal

su jtel
cd /home/jtel/shared/JTELCarrierPortal
git status
git rev-parse HEAD
exit

If this indicates that files have been modified, caution is advised.

The project engineer must first check whether the changes are local bug fixes that have been incorporated into the release. If this is the case, the changes can be deleted before updating. If this is not done, the update may fail!

su jtel
cd /home/jtel/shared/JTELCarrierPortal
git checkout -- .
exit

However, a general procedure cannot be described in such a case. In this case, the procedure must be checked on a case-by-case basis.

If file mode changes should be ignored, the following command can be used:

git config core.fileMode false

Updating the repositories (always on non-authoritative share)

The following command sequence updates all software directories to the latest version of the respective chosen release.

su jtel
cd /srv/jtel/shared/JTELCarrierPortal
git pull
cd /srv/jtel/shared/JTEL
git pull
cd /srv/jtel/shared/Customer_Directory
git pull
exit

If errors occur in GIT, for example:

Merge-Conflict in (file name)

Oder

fatal: Deny the merge of unrelated histories.

the local repository must be reset. This can be done as follows (example for stable-3.18 on JTELCarrierPortal)

su jtel
cd /srv/jtel/shared/JTELCarrierPortal
git reset --hard origin/release/stable-3.18

Checking out the desired release

After the GIT PULL is done, the corresponding release must be checked out if a release other than git status is desired.

su jtel
cd /srv/jtel/shared/JTELCarrierPortal
git checkout release/stable-3.18
cd /srv/jtel/shared/JTEL
git checkout release/stable-3.18
cd /srv/jtel/shared/Customer_Directory
git checkout release/stable-3.18
exit

git fetch --unshallow

git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

git fetch origin

Copy files to Authoritative Share

Only if there is an Authoritative / Non-Authoritative share.

This is done with the following script

cd /srv/jtel/shared/JTELCarrierPortal/
bash update_fileshare.sh

Run update scripts

The following update script makes all necessary changes. This is executed as root or jtel with sudo. Here with jtel user:

cd /home/jtel/shared/JTELCarrierPortal/Update
sudo ./update_release.sh

You may be asked for the sudo password first. 

The script asks for the corresponding master database, user, password, and name of the customer directory where specific software for the customer may be located.

The questions can be answered, usually with the following answers, knowledge of <password> for the database assumed:

• mysql host address: acd-dbm
• mysql user name: root
• mysql password: <password>
• customer directory:

If there is no customer directory, this can be left empty. The presence of custom software can be checked with ls /home/jtel/shared - there you can see the names of the directories. Enter only the name of the directory! For example acme

The update is then carried out.

Download Binaries

The binaries are downloaded with the following command:

cd /home/jtel/shared/JTELCarrierPortal/Update
su jtel
bash get_binaries.sh

Update of the Platform UDP Listener

Master Databases

With release 3.18 and higher, Java 8 is used. The (existing) Java 7 must be uninstalled and Java 8 will be installed.

We use the openjdk so that it remains updateable under Linux.

service jtel-listener stop
yum -y remove jdk.x86_64
yum -y install https://cdn.jtel.de/downloads/java/jdk-8u202-linux-x64.rpm

The update of the Platform UDP Listener must be executed as user "root" and is performed on the master databases by entering the following command:

updatepl.sh

This command updates the listener and starts it. The success can be checked by viewing the file /home/jtel/PlatformUDPlistener/log/listener.log.

Telephony servers

Here you have to check if Java 8 is installed. If necessary, download Java 8 from one of the sources here (these are BCL licensed versions, i.e. license-free):

• https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html
• https://cdn.jtel.de/downloads/java/jdk-8u202-windows-x64.exe

The start script for the platform listener copies the new version of the listener itself.

Updating the WEB servers

The JBOSS server is completely replaced by a Wildfly server.

Backup Hazelcast configuration

If no backup already exists:

cp /home/jtel/jboss-as-7.1.1.FINAL/standalone/configuration/hazelcast.xml /home/jtel/shared

Uninstalling JBOSS Server and Java 7

As root:

service jboss stop
yum -y remove jdk.x86_64
cd /home/jtel
chkconfig jboss off
rm -f /etc/init.d/jboss
rm -f /etc/cron.daily/jboss-logmaint.sh
rm -f /etc/cron.daily/jboss-restart.sh
rm -f /usr/local/bin/updatejb.sh
rm -Rf jboss-as-7.1.1.FINAL

Install Java 8 and Wildfly

yum -y install https://cdn.jtel.de/downloads/java/jdk-8u202-linux-x64.rpm
wget http://cdn.jtel.de/downloads/jboss/wildfly-18.0.1.Final.0305.tar.gz
tar xzf wildfly-18.0.1.Final.0305.tar.gz
rm -f wildfly-18.0.1.Final.0305.tar.gz
mkdir -p wildfly-18.0.1.Final/standalone/deployments
chown -R jtel:jtel wildfly-18.0.1.Final
mkdir -p /home/jtel/upload
chown -R jtel:jtel /home/jtel/upload
ln -s /home/jtel/wildfly-18.0.1.Final wildfly-current
cp /home/jtel/wildfly-current/init.d/wildfly /etc/init.d/wildfly
ln -s /home/jtel/wildfly-current/default/wildfly /etc/default/wildfly
cd /etc/cron.daily
ln -s /home/jtel/wildfly-current/bin/jboss-logmaint.sh
ln -s /home/jtel/wildfly-current/bin/jboss-restart.sh
cd /usr/local/bin
ln -s /home/jtel/wildfly-current/bin/updatejb.sh
chkconfig wildfly on

Configure Wildfly

The hazelcast.xml must be copied and the default settings for the database must be adjusted.

In the following replace <password> please accordingly:

DBPRI=acd-dbm
DBSTA=acd-dbs
DBREP=acd-dbr
DBPWD=<password>
sed -i -e "s/DATA_PRIMARY/${DBPRI}/g" -e "s/DATA_STATS/${DBSTA}/g" -e "s/DATA_REPORTS/${DBREP}/g" -e "s/DB_PASSWORD/${DBPWD}/g" /home/jtel/wildfly-current/standalone/configuration/standalone.xml
unset DBPWD
unset DBREP
unset DBSTA
unset DBPRI
cp /home/jtel/shared/hazelcast.xml /home/jtel/wildfly-current/standalone/configuration/

Updating the portal software

The update of the portal software must be executed as user "root" on the web server and is done by entering the following command:

updatejb.sh

This command updates the portal software and starts the wildfly server. The success can be checked by viewing the file /home/jtel/wildfly-current/standalone/log/server.log.

 There you should see the following entries in the log file if the update was successful:

Updating Client Messenger

If Chat Server is installed.

For  jtel portal versions >= 3.25, make sure to edit systemd and/or init.d:

If you are using init.d:

vi /home/jtel/ClientMessenger/init.d/jtel-clientmessenger

# make sure this line exists
-Djava.net.preferIPv4Stack=true \

# make sure this line contains \&serverTimezone=Europe/Berlin
-Dde.jtel.platform.clientmessenger.connection=jdbc:mysql://acd-dbm/JTELWeb?user=root\&password=<password>\&characterEncoding=utf8\&serverTimezone=Europe/Berlin \

# copy to etc
cp /home/jtel/ClientMessenger/init.d/jtel-clientmessenger /etc/init.d/
chmod 755 /etc/init.d/jtel-clientmessenger 

If you are using systemd:

vi /home/jtel/ClientMessenger/systemd/jtel-clientmessenger.service  

# make sure this line exists
-Djava.net.preferIPv4Stack=true \

# make sure this line contains &serverTimezone=Europe/Berlin
-Dde.jtel.platform.clientmessenger.connection=jdbc:mysql://acd-dbm/JTELWeb?user=root&password=<Password>&characterEncoding=utf8&serverTimezone=Europe/Berlin \

# copy to systemd
cp /home/jtel/ClientMessenger/systemd/jtel-clientmessenger.service /etc/systemd/system/jtel-clientmessenger.service
systemctl daemon-reload

Then update:

updatesc.sh

Update of the 8Server

The 8Server is updated in Windows Explorer using GIT. To do this, right-click on the directory C:\8Server\deploy to perform a GIT pull and switch to the desired version after the pull.

If you do not want to update to the last version, but to a specific version, you can do this via "git checkout", by indicating the release.

After updating, the version must then be installed. For a daemon server, this is done with the command:

C:
cd \8Server\Install
copy_keyfile_en_daemon.cmd

For a telephony server with Aculab Media Service, this is done with the following command:

C:
cd \8Server\Install
copy_keyfile_en_mcp.cmd

Afterwards the 8Server can be started.

Updating the connectors

Basically: the corresponding files from the software directory in the STORE must be copied into the target directory.

These usually come from JTELCarrierPortal/WebServices/Install

Final work

Once the update has been completed and the system restarted without errors, the following tasks must be performed:

• Login to the portal. Short test to see that agents and supervisors also get to see what they expect.
• Test call! Ideally there is a test number. If not, call one of the regular service numbers. Check that the 8Server answers correctly.
  
  • Test call with observation of the portal.
    Log in as a user, and see if a call is displayed correctly in the portal when it is delivered to the agent.