Versions Compared

Old Version 39

changes.mady.by.user jtel Support

Saved on

compared with

New Version 40

changes.mady.by.user jtel Support

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Sv translation
languageen

Notes

It is not trivial to resize this configuration, it is recommended to use Role STORE - DRBD File System (Redundant with LVM).

Create DRBD Partitions on disk (Both Nodes)

The commands below assume that /dev/sdb will be used for the DRBD partition.

Code Block
device=/dev/sdb
dd if=/dev/zero of=${device} obs=512 count=100
dd if=/dev/zero of=${device} obs=512 count=100 seek=$(( $(blockdev --getsz ${device}) -100 ))
parted ${device} "mklabel gpt"
parted ${device} "mkpart primary 0% 100%"

Verify that the partition is created:

Code Block
fdisk -l /dev/sdb
 
-->
 
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdb: 274.9 GB, 274877906944 bytes, 536870912 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt
Disk identifier: E7FF3D92-84BB-44E1-B0B0-26150DB80639


#         Start          End    Size  Type            Name
 1         2048    536868863    256G  Microsoft basic primary

Install DRBD Repos (Both Nodes)

Code Block
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm

Install DRBD Modules (Both Nodes)

Code Block
yum -y install drbd84-utils kmod-drbd84

Configure Firewall (Both Nodes) 

Code Block
firewall-cmd --zone=public --add-port=7788-7799/tcp --permanent
firewall-cmd --reload

Configure DRBD (Both Nodes)

NOTE: The following commands requires the hostname of both machines and the IP Address. These are obtained as follows:

Code Block
ip addr
hostname

Create a DRBD config file for jtelshared on /dev/sdb

Code Block
title/etc/drbd.d/jtelshared.res
cat <<EOFF > /etc/drbd.d/jtelshared.res
resource jtelshared {
    protocol C;
    meta-disk internal;
    device /dev/drbd1;
    syncer {
        verify-alg sha1;
    }
    net {
        allow-two-primaries;
    }
    on acd-store1 {
        disk   /dev/sdb1;
        address 10.4.8.71:7789;
    }
    on acd-store2 {
        disk   /dev/sdb1;
        address 10.4.8.171:7789;
    }
    startup {
        become-primary-on both;
    }
}
EOFF

Note: it has been observed, that the fully qualified host name is required in the configuration file. If the first step below fails, then try with the fully qualified host name.

Create Metadata and start (Both Nodes)

Code Block
drbdadm create-md jtelshared
drbdadm up jtelshared

Make one node primary (First Node)

Code Block
drbdadm primary jtelshared --force

Tune the transfer (Second Node)

Code Block
drbdadm disk-options --c-plan-ahead=0 --resync-rate=110M jtelshared

Create filesystem (First Node)

Code Block
mkfs.xfs -L data /dev/drbd1

Create fstab entry for file system (Both Nodes)

Add the following line to /etc/fstab

Code Block
/dev/drbd/by-res/jtelshared/0  /srv/jtel/shared         xfs noauto,noatime,nodiratime  0   0

Mount the file system (First Node)

Code Block
mkdir /srv/jtel
mkdir /srv/jtel/shared
chown -R jtel:jtel /srv/jtel
mount /srv/jtel/shared

Wait for initial sync to complete (Any Node)

Code Block
cat /proc/drbd
 
-->
 
# When not yet done:
 
version: 8.4.10-1 (api:1/proto:86-101)
GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-15 14:23:22

 1: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:3955712 dw:3950592 dr:0 al:8 bm:0 lo:5 pe:0 ua:5 ap:0 ep:1 wo:f oos:264474588
        [>....................] sync'ed:  1.5% (258272/262132)M
        finish: 2:08:08 speed: 34,388 (25,652) want: 112,640 K/sec


-->

# When done:

version: 8.4.10-1 (api:1/proto:86-101)
GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-15 14:23:22

 1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:15626582 dw:284051762 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Untune the transfer (Second Node)

Code Block
drbdadm adjust jtelshared

Make second node primary and mount the file system (Second node)

Code Block
mkdir /srv/jtel
mkdir /srv/jtel/shared
chown -R jtel:jtel /srv/jtel
drbdadm primary jtelshared
mount /srv/jtel/shared

Install Samba and lsof (Both Nodes)

Code Block
titleInstall SAMBA
yum -y install samba samba-client lsof

Configure Samba (Both Nodes)

Code Block
titleConfigure SAMBA
cat <<EOFF > /etc/samba/smb.conf
[global]
        workgroup = SAMBA
        security = user
        passdb backend = tdbsam
        printing = cups
        printcap name = cups
        load printers = yes
        cups options = raw
        min protocol = NT1
        ntlm auth = yes

[homes]
        comment = Home Directories
        valid users = %S, %D%w%S
        browseable = No
        read only = No
        inherit acls = Yes

[printers]
        comment = All Printers
        path = /var/tmp
        printable = Yes
        create mask = 0600
        browseable = No

[print$]
        comment = Printer Drivers
        path = /var/lib/samba/drivers
        write list = root
        create mask = 0664
        directory mask = 0775

[shared]
    comment = jtel ACD Shared Directory
    read only = no
    public = yes
    writable = yes
    locking = yes
    path = /srv/jtel/shared
    guest ok = yes
    create mask = 0644
    directory mask = 0755
    force user = jtel
    force group = jtel
    acl allow execute always = True

EOFF

Setup SeLinux, jtel User access and Firewall for Samba (Both Nodes)

Replace <password> with the actual password for the jtel user:

Code Block
titleSeLinux, jtel User, Firewall
setsebool -P samba_enable_home_dirs=on samba_export_all_rw=on use_samba_home_dirs=on use_nfs_home_dirs=on
printf '<password>\n<password>\n' | smbpasswd -a -s jtel
firewall-cmd --zone=public --add-port=445/tcp --add-port=139/tcp --add-port=138/udp --add-port=137/udp --permanent
firewall-cmd --reload

If necessary, add further users to samba:

Code Block
titleMore SAMBA users
useradd -m Administrator
printf 'F1r3B²11\nF1r3B²11\n' | smbpasswd -a -s Administrator

Test SAMBA (Both Nodes)

This test should be performed on the node which currently has /srv/jtel/shared mounted:

Code Block
titleTest SAMBA
mount /srv/jtel/shared
service nmb start
service smb start

 
# Now check access to the SMB share via (for example) one of the windows machines.
 
service smb stop
service nmb stop
umount /srv/jtel/shared
 
# do same again on other node

Unmount and disable SAMBA (Both Nodes)

Code Block
titleUnmount
service smb stop
service nmb stop
umount /srv/jtel/shared
systemctl disable smb

Install PCS Services (Both Nodes)

See Redundancy - Installing PCS Cluster.

Setup virtual IP (One Node Only!)

Change the following to set the virtual IP which should be shared between the nodes.

Code Block
titleSet virtual IP
KE_VIP=10.4.8.22

Configure PCS Resources (One Node Only!)

Configure the PCS resources with the following commands:

Code Block
titleConfigure PCS Resources
pcs resource create ClusterDataJTELSharedMount ocf:heartbeat:Filesystem device="/dev/drbd/by-res/jtelshared/0" directory="/srv/jtel/shared" fstype="xfs" --group=jtel_portal_group
pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=${KE_VIP} cidr_netmask=32 op monitor interval=30s --group=jtel_portal_group
pcs resource create samba systemd:smb op monitor interval=30s --group=jtel_portal_group
pcs constraint order start ClusterDataJTELSharedMount then ClusterIP
pcs constraint order start ClusterIP then samba

Test

Test as follows:

Code Block
titleTest pcs status
pcs status
 
--> shows the status of the newly created resources on both nodes, one node should be active. 
 
Cluster name: portal
Stack: corosync
Current DC: uk-acd-store2 (version 1.1.16-12.el7_4.8-94ff4df) - partition with quorum
Last updated: Mon Mar 19 15:40:24 2018
Last change: Mon Mar 19 15:40:16 2018 by root via cibadmin on uk-acd-store1

2 nodes configured
3 resources configured

Online: [ uk-acd-store1 uk-acd-store2 ]

Full list of resources:

 Resource Group: jtel_portal_group
     ClusterDataJTELSharedMount (ocf::heartbeat:Filesystem):    Started uk-acd-store1
     ClusterIP  (ocf::heartbeat:IPaddr2):       Started uk-acd-store1
     samba      (systemd:smb):  Started uk-acd-store1

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

Test the file mount:

Code Block
titleTest file mount
# From the windows machines:
 
dir \\uk-acd-store\shared

Test manual failover:

Code Block
titleTest file mount
# Failover to node 2
pcs cluster standby uk-acd-store1 
 
# ... (wait)
 
pcs status
 
# Then test the availability of the files from the windows machines.
# Create a new file before failing back (to make sure DRBD working ok).
 
# Fail back to node 1
pcs cluster unstandby uk-acd-store1
pcs cluster standby uk-acd-store2
 
# ... (wait)

pcs status

# Then test the availability of the files from the windows machines.
# Check that the new file created above is available.
 
# Unstandby node 2

pcs cluster unstandby uk-acd-store2

Manually link /home/jtel/shared (Both Nodes)

Code Block
titlelink /home/jtel/shared
ln -s /srv/jtel/shared /home/jtel/shared


Sv translation
languagede

Notes

It is not trivial to resize this configuration, it is recommended to use Role STORE - DRBD File System (Redundant with LVM).

Create DRBD Partitions on disk (Both Nodes)

The commands below assume that /dev/sdb will be used for the DRBD partition.

Code Block
device=/dev/sdb
dd if=/dev/zero of=${device} obs=512 count=100
dd if=/dev/zero of=${device} obs=512 count=100 seek=$(( $(blockdev --getsz ${device}) -100 ))
parted ${device} "mklabel gpt"
parted ${device} "mkpart primary 0% 100%"

Verify that the partition is created:

Code Block
fdisk -l /dev/sdb
 
-->
 
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdb: 274.9 GB, 274877906944 bytes, 536870912 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt
Disk identifier: E7FF3D92-84BB-44E1-B0B0-26150DB80639


#         Start          End    Size  Type            Name
 1         2048    536868863    256G  Microsoft basic primary

Install DRBD Repos (Both Nodes)

Code Block
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm

Install DRBD Modules (Both Nodes)

Code Block
yum -y install drbd84-utils kmod-drbd84

Configure Firewall (Both Nodes) 

Code Block
firewall-cmd --zone=public --add-port=7788-7799/tcp --permanent
firewall-cmd --reload

Configure DRBD (Both Nodes)

NOTE: The following commands requires the hostname of both machines and the IP Address. These are obtained as follows:

Code Block
ip addr
hostname

Create a DRBD config file for jtelshared on /dev/sdb

Code Block
title/etc/drbd.d/jtelshared.res
cat <<EOFF > /etc/drbd.d/jtelshared.res
resource jtelshared {
    protocol C;
    meta-disk internal;
    device /dev/drbd1;
    syncer {
        verify-alg sha1;
    }
    net {
        allow-two-primaries;
    }
    on acd-store1 {
        disk   /dev/sdb1;
        address 10.4.8.71:7789;
    }
    on acd-store2 {
        disk   /dev/sdb1;
        address 10.4.8.171:7789;
    }
    startup {
        become-primary-on both;
    }
}
EOFF

Note: it has been observed, that the fully qualified host name is required in the configuration file. If the first step below fails, then try with the fully qualified host name.

Create Metadata and start (Both Nodes)

Code Block
drbdadm create-md jtelshared
drbdadm up jtelshared

Make one node primary (First Node)

Code Block
drbdadm primary jtelshared --force

Tune the transfer (Second Node)

Code Block
drbdadm disk-options --c-plan-ahead=0 --resync-rate=110M jtelshared

Create filesystem (First Node)

Code Block
mkfs.xfs -L data /dev/drbd1

Create fstab entry for file system (Both Nodes)

Add the following line to /etc/fstab

Code Block
/dev/drbd/by-res/jtelshared/0  /srv/jtel/shared         xfs noauto,noatime,nodiratime  0   0

Mount the file system (First Node)

Code Block
mkdir /srv/jtel
mkdir /srv/jtel/shared
chown -R jtel:jtel /srv/jtel
mount /srv/jtel/shared

Wait for initial sync to complete (Any Node)

Code Block
cat /proc/drbd
 
-->
 
# When not yet done:
 
version: 8.4.10-1 (api:1/proto:86-101)
GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-15 14:23:22

 1: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:3955712 dw:3950592 dr:0 al:8 bm:0 lo:5 pe:0 ua:5 ap:0 ep:1 wo:f oos:264474588
        [>....................] sync'ed:  1.5% (258272/262132)M
        finish: 2:08:08 speed: 34,388 (25,652) want: 112,640 K/sec


-->

# When done:

version: 8.4.10-1 (api:1/proto:86-101)
GIT-hash: a4d5de01fffd7e4cde48a080e2c686f9e8cebf4c build by mockbuild@, 2017-09-15 14:23:22

 1: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:15626582 dw:284051762 dr:0 al:8 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Untune the transfer (Second Node)

Code Block
drbdadm adjust jtelshared

Make second node primary and mount the file system (Second node)

Code Block
mkdir /srv/jtel
mkdir /srv/jtel/shared
chown -R jtel:jtel /srv/jtel
drbdadm primary jtelshared
mount /srv/jtel/shared

Install Samba and lsof (Both Nodes)

Code Block
titleInstall SAMBA
yum -y install samba samba-client lsof

Configure Samba (Both Nodes)

Code Block
titleConfigure SAMBA
cat <<EOFF > /etc/samba/smb.conf
[global]
        workgroup = SAMBA
        security = user
        passdb backend = tdbsam
        printing = cups
        printcap name = cups
        load printers = yes
        cups options = raw
        min protocol = NT1
        ntlm auth = yes

[homes]
        comment = Home Directories
        valid users = %S, %D%w%S
        browseable = No
        read only = No
        inherit acls = Yes

[printers]
        comment = All Printers
        path = /var/tmp
        printable = Yes
        create mask = 0600
        browseable = No

[print$]
        comment = Printer Drivers
        path = /var/lib/samba/drivers
        write list = root
        create mask = 0664
        directory mask = 0775

[shared]
    comment = jtel ACD Shared Directory
    read only = no
    public = yes
    writable = yes
    locking = yes
    path = /srv/jtel/shared
    guest ok = yes
    create mask = 0644
    directory mask = 0755
    force user = jtel
    force group = jtel
    acl allow execute always = True

EOFF

Setup SeLinux, jtel User access and Firewall for Samba (Both Nodes)

Replace <password> with the actual password for the jtel user:

Code Block
titleSeLinux, jtel User, Firewall
setsebool -P samba_enable_home_dirs=on samba_export_all_rw=on use_samba_home_dirs=on use_nfs_home_dirs=on
printf '<password>\n<password>\n' | smbpasswd -a -s jtel
firewall-cmd --zone=public --add-port=445/tcp --add-port=139/tcp --add-port=138/udp --add-port=137/udp --permanent
firewall-cmd --reload

If necessary, add further users to samba:

Code Block
titleMore SAMBA users
useradd -m Administrator
printf 'F1r3B²11\nF1r3B²11\n' | smbpasswd -a -s Administrator

Test SAMBA (Both Nodes)

This test should be performed on the node which currently has /srv/jtel/shared mounted:

Code Block
titleTest SAMBA
mount /srv/jtel/shared
service nmb start
service smb start

 
# Now check access to the SMB share via (for example) one of the windows machines.
 
service smb stop
service nmb stop
umount /srv/jtel/shared
 
# do same again on other node

Unmount and disable SAMBA (Both Nodes)

Code Block
titleUnmount
service smb stop
service nmb stop
umount /srv/jtel/shared
systemctl disable smb

Install PCS Services (Both Nodes)

See Redundancy - Installing PCS Cluster.

Setup virtual IP (One Node Only!)

Change the following to set the virtual IP which should be shared between the nodes.

Code Block
titleSet virtual IP
KE_VIP=10.4.8.22

Configure PCS Resources (One Node Only!)

Configure the PCS resources with the following commands:

Code Block
titleConfigure PCS Resources
pcs resource create ClusterDataJTELSharedMount ocf:heartbeat:Filesystem device="/dev/drbd/by-res/jtelshared/0" directory="/srv/jtel/shared" fstype="xfs" --group=jtel_portal_group
pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=${KE_VIP} cidr_netmask=32 op monitor interval=30s --group=jtel_portal_group
pcs resource create samba systemd:smb op monitor interval=30s --group=jtel_portal_group
pcs constraint order start ClusterDataJTELSharedMount then ClusterIP
pcs constraint order start ClusterIP then samba

Test

Test as follows:

Code Block
titleTest pcs status
pcs status
 
--> shows the status of the newly created resources on both nodes, one node should be active. 
 
Cluster name: portal
Stack: corosync
Current DC: uk-acd-store2 (version 1.1.16-12.el7_4.8-94ff4df) - partition with quorum
Last updated: Mon Mar 19 15:40:24 2018
Last change: Mon Mar 19 15:40:16 2018 by root via cibadmin on uk-acd-store1

2 nodes configured
3 resources configured

Online: [ uk-acd-store1 uk-acd-store2 ]

Full list of resources:

 Resource Group: jtel_portal_group
     ClusterDataJTELSharedMount (ocf::heartbeat:Filesystem):    Started uk-acd-store1
     ClusterIP  (ocf::heartbeat:IPaddr2):       Started uk-acd-store1
     samba      (systemd:smb):  Started uk-acd-store1

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

Test the file mount:

Code Block
titleTest file mount
# From the windows machines:
 
dir \\uk-acd-store\shared

Test manual failover:

Code Block
titleTest file mount
# Failover to node 2
pcs cluster standby uk-acd-store1 
 
# ... (wait)
 
pcs status
 
# Then test the availability of the files from the windows machines.
# Create a new file before failing back (to make sure DRBD working ok).
 
# Fail back to node 1
pcs cluster unstandby uk-acd-store1
pcs cluster standby uk-acd-store2
 
# ... (wait)

pcs status

# Then test the availability of the files from the windows machines.
# Check that the new file created above is available.
 
# Unstandby node 2

pcs cluster unstandby uk-acd-store2

Manually link /home/jtel/shared (Both Nodes)

Code Block
titlelink /home/jtel/shared
ln -s /srv/jtel/shared /home/jtel/shared