Slave Disk FullThere are several reasons why the slave disk can become full. Following is a summary of the different cases and further below is a detailed explanation: Case |
---|
Temp directory full |
---|
Lots of "relay logs" available |
---|
ibdata files on the slave "very large" |
---|
Still no disk space > 20% free |
---|
tmp Directory Is FullIntroductionEvery time a query creates a tmp table, it is written to the temp directory, usually /tmp. This happens when the maximum size exceeds the maximum "in memory" table size. This is defined with the variables tmp_table_size aswell as max_heap_table_size . The tables in /tmp are kept until the respective DB connection is closed or a DROP TEMPORARY TABLE is called. If the /tmp directory is full, it is likely that a DROP TEMPORARY TABLE is missing somewhere. This can also happen through customer queries to the DB. The installation of tmpwatch creates help in a permanent way Hints: - On CentOS 6 the /tmp directory is then by default freed from files that have not been accessed for > 10 days
- Circumstantially this may not be sufficient
- On CentOS 7 the /tmp directory is then freed by default from files that have not been accessed for> 1 day
Procedure- Space must first be made available. Just mercilessly delete everything from /tmp.
- If this provides enough space, then restart the mysql service first: service mysql restart
- If there is enough space (at least about 20%), then proceed with replication as described above.
Lots of "relay logs" availableIntroductionMySQL first writes the relay logs from the master to a file. Once replication is interrupted, but the slave relay process continues to operate, the disk is filled by relay logs. This step should be done in any case, especially before the next one (ibdata too large) to make room. ProcedureThe files for the database are usually located in /var/lib/mysql If not, the location can be found in /etc/my.cnf The corresponding entry is datadir=(pfad) Delete all relay logs: Info |
---|
cd /var/lib/mysql rm mysqld-relay-bin* |
Restart MySQL service Code Block |
---|
service mysqld restart |
If enough space is available (at least 20%) then proceed with the slave recovery as described above. ibdata files on the slave "very large"IntroductionThe files for the database are usually located in /var/lib/mysql Due to not clearly documented MySQL internas, the file /var/lib/mysql/ibdata1 can be huge compared to the master database. To remedy this, you have to proceed a little more rigorously. ProcedureIf no less than 100% disk can be achieved by the steps above: Unsubscribe the MySQL service from the autostart: Code Block |
---|
systemctl disable mysqld |
Restart the computer with : Code Block |
---|
systemctl reboot now |
Only works if less than 100% disk is reached (possibly after reboot above) then Log on to the mysql server: Code Block |
---|
mysql -u root -p |
Drop all JTEL databases: Code Block |
---|
SET FOREIGN_KEY_CHECKS=0;
DROP DATABASE JTELLog;
DROP DATABASE JTELStats;
DROP DATABASE JTELWeb;
SET FOREIGN_KEY_CHECKS=1; |
Press CTRL+C to return to the command line, and then Code Block |
---|
service mysqld stop
rm /var/lib/mysql/ibdata*
rm /var/lib/mysql/ib_log_* |
Start MySql Server, and enable it again if necessary: Code Block |
---|
service mysqld start
service mysqld enable |
Check disk space, and proceed with Slave Restore as described above. Still no disk space > 20% freeIntroductionThe slave disk space is still full, more than 80% of the space on the drive is used by normal operation SolutionIn this case the slave is simply too small. The hard disk must be expanded (as with extending the STORE role, apply only to the logical volume where the MySQL database data resides). Then perform the steps again, if the disk is too small, then restore the slave as described above. Or the slave is completely rebuilt with a larger plate.
|