MAKING RAW MyISAM BACKUP IN MySQL (Article -19).
MyISAM was the default storage engine for the MySQL relational database management system versions prior to 5.5 released in December 2009.
it is based on the older ISAM code.
Filesystem
Each MyISAM table is stored on disk in three files (if it is not partitioned). The files have names that begin with the table name and have an extension to indicate the file type. MySQL uses a .frm file to store the definition of the table, but this file is not a part of the MyISAM engine; instead it is a part of the server. The data file has a .MYD (MYData) extension. The index file has a .MYI (MYIndex) extension.
MyISAM FEATURES
MyISAM
tables have the following characteristics:
All data values are stored with the low byte first. This makes the data machine and operating system independent. The only requirements for binary portability are that the machine uses two’s-complement signed integers and IEEE floating-point format. These requirements are widely used among mainstream machines. Binary compatibility might not be applicable to embedded systems, which sometimes have peculiar processors.There is no significant speed penalty for storing data low byte first; the bytes in a table row normally are unaligned and it takes little more processing to read an unaligned byte in order than in reverse order. Also, the code in the server that fetches column values is not time critical compared to other code.
All numeric key values are stored with the high byte first to permit better index compression.
Large files (up to 63-bit file length) are supported on file systems and operating systems that support large files.
There is a limit of (232)2 (1.844E+19) rows in a MyISAM
table.
The maximum number of indexes per MyISAM
table is 64.The maximum number of columns per index is 16.
The maximum key length is 1000 bytes. This can also be changed by changing the source and recompiling. For the case of a key longer than 250 bytes, a larger key block size than the default of 1024 bytes is used.
When rows are inserted in sorted order (as when you are using an AUTO_INCREMENT
column), the index tree is split so that the high node only contains one key. This improves space utilization in the index tree.
Internal handling of one AUTO_INCREMENT
column per table is supported. MyISAM
automatically updates this column for INSERT
and UPDATE
operations. This makes AUTO_INCREMENT
columns faster (at least 10%). Values at the top of the sequence are not reused after being deleted. (When an AUTO_INCREMENT
column is defined as the last column of a multiple-column index, reuse of values deleted from the top of a sequence does occur.) TheAUTO_INCREMENT
value can be reset with ALTER TABLE
or myisamchk.
Dynamic-sized rows are much less fragmented when mixing deletes with updates and inserts. This is done by automatically combining adjacent deleted blocks and by extending blocks if the next block is deleted.
MyISAM
supports concurrent inserts: If a table has no free blocks in the middle of the data file, you can INSERT
new rows into it at the same time that other threads are reading from the table. A free block can occur as a result of deleting rows or an update of a dynamic length row with more data than its current contents. When all free blocks are used up (filled in), future inserts become concurrent again. See Section 8.11.3, “Concurrent Inserts”.
You can put the data file and index file in different directories on different physical devices to get more speed with the DATA DIRECTORY
and INDEX DIRECTORY
table options to CREATE TABLE
. See Section 13.1.20, “CREATE TABLE Syntax”.
BLOB
and TEXT
columns can be indexed.
NULL
values are permitted in indexed columns. This takes 0 to 1 bytes per key.
Each character column can have a different character set. See Chapter 10, Character Sets, Collations, Unicode.
There is a flag in the MyISAM
index file that indicates whether the table was closed correctly. If mysqld is started with the --myisam-recover-options
option, MyISAM
tables are automatically checked when opened, and are repaired if the table wasn’t closed properly.
How to create tables in MyISAM storage engine ?
MySQL 5.6 on-wards , it is normally necessary to use ENGINE
to specify the MyISAM
storage engine because InnoDB
is the default engine.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
mysql> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | crm | | mysql | | performance_schema | | sys | | world | +--------------------+ 6 rows in set (0.00 sec) mysql> create database AJAY_MyISAM; Query OK, 1 row affected (0.00 sec) mysql> use AJAY_MyISAM; Database changed |
Now lets create an object inside AJAY_MyISAM database.
1 2 |
mysql> create table test1(empno int,ename varchar(10)) ENGINE=MYISAM; Query OK, 0 rows affected (0.00 sec) |
Here we created a object called test1 under MyISAM storage engine.
Now check table information.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
mysql> show table status\G *************************** 1. row *************************** Name: test1 Engine: MyISAM Version: 10 Row_format: Dynamic Rows: 0 Avg_row_length: 0 Data_length: 0 Max_data_length: 281474976710655 Index_length: 1024 Data_free: 0 Auto_increment: NULL Create_time: 2018-12-12 14:58:15 Update_time: 2018-12-12 14:58:15 Check_time: NULL Collation: latin1_swedish_ci Checksum: NULL Create_options: Comment: 1 row in set (0.00 sec) mysql> select engine from information_schema.tables where table_name='test1'; +--------+ | engine | +--------+ | MyISAM | +--------+ 1 row in set (0.01 sec) mysql> select engine ,table_name,table_rows from information_schema.tables where TABLE_SCHEMA='AJAY_MyISAM'; +--------+------------+------------+ | engine | table_name | table_rows | +--------+------------+------------+ | MyISAM | test1 | 0 | +--------+------------+------------+ |
Lets create one more table under InnoDB storage and check the differences with physical files.
1 2 3 4 5 6 7 8 9 10 11 |
mysql> create table test2 (empno int); Query OK, 0 rows affected (0.02 sec) mysql> select engine ,table_name,table_rows from information_schema.tables where TABLE_SCHEMA='AJAY_MyISAM'; +--------+------------+------------+ | engine | table_name | table_rows | +--------+------------+------------+ | MyISAM | test1 | 0 | | InnoDB | test2 | 0 | +--------+------------+------------+ 2 rows in set (0.00 sec) |
Goto datadir and check the files
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
mysql> show variables like '%datadir%'; +---------------+-----------------+ | Variable_name | Value | +---------------+-----------------+ | datadir | /var/lib/mysql/ | +---------------+-----------------+ 1 row in set (0.01 sec) mysql> exit Bye [root@ip-172-31-47-50 ~]# cd /var/lib/mysql/ [root@ip-172-31-47-50 mysql]# ll total 123636 drwxr-x---. 2 mysql mysql 105 Dec 12 15:32 AJAY_MyISAM -rw-r-----. 1 mysql mysql 56 Dec 3 13:57 auto.cnf -rw-------. 1 mysql mysql 1679 Dec 3 13:57 ca-key.pem -rw-r--r--. 1 mysql mysql 1107 Dec 3 13:57 ca.pem -rw-r--r--. 1 mysql mysql 1107 Dec 3 13:57 client-cert.pem -rw-------. 1 mysql mysql 1679 Dec 3 13:57 client-key.pem drwxr-x---. 2 mysql mysql 68 Dec 3 15:09 crm -rw-r-----. 1 mysql mysql 330 Dec 11 15:15 ib_buffer_pool -rw-r-----. 1 mysql mysql 12582912 Dec 12 15:32 ibdata1 -rw-r-----. 1 mysql mysql 50331648 Dec 12 15:32 ib_logfile0 -rw-r-----. 1 mysql mysql 50331648 Dec 3 13:57 ib_logfile1 -rw-r-----. 1 mysql mysql 12582912 Dec 12 14:57 ibtmp1 -rw-r-----. 1 mysql mysql 1604 Dec 11 15:15 ip-172-31-47-50-bin.000002 -rw-r-----. 1 mysql mysql 691038 Dec 12 15:32 ip-172-31-47-50-bin.000003 -rw-r-----. 1 mysql mysql 58 Dec 12 13:50 ip-172-31-47-50-bin.index drwxr-x---. 2 mysql mysql 4096 Dec 3 13:57 mysql srwxrwxrwx. 1 mysql mysql 0 Dec 12 13:50 mysql.sock -rw-------. 1 mysql mysql 5 Dec 12 13:50 mysql.sock.lock drwxr-x---. 2 mysql mysql 8192 Dec 3 13:57 performance_schema -rw-------. 1 mysql mysql 1679 Dec 3 13:57 private_key.pem -rw-r--r--. 1 mysql mysql 451 Dec 3 13:57 public_key.pem -rw-r--r--. 1 mysql mysql 1107 Dec 3 13:57 server-cert.pem -rw-------. 1 mysql mysql 1679 Dec 3 13:57 server-key.pem drwxr-x---. 2 mysql mysql 8192 Dec 3 13:57 sys drwxr-x---. 2 mysql mysql 144 Dec 12 14:12 world |
There is directory create for AJAY_MyISAM database with the same name .
Here Test1 object craeted with three files which is under MyISAM.
Now will see how we will backup MyISAM Storage Engine tables.
Backup procedure :
1.Lock the tables while server is running
2.Perform copy in physical location.
3.Release locks on tables after copy.
Step 1 :
Lock the tables which you want to take the backup.
1 2 3 4 5 6 |
mysql> use AJAY_MyISAM; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> lock tables test2 Read; Query OK, 0 rows affected (0.00 sec) |
How to get locked tables information ?
To check locked tables information under database.
1 2 3 4 5 6 7 |
mysql> show open tables from AJAY_MyISAM; +-------------+-------+--------+-------------+ | Database | Table | In_use | Name_locked | +-------------+-------+--------+-------------+ | AJAY_MyISAM | test1 | 1 | 0 | | AJAY_MyISAM | test2 | 0 | 0 | +-------------+-------+--------+-------------+ |
Here In_use shows 1 for test1 table which is locked .
Step 2:
Perform copy of a table files.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[root@ip-172-31-47-50 AJAY_MyISAM]# ll total 128 -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 mysql mysql 8594 Dec 12 14:58 test1.frm -rw-r-----. 1 mysql mysql 0 Dec 12 14:58 test1.MYD -rw-r-----. 1 mysql mysql 1024 Dec 12 14:58 test1.MYI -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd [root@ip-172-31-47-50 AJAY_MyISAM]# mkdir backup [root@ip-172-31-47-50 AJAY_MyISAM]# ll total 128 drwxr-xr-x. 2 root root 6 Dec 12 16:01 backup -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 mysql mysql 8594 Dec 12 14:58 test1.frm -rw-r-----. 1 mysql mysql 0 Dec 12 14:58 test1.MYD -rw-r-----. 1 mysql mysql 1024 Dec 12 14:58 test1.MYI -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd [root@ip-172-31-47-50 AJAY_MyISAM]# cp test1* backup/ [root@ip-172-31-47-50 AJAY_MyISAM]# cd backup/ [root@ip-172-31-47-50 backup]# ll total 16 -rw-r-----. 1 root root 8594 Dec 12 16:02 test1.frm -rw-r-----. 1 root root 0 Dec 12 16:02 test1.MYD -rw-r-----. 1 root root 1024 Dec 12 16:02 test1.MYI [root@ip-172-31-47-50 backup]# |
Step 3 :
Release locks on tables after copy.
1 2 |
mysql> unlock tables; Query OK, 0 rows affected (0.00 sec) |
Now we will try to perform restore of test1 table.
Step 4 :
Drop the object test1 which you took backup.
1 2 3 4 5 6 7 8 9 |
mysql> drop table test1; Query OK, 0 rows affected (0.00 sec) mysql> exit [root@ip-172-31-47-50 AJAY_MyISAM]# ll total 112 drwxr-xr-x. 2 root root 57 Dec 12 16:02 backup -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd |
In physical location also files has been deleted.
Now we need to copy the files from backup location
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[root@ip-172-31-47-50 AJAY_MyISAM]# cd backup/ [root@ip-172-31-47-50 backup]# ll total 16 -rw-r-----. 1 root root 8594 Dec 12 16:02 test1.frm -rw-r-----. 1 root root 0 Dec 12 16:02 test1.MYD -rw-r-----. 1 root root 1024 Dec 12 16:02 test1.MYI [root@ip-172-31-47-50 backup]# cp test1* .. [root@ip-172-31-47-50 backup]# cd .. [root@ip-172-31-47-50 AJAY_MyISAM]# ll total 128 drwxr-xr-x. 2 root root 57 Dec 12 16:02 backup -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 root root 8594 Dec 12 16:55 test1.frm -rw-r-----. 1 root root 0 Dec 12 16:55 test1.MYD -rw-r-----. 1 root root 1024 Dec 12 16:55 test1.MYI -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd [root@ip-172-31-47-50 AJAY_MyISAM]# |
Now connect to mysql and check tables are appear or not ?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
[root@ip-172-31-47-50 AJAY_MyISAM]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 16 Server version: 5.7.24-log MySQL Community Server (GPL) Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use AJAY_MyISAM; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +-----------------------+ | Tables_in_AJAY_MyISAM | +-----------------------+ | test1 | | test2 | +-----------------------+ 2 rows in set (0.01 sec) |
Lets try to select the table test1.
1 2 3 |
mysql> select * from test1; ERROR 1017 (HY000): Can't find file: './AJAY_MyISAM/test1.frm' (errno: 13 - Permission denied) mysql> exit |
The above error will occur after restoring the MyISAM storage engine files.
To troubleshoot the error we need to change the permissions of files.
1 2 3 4 5 6 7 8 9 |
[root@ip-172-31-47-50 AJAY_MyISAM]# ll total 252 drwxr-xr-x. 2 root root 57 Dec 12 16:02 backup -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 root root 8594 Dec 12 16:55 test1.frm -rw-r-----. 1 root root 0 Dec 12 16:55 test1.MYD -rw-r-----. 1 root root 1024 Dec 12 16:55 test1.MYI -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd |
I have copied all the files as ROOT user so files permissions has been changed.
Lets change the permission to mysql user and mysql group.
1 2 3 4 5 6 7 8 9 10 |
[root@ip-172-31-47-50 AJAY_MyISAM]# chown -R mysql:mysql test1* [root@ip-172-31-47-50 AJAY_MyISAM]# ll total 252 drwxr-xr-x. 2 root root 57 Dec 12 16:02 backup -rw-r-----. 1 mysql mysql 65 Dec 12 14:40 db.opt -rw-r-----. 1 mysql mysql 8594 Dec 12 16:55 test1.frm -rw-r-----. 1 mysql mysql 0 Dec 12 16:55 test1.MYD -rw-r-----. 1 mysql mysql 1024 Dec 12 16:55 test1.MYI -rw-r-----. 1 mysql mysql 8562 Dec 12 15:32 test2.frm -rw-r-----. 1 mysql mysql 98304 Dec 12 15:32 test2.ibd |
Now connect to database and check table test1 info.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
mysql> use AJAY_MyISAM; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> insert into test1 values(10,'AJAY'); Query OK, 1 row affected (0.00 sec) mysql> insert into test1 values(20,'KUMAR'); Query OK, 1 row affected (0.00 sec) mysql> SELECT * FROM TEST1; ERROR 1146 (42S02): Table 'AJAY_MyISAM.TEST1' doesn't exist mysql> SELECT * FROM test1; +-------+-------+ | empno | ename | +-------+-------+ | 10 | AJAY | | 20 | KUMAR | +-------+-------+ 2 rows in set (0.00 sec) |
Thank you…………………