MySQL DBA Tasks
By Will Mayall
16-Nov-2012
Contents
MySQL Daily Tasks .............................................................................................................................. 1
Get Table Status ................................................................................................................................ 2
Get Index Status ................................................................................................................................ 5
Get Database Sizes ........................................................................................................................... 6
Get Table And Index Lengths ...................................................................................................... 7
Get Engine Status .............................................................................................................................. 9
Show Global Variables .................................................................................................................. 17
Show Global Status ......................................................................................................................... 18
Backing Up MySQL .............................................................................................................................. 18
Run Show Full Processlist ......................................................................................................... 19
Analyze Tables ................................................................................................................................... 20
Optimize Tables................................................................................................................................. 21
Repair MyISAM Tables..................................................................................................................... 22
MySQL Tuning Primer ....................................................................................................................... 23
Percona Toolkit................................................................................................................................. 23
MySQL Daily Tasks
When there are no emergencies, the MySQL DBA should execute the following scripts on a daily basis or
when feasible and read through the output. This document will contain the scripts, followed by what to
look for while reviewing the output of the scripts.
1
Get Table Status
#!/bin/ksh
#Stolen from Will Mayall 16-Sep-2011
#syntax ./get_table_status.ksh host username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch TABLE_STATUS_${dstr}.txt
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA from
TABLES where TABLE_SCHEMA not in
('mysql','performance_schema','temp','innodb','information_schema') order by 1;" |grep -v
TABLE_SCHEMA |awk '{print $1}')
for table_name in $connections
do
mysql -h$1 -u$2 -p$3 -v -e "\s;show table status from $table_name\G" >> TABLE_STATUS_${dstr}.txt
done
The output of the scripts looks like:
m0188ussac1:~/anthology_preprod/logs$ more TABLE_STATUS_11.16.16_12.txt
-------------mysql
Ver 14.14 Distrib 5.1.48, for sun-solaris2.8 (sparc) using
Connection id:
EditLine wrapper
206
Current database:
Current user:
anthology@66.54.56.1
SSL:
Not in use
Current pager:
stdout
Using outfile:
''
Using delimiter:
;
Server version:
5.5.20-log Source distribution
Protocol version:
10
Connection:
anthology.csodmi3f6rfa.us-east-1.rds.amazonaws.com via TCP/IP
Server characterset:
utf8
Db
utf8
characterset:
2
Client characterset:
utf8
Conn.
utf8
characterset:
TCP port:
3306
Uptime:
8 hours 32 min 18 sec
Threads: 2 Questions: 13344 Slow queries: 0
Queries per second avg: 0.434
Opens: 2211
Flush tables: 1
--------------
-------------show table status from anthologydb
*************************** 1. row ***************************
Name: ACTIVITY_TYPE
Engine: InnoDB
Version: 10
Row_format: Compact
Rows: 2
Avg_row_length: 8192
Data_length: 16384
Max_data_length: 0
Index_length: 0
Data_free: 0
Auto_increment: 3
Create_time: 2012-08-21 00:46:10
Update_time: NULL
Check_time: NULL
Collation: utf8_general_ci
Checksum: NULL
Create_options:
Comment:
3
Open tables: 40
The script will create a unique name for the output file like
TABLE_STATUS_11.16.16_12.txt.
The Amazon EC2/RDS version appears at:
Server version:
5.5.20-log Source distribution
The Host name appears at:
Connection:
anthology.csodmi3f6rfa.us-east-1.rds.amazonaws.com via TCP/IP
The Questions are the number of queries that have executed since the
last flush status:
Questions: 13344
The next section will show the status of each application table; this
document will show only one table.
show table status from anthologydb;
Look for the following columns; note that the InnoDB engine will not
populate some of the columns, as the command was designed for the
MyISAM storage engine, when an optimize is executed on a MyISAM table
the Check_time will be updated.
Create_time: 2012-08-21 00:46:10
Update_time: NULL
Check_time: NULL
Comment:
The Comment: column should be empty, if the Table is corrupted, the
Comment column will be populated with the reason for the corruption.
MyISAM tables can be repaired using the repair table command which
will be discussed later in this document. The InnoDB tables can NOT
be repaired. A restore from a previous snapshot or a restore from a
mysqldump would need to be executed. Backups will be discussed later
in this document.
NOTE: TO REPAIR A MyISAM TABLE OR RUN AN OPTIMIZE TABLE, YOU WILL NEED
TWICE AS MUCH DISK SPACE AS THE TABLE PLUS INDEX PLUS A BIT OF
OVERHEAD! IF YOU RUN OUT OF SPACE, THE TABLE AND INDEX WILL BECOME
CORRUPTED!
4
Get Index Status
#!/bin/ksh
#Stolen from Will Mayall 16-Sep-2011
#syntax ./get_index_status.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch INDEX_STATUS_${dstr}.txt
mysql -h$1 -u$2 -p$3 -v -e "\s" >> INDEX_STATUS_${dstr}.txt
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA,
TABLE_NAME from TABLES where TABLE_SCHEMA not in
('mysql','performance_schema','temp','innodb','information_schema') order by 1;" |grep -v
TABLE_SCHEMA |awk '{print $1 "." $2}')
for table_name in $connections
do
mysql -h$1 -u$2 -p$3 -v -e "show index from $table_name\G" >> INDEX_STATUS_${dstr}.txt
done
The get_index_status.ksh script runs the mysql \s command to get
status, which was discussed in the previous section:
status
(\s) Get status information from the server.
The output looks like:
*************************** 1. row ***************************
Table: ACTIVITY_TYPE
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: activityTypeId
Collation: A
Cardinality: 2
Sub_part: NULL
Packed: NULL
Null:
Index_type: BTREE
Comment:
5
Index_comment:
--------------
Look at the Comment and Index_comment section, they should be empty.
If the Index is corrupted a comment will be noted. If the comment has
Disabled, then you will have to run the enable index command. If the
comment notes corrupted, you will have to rebuild the index.
NOTE: TO REPAIR A MyISAM INDEX OR RUN AN OPTIMIZE TABLE, YOU WILL NEED
TWICE AS MUCH DISK SPACE AS THE TABLE PLUS INDEX PLUS A BIT OF
OVERHEAD! IF YOU RUN OUT OF SPACE, THE TABLE AND INDEX WILL BECOME
CORRUPTED!
Get Database Sizes
#!/bin/ksh
#Stolen from Will Mayall 01-NOV-2012
#Syntax
#./get_database_sizes.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch
GET_DATABASE_SIZES_${dstr}.txt
mysql -h$1 -u$2 -p$3 -t -e
"\. ../sql/get_database_sizes.sql" >> GET_DATABASE_SIZES_${dstr}.txt
The get_database_sizes.ksh calls the ../sql/get_database_sizes.sql,
which looks like:
\s
SELECT table_schema "Data Base Name",sum( data_length + index_length ) / 1024 / 1024 "Data Base
Size in MB",sum( data_free )/ 1024 / 1024 "Free Space in MB"
FROM information_schema.TABLES
GROUP BY table_schema with rollup;
The get_database_sizes.ksh script runs the mysql \s command to get
status, which was discussed in the previous section:
status
(\s) Get status information from the server.
The output looks like:
6
+--------------------+----------------------+------------------+
| Data Base Name
| Data Base Size in MB | Free Space in MB |
+--------------------+----------------------+------------------+
| anthologydb
|
2.65625000 |
4.00000000 |
| information_schema |
0.00878906 |
0.00000000 |
| mysql
|
5.40530014 |
0.00028229 |
| performance_schema |
0.00000000 |
0.00000000 |
| NULL
8.07033920 |
4.00028229 |
|
+--------------------+----------------------+------------------+
The Free Space in MB column displays how much Free Space is available
until the next autoextend. MySQL by Default extends in 8MB
increments:
mysql> show variables like 'innodb_autoextend_increment';
+-----------------------------+-------+
| Variable_name
| Value |
+-----------------------------+-------+
| innodb_autoextend_increment | 8
|
+-----------------------------+-------+
Under Data Base Name, The NULL is the result of the WITH ROLLUP
function which calculates the Total size of all databases and Free
Space. By keeping these metrics we will be able to predict disk
growth.
Get Table And Index Lengths
#!/bin/ksh
#Stolen from Will Mayall 01-NOV-2012
#Syntax
#./get_table_index_length.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch
GET_DATA_LENGTH_SIZES_${dstr}.txt
mysql -h$1 -u$2 -p$3 -t -e "\. ../sql/get_table_index_length.sql" >>
GET_DATA_LENGTH_SIZES_${dstr}.txt
7
The get_table_index_length.ksh calls the script ../sql/
get_table_index_length.sql, which looks like:
\s
SELECT table_schema,table_name,`engine`
,ROUND(data_length/1024/1024,2) total_size_mb,
ROUND(index_length/1024/1024,2) total_index_size_mb, table_rows
FROM information_schema.TABLES
WHERE table_schema not in ('performance_schema','temp','innodb','mysql','information_schema')
order by 1,2;
The get_table_index_length.ksh script runs the mysql \s command to get
status, which was discussed in the previous section:
status
(\s) Get status information from the server.
The output looks like:
+--------------+------------------------+--------+---------------+---------------------+------------+
| table_schema | table_name
| engine | total_size_mb | total_index_size_mb | table_rows |
+--------------+------------------------+--------+---------------+---------------------+------------+
| anthologydb | ACTIVITY_TYPE
| InnoDB |
0.02 |
0.00 |
2 |
| anthologydb | COLLECTIONS
| InnoDB |
0.06 |
0.05 |
460 |
| anthologydb | COLLECTIONS_FEATURED
| InnoDB |
0.02 |
0.05 |
1 |
| anthologydb | FACEBOOK_FRIENDS
| InnoDB |
0.02 |
0.02 |
0 |
| anthologydb | LAST_ACTIVITY
| InnoDB |
0.02 |
0.05 |
0 |
| anthologydb | LEADERBOARD
| InnoDB |
0.06 |
0.02 |
923 |
| anthologydb | LEADERBOARD_RANK
| InnoDB |
0.02 |
0.02 |
0 |
| anthologydb | LOCALES
| InnoDB |
0.02 |
0.03 |
110 |
| anthologydb | OPEN_GRAPH_CONNECTIONS | InnoDB |
0.02 |
0.05 |
0 |
| anthologydb | SHOES
| InnoDB |
0.09 |
0.14 |
1151 |
| anthologydb | SHOES_DETAIL
| InnoDB |
1.52 |
0.13 |
1966 |
| anthologydb | USERS
| InnoDB |
0.17 |
0.11 |
2176 |
+--------------+------------------------+--------+---------------+---------------------+------------+
8
The output displays the Schema Name, Table Name, Table Size, Index
Size, and Number of rows in the Table. By keeping these metrics we
will be able to predict growth.
Get Engine Status
#!/bin/ksh
#Stolen from Will Mayall 01-NOV-2012
#Syntax
#./get_engine_status.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch
GET_ENGINE_STATUS_${dstr}.txt
mysql -h$1 -u$2 -p$3 -v -e "\. ../sql/get_engine_status.sql" >> GET_ENGINE_STATUS_${dstr}.txt
The get_engine_status.ksh calls the script ../sql/ get_engine_status.sql,
which looks like:
\s
show engine innodb status\Gselect sleep(60);show engine innodb status\G
The get_engine_status.ksh script runs the mysql \s command to get
status, which was discussed in the previous section:
status
The
and
the
the
(\s) Get status information from the server.
script executes the show engine innodb status, sleeps 60 seconds
executes the command again. By taking snapshots, we can calculate
size of the redo logs. We can also see what is happening within
InnoDB Storage Engine.
The output looks like:
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
121116 16:13:39 INNODB MONITOR OUTPUT
=====================================
9
Per second averages calculated from the last 1 seconds
----------------BACKGROUND THREAD
----------------srv_master_thread loops: 3076 1_second, 3076 sleeps, 102 10_second, 2063 background, 2063 flush
srv_master_thread log flush and writes: 3075
---------SEMAPHORES
---------OS WAIT ARRAY INFO: reservation count 114, signal count 114
Mutex spin waits 158, rounds 4740, OS waits 5
RW-shared spins 109, rounds 3270, OS waits 109
RW-excl spins 0, rounds 0, OS waits 0
Spin rounds per wait: 30.00 mutex, 30.00 RW-shared, 0.00 RW-excl
-----------TRANSACTIONS
-----------Trx id counter 479D0
Purge done for trx's n:o < 479C0 undo n:o < 0
History list length 1220
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 224, OS thread handle 0x49b34940, query id 13452 66.54.56.1 anthology
show engine innodb status
---TRANSACTION 479CF, not started
MySQL thread id 1, OS thread handle 0x49af3940, query id 13447 localhost 127.0.0.1 rdsadmin
-------FILE I/O
-------I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
10
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
564 OS file reads, 841 OS file writes, 637 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
------------------------------------INSERT BUFFER AND ADAPTIVE HASH INDEX
------------------------------------Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:
insert 0, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 11641807, node heap has 1 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
--LOG
--Log sequence number 126416675
Log flushed up to
126416675
Last checkpoint at
126416675
0 pending log writes, 0 pending chkp writes
325 log i/o's done, 0.00 log i/o's/second
---------------------BUFFER POOL AND MEMORY
11
---------------------Total memory allocated 6020382720; in additional pool allocated 0
Dictionary memory allocated 137980
Buffer pool size
359039
Free buffers
358498
Database pages
540
Old database pages 219
Modified db pages
0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 539, created 1, written 413
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 540, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
-------------ROW OPERATIONS
-------------0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread process no. 2496, id 1094109504, state: waiting for server activity
Number of rows inserted 1, updated 99, deleted 1, read 2154
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
---------------------------END OF INNODB MONITOR OUTPUT
============================
-------------select sleep(60)
12
--------------
sleep(60)
0
-------------show engine innodb status
--------------
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
121116 16:14:39 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 60 seconds
----------------BACKGROUND THREAD
----------------srv_master_thread loops: 3080 1_second, 3080 sleeps, 102 10_second, 2067 background, 2067 flush
srv_master_thread log flush and writes: 3079
---------SEMAPHORES
---------OS WAIT ARRAY INFO: reservation count 114, signal count 114
Mutex spin waits 158, rounds 4740, OS waits 5
RW-shared spins 109, rounds 3270, OS waits 109
RW-excl spins 0, rounds 0, OS waits 0
Spin rounds per wait: 30.00 mutex, 30.00 RW-shared, 0.00 RW-excl
-----------TRANSACTIONS
------------
13
Trx id counter 479D4
Purge done for trx's n:o < 479C0 undo n:o < 0
History list length 1220
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 0, not started
MySQL thread id 224, OS thread handle 0x49b34940, query id 13478 66.54.56.1 anthology
show engine innodb status
---TRANSACTION 479D3, not started
MySQL thread id 1, OS thread handle 0x49af3940, query id 13477 localhost 127.0.0.1 rdsadmin
-------FILE I/O
-------I/O thread 0 state: waiting for i/o request (insert buffer thread)
I/O thread 1 state: waiting for i/o request (log thread)
I/O thread 2 state: waiting for i/o request (read thread)
I/O thread 3 state: waiting for i/o request (read thread)
I/O thread 4 state: waiting for i/o request (read thread)
I/O thread 5 state: waiting for i/o request (read thread)
I/O thread 6 state: waiting for i/o request (write thread)
I/O thread 7 state: waiting for i/o request (write thread)
I/O thread 8 state: waiting for i/o request (write thread)
I/O thread 9 state: waiting for i/o request (write thread)
Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,
ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0
Pending flushes (fsync) log: 0; buffer pool: 0
564 OS file reads, 841 OS file writes, 637 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
------------------------------------INSERT BUFFER AND ADAPTIVE HASH INDEX
------------------------------------Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:
14
insert 0, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 11641807, node heap has 1 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
--LOG
--Log sequence number 126416675
Log flushed up to
126416675
Last checkpoint at
126416675
0 pending log writes, 0 pending chkp writes
325 log i/o's done, 0.00 log i/o's/second
---------------------BUFFER POOL AND MEMORY
---------------------Total memory allocated 6020382720; in additional pool allocated 0
Dictionary memory allocated 137980
Buffer pool size
359039
Free buffers
358498
Database pages
540
Old database pages 219
Modified db pages
0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 539, created 1, written 413
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 540, unzip_LRU len: 0
15
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
-------------ROW OPERATIONS
-------------0 queries inside InnoDB, 0 queries in queue
1 read views open inside InnoDB
Main thread process no. 2496, id 1094109504, state: waiting for server activity
Number of rows inserted 1, updated 99, deleted 1, read 2158
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.07 reads/s
---------------------------END OF INNODB MONITOR OUTPUT
============================
If a Foreign Key Error occurs it will be list in a section called
Foreign Keys. The LOGS section can tell you how busy the database is
at that point in time. The sleep 60 seconds gets us 2 peaks at the
LOG.
--LOG
--Log sequence number 126416675
Log flushed up to
126416675
Last checkpoint at
126416675
0 pending log writes, 0 pending chkp writes
325 log i/o's done, 0.00 log i/o's/second
----------------------
Notice the first LOG, Log sequence number and the second Log sequence
number (Below) are identical, which means there is NO activity within
that 60 second interval. If the database were busy, you could use the
following formula to predict an optimal redo log size:
16
select (log2 – log1) /60/60;
mysql> select (126416675 - 126416675) /60/60;
+--------------------------------+
| (126416675 - 126416675) /60/60 |
+--------------------------------+
|
0.00000000 |
+--------------------------------+
--LOG
--Log sequence number 126416675
Log flushed up to
126416675
Last checkpoint at
126416675
0 pending log writes, 0 pending chkp writes
325 log i/o's done, 0.00 log i/o's/second
----------------------
The current redo log size is 132MB, which seems reasonable for a busy
database.
mysql> show variables like 'innodb_log_file_size';
+----------------------+-----------+
| Variable_name
| Value
|
+----------------------+-----------+
| innodb_log_file_size | 134217728 |
+----------------------+-----------+
Show Global Variables
#!/bin/ksh
#Stolen from Will Mayall 01-NOV-2012
#Syntax
#./show_global_variables.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
17
export dstr
touch
SHOW_GLOBAL_VARIABLES_${dstr}.txt
mysql -h$1 -u$2 -p$3 -t -e "\. ../sql/show_global_variables.sql" >>
SHOW_GLOBAL_VARIABLES_${dstr}.txt
The show_global_variables.ksh calls the
script../sql/show_global_variables.sql, which looks like:
show global variables;
The global variables are the current configuration setting for
RDS/MySQL. Please refer to my document titled “RDS DB Parameters and
Security Group Setup.”
Show Global Status
#!/bin/ksh
#Stolen from Will Mayall 01-NOV-2012
#Syntax
#./show_global_status.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch
SHOW_GLOBAL_STATUS_${dstr}.txt
mysql -h$1 -u$2 -p$3 -t -e "\. ../sql/show_global_status.sql" >> SHOW_GLOBAL_STATUS_${dstr}.txt
The show_global_status.ksh calls the scripts
../sql/show_global_status.sql, which looks like:
show global status;
There are all sorts of information about what is happening to MySQL
when the script is executed. Please refer to my document titled
“Calculating MySQL Metrics.”
Backing Up MySQL
#!/bin/ksh
#Stolen from Will Mayall 16-Sep-2011
#syntax ./backup_tables.ksh host username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch BACKUP_TABLES.ksh
18
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA from
TABLES where TABLE_SCHEMA not in
('mysql','performance_schema','temp','innodb','information_schema') order by 1;" |grep -v
TABLE_SCHEMA |grep -v TABLE_NAME|awk '{print $1}')
for table_name in $connections
do
mysql -v -e "show binary logs;" > BINARY_LOG_LIST_${dstr}.txt
echo "mysqldump $table_name" >> BACKUP_TABLES.ksh
chmod 700 BACKUP_TABLES.ksh
./BACKUP_TABLES.ksh > BACKUP_TABLES_${dstr}.sql
rm BACKUP_TABLES.ksh
done
The script executes a show binary logs so we have an idea of the
current off line redo logs. With RDS we do NOT have the SUPER
PRIVILEGE so we can NOT FLUSH LOGS, and we have NO SHELL access, so we
can NOT restore the redo logs to Point-In-Time. Amazon says they can
recover to a Point within 5 minutes of a crash. My experience tells
me that taking a snapshot of a live database can corrupt the database
unless Amazon is using a proprietary Hotbackup script when the
snapshot is taken. The script will back up just the Application
database(s).
The backup_tables.ksh is a good way to move Development changes into
Production, and ensure that Nike owns their data if something were to
happen which closes down Amazon.
Run Show Full Processlist
#!/bin/ksh
#Stolen from Will Mayall wamayall@yahoo.com
#./run_show_full_processlist.ksh > SHOW_PROCCESSLIST.doc &
x=1
while [ $x -le 6 ]
#Loop for 1 Hour (60sec * 60 = 3600)
do
echo "START TIME"
date
echo "show full processlist;" | mysql
x=$(( $x + 1 ))
19
echo "END DATE"
date
sleep 1
#snooze for a second
done
#End when 3600 tries is reached
When the database becomes busy, the run_show_full_processlist.ksh
script will execute the show full processlist command sleep one second
and re-execute, there is a loop counter to stop the script when the
value is met. The current value is 6 executions. If there are
queries taking over a second, this script will capture them.
NOTE: Executing this script could use a lot of disk space on a busy
Database.
Analyze Tables
#syntax ./analyze_tables.ksh localhost username password
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch ANALYZE_TABLES_${dstr}.txt
mysql -h$1 -u$2 -p$3 -v -e "\s;\G" >> ANALYZE_TABLES_${dstr}.txt
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA,
TABLE_NAME from TABLES where TABLE_SCHEMA not in
('mysql','performance_schema','temp','innodb','information_schema') order by 1;" |grep -v
TABLE_SCHEMA |awk '{print $1 "." $2}')
for table_name in $connections
do
mysql -h$1 -u$2 -p$3 -v -e "analyze table $table_name\G" >> ANALYZE_TABLES_${dstr}.txt
done
The analyze_tables.ksh script analyzes the application database
tables. It is meant to be run on tables that use the InnoDB Storage
Engine. Analyze will rebuild the Table statistics and compress the
data in the ibdata1 datafile.
The script also executes the \s within mysql to get server
information.
The output looks like:
analyze table anthologydb.COLLECTIONS
20
--------------
*************************** 1. row ***************************
Table: anthologydb.COLLECTIONS
Op: analyze
Msg_type: status
Msg_text: OK
-------------analyze table anthologydb.COLLECTIONS_FEATURED
--------------
A Msg_text of OK is good. As previously stated, you can NOT repair
InnoDB tables. This script will do nothing to MyISAM tables (see
Optimize Tables in the next section).
Optimize Tables
#!/bin/ksh
#Stolen from Will Mayall 16-Sep-2011
#syntax ./optimize_tables.ksh localhost username password
#For MyISAM ENGINE
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch OPTIMIZE_TABLES_${dstr}.txt
#mysql -h$1 -u$2 -p$3 -v -e "\s;\G" >> OPTIMIZE_TABLES_XXOCT2012.txt
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA,
TABLE_NAME from TABLES where TABLE_SCHEMA not in
('mysql','performance_schema','temp','innodb','information_schema') order by 1;" |grep -v
TABLE_SCHEMA |awk '{print $1 "." $2}')
for table_name in $connections
do
mysql -h$1 -u$2 -p$3 -v -e "optimize table $table_name;" >> OPTIMIZE_TABLES_${dstr}.txt
done
21
The optimize_tables.ksh script optimizes the application database
tables. It is meant to be run on tables that use the MyISAM Storage
Engine. Optimize will rebuild the table statistics, by copying the
data and indexes to a Temporary Table and compress the data and
Indexes, then renames the Temporary File back to *.MYD and *.MYI.
There must be double the disk space of the table and index plus
overhead or the table will become corrupt!
The script also executes the \s within mysql to get server
information.
The output looks like:
-------------optimize table hyperevo.account
--------------
Table
Op
Msg_type
hyperevo.account
Msg_text
optimize
status
OK
status
Table is already up to date
-------------optimize table hyperevo.account
--------------
Table
Op
Msg_type
hyperevo.account
Msg_text
optimize
--------------
A Msg_text of OK is good, if there have been NO activity to the table
since the last optimize the Msg_text will display The Table is already
up to date. An optimize should be run when there is a lot of data
being changed.
Repair MyISAM Tables
#!/bin/ksh
#Stolen from Will Mayall 16-Sep-2011
#syntax ./repair_myisam_tables.ksh localhost username password
22
#For MyISAM ENGINE
dstr=`date +'%m.%d.%H_%M'`
export dstr
touch REPAIR_TABLES_${dstr}.txt
mysql -h$1 -u$2 -p$3 -v -e "\s;\G" >> REPAIR_TABLES_${dstr}.txt
connections=$(mysql -h$1 -u$2 -p$3 -e "use information_schema;select distinct TABLE_SCHEMA,
TABLE_NAME from TABLES where TABLE_SCHEMA not in ('mysql','pe
rformance_schema','temp','innodb','information_schema') order by 1;" |grep -v TABLE_SCHEMA |awk
'{print $1 "." $2}')
for table_name in $connections
do
mysql -h$1 -u$2 -p$3 -v -e "repair table $table_name;" >> REPAIR_TABLES_${dstr}.txt
done
The repair_myisam_tables.ksh script rebuilds the application database
tables. It is meant to be run on tables that use the MyISAM Storage
Engine. Repair will rebuild the table statistics, by copying the data
and indexes to a Temporary Table and compress the data and Indexes,
then renames the Temporary File back to *.MYD and *.MYI. There must
be double the disk space of the table and index plus overhead or the
table will become corrupt!
MySQL Tuning Primer
The mysqltuner.pl is a freeware Perl script that looks at the system
configuration and outputs advise on tuning the MySQL variables.
Googling mysqltuner.pl will bring up many results where you can
download the script.
Percona Toolkit
Percona is a company that has their own version of MySQL, and have
written many useful tools to check MySQL and Server performance.
Since we are using RDS only two scripts seem to be useful,
pt-variable-advisor and pt-mysql-summary. Just google Percona and
click on the link for Tools.
23