MySQL Test Framework for Support and Bugs Work
0 likes713 views
MySQL Test Framework (MTR) provides unit test suite for MySQL. Tests in the framework are written by MySQL Server developers and contributors and run to ensure build is working correctly. I found this is not the only thing which can be done with MTR. I regularly use it in my Support job to help customers and verify bug reports. With MySQL Test Framework I can: Create complicated environment in single step and re-use it later Test same scenario on dozens of MySQL/Percona/MariaDB server versions with single command Test concurrent scenarios Test errors and return codes Work with results, external commands, stored routines Everything with single script which can be reused on any machine any time with any MySQL/Percona/MariaDB Server version. In this session I will show my way of working with MySQL Test Framework and I hope you will love it as I do! Presented at Fosdem in 2018: https://fosdem.org/2018/schedule/event/mysql_test_framework/
![sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_9.cnf
!include ../../rpl/my.cnf
[mysqld.1]
log-slave-updates
gtid_mode=ON
enforce-gtid-consistency
Complicated Replication Options
19](https://image.slidesharecdn.com/mtr-180209160704/85/MySQL-Test-Framework-for-Support-and-Bugs-Work-33-320.jpg?cb=1518192503)
![sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_9.cnf
!include ../../rpl/my.cnf
[mysqld.1]
log-slave-updates
gtid_mode=ON
enforce-gtid-consistency
[mysqld.2]
master-info-repository=TABLE
relay-log-info-repository=TABLE
log-slave-updates
gtid_mode=ON
enforce-gtid-consistency
[mysqld.3]
# Third server options
[ENV]
SERVER_MYPORT_3= @mysqld.3.port
SERVER_MYSOCK_3= @mysqld.3.socket
Complicated Replication Options
19](https://image.slidesharecdn.com/mtr-180209160704/85/MySQL-Test-Framework-for-Support-and-Bugs-Work-34-320.jpg?cb=1518192503)
![sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr --suite=your_suite your_test
Logging: ./mtr
MySQL Version 5.7.19
...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
create table t1(f1 int);
insert into t1 values(1);
select * from t1;
f1
1
drop table t1;
your_suite.your_test [ pass ] 45802
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 45.802 of 61 seconds executing testcases
Run by Single Command
20](https://image.slidesharecdn.com/mtr-180209160704/85/MySQL-Test-Framework-for-Support-and-Bugs-Work-35-320.jpg?cb=1518192503)
![• Record result:
sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr --record --suite=your_suite your_test
Logging: ./mtr
MySQL Version 5.7.19
Checking supported features...
...
==============================================================================
TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
your_suite.your_test [ pass ] 45802
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 45.802 of 61 seconds executing testcases
Completed: All 1 tests were successful.
Run by Single Command
20](https://image.slidesharecdn.com/mtr-180209160704/85/MySQL-Test-Framework-for-Support-and-Bugs-Work-36-320.jpg?cb=1518192503)
![Introduction into MySQL Query Tuning for Dev[Op]s](https://cdn.slidesharecdn.com/ss_thumbnails/qtdevops-210717011329-thumbnail.jpg?width=560&fit=bounds)
![Introduction to MySQL Query Tuning for Dev[Op]s](https://cdn.slidesharecdn.com/ss_thumbnails/qtdevops-191005204425-thumbnail.jpg?width=560&fit=bounds)
More Related Content
More from Sveta Smirnova (20)
Recently uploaded (20)
MySQL Test Framework for Support and Bugs Work
- 1. MySQL Test Framework for Troubleshooting February, 04, 2018 Sveta Smirnova
- 2. What my Family Thinks I Do 2
- 3. What my Boss Thinks I Do 3
- 4. What I Really Do 4
- 5. • Why customer’s SQL works wrongly I Investigate 5
- 6. • Why customer’s SQL works wrongly • Is this true with all versions? I Investigate 5
- 7. • Why customer’s SQL works wrongly • Is this true with all versions? • With all branches? I Investigate 5
- 8. • Customer’s server version I Often Need to Test 6
- 9. • Customer’s server version • Same major branch, but latest version I Often Need to Test 6
- 10. • Customer’s server version • Same major branch, but latest version • Latest version I Often Need to Test 6
- 11. • Customer’s server version • Same major branch, but latest version • Latest version • Upstream I Often Need to Test 6
- 12. • Customer’s server version • Same major branch, but latest version • Latest version • Upstream • Other forks (e.g. MariaDB) I Often Need to Test 6
- 13. • Customer’s server version • Same major branch, but latest version • Latest version • Upstream • Other forks (e.g. MariaDB) • Same scenario I Often Need to Test 6
- 14. Not an Option! Test Manually? 7
- 15. • MySQL Sandbox • Complicated installations with single command • Scripts to start/stop/restart mysqld • Does NOT enter code for you! Solutions 8
- 16. • MySQL Sandbox • Docker • Runs everything you wish with single command • Not much version variations pre-installed • You need to write Dockerfile if you want Custom setup Custom options Run SQL inside Docker • Does NOT check results! Solutions 8
- 17. • MySQL Sandbox • Docker • MySQL Test Framework Solutions 8
- 18. $cat mtr_test_1.test #--source include/have_innodb.inc source include/have_innodb.inc; CREATE TABLE ‘t‘ ( ‘id‘ int(11) NOT NULL, ‘f‘ varchar(100) DEFAULT NULL, PRIMARY KEY (‘id‘) ) ENGINE=InnoDB; insert into t values(12345, ’value1’), (54321, ’value2’); select * from t; show create table t; drop table if exists t; SQL you are Used to 9
- 19. sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/mtr_test_2-master.opt --transaction-isolation=read-committed Custom Options 10
- 20. --source include/have_innodb.inc CREATE TABLE ‘t‘ (‘id‘ int(11) NOT NULL, ‘f‘ varchar(100) DEFAULT NULL, PRIMARY KEY (‘id‘)) ENGINE=InnoDB; insert into t values(12345, ’value1’), (54321, ’value2’); begin; select * from t; --connect(addconroot,localhost,root,,) --connection addconroot begin; update t set f=’foo’ where id=12345; commit; --connection default select * from t; drop table if exists t; Concurrent Execution 11
- 21. --source include/have_innodb.inc CREATE TABLE ‘t‘ (‘id‘ int(11) NOT NULL, ‘f‘ varchar(100) DEFAULT NULL, PRIMARY KEY (‘id‘)) ENGINE=InnoDB; insert into t values(12345, ’value1’), (54321, ’value2’); begin; select * from t where id=12345 lock in share mode; --connect(addconroot,localhost,root,,) --connection addconroot set innodb_lock_wait_timeout=3; begin; --error 1205 update t set f=’value3’ where id=12345; rollback; --connection default rollback; drop table if exists t; Error Handling 12
- 22. ... --connect(addconroot,localhost,root,,) --connection addconroot begin; select * from t where id=54321 for update; --connection default --send update t set f=’value3’ where id=54321 --connection addconroot update t set f=’value3’ where id=12345; --connection default --error 1213 --reap ... Result Processing 13
- 23. ... --exec $MYSQL_DUMP test > $MYSQL_TEST_DIR/var/tmp/mtr_amer_5.sql --exec ls $MYSQL_TEST_DIR/var/tmp/ drop table t; show tables; --exec $MYSQL test < $MYSQL_TEST_DIR/var/tmp/mtr_amer_5.sql show tables; select * from t; ... External Commands 14
- 24. --source include/have_innodb.inc CREATE TABLE ‘t‘ (‘id‘ int(11) NOT NULL, ‘f‘ varchar(100) DEFAULT NULL, PRIMARY KEY (‘id‘)) ENGINE=InnoDB; --disable_query_log --let $c=1 while ($c<100) { --eval insert into t values($c, md5($c)) --inc $c } --enable_query_log Flow Control 15
- 25. sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_7.test --source include/master-slave.inc --source include/have_innodb.inc # We are on master CREATE TABLE ‘t‘ (‘id‘ int(11) NOT NULL, ‘f‘ varchar(100) DEFAULT NULL, PRIMARY KEY (‘id‘)) ENGINE=InnoDB; insert into t values(12345, ’value1’), (54321, ’value2’); select * from t; show create table t; --connection slave --vertical_results show slave status; --connection master --horizontal_results update t set f=’foo’; Replication from the Box 16
- 26. #We are on master --sync_slave_with_master #We are on slave select * from t; --connection master drop table if exists t; --sync_slave_with_master stop slave; Replication from the Box 16
- 27. • On master sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_7-master.opt --gtid_mode=ON --log-slave-updates --enforce-gtid-consistency --binlog-format=row Replication Options 17
- 28. • On master sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_7-master.opt --gtid_mode=ON --log-slave-updates --enforce-gtid-consistency --binlog-format=row • On slave sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_7-slave.opt --gtid_mode=ON --log-slave-updates --enforce-gtid-consistency --binlog-format=row Replication Options 17
- 29. --let $rpl_topology= 1->2,2->3,3->1 --source include/rpl_init.inc Any Complicated Setup 18
- 30. --let $rpl_topology= 1->2,2->3,3->1 --source include/rpl_init.inc # On server 1 --let $rpl_connection_name= server_1 --source include/rpl_connection.inc create table t1(id int) engine=innodb; insert into t1 values(1); Any Complicated Setup 18
- 31. --let $rpl_topology= 1->2,2->3,3->1 --source include/rpl_init.inc # On server 1 --let $rpl_connection_name= server_1 --source include/rpl_connection.inc create table t1(id int) engine=innodb; insert into t1 values(1); # On server 2 --let $rpl_connection_name= server_2 --source include/rpl_connection.inc create table t2(id int) engine=innodb; insert into t2 values(2); # On server 3 --let $rpl_connection_name= server_3 --source include/rpl_connection.inc create table t3(id int) engine=innodb; insert into t3 values(3); Any Complicated Setup 18
- 32. sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_9.cnf !include ../../rpl/my.cnf Complicated Replication Options 19
- 33. sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_9.cnf !include ../../rpl/my.cnf [mysqld.1] log-slave-updates gtid_mode=ON enforce-gtid-consistency Complicated Replication Options 19
- 34. sveta@delly:~/build/ps-5.7/mysql-test$ cat suite/mtr_test/t/rpl_mtr_test_9.cnf !include ../../rpl/my.cnf [mysqld.1] log-slave-updates gtid_mode=ON enforce-gtid-consistency [mysqld.2] master-info-repository=TABLE relay-log-info-repository=TABLE log-slave-updates gtid_mode=ON enforce-gtid-consistency [mysqld.3] # Third server options [ENV] SERVER_MYPORT_3= @mysqld.3.port SERVER_MYSOCK_3= @mysqld.3.socket Complicated Replication Options 19
- 35. sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr --suite=your_suite your_test Logging: ./mtr MySQL Version 5.7.19 ... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 create table t1(f1 int); insert into t1 values(1); select * from t1; f1 1 drop table t1; your_suite.your_test [ pass ] 45802 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 45.802 of 61 seconds executing testcases Run by Single Command 20
- 36. • Record result: sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr --record --suite=your_suite your_test Logging: ./mtr MySQL Version 5.7.19 Checking supported features... ... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 your_suite.your_test [ pass ] 45802 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 45.802 of 61 seconds executing testcases Completed: All 1 tests were successful. Run by Single Command 20
- 37. • mysql-scripts On Any Number of Servers 21
- 38. • mysql-scripts • Run tests: do test.sh On Any Number of Servers 21
- 39. • mysql-scripts • Run tests: do test.sh • With single version $ cp suite/mtr_test/t/mtr_test_1.test ~/src/tests/t $ do_test.sh -b ps-5.7 Logging: ./mysql-test-run.pl --record --force mtr_test_1 MySQL Version 5.7.19 ... On Any Number of Servers 21
- 40. • mysql-scripts • Run tests: do test.sh • With multiple versions $ do_test.sh -b ps-5.7 -b ps-5.6 -b ps-5.5 Logging: ./mysql-test-run.pl --record --force mtr_test_1 MySQL Version 5.7.19 Checking supported features... ... On Any Number of Servers 21
- 41. • mysql-scripts • Run tests: do test.sh • Archive: ar test.sh On Any Number of Servers 21
- 42. • mysql-scripts • Run tests: do test.sh • Archive: ar test.sh • Copy to remote box: scp test.sh On Any Number of Servers 21
- 43. • SQL Syntax • Custom commands • Any complicated setups • Runs everything automatically • Checks results MySQL Test Framework 22