- developers - lists.mariadb.org

[Maria-developers] Rev 36: Re-arranged killing of monitoring processes. in file:///Users/hakan/work/monty_program/mariadb-tools/
by Hakan Kuecuekyilmaz 12 Apr '10

12 Apr '10

At file:///Users/hakan/work/monty_program/mariadb-tools/ ------------------------------------------------------------ revno: 36 revision-id: hakan(a)askmonty.org-20100412232832-2ylrw3ldo1cvdc5p parent: hakan(a)askmonty.org-20100412135900-1v7jsdxf3ahxvwrd committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: mariadb-tools timestamp: Tue 2010-04-13 01:28:32 +0200 message: Re-arranged killing of monitoring processes. === modified file 'sysbench/run-sysbench-myisam.sh' --- a/sysbench/run-sysbench-myisam.sh 2010-04-09 19:13:59 +0000 +++ b/sysbench/run-sysbench-myisam.sh 2010-04-12 23:28:32 +0000 @@ -380,7 +380,7 @@ SERVER_STATUS_PID=$! if [ $PROFILE_IT -eq 1 ]; then - $SUDO opcontrol --setup --separate=lib,kernel,thread --no-vmlinux + $SUDO opcontrol --setup --no-vmlinux --separate=lib,kernel,thread $SUDO opcontrol --start-daemon if [ $? != 0 ]; then echo "[WARNING]: Could not start oprofile daemonl." @@ -410,15 +410,15 @@ sync; sync; sync sleep 1 - - kill -9 $SERVER_STATUS_PID - kill -9 $MPSTAT_PID - kill -9 $IOSTAT_PID - + grep "write requests:" ${THIS_RESULT_DIR}/result${k}.txt | awk '{ print $4 }' | sed -e 's/(//' >> ${THIS_RESULT_DIR}/results.txt echo 'SELECT * FROM INFORMATION_SCHEMA.KEY_CACHES' | $MYSQL $MYSQL_OPTIONS > ${THIS_RESULT_DIR}/key_cache_stats${k}.txt + kill -9 $IOSTAT_PID + kill -9 $MPSTAT_PID + kill -9 $SERVER_STATUS_PID + k=$(($k + 1)) done === modified file 'sysbench/run-sysbench.sh' --- a/sysbench/run-sysbench.sh 2010-04-12 13:59:00 +0000 +++ b/sysbench/run-sysbench.sh 2010-04-12 23:28:32 +0000 @@ -371,7 +371,7 @@ SERVER_STATUS_PID=$! if [ $PROFILE_IT -eq 1 ]; then - $SUDO opcontrol --setup --separate=lib,kernel,thread --no-vmlinux + $SUDO opcontrol --setup --no-vmlinux --separate=lib,kernel,thread $SUDO opcontrol --start-daemon if [ $? != 0 ]; then echo "[WARNING]: Could not start oprofile daemonl." @@ -402,11 +402,11 @@ sync; sync; sync sleep 1 + grep "write requests:" ${THIS_RESULT_DIR}/result${k}.txt | awk '{ print $4 }' | sed -e 's/(//' >> ${THIS_RESULT_DIR}/results.txt + + kill -9 $IOSTAT_PID + kill -9 $MPSTAT_PID kill -9 $SERVER_STATUS_PID - kill -9 $MPSTAT_PID - kill -9 $IOSTAT_PID - - grep "write requests:" ${THIS_RESULT_DIR}/result${k}.txt | awk '{ print $4 }' | sed -e 's/(//' >> ${THIS_RESULT_DIR}/results.txt k=$(($k + 1)) done

1 0

[Maria-developers] Updated (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 20 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 23:24)=-=- Low Level Design modified. --- /tmp/wklog.100.old.20140 2010-04-12 23:24:16.000000000 +0000 +++ /tmp/wklog.100.new.20140 2010-04-12 23:24:16.000000000 +0000 @@ -60,6 +60,8 @@ test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. + warm_up_time int unsigned, -- Warm up time in seconds. + warm_up_threads int unsigned, -- Number of threads used for warm up. PRIMARY KEY (id), KEY (host), KEY (run_date -=-=(Hakan - Mon, 12 Apr 2010, 21:06)=-=- Added: * OProfile hook * iostat hook * mpstat hook * Copy away .err file for crash detection * mysqladmin --sleep 10 status, for getting in test query statistics Worked 8 hours and estimate 20 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. warm_up_time int unsigned, -- Warm up time in seconds. warm_up_threads int unsigned, -- Number of threads used for warm up. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 20 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 23:24)=-=- Low Level Design modified. --- /tmp/wklog.100.old.20140 2010-04-12 23:24:16.000000000 +0000 +++ /tmp/wklog.100.new.20140 2010-04-12 23:24:16.000000000 +0000 @@ -60,6 +60,8 @@ test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. + warm_up_time int unsigned, -- Warm up time in seconds. + warm_up_threads int unsigned, -- Number of threads used for warm up. PRIMARY KEY (id), KEY (host), KEY (run_date -=-=(Hakan - Mon, 12 Apr 2010, 21:06)=-=- Added: * OProfile hook * iostat hook * mpstat hook * Copy away .err file for crash detection * mysqladmin --sleep 10 status, for getting in test query statistics Worked 8 hours and estimate 20 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. warm_up_time int unsigned, -- Warm up time in seconds. warm_up_threads int unsigned, -- Number of threads used for warm up. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] bzr commit into Mariadb 5.2, with Maria 2.0:maria/5.2 branch (knielsen:2764)
by knielsen＠knielsen-hq.org 12 Apr '10

12 Apr '10

#At lp:maria/5.2 2764 knielsen(a)knielsen-hq.org 2010-04-12 MBug#524625: OQGraph error with binary logging enabled Set the correct engine flags for OQGraph to get binlogging working. removed: mysql-test/suite/oqgraph/t/basic-master.opt added: mysql-test/suite/oqgraph/r/binlog.result mysql-test/suite/oqgraph/t/binlog.test mysql-test/suite/oqgraph/t/suite.opt modified: storage/oqgraph/ha_oqgraph.cc === added file 'mysql-test/suite/oqgraph/r/binlog.result' --- a/mysql-test/suite/oqgraph/r/binlog.result 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/oqgraph/r/binlog.result 2010-04-12 23:23:51 +0000 @@ -0,0 +1,18 @@ +drop table if exists graph; +CREATE TABLE graph ( +latch SMALLINT UNSIGNED NULL, +origid BIGINT UNSIGNED NULL, +destid BIGINT UNSIGNED NULL, +weight DOUBLE NULL, +seq BIGINT UNSIGNED NULL, +linkid BIGINT UNSIGNED NULL, +KEY (latch, origid, destid) USING HASH, +KEY (latch, destid, origid) USING HASH +) ENGINE=OQGRAPH; +SET binlog_format = row; +insert into graph(origid, destid) values (1,3), (3,1); +SET binlog_format = statement; +insert into graph(origid, destid) values (3,4), (4,3); +SET binlog_format = mixed; +insert into graph(origid, destid) values (3,5), (5,3); +drop table graph; === removed file 'mysql-test/suite/oqgraph/t/basic-master.opt' --- a/mysql-test/suite/oqgraph/t/basic-master.opt 2010-01-04 08:27:50 +0000 +++ b/mysql-test/suite/oqgraph/t/basic-master.opt 1970-01-01 00:00:00 +0000 @@ -1,2 +0,0 @@ -$OQGRAPH_PLUGIN_OPT -$OQGRAPH_PLUGIN_LOAD === added file 'mysql-test/suite/oqgraph/t/binlog.test' --- a/mysql-test/suite/oqgraph/t/binlog.test 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/oqgraph/t/binlog.test 2010-04-12 23:23:51 +0000 @@ -0,0 +1,28 @@ +-- source suite/oqgraph/include/have_oqgraph_engine.inc +-- source include/have_log_bin.inc + +--disable_warnings +drop table if exists graph; +--enable_warnings + +CREATE TABLE graph ( + latch SMALLINT UNSIGNED NULL, + origid BIGINT UNSIGNED NULL, + destid BIGINT UNSIGNED NULL, + weight DOUBLE NULL, + seq BIGINT UNSIGNED NULL, + linkid BIGINT UNSIGNED NULL, + KEY (latch, origid, destid) USING HASH, + KEY (latch, destid, origid) USING HASH + ) ENGINE=OQGRAPH; + +# MBug#524625: OQGraph error with binary logging enabled +# Test that OQGraph works with different binlogging modes. +SET binlog_format = row; +insert into graph(origid, destid) values (1,3), (3,1); +SET binlog_format = statement; +insert into graph(origid, destid) values (3,4), (4,3); +SET binlog_format = mixed; +insert into graph(origid, destid) values (3,5), (5,3); + +drop table graph; === added file 'mysql-test/suite/oqgraph/t/suite.opt' --- a/mysql-test/suite/oqgraph/t/suite.opt 1970-01-01 00:00:00 +0000 +++ b/mysql-test/suite/oqgraph/t/suite.opt 2010-04-12 23:23:51 +0000 @@ -0,0 +1,2 @@ +$OQGRAPH_PLUGIN_OPT +$OQGRAPH_PLUGIN_LOAD === modified file 'storage/oqgraph/ha_oqgraph.cc' --- a/storage/oqgraph/ha_oqgraph.cc 2010-01-04 13:32:42 +0000 +++ b/storage/oqgraph/ha_oqgraph.cc 2010-04-12 23:23:51 +0000 @@ -357,7 +357,8 @@ ulong ha_oqgraph::table_flags() const #endif { return (HA_NO_BLOBS | HA_NULL_IN_KEY | - HA_REC_NOT_IN_SEQ | HA_CAN_INSERT_DELAYED); + HA_REC_NOT_IN_SEQ | HA_CAN_INSERT_DELAYED | + HA_BINLOG_STMT_CAPABLE | HA_BINLOG_ROW_CAPABLE); } ulong ha_oqgraph::index_flags(uint inx, uint part, bool all_parts) const

1 0

[Maria-developers] Progress (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 20 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 21:06)=-=- Added: * OProfile hook * iostat hook * mpstat hook * Copy away .err file for crash detection * mysqladmin --sleep 10 status, for getting in test query statistics Worked 8 hours and estimate 20 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 20 ESTIMATE.......: 20 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 21:06)=-=- Added: * OProfile hook * iostat hook * mpstat hook * Copy away .err file for crash detection * mysqladmin --sleep 10 status, for getting in test query statistics Worked 8 hours and estimate 20 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 12 ESTIMATE.......: 28 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Updated (by Hakan): Benchmark suite for sysbench (100)
by worklog-noreply＠askmonty.org 12 Apr '10

12 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Benchmark suite for sysbench CREATION DATE..: Thu, 04 Mar 2010, 17:46 SUPERVISOR.....: Igor IMPLEMENTOR....: Hakan COPIES TO......: CATEGORY.......: Other TASK ID........: 100 (http://askmonty.org/worklog/?tid=100) VERSION........: Benchmarks-3.0 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 12 ESTIMATE.......: 28 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: -=-=(Hakan - Mon, 12 Apr 2010, 21:04)=-=- Low Level Design modified. --- /tmp/wklog.100.old.13854 2010-04-12 21:04:41.000000000 +0000 +++ /tmp/wklog.100.new.13854 2010-04-12 21:04:41.000000000 +0000 @@ -1,20 +1,24 @@ All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers -This is implemented in run-sysbench.sh. Currently it supports: - ** Optionally pull of latest source from Launchpad and compile - ** Starting the server +This is implemented in run-sysbench.sh. Currently we supports: + ** Optionally pulling of latest source from Launchpad and compile. + ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. - ** Number of concurrent clients is hardcoded + ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc - ** Set random seed of sysbench to have better comparison - ** Restart mysqld from scratch for each run and copy away + ** Setting random seed of sysbench to have better comparison. + ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. + ** Between each run, we run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Writing out mysqld and sysbench options for reference. + ** OProfile hook. + ** iostat hook. + ** mpstat -u (CPU utilization) hook. + ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: @@ -32,9 +36,6 @@ done Open items: - ** OProfile hook - ** iostat hook - ** sar -u (CPU utilization) hook ** Crash detection ** Error detection @@ -45,9 +46,10 @@ ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt - ** Collect OProfile, iostat, cpu utilization, and machine info + ** Collect OProfile, iostat, cpu utilization, sysbench, per test query +statisticts and machine info. ** Detect errors and crashes. - ** Generate SQL INSERT strings for presentation usage + ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( @@ -97,7 +99,7 @@ sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs - 1 first run + 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV -=-=(Hakan - Tue, 09 Mar 2010, 14:10)=-=- * Added run-sysbench-myisam.sh for running MyISAM related benchmarks with sysbench. Worked 4 hours and estimate 28 hours remain (original estimate unchanged). -=-=(Hakan - Tue, 09 Mar 2010, 14:09)=-=- Low Level Design modified. --- /tmp/wklog.100.old.18803 2010-03-09 14:09:37.000000000 +0000 +++ /tmp/wklog.100.new.18803 2010-03-09 14:09:37.000000000 +0000 @@ -9,6 +9,12 @@ ** Number of concurrent clients is hardcoded ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc + ** Set random seed of sysbench to have better comparison + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. The main loop of run-sysbench.sh is: @@ -31,12 +37,6 @@ ** sar -u (CPU utilization) hook ** Crash detection ** Error detection - ** Set random seed of sysbench to have better comparision - ** Restart mysqld from scratch for each run and copy away - DATA_DIR of the database for faster starts. - ** Between each run, run sync and clear file system caches with - echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) - ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Hakan - Tue, 09 Mar 2010, 14:06)=-=- Added: ** Set random seed of sysbench to have better comparison ** Restart mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Write out mysqld and sysbench options for reference. Worked 8 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Hakan - Mon, 08 Mar 2010, 12:27)=-=- Low Level Design modified. --- /tmp/wklog.100.old.21404 2010-03-08 12:27:08.000000000 +0000 +++ /tmp/wklog.100.new.21404 2010-03-08 12:27:08.000000000 +0000 @@ -36,6 +36,7 @@ DATA_DIR of the database for faster starts. ** Between each run, run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + ** Write out mysqld and sysbench options for reference. * Analyze numbers This is implemented in analyze-sysbench.php -=-=(Guest - Thu, 04 Mar 2010, 18:15)=-=- Low Level Design modified. --- /tmp/wklog.100.old.2695 2010-03-04 18:15:46.000000000 +0000 +++ /tmp/wklog.100.new.2695 2010-03-04 18:15:46.000000000 +0000 @@ -1 +1,112 @@ +All scripts can be found at lp:mariadb-tools/sysbench +* Run sysbench tests on a machine and collect numbers +This is implemented in run-sysbench.sh. Currently it supports: + ** Optionally pull of latest source from Launchpad and compile + ** Starting the server + ** Running each sysbench test for $LOOP_COUNT times and + $RUN_TIME time. + ** Number of concurrent clients is hardcoded + ** Machine specific configuration like location of binaries and + directories needed are in separate config files located at conf/<hostname>.inc + +The main loop of run-sysbench.sh is: + +start_mysqld +for SYSBENCH_TEST in $SYSBENCH_TESTS + for THREADS in $NUM_THREADS + while [ $k -lt $LOOP_COUNT ] + drop schema sbtest + create schema sbtest + + $SYSBENCH $SYSBENCH_OPTIONS prepare + $SYSBENCH $SYSBENCH_OPTIONS run + done + done +done + +Open items: + ** OProfile hook + ** iostat hook + ** sar -u (CPU utilization) hook + ** Crash detection + ** Error detection + ** Set random seed of sysbench to have better comparision + ** Restart mysqld from scratch for each run and copy away + DATA_DIR of the database for faster starts. + ** Between each run, run sync and clear file system caches with + echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) + +* Analyze numbers +This is implemented in analyze-sysbench.php + +Open items: + ** Read result files from + ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt + + ** Collect OProfile, iostat, cpu utilization, and machine info + ** Detect errors and crashes. + ** Generate SQL INSERT strings for presentation usage + +The layout for storing the numbers is: + CREATE TABLE sysbench_run ( + id int unsigned NOT NULL auto_increment, + host varchar(80), -- Hostname we ran the test on. + run_date date, -- The day we ran the test. + sysbench_version varchar(32), -- Version of sysbench we used. + test_name varchar(32), -- Name of the sysbench test. + run_time int unsigned, -- Run time in seconds. + runs int unsigned, -- Number of iterations of the test. + PRIMARY KEY (id), + KEY (host), + KEY (run_date + ); + + CREATE TABLE sysbench_comment ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + compile_info text, -- Compile options we used. + machine_info text, -- Details about the hardware. + sysbench_options text, -- The sysbench options we used. + PRIMARY KEY (id), + KEY (sysbench_run_id) + ); + + CREATE TABLE sysbench_result ( + id int unsigned NOT NULL auto_increment, + sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. + concurrency int unsigned, -- Concurrency level we used. + result decimal(7,2), -- The actual result. + io varchar(80), -- The IO from iostat. + cpu varchar(80), -- CPU utilization. + profile text, -- Profiling information. + error text, -- Error messages and stack traces. + PRIMARY KEY (id), + KEY (sysbench_run_id + ); + +* Generate a report out of the numbers +This script will generate a HTML version for putting up on the web and a +txt version for email usage. + +Open items: + ** Generate an overview table in the form of + Number of threads + 1 4 8 16 32 64 128 +sysbench test + delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 + mean value of runs + 1 first run + 2 second run + 3 third run + STDEV + STDEV in % of mean + CPU utilization (usr/sys/wait/idle) + IO (read/write) + +For HTML version additionally generate a graph with JPGraph. + +* Get machine(s) and run the test on a weekly basis and for each release +comparing with the prior release. + +* Email weekly reports and blog about it. DESCRIPTION: Create a benchmark suite for running sysbench * Run sysbench tests on a machine and collect numbers * Analyze numbers * Generate a report out of the numbers LOW-LEVEL DESIGN: All scripts can be found at lp:mariadb-tools/sysbench * Run sysbench tests on a machine and collect numbers This is implemented in run-sysbench.sh. Currently we supports: ** Optionally pulling of latest source from Launchpad and compile. ** Starting and killing the server. ** Running each sysbench test for $LOOP_COUNT times and $RUN_TIME time. ** Number of concurrent clients is currently hardcoded. ** Machine specific configuration like location of binaries and directories needed are in separate config files located at conf/<hostname>.inc ** Setting random seed of sysbench to have better comparison. ** Restarting mysqld from scratch for each run and copy away DATA_DIR of the database for faster starts. ** Between each run, we run sync and clear file system caches with echo 3 > /proc/sys/vm/drop_caches (http://linux.die.net/man/5/proc) ** Writing out mysqld and sysbench options for reference. ** OProfile hook. ** iostat hook. ** mpstat -u (CPU utilization) hook. ** mysqladmin --sleep 10 status, for getting in test query statistics. The main loop of run-sysbench.sh is: start_mysqld for SYSBENCH_TEST in $SYSBENCH_TESTS for THREADS in $NUM_THREADS while [ $k -lt $LOOP_COUNT ] drop schema sbtest create schema sbtest $SYSBENCH $SYSBENCH_OPTIONS prepare $SYSBENCH $SYSBENCH_OPTIONS run done done done Open items: ** Crash detection ** Error detection * Analyze numbers This is implemented in analyze-sysbench.php Open items: ** Read result files from ${RESULT_DIR}/${TODAY}/${PRODUCT}/${SYSBENCH_TEST}/${THREADS}/results.txt ** Collect OProfile, iostat, cpu utilization, sysbench, per test query statisticts and machine info. ** Detect errors and crashes. ** Generate SQL INSERT strings for presentation usage. The layout for storing the numbers is: CREATE TABLE sysbench_run ( id int unsigned NOT NULL auto_increment, host varchar(80), -- Hostname we ran the test on. run_date date, -- The day we ran the test. sysbench_version varchar(32), -- Version of sysbench we used. test_name varchar(32), -- Name of the sysbench test. run_time int unsigned, -- Run time in seconds. runs int unsigned, -- Number of iterations of the test. PRIMARY KEY (id), KEY (host), KEY (run_date ); CREATE TABLE sysbench_comment ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. compile_info text, -- Compile options we used. machine_info text, -- Details about the hardware. sysbench_options text, -- The sysbench options we used. PRIMARY KEY (id), KEY (sysbench_run_id) ); CREATE TABLE sysbench_result ( id int unsigned NOT NULL auto_increment, sysbench_run_id int unsigned NOT NULL, -- FK pointing to sysbench_run. concurrency int unsigned, -- Concurrency level we used. result decimal(7,2), -- The actual result. io varchar(80), -- The IO from iostat. cpu varchar(80), -- CPU utilization. profile text, -- Profiling information. error text, -- Error messages and stack traces. PRIMARY KEY (id), KEY (sysbench_run_id ); * Generate a report out of the numbers This script will generate a HTML version for putting up on the web and a txt version for email usage. Open items: ** Generate an overview table in the form of Number of threads 1 4 8 16 32 64 128 sysbench test delete 121.52 144.77 117.70 115.15 100.48 75.39 66.56 mean value of runs 1 first run (hover for in test query statistics) 2 second run 3 third run STDEV STDEV in % of mean CPU utilization (usr/sys/wait/idle) IO (read/write) For HTML version additionally generate a graph with JPGraph. * Get machine(s) and run the test on a weekly basis and for each release comparing with the prior release. * Email weekly reports and blog about it. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Rev 35: We do the warm up run with 3 threads. in file:///Users/hakan/work/monty_program/mariadb-tools/
by Hakan Kuecuekyilmaz 12 Apr '10

12 Apr '10

At file:///Users/hakan/work/monty_program/mariadb-tools/ ------------------------------------------------------------ revno: 35 revision-id: hakan(a)askmonty.org-20100412135900-1v7jsdxf3ahxvwrd parent: hakan(a)askmonty.org-20100409191359-w2dpenia3bnmlypr committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: mariadb-tools timestamp: Mon 2010-04-12 15:59:00 +0200 message: We do the warm up run with 3 threads. === modified file 'sysbench/run-sysbench.sh' --- a/sysbench/run-sysbench.sh 2010-04-09 19:13:59 +0000 +++ b/sysbench/run-sysbench.sh 2010-04-12 13:59:00 +0000 @@ -323,7 +323,7 @@ echo "[$(date "+%Y-%m-%d %H:%M:%S")] Running ${SYSBENCH_TESTS[$i]} with $THREADS threads and $LOOP_COUNT iterations for $PRODUCT" | tee ${THIS_RESULT_DIR}/results.txt echo '' >> ${THIS_RESULT_DIR}/results.txt - SYSBENCH_OPTIONS_WARM_UP="${SYSBENCH_OPTIONS} --num-threads=1 --max-time=$WARM_UP_TIME" + SYSBENCH_OPTIONS_WARM_UP="${SYSBENCH_OPTIONS} --num-threads=3 --max-time=$WARM_UP_TIME" SYSBENCH_OPTIONS_RUN="${SYSBENCH_OPTIONS} --num-threads=$THREADS --max-time=$RUN_TIME" # Check whether we want a profiled run.

1 0

[Maria-developers] Updated (by Serg): options for CREATE TABLE (43)
by worklog-noreply＠askmonty.org 11 Apr '10

11 Apr '10

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: options for CREATE TABLE CREATION DATE..: Tue, 11 Aug 2009, 17:02 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: Sanja COPIES TO......: Sergei, Monty CATEGORY.......: Server-BackLog TASK ID........: 43 (http://askmonty.org/worklog/?tid=43) VERSION........: Server-5.2 STATUS.........: Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 32 PROGRESS NOTES: -=-=(Serg - Sun, 11 Apr 2010, 11:58)=-=- Low Level Design modified. --- /tmp/wklog.43.old.24176 2010-04-11 11:58:36.000000000 +0000 +++ /tmp/wklog.43.new.24176 2010-04-11 11:58:36.000000000 +0000 @@ -2,6 +2,5 @@ Handler have access to the options during creation and after opening frm. -It should has C interface. -=-=(Serg - Thu, 04 Mar 2010, 16:15)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.32502 2010-03-04 16:15:12.000000000 +0000 +++ /tmp/wklog.43.new.32502 2010-03-04 16:15:12.000000000 +0000 @@ -1,13 +1,13 @@ -Table definition ca looks like following +Table definition will look like the following: CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key options where -'=' was not obligatory. Behaviour and way of storage for existing options will -be left as is. +Exceptions should be made for old table and key options where +'=' was not obligatory. These options will behave as they did, +no changes from the user point of view. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -37,12 +37,11 @@ default charset default collation DATA DIRECTORY +INDEX DIRECTORY TABLESPACE STORAGE -For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can -be separated from them by '=' sign. - -Incorrect option during creation/altering table lead only to worning. During -opening unrecognised options should be ignored. +Incorrect option during creation/altering table lead only to a warning. +Strict mode makes them errors, as expected. +When opening a table all unrecognised options should be ignored. -=-=(Sergei - Mon, 18 Jan 2010, 22:12)=-=- Observers changed: Monty,Sergei -=-=(Monty - Tue, 12 Jan 2010, 17:04)=-=- Version updated. --- /tmp/wklog.43.old.21802 2010-01-12 17:04:31.000000000 +0200 +++ /tmp/wklog.43.new.21802 2010-01-12 17:04:31.000000000 +0200 @@ -1 +1 @@ -Server-5.1 +Server-5.2 -=-=(Guest - Fri, 18 Sep 2009, 14:24)=-=- Low Level Design modified. --- /tmp/wklog.43.old.31654 2009-09-18 14:24:41.000000000 +0300 +++ /tmp/wklog.43.new.31654 2009-09-18 14:24:41.000000000 +0300 @@ -2,4 +2,6 @@ Handler have access to the options during creation and after opening frm. +It should has C interface. + -=-=(Sanja - Fri, 18 Sep 2009, 13:32)=-=- Low Level Design modified. --- /tmp/wklog.43.old.29567 2009-09-18 13:32:41.000000000 +0300 +++ /tmp/wklog.43.new.29567 2009-09-18 13:32:41.000000000 +0300 @@ -1 +1,5 @@ +Options stored in a list as pairs key, value which are LEX_STRING. + +Handler have access to the options during creation and after opening frm. + -=-=(Sanja - Fri, 18 Sep 2009, 13:21)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.29107 2009-09-18 13:21:19.000000000 +0300 +++ /tmp/wklog.43.new.29107 2009-09-18 13:21:19.000000000 +0300 @@ -6,7 +6,8 @@ table_option1=tval1, table_option2=tval2; Exclusion should be made for old table and key options where -'=' was not obligatory. +'=' was not obligatory. Behaviour and way of storage for existing options will +be left as is. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num @@ -42,6 +43,6 @@ For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. - - +Incorrect option during creation/altering table lead only to worning. During +opening unrecognised options should be ignored. -=-=(Guest - Tue, 11 Aug 2009, 19:57)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.31856 2009-08-11 19:57:38.000000000 +0300 +++ /tmp/wklog.43.new.31856 2009-08-11 19:57:38.000000000 +0300 @@ -5,8 +5,43 @@ key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; -Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +Exclusion should be made for old table and key options where '=' was not obligatory. +Old key options: +KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num +WITH PARSER <name> -> PARSER=name + +Old table options: +ENGINE name -> ENGINE=name +TYPE name -> TYPE=name +MAX_ROWS num -> MAX_ROWS=num +MIX_ROWS num -> MIX_ROWS=num +AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num +PASSWORD string -> PASSWORD=string +COMMENT string -> COMMENT=string +AUTO_INCREMENT num -> AUTO_INCREMENT=num +PACK_KEYS num/default -> PACK_KEYS=num/default +CHECKSUM num -> CHECKSUM=num +TABLE_CHECKSUM num -> TABLE_CHECKSUM=num +PAGE_CHECKSUM num -> PAGE_CHECKSUM=num +DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num +ROW_FORMAT name -> ROW_FORMAT=name +INSERT_METHOD name -> INSERT_METHOD=name +KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num +TRANSACTIONAL num -> TRANSACTIONAL=num + +Table options which will be left hardcoded +UNION +default charset +default collation +DATA DIRECTORY +TABLESPACE +STORAGE + For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can be separated from them by '=' sign. + + + + -=-=(Guest - Tue, 11 Aug 2009, 19:36)=-=- High-Level Specification modified. --- /tmp/wklog.43.old.30883 2009-08-11 19:36:45.000000000 +0300 +++ /tmp/wklog.43.new.30883 2009-08-11 19:36:45.000000000 +0300 @@ -1 +1,12 @@ +Table definition ca looks like following +CREATE TABLE table + (field int ... field_opt1=fval1 field_opt2=fval2, + key key1(field) key_opt1=kval1 key_opt2=kval2) + table_option1=tval1, table_option2=tval2; + +Exclusion should be made for old table and key (KEY_BLOCK_SIZE) options where +'=' was not obligatory. + +For fields options can go with field attributes (NOT NULL, UNIQUE and so on) can +be separated from them by '=' sign. DESCRIPTION: Add ability to create table with additional option which can be passed to engine. Also make current options such as TRANSACTIONAL working via this mechanism. HIGH-LEVEL SPECIFICATION: Table definition will look like the following: CREATE TABLE table (field int ... field_opt1=fval1 field_opt2=fval2, key key1(field) key_opt1=kval1 key_opt2=kval2) table_option1=tval1, table_option2=tval2; Exceptions should be made for old table and key options where '=' was not obligatory. These options will behave as they did, no changes from the user point of view. Old key options: KEY_BLOCK_SIZE <num> -> KEY_BLOCK_SIZE=num WITH PARSER <name> -> PARSER=name Old table options: ENGINE name -> ENGINE=name TYPE name -> TYPE=name MAX_ROWS num -> MAX_ROWS=num MIX_ROWS num -> MIX_ROWS=num AVG_ROW_LENGTH num -> AVG_ROW_LENGTH=num PASSWORD string -> PASSWORD=string COMMENT string -> COMMENT=string AUTO_INCREMENT num -> AUTO_INCREMENT=num PACK_KEYS num/default -> PACK_KEYS=num/default CHECKSUM num -> CHECKSUM=num TABLE_CHECKSUM num -> TABLE_CHECKSUM=num PAGE_CHECKSUM num -> PAGE_CHECKSUM=num DELAY_KEY_WRITE num -> DELAY_KEY_WRITE=num ROW_FORMAT name -> ROW_FORMAT=name INSERT_METHOD name -> INSERT_METHOD=name KEY_BLOCK_SIZE num -> KEY_BLOCK_SIZE=num TRANSACTIONAL num -> TRANSACTIONAL=num Table options which will be left hardcoded UNION default charset default collation DATA DIRECTORY INDEX DIRECTORY TABLESPACE STORAGE Incorrect option during creation/altering table lead only to a warning. Strict mode makes them errors, as expected. When opening a table all unrecognised options should be ignored. LOW-LEVEL DESIGN: Options stored in a list as pairs key, value which are LEX_STRING. Handler have access to the options during creation and after opening frm. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0