- developers - lists.mariadb.org

[Maria-developers] Information for release notes:
by Kristian Nielsen 18 Nov '09

18 Nov '09

Hi Daniel, Here is the best I have for the release notes for 5.1.39: 1. My manually extracted "relevant" information from the bzr history. 2. Raw bzr dump of changes (short). 3. Raw bzr dump of changes (long). Hope this helps. - Kristian. ----------------------------------------------------------------------- Changes: - Includes MySQL 5.1.39 (check MySQL release notes for details of changes since MySQL 5.1.38) - Includes XtraDB 8 (check XtraDB release notes for details of changes since XtraDB 6) - RPMs for Centos 5 now available. - Includes FederatedX as replacement for old Federated storage engine. - Test suite speedups. - Binary tarballs now build on Ubuntu Hardy, so now depends on glibc 2.7 rather than 2.9 and so should work on more systems. - Some (still not all) compiler warnings eliminated. Bug fixes: - MBug#443014: Too many 'skipped' messages in mysql-test-run ----------------------------------------------------------------------- You have 144 extra revision(s): 2768: knielsen@knielsen... 2009-11-14 XtraDB 8 after-merge fix: Fix windows ... 2767: knielsen@knielsen... 2009-11-14 XtraDB 8 after-merge fixes: fix forgot... 2766: knielsen@knielsen... 2009-11-13 XtraDB 8 after-merge fix: Add missing ... 2765: knielsen@knielsen... 2009-11-13 [merge] Merge XtraDB 8. 2763.1.2: knielsen@knielsen... 2009-11-10 Fix double declaration of Makefi... 2763.1.1: knielsen@knielsen... 2009-11-10 Apply Monty's fix for warning ab... 2764: knielsen@knielsen... 2009-11-13 [merge] Merge XtraDB 8 into MariaDB. 0.6.5: Aleksandr Kuzminsky 2009-09-30 Sync with rev. 107 0.6.4: Aleksandr Kuzminsky 2009-09-23 Sync with rev.105 0.6.3: Aleksandr Kuzminsky 2009-09-07 sync with rev.96 2763: knielsen@knielsen... 2009-11-07 Bump revision number for MariaDB 5.1.3... 2762: knielsen@knielsen... 2009-11-07 Result file update for new FederatedX 2761: knielsen@knielsen... 2009-11-06 [merge] Merge 5.1-release branch into ... 2737.1.59: knielsen@knielsen... 2009-11-05 Fix running test suite from ins... 2737.1.58: Arjen Lentz 2009-10-31 libevent fixup required for Debian 4 (Etch) 2760: knielsen@knielsen... 2009-11-06 [merge] Merge Mysql 5.1.39 merge into ... 2743.1.5: Michael Widenius 2009-11-02 Fixed problem with schema tables and... 2743.1.4: knielsen@knielsen... 2009-10-16 When running with --skip-safemal... 2743.1.3: knielsen@knielsen... 2009-10-16 After-merge fixes for MySQL 5.1.... 2743.1.2: knielsen@knielsen... 2009-10-15 [merge] Merge with latest MariaD... 2743.1.1: knielsen@knielsen... 2009-10-15 [merge] Merge MySQL 5.1.39 into ... 2502.733.86: Jonathan Perkin 2009-09-04 Install mysqld.lib 2502.733.85: Jonathan Perkin 2009-09-03 [merge] Merge from mysql-5.1.38-... 2502.733.84: Georgi Kodinov 2009-09-02 [merge] null merge of 5.0 fixes b... 1810.3943.19: Georgi Kodinov 2009-09-02 Backported the --parallel=str ... 1810.3943.18: Georgi Kodinov 2009-09-02 [merge] merge 1810.3948.2: Georgi Kodinov 2009-09-02 [merge] merge 1810.3948.1: Georgi Kodinov 2009-09-02 moved version to 5.0-main 2502.733.83: Georgi Kodinov 2009-09-02 fixed a valgrind warning in parti... 2502.733.82: Georgi Kodinov 2009-09-02 [merge] automerge 2502.774.2: Davi Arnaut 2009-09-02 [merge] Manual merge. 1810.3943.17: Davi Arnaut 2009-09-02 Post-merge fix. Observe C decla... 1810.3943.16: Tatiana A. Nurnberg 2009-08-31 [merge] auto-merge 2502.774.1: Davi Arnaut 2009-09-02 Increase thread stack size on HP-UX... 2502.733.81: Georgi Kodinov 2009-09-02 fixed a valgrind warning in parti... 2502.733.80: Georgi Kodinov 2009-09-02 Fixed win32 compilation warnings 2502.733.79: Georgi Kodinov 2009-09-02 fixed compilation warnings 2502.733.78: Mattias Jonsson 2009-09-01 [merge] merge to update with lat... 2502.773.2: Georgi Kodinov 2009-09-01 Fixed a problem in how BUILD/che... 2502.773.1: Davi Arnaut 2009-09-01 Bug#45611: Minor code cleanup 2502.733.77: Mattias Jonsson 2009-09-01 post push fix for bug#20577 and ... 2502.733.76: Mattias Jonsson 2009-09-01 Post fix patch for bug#20577 and... 2502.733.75: Tatiana A. Nurnberg 2009-08-31 [merge] auto-merge 2502.772.4: Georgi Kodinov 2009-08-31 [merge] automerge 1810.3947.4: Georgi Kodinov 2009-08-31 [merge] merge 5.0-main -> 5.0... 2502.772.3: Georgi Kodinov 2009-08-31 [merge] automerge 5.1-main -> 5.... 2502.772.2: Georgi Kodinov 2009-08-31 [merge] automerge 1810.3947.3: Georgi Kodinov 2009-08-27 Bug #46749: Segfault in add_k... 2502.772.1: Anurag Shekhar 2009-08-31 [merge] merging bugfix from 5.0 1810.3947.2: Anurag Shekhar 2009-08-31 [merge] merging with changes ... 1810.3947.1: Anurag Shekhar 2009-08-24 Bug #44723 Larger read_buffer... 2502.733.74: Tatiana A. Nurnberg 2009-08-31 [merge] manual merge 1810.3943.15: Tatiana A. Nurnberg 2009-08-31 Bug#35132: MySQLadmin --w... 2502.733.73: Dao-Gang.Qu(a)sun.com 2009-08-31 Bug #44331 Restore of datab... 2502.733.72: Staale Smedseng 2009-08-30 [merge] Merge from 5.1-bugteam 2502.770.3: Alexey Kopytov 2009-08-30 [merge] Automerge. 2502.771.1: Alexey Kopytov 2009-08-30 Bug #46607: Assertion failed: ... 2502.770.2: Dao-Gang.Qu(a)sun.com 2009-08-29 Bug #44331 Restore of data... 2502.770.1: Davi Arnaut 2009-08-28 Reduce test case runtime. 2502.733.71: Staale Smedseng 2009-08-28 [merge] Merge from 5.0 for 43414 1810.3943.14: Staale Smedseng 2009-08-28 Bug #43414 Parenthesis (and o... 2502.733.70: Davi Arnaut 2009-08-28 Fix for a few assorted compiler warn... 2502.733.69: Mattias Jonsson 2009-08-28 [merge] merge 2502.767.3: Mattias Jonsson 2009-08-28 [merge] Manual merge between bu... 2502.769.1: Mattias Jonsson 2009-08-26 Bug#46362: Endpoint should be... 2502.767.2: Mattias Jonsson 2009-08-26 [merge] merge 2502.768.1: Mattias Jonsson 2009-08-07 Bug#32430: 'show innodb statu... 2502.767.1: Mattias Jonsson 2009-08-26 [merge] merge 2502.766.1: Mattias Jonsson 2009-08-26 Bug#20577: Partitions: use of... 2502.733.68: Alfranio Correia 2009-08-28 [merge] merge mysql-5.0-bugteam... 1810.3943.13: Alfranio Correia 2009-08-28 [merge] auto-merge mysql-5.0... 1810.3789.10: Alfranio Correia 2009-08-27 BUG#46861 Auto-closing of ... 2502.733.67: Alfranio Correia 2009-08-27 BUG#46864 Incorrect update of I... 2502.733.66: Sergey Glukhov 2009-08-27 [merge] 5.0-bugteam->5.1-bugteam ... 1810.3943.12: Sergey Glukhov 2009-08-27 Bug#46184 Crash, SELECT ... FR... 2502.733.65: Alfranio Correia 2009-08-27 Post-fix for BUG#28976. 2502.733.64: Georgi Kodinov 2009-08-27 [merge] merged 5.0-bugteam -> 5.1... 1810.3943.11: Georgi Kodinov 2009-08-24 Bug #37044: Read overflow in o... 1810.3943.10: Georgi Kodinov 2009-08-19 Bug #46807: subselect test fai... 2502.733.63: Alfranio Correia 2009-08-27 BUG#28976 Mixing trans and non-... 2502.733.62: Davi Arnaut 2009-08-24 Bug#45261: Crash, stored procedure +... 2502.733.61: Alfranio Correia 2009-08-24 [merge] auto-merge mysql-5.1-bu... 2502.765.1: Alfranio Correia 2009-08-19 BUG#45694 Deadlock in replicat... 2502.733.60: Alfranio Correia 2009-08-24 [merge] auto-merge mysql-5.1-bu... 2502.764.1: Alfranio Correia 2009-08-13 BUG#46130 Slave does not corre... 2502.733.59: Mattias Jonsson 2009-08-21 [merge] merge 2502.763.1: Mattias Jonsson 2009-08-21 Bug#46639: 1030 (HY000): Got er... 2502.733.58: Georgi Kodinov 2009-08-21 [merge] reverted the fix for bug ... 1810.3943.9: Georgi Kodinov 2009-08-21 [merge] automerge 1810.3946.1: Martin Hansson 2009-08-21 [merge] Merge. 1810.3943.8: Georgi Kodinov 2009-08-21 Revert of the fix for bug #46019. 2502.733.57: Martin Hansson 2009-08-21 [merge] Merge. 2502.762.1: Martin Hansson 2009-08-20 [merge] Bug#46616: Merge 1810.3945.1: Martin Hansson 2009-08-20 Bug#46616: Assertion `!table-... 2502.733.56: Ramil Kalimullin 2009-08-21 Fix for bug #46456 [Ver->Prg]: ... 2502.733.55: Georgi Kodinov 2009-08-20 [merge] merge of bug #46019 to 5.... 1810.3943.7: Georgi Kodinov 2009-08-19 Bug #46019: ERROR 1356 When sel... 2502.733.54: Georgi Kodinov 2009-07-09 Bug #45962: memory leak after 'so... 2502.733.53: Georgi Kodinov 2009-08-19 [merge] null merge of the backpor... 1810.3943.6: Georgi Kodinov 2009-08-19 backport of Chad's fix for bug ... 2502.733.52: Georgi Kodinov 2009-08-17 [merge] automere 1810.3943.5: Georgi Kodinov 2009-08-17 [merge] automerge 2502.733.51: Georgi Kodinov 2009-08-17 [merge] automerge 2502.733.50: Ramil Kalimullin 2009-08-14 [merge] Automerge 2502.761.1: Davi Arnaut 2009-08-13 [merge] Merge from mysql-5.0-bugteam. 1810.3943.4: Davi Arnaut 2009-08-13 Bug#46013: rpl_extraColmaster_my... 2502.733.49: Ramil Kalimullin 2009-08-14 Fix for bug #46614: Assertion i... 2502.733.48: Li-Bing.Song(a)sun.com 2009-08-13 BUG#45574 CREATE IF NOT EXI... 2502.733.47: Mattias Jonsson 2009-08-12 [merge] merge 2502.760.2: Konstantin Osipov 2009-08-12 A follow up patch for the fol... 2502.760.1: Konstantin Osipov 2009-08-12 A follow up patch for Bug#458... 2502.733.46: Mattias Jonsson 2009-08-12 [merge] manual merge 2502.759.1: Mattias Jonsson 2009-08-06 Bug#46478: timestamp field inco... 2502.733.45: Mattias Jonsson 2009-08-12 [merge] merge 2502.758.1: Mattias Jonsson 2009-01-08 Bug#39893: Crash if select on a... 2502.733.44: Li-Bing.Song(a)sun.com 2009-08-12 [merge] Manual Merge 1810.3943.3: Li-Bing.Song(a)sun.com 2009-08-12 BUG#45516 SQL thread does... 2502.733.43: Davi Arnaut 2009-08-11 [merge] Merge from mysql-5.0-bugteam. 2759: sanja(a)askmonty.org 2009-11-03 Add federatedx files for build 2758: sanja(a)askmonty.org 2009-11-03 Make federated compiling under windows 2757: sanja(a)askmonty.org 2009-11-02 Fixed federatedx building under windows 2756: Hakan Kuecuekyilmaz 2009-11-01 [merge] Merge. 2754.1.2: Hakan Kuecuekyilmaz 2009-11-01 [merge] Merge. 2754.1.1: Hakan Kuecuekyilmaz 2009-11-01 BUILD/compile-dist && make dist w... 2755: sanja(a)askmonty.org 2009-10-31 Compilation under windows x64 made possi... 2754: Michael Widenius 2009-10-30 [merge] Merge with 5.1-release 2737.1.57: Sergey Petrunya 2009-10-30 Dummy change to push the release tag. 2753: Michael Widenius 2009-10-30 [merge] Merge of FederatedX code 2751.1.2: Michael Widenius 2009-10-30 Added ignore for symlinked build file 2751.1.1: Michael Widenius 2009-10-30 Added federatedx storage engine 2752: Sergey Petrunya 2009-10-29 MWL#17: Table elimination 2751: Michael Widenius 2009-10-29 Compile by default MySQL clients with libm... 2750: Michael Widenius 2009-10-28 Added missing 'source' command. 2749: Michael Widenius 2009-10-28 Speed up of test suite: 2748: Michael Widenius 2009-10-26 [merge] Automatic merge with maria-merge 2747: Michael Widenius 2009-10-26 Fixed compiler warning message 2746: Michael Widenius 2009-10-14 [merge] Automatic merge with 5.1-merge 2745: Michael Widenius 2009-10-13 Removed not used declarations 2744: Michael Widenius 2009-10-13 Force restart before maria_showlog_error t... 2743: knielsen@knielsen... 2009-10-06 [merge] Merge a bunch of small fixes f... 2742: Michael Widenius 2009-10-06 [merge] Merge with bug fix 2740.1.2: Michael Widenius 2009-10-06 [merge] Merge with trunk 2740.1.1: Michael Widenius 2009-10-06 Don't call _ma_decrement_open_count(... 2741: knielsen@knielsen... 2009-10-05 MBug#443014: Too many 'skipped' messag... ----------------------------------------------------------------------- You have 144 extra revision(s): ------------------------------------------------------------ revno: 2768 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Sat 2009-11-14 10:53:18 +0100 message: XtraDB 8 after-merge fix: Fix windows build. ------------------------------------------------------------ revno: 2767 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Sat 2009-11-14 02:15:03 +0100 message: XtraDB 8 after-merge fixes: fix forgotten manual merge of patch file. ------------------------------------------------------------ revno: 2766 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Fri 2009-11-13 23:53:04 +0100 message: XtraDB 8 after-merge fix: Add missing header to `make dist`. ------------------------------------------------------------ revno: 2765 [merge] committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Fri 2009-11-13 22:27:39 +0100 message: Merge XtraDB 8. ------------------------------------------------------------ revno: 2763.1.2 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1-release timestamp: Tue 2009-11-10 14:55:48 +0100 message: Fix double declaration of Makefile variable for FederatedX. This eliminates a warning from Automake about the double declaration. ------------------------------------------------------------ revno: 2763.1.1 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1-release timestamp: Tue 2009-11-10 09:35:39 +0100 message: Apply Monty's fix for warning about wrong FEDERATED typecode. ------------------------------------------------------------ revno: 2764 [merge] committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Fri 2009-11-13 22:26:08 +0100 message: Merge XtraDB 8 into MariaDB. ------------------------------------------------------------ revno: 0.6.5 committer: Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> branch nick: release-8 timestamp: Wed 2009-09-30 18:33:51 +0000 message: Sync with rev. 107 ------------------------------------------------------------ revno: 0.6.4 committer: Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> branch nick: release-8 timestamp: Wed 2009-09-23 00:06:02 +0000 message: Sync with rev.105 ------------------------------------------------------------ revno: 0.6.3 committer: Aleksandr Kuzminsky <aleksandr.kuzminsky(a)percona.com> branch nick: percona-xtradb timestamp: Mon 2009-09-07 10:22:53 +0000 message: sync with rev.96 ------------------------------------------------------------ revno: 2763 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1-release timestamp: Sat 2009-11-07 16:56:51 +0100 message: Bump revision number for MariaDB 5.1.39 release. ------------------------------------------------------------ revno: 2762 committer: knielsen(a)knielsen-hq.org branch nick: tmp timestamp: Sat 2009-11-07 07:29:10 +0100 message: Result file update for new FederatedX ------------------------------------------------------------ revno: 2761 [merge] committer: knielsen(a)knielsen-hq.org branch nick: tmp timestamp: Fri 2009-11-06 18:24:38 +0100 message: Merge 5.1-release branch into MariaDB trunk. ------------------------------------------------------------ revno: 2737.1.59 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1-release timestamp: Thu 2009-11-05 14:40:01 +0100 message: Fix running test suite from installed package. Some files are in different locations in this case. ------------------------------------------------------------ revno: 2737.1.58 committer: Arjen Lentz <arjen(a)openquery.com> branch nick: mariadb-openquery-fixes timestamp: Sat 2009-10-31 15:29:16 +1000 message: libevent fixup required for Debian 4 (Etch) Also missing copyright for copied macros. Fix by Antony Curtis. ------------------------------------------------------------ revno: 2760 [merge] committer: knielsen(a)knielsen-hq.org branch nick: tmp timestamp: Fri 2009-11-06 18:22:32 +0100 message: Merge Mysql 5.1.39 merge into MariaDB trunk ------------------------------------------------------------ revno: 2743.1.5 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mariadb-bug timestamp: Mon 2009-11-02 11:30:21 +0200 message: Fixed problem with schema tables and DECIMAL ------------------------------------------------------------ revno: 2743.1.4 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Fri 2009-10-16 17:01:36 +0200 message: When running with --skip-safemalloc, still do some basic, but cheap, overrun checks. This greatly helps with eg. slow hosts in Buildbot. ------------------------------------------------------------ revno: 2743.1.3 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Fri 2009-10-16 14:45:42 +0200 message: After-merge fixes for MySQL 5.1.39 merge. Some pbxt test suite files needed updating due to similar changes after bugfixes in MySQL 5.1.39. ------------------------------------------------------------ revno: 2743.1.2 [merge] committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Thu 2009-10-15 23:52:31 +0200 message: Merge with latest MariaDB 5.1 trunk. ------------------------------------------------------------ revno: 2743.1.1 [merge] committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Thu 2009-10-15 23:38:29 +0200 message: Merge MySQL 5.1.39 into MariaDB 5.1. ------------------------------------------------------------ revno: 2502.733.86 committer: Jonathan Perkin <jperkin(a)sun.com> branch nick: mysql-5.1.39-release timestamp: Fri 2009-09-04 17:45:07 +0200 message: Install mysqld.lib ------------------------------------------------------------ revno: 2502.733.85 [merge] committer: Jonathan Perkin <jperkin(a)sun.com> branch nick: mysql-5.1 timestamp: Thu 2009-09-03 01:48:06 +0200 message: Merge from mysql-5.1.38-release ------------------------------------------------------------ revno: 2502.733.84 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-vg-5.1-bugteam timestamp: Wed 2009-09-02 18:45:33 +0300 message: null merge of 5.0 fixes backported from 5.1 ------------------------------------------------------------ revno: 1810.3943.19 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Wed 2009-09-02 16:36:52 +0300 message: Backported the --parallel=str option from mtr2 for backward compatibility with the newer pb2 testing environments ------------------------------------------------------------ revno: 1810.3943.18 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Wed 2009-09-02 16:07:52 +0300 message: merge ------------------------------------------------------------ revno: 1810.3948.2 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Wed 2009-09-02 15:33:18 +0300 message: merge ------------------------------------------------------------ revno: 1810.3948.1 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Wed 2009-09-02 13:23:37 +0300 message: moved version to 5.0-main ------------------------------------------------------------ revno: 2502.733.83 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-vg-5.1-bugteam timestamp: Wed 2009-09-02 18:42:08 +0300 message: fixed a valgrind warning in partition_pruning ------------------------------------------------------------ revno: 2502.733.82 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-vg-5.1-bugteam timestamp: Wed 2009-09-02 15:22:32 +0300 message: automerge ------------------------------------------------------------ revno: 2502.774.2 [merge] committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Wed 2009-09-02 09:12:18 -0300 message: Manual merge. ------------------------------------------------------------ revno: 1810.3943.17 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.0-bugteam timestamp: Wed 2009-09-02 09:02:22 -0300 message: Post-merge fix. Observe C declaration placement rules. ------------------------------------------------------------ revno: 1810.3943.16 [merge] committer: Tatiana A. Nurnberg <azundris(a)mysql.com> branch nick: 50-35132m timestamp: Mon 2009-08-31 12:56:01 -0700 message: auto-merge ------------------------------------------------------------ revno: 2502.774.1 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Wed 2009-09-02 08:45:48 -0300 message: Increase thread stack size on HP-UX when built with debug. ------------------------------------------------------------ revno: 2502.733.81 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-vg-5.1-bugteam timestamp: Wed 2009-09-02 15:20:47 +0300 message: fixed a valgrind warning in partitioning code ------------------------------------------------------------ revno: 2502.733.80 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Wed 2009-09-02 13:22:47 +0300 message: Fixed win32 compilation warnings ------------------------------------------------------------ revno: 2502.733.79 committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Wed 2009-09-02 13:09:01 +0300 message: fixed compilation warnings ------------------------------------------------------------ revno: 2502.733.78 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Tue 2009-09-01 15:54:59 +0200 message: merge to update with latest mysql-5.1-bugteam ------------------------------------------------------------ revno: 2502.773.2 committer: Georgi Kodinov <joro(a)sun.com> branch nick: test64-5.1-bugteam timestamp: Tue 2009-09-01 16:39:13 +0300 message: Fixed a problem in how BUILD/check_cpu handles Core 2 Duo processors. This fixes the regression introduced in 5.1 that prevents 64 bit builds on Intel while still keeping the core2 hack operational so the cluster can build. ------------------------------------------------------------ revno: 2502.773.1 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Tue 2009-09-01 09:32:26 -0300 message: Bug#45611: Minor code cleanup Remove a self assignment. Rework a few constructs to avoid a potential overflow. Based upon patch contributed by Michal Hrusecky ------------------------------------------------------------ revno: 2502.733.77 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Tue 2009-09-01 14:53:27 +0200 message: post push fix for bug#20577 and bug#46362, disabling warnings ------------------------------------------------------------ revno: 2502.733.76 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Tue 2009-09-01 13:04:56 +0200 message: Post fix patch for bug#20577 and bug#46362. On 64-bits machines the calculation gets the wrong types and results in very large numbers. Fixed by explicitly cast month to (int) ------------------------------------------------------------ revno: 2502.733.75 [merge] committer: Tatiana A. Nurnberg <azundris(a)mysql.com> branch nick: 51-35132m timestamp: Mon 2009-08-31 12:54:22 -0700 message: auto-merge ------------------------------------------------------------ revno: 2502.772.4 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Mon 2009-08-31 17:11:54 +0300 message: automerge ------------------------------------------------------------ revno: 1810.3947.4 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Mon 2009-08-31 17:08:10 +0300 message: merge 5.0-main -> 5.0-bugteam ------------------------------------------------------------ revno: 2502.772.3 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Mon 2009-08-31 17:09:09 +0300 message: automerge 5.1-main -> 5.1-bugteam ------------------------------------------------------------ revno: 2502.772.2 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46749-5.1-bugteam timestamp: Mon 2009-08-31 16:40:35 +0300 message: automerge ------------------------------------------------------------ revno: 1810.3947.3 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46749-5.0-bugteam timestamp: Thu 2009-08-27 14:40:42 +0300 message: Bug #46749: Segfault in add_key_fields() with outer subquery level field references This error requires a combination of factors : 1. An "impossible where" in the outermost SELECT 2. An aggregate in the outermost SELECT 3. A correlated subquery with a WHERE clause that includes an outer field reference as a top level WHERE sargable predicate When JOIN::optimize detects an "impossible WHERE" it will bail out without doing the rest of the work and initializations. It will not call make_join_statistics() as well. And make_join_statistics fills in various structures for each table referenced. When processing the result of the "impossible WHERE" the query must send a single row of data if there are aggregate functions in it. In this case the server marks all the aggregates as having received no rows and calls the relevant Item::val_xxx() method on the SELECT list. However if this SELECT list happens to contain a correlated subquery this subquery is evaluated in a normal evaluation mode. And if this correlated subquery has a reference to a field from the outermost "impossible where" SELECT the add_key_fields will mistakenly consider the outer field reference as a "local" field reference when looking for sargable predicates. But since the SELECT where the outer field reference refers to is not completely initialized due to the "impossible WHERE" in this level we'll get a NULL pointer reference. Fixed by making a better condition for discovering if a field is "local" to the SELECT level being processed. It's not enough to look for OUTER_REF_TABLE_BIT in this case since for outer references to constant tables the Item_field::used_tables() will return 0 regardless of whether the field reference is from the local SELECT or not. ------------------------------------------------------------ revno: 2502.772.1 [merge] committer: Anurag Shekhar <anurag.shekhar(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Mon 2009-08-31 13:26:09 +0530 message: merging bugfix from 5.0 ------------------------------------------------------------ revno: 1810.3947.2 [merge] committer: Anurag Shekhar <anurag.shekhar(a)sun.com> branch nick: mysql-5.0-bugteam-44723 timestamp: Mon 2009-08-31 13:02:31 +0530 message: merging with changes in bugteam branch. ------------------------------------------------------------ revno: 1810.3947.1 committer: Anurag Shekhar <anurag.shekhar(a)sun.com> branch nick: mysql-5.0-bugteam-44723 timestamp: Mon 2009-08-24 13:15:51 +0530 message: Bug #44723 Larger read_buffer_size values can cause performance decrease for INSERTs Bulk inserts (multiple row, CREATE ... SELECT, INSERT ... SELECT) into MyISAM tables were performed inefficiently. This was mainly affecting use cases where read_buffer_size was considerably large (>256K) and low number of rows was inserted (e.g. 30-100). The problem was that during I/O cache initialization (this happens before each bulk insert) allocated I/O buffer was unnecessarily initialized to '\0'. This was happening because of mess in flag values. MyISAM informs I/O cache to wait for free space (if out of disk space) by passing MY_WAIT_IF_FULL flag. Since MY_WAIT_IF_FULL and MY_ZEROFILL have the same values, memory allocator was initializing memory to '\0'. The performance gain provided with this patch may only be visible with non-debug binaries, since safemalloc always initializes allocated memory to 0xA5A5... ------------------------------------------------------------ revno: 2502.733.74 [merge] committer: Tatiana A. Nurnberg <azundris(a)mysql.com> branch nick: 51-35132m timestamp: Mon 2009-08-31 12:40:33 -0700 message: manual merge ------------------------------------------------------------ revno: 1810.3943.15 committer: Tatiana A. Nurnberg <azundris(a)mysql.com> branch nick: 50-35132m timestamp: Mon 2009-08-31 10:01:13 -0700 message: Bug#35132: MySQLadmin --wait ping always crashes on Windows systems Failing to connect would release parts of the MYSQL struct. We would then proceed to try again to connect without re- initializing the struct. We prevent the unwanted freeing of data we'll still need now. ------------------------------------------------------------ revno: 2502.733.73 committer: <Dao-Gang.Qu(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Mon 2009-08-31 10:26:01 +0800 message: Bug #44331 Restore of database with events produces warning in replication Update the test case for BUG#44331 to fix the push build failure. ------------------------------------------------------------ revno: 2502.733.72 [merge] committer: Staale Smedseng <staale.smedseng(a)sun.com> branch nick: 43414-51 timestamp: Sun 2009-08-30 19:01:48 +0200 message: Merge from 5.1-bugteam ------------------------------------------------------------ revno: 2502.770.3 [merge] committer: Alexey Kopytov <Alexey.Kopytov(a)Sun.com> branch nick: mysql-5.1-bugteam timestamp: Sun 2009-08-30 11:38:49 +0400 message: Automerge. ------------------------------------------------------------ revno: 2502.771.1 committer: Alexey Kopytov <Alexey.Kopytov(a)Sun.com> branch nick: my51-bug46607 timestamp: Sun 2009-08-30 11:03:37 +0400 message: Bug #46607: Assertion failed: (cond_type == Item::FUNC_ITEM) results in server crash check_group_min_max_predicates() assumed the input condition item to be one of COND_ITEM, SUBSELECT_ITEM, or FUNC_ITEM. Since a condition of the form "field" is also a valid condition equivalent to "field <> 0", using such a condition in a query where the loose index scan was chosen resulted in a debug assertion failure. Fixed by handling conditions of the FIELD_ITEM type in check_group_min_max_predicates(). ------------------------------------------------------------ revno: 2502.770.2 committer: <Dao-Gang.Qu(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Sat 2009-08-29 16:52:22 +0800 message: Bug #44331 Restore of database with events produces warning in replication If an EVENT is created without the DEFINER clause set explicitly or with it set to CURRENT_USER, the master and slaves become inconsistent. This issue stems from the fact that in both cases, the DEFINER is set to the CURRENT_USER of the current thread. On the master, the CURRENT_USER is the mysqld's user, while on the slave, the CURRENT_USER is empty for the SQL Thread which is responsible for executing the statement. To fix the problem, we do what follows. If the definer is not set explicitly, a DEFINER clause is added when writing the query into binlog; if 'CURRENT_USER' is used as the DEFINER, it is replaced with the value of the current user before writing to binlog. ------------------------------------------------------------ revno: 2502.770.1 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-28 18:49:16 -0300 message: Reduce test case runtime. ------------------------------------------------------------ revno: 2502.733.71 [merge] committer: Staale Smedseng <staale.smedseng(a)sun.com> branch nick: 43414-51 timestamp: Fri 2009-08-28 18:21:54 +0200 message: Merge from 5.0 for 43414 ------------------------------------------------------------ revno: 1810.3943.14 committer: Staale Smedseng <staale.smedseng(a)sun.com> branch nick: 43414-50 timestamp: Fri 2009-08-28 17:51:31 +0200 message: Bug #43414 Parenthesis (and other) warnings compiling MySQL with gcc 4.3.2 This patch fixes a number of GCC warnings about variables used before initialized. A new macro UNINIT_VAR() is introduced for use in the variable declaration, and LINT_INIT() usage will be gradually deprecated. (A workaround is used for g++, pending a patch for a g++ bug.) GCC warnings for unused results (attribute warn_unused_result) for a number of system calls (present at least in later Ubuntus, where the usual void cast trick doesn't work) are also fixed. ------------------------------------------------------------ revno: 2502.733.70 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-28 12:06:59 -0300 message: Fix for a few assorted compiler warnings. ------------------------------------------------------------ revno: 2502.733.69 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam_2 timestamp: Fri 2009-08-28 13:54:17 +0200 message: merge ------------------------------------------------------------ revno: 2502.767.3 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Fri 2009-08-28 12:55:59 +0200 message: Manual merge between bug#46362 and bug#20577. ------------------------------------------------------------ revno: 2502.769.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: b46362-51-bugteam timestamp: Wed 2009-08-26 12:51:23 +0200 message: Bug#46362: Endpoint should be set to false for TO_DAYS(DATE) There were a problem since pruning uses the field for comparison (while evaluate_join_record uses longlong), resulting in pruning failures when comparing DATE to DATETIME. Fix was to always comparing DATE vs DATETIME as DATETIME, by adding ' 00:00:00' to the DATE string. And adding optimization for comparing with 23:59:59, so that DATETIME_col > '2001-02-03 23:59:59' -> TO_DAYS(DATETIME_col) > TO_DAYS('2001-02-03 23:59:59') instead of '>='. ------------------------------------------------------------ revno: 2502.767.2 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Wed 2009-08-26 14:45:50 +0200 message: merge ------------------------------------------------------------ revno: 2502.768.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: test-51-bugteam_innodb_plugin timestamp: Fri 2009-08-07 15:08:32 +0200 message: Bug#32430: 'show innodb status' causes errors Invalid (old?) table or database name in logs Post push patch. Bug was that a non partitioned table file was not converted to system_charset, (due to table_name_len was not set). Also missing DBUG_RETURN. And Innodb adds quotes after calling the function, so I added one more mode where explain_filename does not add quotes. But it still appends the [sub]partition name as a comment. Also caught a minor quoting bug, the character '`' was not quoted in the identifier. (so 'a`b' was quoted as `a`b` and not `a``b`, this is mulitbyte characters aware.) ------------------------------------------------------------ revno: 2502.767.1 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Wed 2009-08-26 14:40:18 +0200 message: merge ------------------------------------------------------------ revno: 2502.766.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: b20577-51-bugteam timestamp: Wed 2009-08-26 12:59:49 +0200 message: Bug#20577: Partitions: use of to_days() function leads to selection failures Problem was that the partition containing NULL values was pruned away, since '2001-01-01' < '2001-02-00' but TO_DAYS('2001-02-00') is NULL. Added the NULL partition for RANGE/LIST partitioning on TO_DAYS() function to be scanned too. Also fixed a bug that added ALLOW_INVALID_DATES to sql_mode (SELECT * FROM t WHERE date_col < '1999-99-99' on a RANGE/LIST partitioned table would add it). ------------------------------------------------------------ revno: 2502.733.68 [merge] committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-28 10:45:57 +0100 message: merge mysql-5.0-bugteam --> mysql-5.1-bugteam ------------------------------------------------------------ revno: 1810.3943.13 [merge] committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.0-bugteam timestamp: Fri 2009-08-28 10:29:04 +0100 message: auto-merge mysql-5.0-bugteam (local) --> mysql-5.0-bugteam ------------------------------------------------------------ revno: 1810.3789.10 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: b46861-5.0.72sp timestamp: Thu 2009-08-27 17:28:09 +0100 message: BUG#46861 Auto-closing of temporary tables broken by replicate-rewrite-db When a connection is dropped any remaining temporary table is also automatically dropped and the SQL statement of this operation is written to the binary log in order to drop such tables on the slave and keep the slave in sync. Specifically, the current code base creates the following type of statement: DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `db`.`table`; Unfortunately, appending the database to the table name in this manner circumvents the replicate-rewrite-db option (and any options that check the current database). To solve the issue, we started writing the statement to the binary as follows: use `db`; DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `table`; ------------------------------------------------------------ revno: 2502.733.67 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-27 13:46:29 +0100 message: BUG#46864 Incorrect update of InnoDB table on slave when using trigger with myisam table Slave does not correctly handle "expected errors" leading to inconsistencies between the mater and slave. Specifically, when a statement changes both transactional and non-transactional tables, the transactional changes are automatically rolled back on the master but the slave ignores the error and does not roll them back thus leading to inconsistencies. To fix the problem, we automatically roll back a statement that fails on the slave but note that the transaction is not rolled back unless a "rollback" command is in the relay log file. ------------------------------------------------------------ revno: 2502.733.66 [merge] committer: Sergey Glukhov <Sergey.Glukhov(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-27 15:59:25 +0500 message: 5.0-bugteam->5.1-bugteam merge ------------------------------------------------------------ revno: 1810.3943.12 committer: Sergey Glukhov <Sergey.Glukhov(a)sun.com> branch nick: mysql-5.0-bugteam timestamp: Thu 2009-08-27 15:22:19 +0500 message: Bug#46184 Crash, SELECT ... FROM derived table procedure analyze The crash happens because select_union object is used as result set for queries which have derived tables. select_union use temporary table as data storage and if fields count exceeds 10(count of values for procedure ANALYSE()) then we get a crash on fill_record() function. ------------------------------------------------------------ revno: 2502.733.65 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-27 10:32:27 +0100 message: Post-fix for BUG#28976. Updated main.mysqlbinlog_row_trans's result file as TRUNCATE statements are wrapped in BEGIN...COMMIT. ------------------------------------------------------------ revno: 2502.733.64 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B37044-5.1-bugteam timestamp: Thu 2009-08-27 10:46:35 +0300 message: merged 5.0-bugteam -> 5.1-bugteam ------------------------------------------------------------ revno: 1810.3943.11 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B37044-5.0-bugteam timestamp: Mon 2009-08-24 15:28:03 +0300 message: Bug #37044: Read overflow in opt_range.cc found during "make test" The code was using a special global buffer for the value of IS NULL ranges. This was not always long enough to be copied by a regular memcpy. As a result read buffer overflows may occur. Fixed by setting the null byte to 1 and setting the rest of the field disk image to NULL with a bzero (instead of relying on the buffer and memcpy()). ------------------------------------------------------------ revno: 1810.3943.10 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46807-5.0-bugteam timestamp: Wed 2009-08-19 17:53:43 +0300 message: Bug #46807: subselect test fails on PB-2 with a crash The check for stack overflow was independent of the size of the structure stored in the heap. Fixed by adding sizeof(PARAM) to the requested free heap size. ------------------------------------------------------------ revno: 2502.733.63 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-27 00:13:03 +0100 message: BUG#28976 Mixing trans and non-trans tables in one transaction results in incorrect binlog Mixing transactional (T) and non-transactional (N) tables on behalf of a transaction may lead to inconsistencies among masters and slaves in STATEMENT mode. The problem stems from the fact that although modifications done to non-transactional tables on behalf of a transaction become immediately visible to other connections they do not immediately get to the binary log and therefore consistency is broken. Although there may be issues in mixing T and M tables in STATEMENT mode, there are safe combinations that clients find useful. In this bug, we fix the following issue. Mixing N and T tables in multi-level (e.g. a statement that fires a trigger) or multi-table table statements (e.g. update t1, t2...) were not handled correctly. In such cases, it was not possible to distinguish when a T table was updated if the sequence of changes was N and T. In a nutshell, just the flag "modified_non_trans_table" was not enough to reflect that both a N and T tables were changed. To circumvent this issue, we check if an engine is registered in the handler's list and changed something which means that a T table was modified. Check WL 2687 for a full-fledged patch that will make the use of either the MIXED or ROW modes completely safe. ------------------------------------------------------------ revno: 2502.733.62 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: 45261-5.1 timestamp: Mon 2009-08-24 16:47:08 -0300 message: Bug#45261: Crash, stored procedure + decimal The problem was that creating a DECIMAL column from a decimal value could lead to a failed assertion as decimal values can have a higher precision than those attached to a table. The assert could be triggered by creating a table from a decimal with a large (> 30) scale. Also, there was a problem in calculating the number of digits in the integral and fractional parts if both exceeded the maximum number of digits permitted by the new decimal type. The solution is to ensure that truncation procedure is executed when deducing a DECIMAL column from a decimal value of higher precision. If the integer part is equal to or bigger than the maximum precision for the DECIMAL type (65), the integer part is truncated to fit and the fractional becomes zero. Otherwise, the fractional part is truncated to fit into the space left after the integer part is copied. This patch borrows code and ideas from Martin Hansson's patch. ------------------------------------------------------------ revno: 2502.733.61 [merge] committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam-push timestamp: Mon 2009-08-24 11:37:44 +0100 message: auto-merge mysql-5.1-bugteam (local) --> mysql-5.1-bugteam ------------------------------------------------------------ revno: 2502.765.1 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Wed 2009-08-19 16:38:18 +0100 message: BUG#45694 Deadlock in replicated statement is not retried If the SQL Thread fails to execute an event due to a temporary error (e.g. ER_LOCK_DEADLOCK) and the option "--slave_transaction_retries" is set the SQL Thread should not be aborted and the transaction should be restarted from the beginning and re-executed. Unfortunately, a wrong interpretation of the THD::is_fatal_error was preventing this behavior. In a nutshell, "this variable is set to TRUE if an execution of a compound statement cannot continue. In particular, it is used to disable access to the CONTINUE or EXIT handlers of stored routines. So even temporary errors may have this variable set. To fix the bug, we have done what follows: DBUG_ENTER("has_temporary_error"); - if (thd->is_fatal_error) - DBUG_RETURN(0); - DBUG_EXECUTE_IF("all_errors_are_temporary_errors", if (thd->main_da.is_error()) { ------------------------------------------------------------ revno: 2502.733.60 [merge] committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam-push timestamp: Mon 2009-08-24 10:24:52 +0100 message: auto-merge mysql-5.1-bugteam (local) --> mysql-5.1-bugteam ------------------------------------------------------------ revno: 2502.764.1 committer: Alfranio Correia <alfranio.correia(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-13 17:21:01 +0100 message: BUG#46130 Slave does not correctly handle "expected errors" In STATEMENT based replication, a statement that failed on the master but that updated non-transactional tables is written to binary log with the error code appended to it. On the slave, the statement is executed and the same error is expected. However, when an "expected error" did not happen on the slave and was either ignored or was related to a concurrency issue on the master, the slave did not rollback the effects of the statement and as such inconsistencies might happen. To fix the problem, we automatically rollback a statement that should have failed on a slave but succeded and whose expected failure is either ignored or stems from a concurrency issue on the master. ------------------------------------------------------------ revno: 2502.733.59 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Fri 2009-08-21 18:00:38 +0200 message: merge ------------------------------------------------------------ revno: 2502.763.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: b46639-51-bugteam timestamp: Fri 2009-08-21 17:38:29 +0200 message: Bug#46639: 1030 (HY000): Got error 124 from storage engine on INSERT ... SELECT ... Problem was that when bulk insert is used on an empty table/partition, it disables the indexes for better performance, but in this specific case it also tries to read from that partition using an index, which is not possible since it has been disabled. Solution was to allow index reads on disabled indexes if there are no records. Also reverted the patch for bug#38005, since that was a workaround in the partitioning engine instead of a fix in myisam. ------------------------------------------------------------ revno: 2502.733.58 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46019-5.1-bugteam timestamp: Fri 2009-08-21 17:41:48 +0300 message: reverted the fix for bug #46019 from 5.1-bugteam ------------------------------------------------------------ revno: 1810.3943.9 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46019-5.0-bugteam timestamp: Fri 2009-08-21 17:12:03 +0300 message: automerge ------------------------------------------------------------ revno: 1810.3946.1 [merge] committer: Martin Hansson <martin.hansson(a)sun.com> branch nick: 5.0bt timestamp: Fri 2009-08-21 14:31:40 +0200 message: Merge. ------------------------------------------------------------ revno: 1810.3943.8 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46019-5.0-bugteam timestamp: Fri 2009-08-21 17:10:55 +0300 message: Revert of the fix for bug #46019. ------------------------------------------------------------ revno: 2502.733.57 [merge] committer: Martin Hansson <martin.hansson(a)sun.com> branch nick: 5.1bt timestamp: Fri 2009-08-21 12:13:03 +0200 message: Merge. ------------------------------------------------------------ revno: 2502.762.1 [merge] committer: Martin Hansson <martin.hansson(a)sun.com> branch nick: 5.1bt-gca timestamp: Thu 2009-08-20 14:30:59 +0200 message: Bug#46616: Merge ------------------------------------------------------------ revno: 1810.3945.1 committer: Martin Hansson <martin.hansson(a)sun.com> branch nick: 5.0bt-gca timestamp: Thu 2009-08-20 13:56:29 +0200 message: Bug#46616: Assertion `!table->auto_increment_field_not_null' on view manipulations The bespoke flag was not properly reset after last call to fill_record. Fixed by resetting in caller mysql_update. ------------------------------------------------------------ revno: 2502.733.56 committer: Ramil Kalimullin <ramil(a)mysql.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-21 10:55:35 +0500 message: Fix for bug #46456 [Ver->Prg]: HANDLER OPEN + TRUNCATE + DROP (temporary) TABLE, crash Problem: if one has an open "HANDLER t1", further "TRUNCATE t1" doesn't close the handler and leaves handler table hash in an inconsistent state, that may lead to a server crash. Fix: TRUNCATE should implicitly close all open handlers. Doc. request: the fact should be described in the manual accordingly. ------------------------------------------------------------ revno: 2502.733.55 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46019-5.1-bugteam timestamp: Thu 2009-08-20 17:11:22 +0300 message: merge of bug #46019 to 5.1-bugteam ------------------------------------------------------------ revno: 1810.3943.7 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B46019-5.0-bugteam timestamp: Wed 2009-08-19 15:14:57 +0300 message: Bug #46019: ERROR 1356 When selecting from within another view that has Group By Table access rights checking function check_grant() assumed that no view is opened when it's called. This is not true with nested views where the inner view needs materialization. In this case the view is already materialized when check_grant() is called for it. This caused check_grant() to not look for table level grants on the materialized view table. Fixed by checking if a view is already materialized and if it is check table level grants using the original table name (not the ones of the materialized temp table). ------------------------------------------------------------ revno: 2502.733.54 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B45962-5.1-bugteam timestamp: Thu 2009-07-09 15:05:30 +0300 message: Bug #45962: memory leak after 'sort aborted' errors When the function exits with an error it was not freeing the local Unique class instance. Fixed my making sure all the places where the function returns from are freeing the Unique instance ------------------------------------------------------------ revno: 2502.733.53 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: B39326-5.1-bugteam timestamp: Wed 2009-08-19 18:16:30 +0300 message: null merge of the backport of Chad's fix for bug #39326 to 5.1-bugteam ------------------------------------------------------------ revno: 1810.3943.6 committer: Georgi Kodinov <joro(a)sun.com> branch nick: B39326-5.0-bugteam timestamp: Wed 2009-08-19 10:59:17 +0300 message: backport of Chad's fix for bug #39326 to 5.0-bugteam ------------------------------------------------------------ revno: 2502.733.52 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Mon 2009-08-17 17:21:08 +0300 message: automere ------------------------------------------------------------ revno: 1810.3943.5 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.0-bugteam timestamp: Mon 2009-08-17 17:13:08 +0300 message: automerge ------------------------------------------------------------ revno: 2502.733.51 [merge] committer: Georgi Kodinov <joro(a)sun.com> branch nick: merge-5.1-bugteam timestamp: Mon 2009-08-17 17:14:51 +0300 message: automerge ------------------------------------------------------------ revno: 2502.733.50 [merge] committer: Ramil Kalimullin <ramil(a)mysql.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-14 14:13:16 +0500 message: Automerge ------------------------------------------------------------ revno: 2502.761.1 [merge] committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-13 17:45:01 -0300 message: Merge from mysql-5.0-bugteam. ------------------------------------------------------------ revno: 1810.3943.4 committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: 45238-5.0 timestamp: Thu 2009-08-13 17:07:20 -0300 message: Bug#46013: rpl_extraColmaster_myisam fails on pb2 Bug#45243: crash on win in sql thread clear_tables_to_lock() -> free() Bug#45242: crash on win in mysql_close() -> free() Bug#45238: rpl_slave_skip, rpl_change_master failed (lost connection) for STOP SLAVE Bug#46030: rpl_truncate_3innodb causes server crash on windows Bug#46014: rpl_stm_reset_slave crashes the server sporadically in pb2 When killing a user session on the server, it's necessary to interrupt (notify) the thread associated with the session that the connection is being killed so that the thread is woken up if waiting for I/O. On a few platforms (Mac, Windows and HP-UX) where the SIGNAL_WITH_VIO_CLOSE flag is defined, this interruption procedure is to asynchronously close the underlying socket of the connection. In order to enable this schema, each connection serving thread registers its VIO (I/O interface) so that other threads can access it and close the connection. But only the owner thread of the VIO might delete it as to guarantee that other threads won't see freed memory (the thread unregisters the VIO before deleting it). A side note: closing the socket introduces a harmless race that might cause a thread attempt to read from a closed socket, but this is deemed acceptable. The problem is that this infrastructure was meant to only be used by server threads, but the slave I/O thread was registering the VIO of a mysql handle (a client API structure that represents a connection to another server instance) as a active connection of the thread. But under some circumstances such as network failures, the client API might destroy the VIO associated with a handle at will, yet the VIO wouldn't be properly unregistered. This could lead to accesses to freed data if a thread attempted to kill a slave I/O thread whose connection was already broken. There was a attempt to work around this by checking whether the socket was being interrupted, but this hack didn't work as intended due to the aforementioned race -- attempting to read from the socket would yield a "bad file descriptor" error. The solution is to add a hook to the client API that is called from the client code before the VIO of a handle is deleted. This hook allows the slave I/O thread to detach the active vio so it does not point to freed memory. ------------------------------------------------------------ revno: 2502.733.49 committer: Ramil Kalimullin <ramil(a)mysql.com> branch nick: mysql-5.1-bugteam timestamp: Fri 2009-08-14 00:49:28 +0500 message: Fix for bug #46614: Assertion in show_create_trigger() on SHOW CREATE TRIGGER + MERGE table Problem: SHOW CREATE TRIGGER erroneously relies on fact that we have the only underlying table for a trigger (wrong for merge tables). Fix: remove erroneous assert(). ------------------------------------------------------------ revno: 2502.733.48 committer: <Li-Bing.Song(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Thu 2009-08-13 10:48:57 +0800 message: BUG#45574 CREATE IF NOT EXISTS is not binlogged if the object exists There is an inconsistency with DROP DATABASE|TABLE|EVENT IF EXISTS and CREATE DATABASE|TABLE|EVENT IF NOT EXISTS. DROP IF EXISTS statements are binlogged even if either the DB, TABLE or EVENT does not exist. In contrast, Only the CREATE EVENT IF NOT EXISTS is binlogged when the EVENT exists. This patch fixes the following cases for all the replication formats: CREATE DATABASE IF NOT EXISTS. CREATE TABLE IF NOT EXISTS, CREATE TABLE IF NOT EXISTS ... LIKE, CREAET TABLE IF NOT EXISTS ... SELECT. ------------------------------------------------------------ revno: 2502.733.47 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Wed 2009-08-12 18:53:33 +0200 message: merge ------------------------------------------------------------ revno: 2502.760.2 committer: Konstantin Osipov <kostja(a)sun.com> branch nick: 5.1-bugteam timestamp: Wed 2009-08-12 17:11:06 +0400 message: A follow up patch for the follow up patch for Bug#45829 "CREATE TABLE TRANSACTIONAL PAGE_CHECKSUM ROW_FORMAT=PAGE accepted, does nothing". Put back stubs for members of structures that are shared between sql/ and pluggable storage engines. to not break ABI unnecessarily. To be NULL-merged into 5.4, where we do break the ABI already. ------------------------------------------------------------ revno: 2502.760.1 committer: Konstantin Osipov <kostja(a)sun.com> branch nick: 5.1-bugteam timestamp: Wed 2009-08-12 14:57:41 +0400 message: A follow up patch for Bug#45829 "CREATE TABLE TRANSACTIONAL PAGE_CHECKSUM ROW_FORMAT=PAGE accepted, does nothing" Remove unused code that would lead to warnings when compiling sql_yacc.yy. ------------------------------------------------------------ revno: 2502.733.46 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Wed 2009-08-12 12:03:05 +0200 message: manual merge ------------------------------------------------------------ revno: 2502.759.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: b46478-51-bugteam timestamp: Thu 2009-08-06 14:28:39 +0200 message: Bug#46478: timestamp field incorrectly defaulted when partition is reoganized. Problem was that table->timestamp_field_type was not changed before copying rows between partitions. fixed by setting it to TIMESTAMP_NO_AUTO_SET as the first thing in fast_alter_partition_table, so that all if-branches is covered. ------------------------------------------------------------ revno: 2502.733.45 [merge] committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: topush-51-bugteam timestamp: Wed 2009-08-12 11:46:08 +0200 message: merge ------------------------------------------------------------ revno: 2502.758.1 committer: Mattias Jonsson <mattias.jonsson(a)sun.com> branch nick: b39893-51-bugteam timestamp: Thu 2009-01-08 15:16:44 +0100 message: Bug#39893: Crash if select on a partitioned table, when partitioning is disabled Problem was that it tried to run partitioning function calls when opening a partitioned table, when it was explicitly disabled. Solution is to check if the partitioning plugin is ready to use before using any partitioning specific calls. ------------------------------------------------------------ revno: 2502.733.44 [merge] committer: <Li-Bing.Song(a)sun.com> branch nick: mysql-5.1-bugteam timestamp: Wed 2009-08-12 13:31:56 +0800 message: Manual Merge ------------------------------------------------------------ revno: 1810.3943.3 committer: <Li-Bing.Song(a)sun.com> branch nick: mysql-5.0-bugteam timestamp: Wed 2009-08-12 11:54:05 +0800 message: BUG#45516 SQL thread does not use database charset properly Replication SQL thread does not set database default charset to thd->variables.collation_database properly, when executing LOAD DATA binlog. This bug can be repeated by using "LOAD DATA" command in STATEMENT mode. This patch adds code to find the default character set of the current database then assign it to thd->db_charset when slave server begins to execute a relay log. The test of this bug is added into rpl_loaddata_charset.test ------------------------------------------------------------ revno: 2502.733.43 [merge] committer: Davi Arnaut <Davi.Arnaut(a)Sun.COM> branch nick: mysql-5.1-bugteam timestamp: Tue 2009-08-11 13:29:45 -0300 message: Merge from mysql-5.0-bugteam. ------------------------------------------------------------ revno: 2759 committer: sanja(a)askmonty.org branch nick: work-maria-5.1-federatedx timestamp: Tue 2009-11-03 16:39:54 +0200 message: Add federatedx files for build ------------------------------------------------------------ revno: 2758 committer: sanja(a)askmonty.org branch nick: work-maria-5.1-federatedx timestamp: Tue 2009-11-03 13:08:09 +0200 message: Make federated compiling under windows ------------------------------------------------------------ revno: 2757 committer: sanja(a)askmonty.org branch nick: work-maria-5.1-win64 timestamp: Mon 2009-11-02 21:17:58 +0200 message: Fixed federatedx building under windows ------------------------------------------------------------ revno: 2756 [merge] committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: maria-local-master timestamp: Sun 2009-11-01 18:41:16 +0100 message: Merge. ------------------------------------------------------------ revno: 2754.1.2 [merge] committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: maria timestamp: Sun 2009-11-01 18:37:37 +0100 message: Merge. ------------------------------------------------------------ revno: 2754.1.1 committer: Hakan Kuecuekyilmaz <hakan(a)askmonty.org> branch nick: maria timestamp: Sun 2009-11-01 16:09:55 +0100 message: BUILD/compile-dist && make dist was not working due to typos. Fixed by renaming a file and editing the typo in Makefile.am ------------------------------------------------------------ revno: 2755 committer: sanja(a)askmonty.org branch nick: work-maria-5.1-win64 timestamp: Sat 2009-10-31 21:22:50 +0200 message: Compilation under windows x64 made possible. ------------------------------------------------------------ revno: 2754 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Fri 2009-10-30 22:28:11 +0200 message: Merge with 5.1-release ------------------------------------------------------------ revno: 2737.1.57 committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.1-build1 timestamp: Fri 2009-10-30 13:50:48 +0300 message: Dummy change to push the release tag. ------------------------------------------------------------ revno: 2753 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-5.1 timestamp: Fri 2009-10-30 22:16:08 +0200 message: Merge of FederatedX code ------------------------------------------------------------ revno: 2751.1.2 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Fri 2009-10-30 20:51:46 +0200 message: Added ignore for symlinked build file ------------------------------------------------------------ revno: 2751.1.1 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Fri 2009-10-30 20:50:56 +0200 message: Added federatedx storage engine Fixed compiler warnings ------------------------------------------------------------ revno: 2752 committer: Sergey Petrunya <psergey(a)askmonty.org> branch nick: maria-5.1 timestamp: Thu 2009-10-29 20:50:33 +0300 message: MWL#17: Table elimination - add debug tests (were accidentally not pushed with the bulk of WL) ------------------------------------------------------------ revno: 2751 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Thu 2009-10-29 02:04:56 +0200 message: Compile by default MySQL clients with libmysqldclient.a (not .so) This makes them suitable for tar archices right away and also are easier to copy Don't disable federated storage engine by default. Don't allow one to disable the Maria storage engine if it's used for temp tables ------------------------------------------------------------ revno: 2750 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Wed 2009-10-28 10:08:54 +0200 message: Added missing 'source' command. ------------------------------------------------------------ revno: 2749 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Wed 2009-10-28 09:52:34 +0200 message: Speed up of test suite: - Added --disable_query_log ; begin ; .... commit; --enable_query_log around all while loops that does insert ------------------------------------------------------------ revno: 2748 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Mon 2009-10-26 13:38:17 +0200 message: Automatic merge with maria-merge ------------------------------------------------------------ revno: 2747 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Mon 2009-10-26 13:35:42 +0200 message: Fixed compiler warning message - Added checking of return value for system(), freopen(), fgets() and chown() - Ensure that calls that require a format strings gets a format string - Other trivial things Updated test suite results (especially for pbxt and embedded server) Removed warning for "Invalid (old?) table or database name 'mysqld.1'" from pbxt tests Speed up some pbxt tests by inserting begin ; commit; around "while loops with inserts" Added mysqld startup option '--debug-flush' Create maria_recovery.trace in data directory instead of current directory ------------------------------------------------------------ revno: 2746 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Wed 2009-10-14 11:09:56 +0300 message: Automatic merge with 5.1-merge ------------------------------------------------------------ revno: 2745 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Tue 2009-10-13 19:57:36 +0300 message: Removed not used declarations ------------------------------------------------------------ revno: 2744 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Tue 2009-10-13 15:47:17 +0300 message: Force restart before maria_showlog_error to get rid of status from previous connections This fixes a race condition in the test system ------------------------------------------------------------ revno: 2743 [merge] committer: knielsen(a)knielsen-hq.org branch nick: mariadb-5.1 timestamp: Tue 2009-10-06 20:37:55 +0200 message: Merge a bunch of small fixes from release tree into main lp:maria. ------------------------------------------------------------ revno: 2742 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: maria-skr timestamp: Tue 2009-10-06 09:20:31 +0300 message: Merge with bug fix ------------------------------------------------------------ revno: 2740.1.2 [merge] committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Tue 2009-10-06 09:15:12 +0300 message: Merge with trunk ------------------------------------------------------------ revno: 2740.1.1 committer: Michael Widenius <monty(a)askmonty.org> branch nick: mysql-maria timestamp: Tue 2009-10-06 09:13:56 +0300 message: Don't call _ma_decrement_open_count() for ma_extra(HA_EXTRA_PREPARE_FOR_DROP). Don't call _mi_decrement_open_count() for mi_extra(HA_EXTRA_PREPARE_FOR_DROP). This ensures that if we empty the key cache and a drop table fails later, the index will be automaticly rebuilt ------------------------------------------------------------ revno: 2741 committer: knielsen(a)knielsen-hq.org branch nick: work-5.1 timestamp: Mon 2009-10-05 14:26:57 +0200 message: MBug#443014: Too many 'skipped' messages in mysql-test-run Remove mysql-test-run.pl unnecessary and confusing 'skipped' messages. - In mysql-test-run.pl, we auto-generate combinations of replication tests. But this sometimes generates combinations that are meaningless, like running a test that requires row-based replication with statement-based. These superfluous combinationes should not be reported as skipped, they should just be deleted. - Remove ndb suites from default suites, as we do not support NDB in MariaDB. Keep skip messages resulting from running mysql-test-run.pl in special ways, eg. --mysqld=--binlog-format=statement. -----------------------------------------------------------------------

2 1

[Maria-developers] Bugs are not being closed in lp:maria
by Henrik Ingo 18 Nov '09

18 Nov '09

Hi I took a look at https://bugs.launchpad.net/maria/+bugs? ...all bugs in MariaDB. For instance the third one https://bugs.launchpad.net/maria/+bug/417751 has been fixed and closed a long time ago. The status has in fact been updated properly, but this is only effective as the bug appears in the xtradb project, not maria project. Hakan, with your new bug triaging hat on, could you also make sure old bugs like this are properly closed so we don't need to see them. henrik -- email: henrik.ingo(a)avoinelama.fi tel: +358-40-5697354 www: www.avoinelama.fi/~hingo book: www.openlife.cc

1 0

[Maria-developers] bzr commit into MariaDB 5.1, with Maria 1.5:maria branch (knielsen:2767)
by knielsen＠knielsen-hq.org 18 Nov '09

18 Nov '09

#At lp:maria 2767 knielsen(a)knielsen-hq.org 2009-11-18 [merge] Merge the last bit of MySQL 5.1.41 into MariaDB. modified: mysys/typelib.c === modified file 'mysys/typelib.c' --- a/mysys/typelib.c 2009-11-16 20:49:51 +0000 +++ b/mysys/typelib.c 2009-11-18 11:47:59 +0000 @@ -192,7 +192,7 @@ my_ulonglong find_typeset(char *x, TYPEL i= x; while (*x && *x != field_separator) x++; - if (x[0] && x[1]) // skip separator if found + if (x[0] && x[1]) /* skip separator if found */ x++; if ((find= find_type(i, lib, 2 | 8) - 1) < 0) DBUG_RETURN(0);

1 0

[Maria-developers] Buildbot categories enabled
by Kristian Nielsen 18 Nov '09

18 Nov '09

Hi, With the lots of builders now included in our Buildbot, the status pages are getting difficult to get an overview of. I therefore enabled the Buildbot category feature. I grouped the builders into the following four categories: main These are the main builders, which are well-working and should always be green package These are the builders for package testing. There are a lot of these, so I put them in their own category. experimental These are the builders that are not stable, ie. they crash/disconnect a lot or have other problems that makes them unreliable for checking for problems in new commits. THESE NEED TO BE FIXED so they can move into category main! new New builders that are not fully operation yet (eg. missing bzr installation or similar). I also added some simple forms to be able to select among these on the main page (reload to clear browser cache): http://askmonty.org/buildbot/ Please someone help with getting the many builders in experimental fixes! http://askmonty.org/buildbot/waterfall?category=experimental&branch=5.1 Eg. all of the Windows builders are currectly experimental because of multiple problems with crashes, hangs, stuck processes, etc. etc. - Kristian.

1 0

[Maria-developers] New (by Monty): Give warnings in case of overflow in integer arithmetic (65)
by worklog-noreply＠askmonty.org 17 Nov '09

17 Nov '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Give warnings in case of overflow in integer arithmetic CREATION DATE..: Wed, 18 Nov 2009, 01:20 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 65 (http://askmonty.org/worklog/?tid=65) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 40 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: DESCRIPTION: Give warnings in case of overflow in integer arithmetic The issue is described here: http://bugs.launchpad.net/maria/+bug/482846 MySQL reason is descried here: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html We can't fix the arithmetic as it would make us incompatible in how MySQL works. We should however see if we can generate warnings when overflows happens. mysql> SELECT 4294967296 * 4294967296; +-------------------------+ | 4294967296 * 4294967296 | +-------------------------+ | 0 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 3037000500 * 3037000500; +-------------------------+ | 3037000500 * 3037000500 | +-------------------------+ | -9223372036709301616 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 4294967296 * 4294967297; +-------------------------+ | 4294967296 * 4294967297 | +-------------------------+ | 4294967296 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 4294967296 * 4294967298; +-------------------------+ | 4294967296 * 4294967298 | +-------------------------+ | 8589934592 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] New (by Monty): Give warnings in case of overflow in integer arithmetic (65)
by worklog-noreply＠askmonty.org 17 Nov '09

17 Nov '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Give warnings in case of overflow in integer arithmetic CREATION DATE..: Wed, 18 Nov 2009, 01:20 SUPERVISOR.....: Bothorsen IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 65 (http://askmonty.org/worklog/?tid=65) VERSION........: Benchmarks-3.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 40 (hours remain) ORIG. ESTIMATE.: 40 PROGRESS NOTES: DESCRIPTION: Give warnings in case of overflow in integer arithmetic The issue is described here: http://bugs.launchpad.net/maria/+bug/482846 MySQL reason is descried here: http://dev.mysql.com/doc/refman/5.1/en/arithmetic-functions.html We can't fix the arithmetic as it would make us incompatible in how MySQL works. We should however see if we can generate warnings when overflows happens. mysql> SELECT 4294967296 * 4294967296; +-------------------------+ | 4294967296 * 4294967296 | +-------------------------+ | 0 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 3037000500 * 3037000500; +-------------------------+ | 3037000500 * 3037000500 | +-------------------------+ | -9223372036709301616 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 4294967296 * 4294967297; +-------------------------+ | 4294967296 * 4294967297 | +-------------------------+ | 4294967296 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> SELECT 4294967296 * 4294967298; +-------------------------+ | 4294967296 * 4294967298 | +-------------------------+ | 8589934592 | +-------------------------+ 1 row in set (0.00 sec) mysql> show warnings; Empty set (0.00 sec) ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Bothorsen): Add a mysqlbinlog option to filter updates to certain tables (40)
by worklog-noreply＠askmonty.org 17 Nov '09

17 Nov '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to filter updates to certain tables CREATION DATE..: Mon, 10 Aug 2009, 13:25 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Psergey CATEGORY.......: Server-RawIdeaBin TASK ID........: 40 (http://askmonty.org/worklog/?tid=40) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 32 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 48 PROGRESS NOTES: -=-=(Bothorsen - Tue, 17 Nov 2009, 17:20)=-=- Alex is closer to a working patch now. Worked 14 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Bothorsen - Thu, 12 Nov 2009, 13:13)=-=- Work hours by Alexi and Bo + estimated time for the task. Worked 16 hours and estimate 46 hours remain (original estimate increased by 14 hours). -=-=(Alexi - Sun, 08 Nov 2009, 15:18)=-=- Low Level Design modified. --- /tmp/wklog.40.old.15787 2009-11-08 15:18:11.000000000 +0200 +++ /tmp/wklog.40.new.15787 2009-11-08 15:18:11.000000000 +0200 @@ -62,7 +62,7 @@ it considers the query to extent to the end of the event. 2. For 'db' (current db) the trailing zero is redundant since the length is already known. -3. db_len = 0 means that this is the current db. +3. In tables_info, db_len = 0 means that this is the current db. When reading Query events from binary log, we can recognize its format by its post-header length: in extended case the post-header includes 4 @@ -75,6 +75,77 @@ + #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 + #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 + +*********************************************************************** +HELP NEEDED +*********************************************************************** +The QUERY_HEADER_LEN is used in the definition of MAX_LOG_EVENT_HEADER: + +log_event.h +~~~~~~~~~~~ +#define MAX_LOG_EVENT_HEADER ( /* in order of Query_log_event::write */ \ + LOG_EVENT_HEADER_LEN + /* write_header */ \ + QUERY_HEADER_LEN + /* write_data */ \ + EXECUTE_LOAD_QUERY_EXTRA_HEADER_LEN + /*write_post_header_for_derived */ \ + MAX_SIZE_LOG_EVENT_STATUS + /* status */ \ + NAME_LEN + 1) + +which is used only for setting + + thd->variables.max_allowed_packet + mysql->net.max_packet_size + +Looks like (but I am not quite sure) that QUERY_HEADER_LEN can simply +(without making any other changes) be substituted in this definition by +QUERY_HEADER_LEN_EXT. + +Below I list all places where MAX_LOG_EVENT_HEADER is used: + +slave.cc +~~~~~~~~ +static int init_slave_thread(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN to the max_allowed_packet on all + slave threads, since a replication event can become this much larger + than the corresponding packet (query) sent from client to master. + */ + thd->variables.max_allowed_packet= global_system_variables.max_allowed_packet + + MAX_LOG_EVENT_HEADER; /* note, incr over the global not session var */ + ... +} +pthread_handler_t handle_slave_io(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN to the max_packet_size on the I/O + thread, since a replication event can become this much larger than + the corresponding packet (query) sent from client to master. + */ + mysql->net.max_packet_size= thd->net.max_packet_size+= MAX_LOG_EVENT_HEADER; + ... +} + +sql_repl.cc +~~~~~~~~~~~ +void mysql_binlog_send(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN, since a binlog event can become + this larger than the corresponding packet (query) sent + from client to master. + */ + thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; + ... +} +bool mysql_show_binlog_events(...) +{ ... + /* + to account binlog event header size + */ + thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; + ... +} + 3. Changes in log events ************************ @@ -84,7 +155,7 @@ This setting is done in Format description event constructor which creates the event for writing to binary log: - if (binlog_with_tables_info) + if (opt_binlog_with_tables_info) post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; else post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; @@ -99,12 +170,12 @@ following manner: switch (binlog_ver) { - case 4: /* MySQL 5.0 and higher */ + #ifndef MYSQL_CLIENT + case 4: /* MySQL 5.0 and higher */ ... -+ #else -+ <error> + break; + #endif + case 1: case 3: ... @@ -132,7 +203,7 @@ -------------------------------- [Creates the event for binlogging] -In case of binlog_with_tables_info = TRUE, set additionally query_len, +In case of opt_binlog_with_tables_info = TRUE, set additionally query_len, tables_info_len, and tables_info members (the constructor is to have an additional 'tables_info' argument). @@ -140,7 +211,7 @@ ---------------- [Writes the event to binlog] -In case of binlog_with_tables_info = TRUE, write additional members +In case of opt_binlog_with_tables_info = TRUE, write additional members (query_len, tables_info_len, and tables_info) to binary log. Also write corresponding whole event length to the common-header. -=-=(Alexi - Sun, 08 Nov 2009, 10:40)=-=- Low Level Design modified. --- /tmp/wklog.40.old.5055 2009-11-08 08:40:02.000000000 +0000 +++ /tmp/wklog.40.new.5055 2009-11-08 08:40:02.000000000 +0000 @@ -3,6 +3,7 @@ 1. Adding --binlog-with-tables-info option ****************************************** +GLOBAL, read-only option. When set, Query events are to be written in the extended binary format which contains tables_info. When not set, Query events -=-=(Alexi - Thu, 05 Nov 2009, 12:37)=-=- Low Level Design modified. --- /tmp/wklog.40.old.11441 2009-11-05 12:37:16.000000000 +0200 +++ /tmp/wklog.40.new.11441 2009-11-05 12:37:16.000000000 +0200 @@ -1,9 +1,18 @@ OPTION: 2.5 Extend Query Events With Tables Info ================================================ -1. Query_log_event Binary Format -******************************** -Changes to be done: +1. Adding --binlog-with-tables-info option +****************************************** + +When set, Query events are to be written in the extended binary +format which contains tables_info. When not set, Query events +are to be written in usual format (without any changes). + +2. Query event extended binary format +************************************* + +When --binlog-with-tables-info is set, Query events are writen +to binary log in the following (extended) format. Query_log_event binary format --------------------------------- @@ -24,12 +33,12 @@ error_code 2 status_vars_len 2 + query_len 2 (see Note 1) -+ tables_info_len 2 (see Note 2) ++ tables_info_len 2 --------------------------------- BODY: status_vars status_vars_len - db db_len + 1 -+ db db_len (see Note 3) ++ db db_len (see Note 2) query query_len + tables_info @@ -37,7 +46,7 @@ --------------------------------- Name Size (bytes) --------------------------------- - db_len 1 (see Note 4) + db_len 1 (see Note 3) db db_len table_name_len 1 table_name table_name_len @@ -48,19 +57,99 @@ table_name table_name_len NOTES -1. Currently Query_log_event format doesn't include 'query_len' because +1. In usual format, Query_log_event doesn't include 'query_len' because it considers the query to extent to the end of the event. -2. If tables_info is not included in the event (--binlog-with-tables-info - option), tables_info_len = 0. -3. The trailing zero is redundant since the length is already known. -4. In case of db = current db, db_len = 0 and db = empty, because - current db is already included in the current event format. +2. For 'db' (current db) the trailing zero is redundant since the length + is already known. +3. db_len = 0 means that this is the current db. + +When reading Query events from binary log, we can recognize its format +by its post-header length: in extended case the post-header includes 4 +additional bytes. + + #define QUERY_HEADER_LEN (QUERY_HEADER_MINIMAL_LEN + 4) ++ #define QUERY_HEADER_LEN_EXT (QUERY_HEADER_LEN + 4) + ... + #define Q_STATUS_VARS_LEN_OFFSET 11 ++ #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 ++ #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 + +3. Changes in log events +************************ + +3.1. Format description event +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Changes needed here concern setting post-header length for Query events. +This setting is done in Format description event constructor which creates +the event for writing to binary log: + + if (binlog_with_tables_info) + post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; + else + post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; + +This change is to be done only for case binlog_ver = 4. + +NOTE. The refered above constructor is allowed to be invoked in a client +context for creating "artificial" Format description events in case of +MySQL < 5.0 (e.g. see mysqlbinlog code). To avoid compilation problems +(because of 'binlog_with_tables_info') and taking into account the +"MySQL < 5.0" restriction, we have to #ifdef out the above code in +following manner: + + switch (binlog_ver) { + case 4: /* MySQL 5.0 and higher */ ++ #ifndef MYSQL_CLIENT + ... ++ #else ++ <error> ++ #endif + case 1: + case 3: + ... + } + +3.2. Query event +~~~~~~~~~~~~~~~~ +Changes needed here include adding tables_info and tables_info_len +members (member for query length already exists) and modifying the +following function-members: + +Query_log_event(buf) constructor +-------------------------------- +[Parses binary format written to the 'buf'] + +Getting post-header length from the Format description event (passed +to the constructor as an argument), define whether buf contains an +extended or usual Query event and parse the buf contents accordingly. + +NOTE. Defining Query event format here should be done with taking into +account that this constructor can be called within a Query-derived +event with the event_type argument != QUERY_EVENT. + +Query_log_event(thd) constructor +-------------------------------- +[Creates the event for binlogging] + +In case of binlog_with_tables_info = TRUE, set additionally query_len, +tables_info_len, and tables_info members (the constructor is to have +an additional 'tables_info' argument). + +write() function +---------------- +[Writes the event to binlog] + +In case of binlog_with_tables_info = TRUE, write additional members +(query_len, tables_info_len, and tables_info) to binary log. Also +write corresponding whole event length to the common-header. + +<To be continued> -2. Where to get tables info from? +4. Where to get tables info from? ********************************* -2.1. Case study: CREATE TABLE -****************************** +4.1. Case study: CREATE TABLE +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** CREATE TABLE table [SELECT ...] @@ -129,4 +218,4 @@ } } -To be continued +<To be continued> -=-=(Alexi - Wed, 04 Nov 2009, 10:21)=-=- Low Level Design modified. --- /tmp/wklog.40.old.6734 2009-11-04 10:21:20.000000000 +0200 +++ /tmp/wklog.40.new.6734 2009-11-04 10:21:20.000000000 +0200 @@ -21,9 +21,9 @@ slave_proxy_id 4 exec_time 4 db_len 1 -+ query_len 2 (see Note 1) error_code 2 status_vars_len 2 ++ query_len 2 (see Note 1) + tables_info_len 2 (see Note 2) --------------------------------- BODY: -=-=(Alexi - Tue, 03 Nov 2009, 11:19)=-=- Low Level Design modified. --- /tmp/wklog.40.old.7187 2009-11-03 11:19:22.000000000 +0200 +++ /tmp/wklog.40.new.7187 2009-11-03 11:19:22.000000000 +0200 @@ -1 +1,132 @@ +OPTION: 2.5 Extend Query Events With Tables Info +================================================ +1. Query_log_event Binary Format +******************************** +Changes to be done: + + Query_log_event binary format + --------------------------------- + Name Size (bytes) + --------------------------------- + COMMON HEADER: + timestamp 4 + type 1 + server_id 4 + total_size 4 + master_position 4 + flags 2 + --------------------------------- + POST HEADER: + slave_proxy_id 4 + exec_time 4 + db_len 1 ++ query_len 2 (see Note 1) + error_code 2 + status_vars_len 2 ++ tables_info_len 2 (see Note 2) + --------------------------------- + BODY: + status_vars status_vars_len +- db db_len + 1 ++ db db_len (see Note 3) + query query_len ++ tables_info + + tables_info binary format + --------------------------------- + Name Size (bytes) + --------------------------------- + db_len 1 (see Note 4) + db db_len + table_name_len 1 + table_name table_name_len + ... + db_len 1 + db db_len + table_name_len 1 + table_name table_name_len + +NOTES +1. Currently Query_log_event format doesn't include 'query_len' because + it considers the query to extent to the end of the event. +2. If tables_info is not included in the event (--binlog-with-tables-info + option), tables_info_len = 0. +3. The trailing zero is redundant since the length is already known. +4. In case of db = current db, db_len = 0 and db = empty, because + current db is already included in the current event format. + +2. Where to get tables info from? +********************************* + +2.1. Case study: CREATE TABLE +****************************** + +*** CREATE TABLE table [SELECT ...] + + bool mysql_create_table_no_lock( + THD *thd, + const char *db, + const char *table_name, ...) + { + ... + // ------------------------------------- + // WL40: To be included in tables_info: + // * db, table_name + // * thd->lex->query_tables (tables refered to in + // the select-part; empty if no select-part) + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + +*** CREATE TABLE table LIKE src-table + + bool mysql_create_like_table( + ... + TABLE_LIST *table, + TABLE_LIST *src_table, + ...) + { + ... + if (thd->current_stmt_binlog_row_based) + { // RBR: In this case we don't replicate temp tables + if (!(create_info->options & HA_LEX_CREATE_TMP_TABLE)) + { + if (src_table->table->s->tmp_table) + { // CREATE normal-table LIKE temp-table: + + // Generate new query without LIKE-part + store_create_info(thd, table, &query, create_info, FALSE); + + // ------------------------------------- + // WL40: To include to tables_info: + // * table (src_table is not included) + // ------------------------------------- + write_bin_log(thd, TRUE, query.ptr(), query.length()); + } + else + { // CREATE normal-table LIKE normal-table + + // ------------------------------------- + // WL40: To include to log_tables_info: + // * table + // * src_table + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + } + // CREATE temp-table LIKE ... + // This case is not replicated + } + else + { // SBR: + // ------------------------------------- + // WL40: To include to tables_info: + // * table + // * src_table + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + } + +To be continued -=-=(Alexi - Mon, 02 Nov 2009, 11:34)=-=- Worked 2 hours on option 2.5 Worked 2 hours and estimate 48 hours remain (original estimate increased by 50 hours). -=-=(Alexi - Mon, 02 Nov 2009, 11:20)=-=- High-Level Specification modified. --- /tmp/wklog.40.old.4848 2009-11-02 11:20:24.000000000 +0200 +++ /tmp/wklog.40.new.4848 2009-11-02 11:20:24.000000000 +0200 @@ -90,3 +90,25 @@ It might be useful to integrate this with the code that already handles --replicate-ignore-db and similar slave options. +2.5 Extend Query Events With Tables Info +---------------------------------------- + +We could extend query events structure with a tables info - a list of tables +which the query refers to: + + <current query event structure> + tables_info_len + dbase_len dbase + table_len table + ... + dbase_len dbase + table_len table + +Note. In case of <dbase> = current data base, we can set dbase_len = 0 + and dbase = empty because current query event structure already + includes current data base name. + +Note. Possibly it is reasonable also to add a --binlog-with-tables-info + option which defines whether tables info must be included to the + query events. + -=-=(Knielsen - Fri, 14 Aug 2009, 15:47)=-=- High-Level Specification modified. --- /tmp/wklog.40.old.10896 2009-08-14 15:47:39.000000000 +0300 +++ /tmp/wklog.40.new.10896 2009-08-14 15:47:39.000000000 +0300 @@ -72,3 +72,21 @@ /* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ... and further processing in mysqlbinlog will be trivial. + +2.4 Implement server functionality to ignore certain tables +----------------------------------------------------------- + +We could add a general facility in the server to ignore certain tables: + + SET SESSION ignored_tables = "db1.t1,db2.t2"; + +This would work similar to --replicate-ignore-table, but in a general way not +restricted to the slave SQL thread. + +It would then be trivial for mysqlbinlog to add such statements at the start +of the output, or probably the user could just do it manually with no need for +additional options for mysqlbinlog. + +It might be useful to integrate this with the code that already handles +--replicate-ignore-db and similar slave options. + ------------------------------------------------------------ -=-=(View All Progress Notes, 14 total)=-=- http://askmonty.org/worklog/index.pl?tid=40&nolimit=1 DESCRIPTION: Replication slave can be set to filter updates to certain tables with --replicate-[wild-]{do,ignore}-table options. This task is about adding similar functionality to mysqlbinlog. HIGH-LEVEL SPECIFICATION: 1. Context ---------- (See http://askmonty.org/wiki/index.php/Scratch/ReplicationOptions for global overview) At the moment, the server has these replication slave options: --replicate-do-table=db.tbl --replicate-ignore-table=db.tbl --replicate-wild-do-table=pattern.pattern --replicate-wild-ignore-table=pattern.pattern They affect both RBR and SBR events. SBR events are checked after the statement has been parsed, the server iterates over list of used tables and checks them againist --replicate instructions. What is interesting is that this scheme still allows to update the ignored table through a VIEW. 2. Table filtering in mysqlbinlog --------------------------------- Per-table filtering of RBR events is easy (as it is relatively easy to extract the name of the table that the event applies to). Per-table filtering of SBR events is hard, as generally it is not apparent which tables the statement refers to. This opens possible options: 2.1 Put the parser into mysqlbinlog ----------------------------------- Once we have a full parser in mysqlbinlog, we'll be able to check which tables are used by a statement, and will allow to show behaviour identical to those that one obtains when using --replicate-* slave options. (It is not clear how much effort is needed to put the parser into mysqlbinlog. Any guesses?) 2.2 Use dumb regexp match ------------------------- Use a really dumb approach. A query is considered to be modifying table X if it matches an expression CREATE TABLE $tablename DROP $tablename UPDATE ...$tablename ... SET // here '...' can't contain the word 'SET' DELETE ...$tablename ... WHERE // same as above ALTER TABLE $tablename .. etc (go get from the grammar) .. The advantage over doing the same in awk is that mysqlbinlog will also process RBR statements, and together with that will provide a working solution for those who are careful with their table names not mixing with string constants and such. (TODO: string constants are of particular concern as they come from [potentially hostile] users, unlike e.g. table aliases which come from [not hostile] developers. Remove also all string constants before attempting to do match?) 2.3 Have the master put annotations ----------------------------------- We could add a master option so that it injects into query a mark that tells which tables the query will affect, e.g. for the query UPDATE t1 LEFT JOIN db3.t2 ON ... WHERE ... the binlog will have /* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ... and further processing in mysqlbinlog will be trivial. 2.4 Implement server functionality to ignore certain tables ----------------------------------------------------------- We could add a general facility in the server to ignore certain tables: SET SESSION ignored_tables = "db1.t1,db2.t2"; This would work similar to --replicate-ignore-table, but in a general way not restricted to the slave SQL thread. It would then be trivial for mysqlbinlog to add such statements at the start of the output, or probably the user could just do it manually with no need for additional options for mysqlbinlog. It might be useful to integrate this with the code that already handles --replicate-ignore-db and similar slave options. 2.5 Extend Query Events With Tables Info ---------------------------------------- We could extend query events structure with a tables info - a list of tables which the query refers to: <current query event structure> tables_info_len dbase_len dbase table_len table ... dbase_len dbase table_len table Note. In case of <dbase> = current data base, we can set dbase_len = 0 and dbase = empty because current query event structure already includes current data base name. Note. Possibly it is reasonable also to add a --binlog-with-tables-info option which defines whether tables info must be included to the query events. LOW-LEVEL DESIGN: OPTION: 2.5 Extend Query Events With Tables Info ================================================ 1. Adding --binlog-with-tables-info option ****************************************** GLOBAL, read-only option. When set, Query events are to be written in the extended binary format which contains tables_info. When not set, Query events are to be written in usual format (without any changes). 2. Query event extended binary format ************************************* When --binlog-with-tables-info is set, Query events are writen to binary log in the following (extended) format. Query_log_event binary format --------------------------------- Name Size (bytes) --------------------------------- COMMON HEADER: timestamp 4 type 1 server_id 4 total_size 4 master_position 4 flags 2 --------------------------------- POST HEADER: slave_proxy_id 4 exec_time 4 db_len 1 error_code 2 status_vars_len 2 + query_len 2 (see Note 1) + tables_info_len 2 --------------------------------- BODY: status_vars status_vars_len - db db_len + 1 + db db_len (see Note 2) query query_len + tables_info tables_info binary format --------------------------------- Name Size (bytes) --------------------------------- db_len 1 (see Note 3) db db_len table_name_len 1 table_name table_name_len ... db_len 1 db db_len table_name_len 1 table_name table_name_len NOTES 1. In usual format, Query_log_event doesn't include 'query_len' because it considers the query to extent to the end of the event. 2. For 'db' (current db) the trailing zero is redundant since the length is already known. 3. In tables_info, db_len = 0 means that this is the current db. When reading Query events from binary log, we can recognize its format by its post-header length: in extended case the post-header includes 4 additional bytes. #define QUERY_HEADER_LEN (QUERY_HEADER_MINIMAL_LEN + 4) + #define QUERY_HEADER_LEN_EXT (QUERY_HEADER_LEN + 4) ... #define Q_STATUS_VARS_LEN_OFFSET 11 + #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 + #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 *********************************************************************** HELP NEEDED *********************************************************************** The QUERY_HEADER_LEN is used in the definition of MAX_LOG_EVENT_HEADER: log_event.h ~~~~~~~~~~~ #define MAX_LOG_EVENT_HEADER ( /* in order of Query_log_event::write */ \ LOG_EVENT_HEADER_LEN + /* write_header */ \ QUERY_HEADER_LEN + /* write_data */ \ EXECUTE_LOAD_QUERY_EXTRA_HEADER_LEN + /*write_post_header_for_derived */ \ MAX_SIZE_LOG_EVENT_STATUS + /* status */ \ NAME_LEN + 1) which is used only for setting thd->variables.max_allowed_packet mysql->net.max_packet_size Looks like (but I am not quite sure) that QUERY_HEADER_LEN can simply (without making any other changes) be substituted in this definition by QUERY_HEADER_LEN_EXT. Below I list all places where MAX_LOG_EVENT_HEADER is used: slave.cc ~~~~~~~~ static int init_slave_thread(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN to the max_allowed_packet on all slave threads, since a replication event can become this much larger than the corresponding packet (query) sent from client to master. */ thd->variables.max_allowed_packet= global_system_variables.max_allowed_packet + MAX_LOG_EVENT_HEADER; /* note, incr over the global not session var */ ... } pthread_handler_t handle_slave_io(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN to the max_packet_size on the I/O thread, since a replication event can become this much larger than the corresponding packet (query) sent from client to master. */ mysql->net.max_packet_size= thd->net.max_packet_size+= MAX_LOG_EVENT_HEADER; ... } sql_repl.cc ~~~~~~~~~~~ void mysql_binlog_send(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN, since a binlog event can become this larger than the corresponding packet (query) sent from client to master. */ thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; ... } bool mysql_show_binlog_events(...) { ... /* to account binlog event header size */ thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; ... } 3. Changes in log events ************************ 3.1. Format description event ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Changes needed here concern setting post-header length for Query events. This setting is done in Format description event constructor which creates the event for writing to binary log: if (opt_binlog_with_tables_info) post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; else post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; This change is to be done only for case binlog_ver = 4. NOTE. The refered above constructor is allowed to be invoked in a client context for creating "artificial" Format description events in case of MySQL < 5.0 (e.g. see mysqlbinlog code). To avoid compilation problems (because of 'binlog_with_tables_info') and taking into account the "MySQL < 5.0" restriction, we have to #ifdef out the above code in following manner: switch (binlog_ver) { + #ifndef MYSQL_CLIENT case 4: /* MySQL 5.0 and higher */ ... break; + #endif case 1: case 3: ... } 3.2. Query event ~~~~~~~~~~~~~~~~ Changes needed here include adding tables_info and tables_info_len members (member for query length already exists) and modifying the following function-members: Query_log_event(buf) constructor -------------------------------- [Parses binary format written to the 'buf'] Getting post-header length from the Format description event (passed to the constructor as an argument), define whether buf contains an extended or usual Query event and parse the buf contents accordingly. NOTE. Defining Query event format here should be done with taking into account that this constructor can be called within a Query-derived event with the event_type argument != QUERY_EVENT. Query_log_event(thd) constructor -------------------------------- [Creates the event for binlogging] In case of opt_binlog_with_tables_info = TRUE, set additionally query_len, tables_info_len, and tables_info members (the constructor is to have an additional 'tables_info' argument). write() function ---------------- [Writes the event to binlog] In case of opt_binlog_with_tables_info = TRUE, write additional members (query_len, tables_info_len, and tables_info) to binary log. Also write corresponding whole event length to the common-header. <To be continued> 4. Where to get tables info from? ********************************* 4.1. Case study: CREATE TABLE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** CREATE TABLE table [SELECT ...] bool mysql_create_table_no_lock( THD *thd, const char *db, const char *table_name, ...) { ... // ------------------------------------- // WL40: To be included in tables_info: // * db, table_name // * thd->lex->query_tables (tables refered to in // the select-part; empty if no select-part) // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } *** CREATE TABLE table LIKE src-table bool mysql_create_like_table( ... TABLE_LIST *table, TABLE_LIST *src_table, ...) { ... if (thd->current_stmt_binlog_row_based) { // RBR: In this case we don't replicate temp tables if (!(create_info->options & HA_LEX_CREATE_TMP_TABLE)) { if (src_table->table->s->tmp_table) { // CREATE normal-table LIKE temp-table: // Generate new query without LIKE-part store_create_info(thd, table, &query, create_info, FALSE); // ------------------------------------- // WL40: To include to tables_info: // * table (src_table is not included) // ------------------------------------- write_bin_log(thd, TRUE, query.ptr(), query.length()); } else { // CREATE normal-table LIKE normal-table // ------------------------------------- // WL40: To include to log_tables_info: // * table // * src_table // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } } // CREATE temp-table LIKE ... // This case is not replicated } else { // SBR: // ------------------------------------- // WL40: To include to tables_info: // * table // * src_table // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } } <To be continued> ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Progress (by Bothorsen): Add a mysqlbinlog option to filter updates to certain tables (40)
by worklog-noreply＠askmonty.org 17 Nov '09

17 Nov '09

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Add a mysqlbinlog option to filter updates to certain tables CREATION DATE..: Mon, 10 Aug 2009, 13:25 SUPERVISOR.....: Monty IMPLEMENTOR....: COPIES TO......: Psergey CATEGORY.......: Server-RawIdeaBin TASK ID........: 40 (http://askmonty.org/worklog/?tid=40) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 32 ESTIMATE.......: 32 (hours remain) ORIG. ESTIMATE.: 48 PROGRESS NOTES: -=-=(Bothorsen - Tue, 17 Nov 2009, 17:20)=-=- Alex is closer to a working patch now. Worked 14 hours and estimate 32 hours remain (original estimate unchanged). -=-=(Bothorsen - Thu, 12 Nov 2009, 13:13)=-=- Work hours by Alexi and Bo + estimated time for the task. Worked 16 hours and estimate 46 hours remain (original estimate increased by 14 hours). -=-=(Alexi - Sun, 08 Nov 2009, 15:18)=-=- Low Level Design modified. --- /tmp/wklog.40.old.15787 2009-11-08 15:18:11.000000000 +0200 +++ /tmp/wklog.40.new.15787 2009-11-08 15:18:11.000000000 +0200 @@ -62,7 +62,7 @@ it considers the query to extent to the end of the event. 2. For 'db' (current db) the trailing zero is redundant since the length is already known. -3. db_len = 0 means that this is the current db. +3. In tables_info, db_len = 0 means that this is the current db. When reading Query events from binary log, we can recognize its format by its post-header length: in extended case the post-header includes 4 @@ -75,6 +75,77 @@ + #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 + #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 + +*********************************************************************** +HELP NEEDED +*********************************************************************** +The QUERY_HEADER_LEN is used in the definition of MAX_LOG_EVENT_HEADER: + +log_event.h +~~~~~~~~~~~ +#define MAX_LOG_EVENT_HEADER ( /* in order of Query_log_event::write */ \ + LOG_EVENT_HEADER_LEN + /* write_header */ \ + QUERY_HEADER_LEN + /* write_data */ \ + EXECUTE_LOAD_QUERY_EXTRA_HEADER_LEN + /*write_post_header_for_derived */ \ + MAX_SIZE_LOG_EVENT_STATUS + /* status */ \ + NAME_LEN + 1) + +which is used only for setting + + thd->variables.max_allowed_packet + mysql->net.max_packet_size + +Looks like (but I am not quite sure) that QUERY_HEADER_LEN can simply +(without making any other changes) be substituted in this definition by +QUERY_HEADER_LEN_EXT. + +Below I list all places where MAX_LOG_EVENT_HEADER is used: + +slave.cc +~~~~~~~~ +static int init_slave_thread(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN to the max_allowed_packet on all + slave threads, since a replication event can become this much larger + than the corresponding packet (query) sent from client to master. + */ + thd->variables.max_allowed_packet= global_system_variables.max_allowed_packet + + MAX_LOG_EVENT_HEADER; /* note, incr over the global not session var */ + ... +} +pthread_handler_t handle_slave_io(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN to the max_packet_size on the I/O + thread, since a replication event can become this much larger than + the corresponding packet (query) sent from client to master. + */ + mysql->net.max_packet_size= thd->net.max_packet_size+= MAX_LOG_EVENT_HEADER; + ... +} + +sql_repl.cc +~~~~~~~~~~~ +void mysql_binlog_send(...) +{ ... + /* + Adding MAX_LOG_EVENT_HEADER_LEN, since a binlog event can become + this larger than the corresponding packet (query) sent + from client to master. + */ + thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; + ... +} +bool mysql_show_binlog_events(...) +{ ... + /* + to account binlog event header size + */ + thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; + ... +} + 3. Changes in log events ************************ @@ -84,7 +155,7 @@ This setting is done in Format description event constructor which creates the event for writing to binary log: - if (binlog_with_tables_info) + if (opt_binlog_with_tables_info) post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; else post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; @@ -99,12 +170,12 @@ following manner: switch (binlog_ver) { - case 4: /* MySQL 5.0 and higher */ + #ifndef MYSQL_CLIENT + case 4: /* MySQL 5.0 and higher */ ... -+ #else -+ <error> + break; + #endif + case 1: case 3: ... @@ -132,7 +203,7 @@ -------------------------------- [Creates the event for binlogging] -In case of binlog_with_tables_info = TRUE, set additionally query_len, +In case of opt_binlog_with_tables_info = TRUE, set additionally query_len, tables_info_len, and tables_info members (the constructor is to have an additional 'tables_info' argument). @@ -140,7 +211,7 @@ ---------------- [Writes the event to binlog] -In case of binlog_with_tables_info = TRUE, write additional members +In case of opt_binlog_with_tables_info = TRUE, write additional members (query_len, tables_info_len, and tables_info) to binary log. Also write corresponding whole event length to the common-header. -=-=(Alexi - Sun, 08 Nov 2009, 10:40)=-=- Low Level Design modified. --- /tmp/wklog.40.old.5055 2009-11-08 08:40:02.000000000 +0000 +++ /tmp/wklog.40.new.5055 2009-11-08 08:40:02.000000000 +0000 @@ -3,6 +3,7 @@ 1. Adding --binlog-with-tables-info option ****************************************** +GLOBAL, read-only option. When set, Query events are to be written in the extended binary format which contains tables_info. When not set, Query events -=-=(Alexi - Thu, 05 Nov 2009, 12:37)=-=- Low Level Design modified. --- /tmp/wklog.40.old.11441 2009-11-05 12:37:16.000000000 +0200 +++ /tmp/wklog.40.new.11441 2009-11-05 12:37:16.000000000 +0200 @@ -1,9 +1,18 @@ OPTION: 2.5 Extend Query Events With Tables Info ================================================ -1. Query_log_event Binary Format -******************************** -Changes to be done: +1. Adding --binlog-with-tables-info option +****************************************** + +When set, Query events are to be written in the extended binary +format which contains tables_info. When not set, Query events +are to be written in usual format (without any changes). + +2. Query event extended binary format +************************************* + +When --binlog-with-tables-info is set, Query events are writen +to binary log in the following (extended) format. Query_log_event binary format --------------------------------- @@ -24,12 +33,12 @@ error_code 2 status_vars_len 2 + query_len 2 (see Note 1) -+ tables_info_len 2 (see Note 2) ++ tables_info_len 2 --------------------------------- BODY: status_vars status_vars_len - db db_len + 1 -+ db db_len (see Note 3) ++ db db_len (see Note 2) query query_len + tables_info @@ -37,7 +46,7 @@ --------------------------------- Name Size (bytes) --------------------------------- - db_len 1 (see Note 4) + db_len 1 (see Note 3) db db_len table_name_len 1 table_name table_name_len @@ -48,19 +57,99 @@ table_name table_name_len NOTES -1. Currently Query_log_event format doesn't include 'query_len' because +1. In usual format, Query_log_event doesn't include 'query_len' because it considers the query to extent to the end of the event. -2. If tables_info is not included in the event (--binlog-with-tables-info - option), tables_info_len = 0. -3. The trailing zero is redundant since the length is already known. -4. In case of db = current db, db_len = 0 and db = empty, because - current db is already included in the current event format. +2. For 'db' (current db) the trailing zero is redundant since the length + is already known. +3. db_len = 0 means that this is the current db. + +When reading Query events from binary log, we can recognize its format +by its post-header length: in extended case the post-header includes 4 +additional bytes. + + #define QUERY_HEADER_LEN (QUERY_HEADER_MINIMAL_LEN + 4) ++ #define QUERY_HEADER_LEN_EXT (QUERY_HEADER_LEN + 4) + ... + #define Q_STATUS_VARS_LEN_OFFSET 11 ++ #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 ++ #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 + +3. Changes in log events +************************ + +3.1. Format description event +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Changes needed here concern setting post-header length for Query events. +This setting is done in Format description event constructor which creates +the event for writing to binary log: + + if (binlog_with_tables_info) + post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; + else + post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; + +This change is to be done only for case binlog_ver = 4. + +NOTE. The refered above constructor is allowed to be invoked in a client +context for creating "artificial" Format description events in case of +MySQL < 5.0 (e.g. see mysqlbinlog code). To avoid compilation problems +(because of 'binlog_with_tables_info') and taking into account the +"MySQL < 5.0" restriction, we have to #ifdef out the above code in +following manner: + + switch (binlog_ver) { + case 4: /* MySQL 5.0 and higher */ ++ #ifndef MYSQL_CLIENT + ... ++ #else ++ <error> ++ #endif + case 1: + case 3: + ... + } + +3.2. Query event +~~~~~~~~~~~~~~~~ +Changes needed here include adding tables_info and tables_info_len +members (member for query length already exists) and modifying the +following function-members: + +Query_log_event(buf) constructor +-------------------------------- +[Parses binary format written to the 'buf'] + +Getting post-header length from the Format description event (passed +to the constructor as an argument), define whether buf contains an +extended or usual Query event and parse the buf contents accordingly. + +NOTE. Defining Query event format here should be done with taking into +account that this constructor can be called within a Query-derived +event with the event_type argument != QUERY_EVENT. + +Query_log_event(thd) constructor +-------------------------------- +[Creates the event for binlogging] + +In case of binlog_with_tables_info = TRUE, set additionally query_len, +tables_info_len, and tables_info members (the constructor is to have +an additional 'tables_info' argument). + +write() function +---------------- +[Writes the event to binlog] + +In case of binlog_with_tables_info = TRUE, write additional members +(query_len, tables_info_len, and tables_info) to binary log. Also +write corresponding whole event length to the common-header. + +<To be continued> -2. Where to get tables info from? +4. Where to get tables info from? ********************************* -2.1. Case study: CREATE TABLE -****************************** +4.1. Case study: CREATE TABLE +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** CREATE TABLE table [SELECT ...] @@ -129,4 +218,4 @@ } } -To be continued +<To be continued> -=-=(Alexi - Wed, 04 Nov 2009, 10:21)=-=- Low Level Design modified. --- /tmp/wklog.40.old.6734 2009-11-04 10:21:20.000000000 +0200 +++ /tmp/wklog.40.new.6734 2009-11-04 10:21:20.000000000 +0200 @@ -21,9 +21,9 @@ slave_proxy_id 4 exec_time 4 db_len 1 -+ query_len 2 (see Note 1) error_code 2 status_vars_len 2 ++ query_len 2 (see Note 1) + tables_info_len 2 (see Note 2) --------------------------------- BODY: -=-=(Alexi - Tue, 03 Nov 2009, 11:19)=-=- Low Level Design modified. --- /tmp/wklog.40.old.7187 2009-11-03 11:19:22.000000000 +0200 +++ /tmp/wklog.40.new.7187 2009-11-03 11:19:22.000000000 +0200 @@ -1 +1,132 @@ +OPTION: 2.5 Extend Query Events With Tables Info +================================================ +1. Query_log_event Binary Format +******************************** +Changes to be done: + + Query_log_event binary format + --------------------------------- + Name Size (bytes) + --------------------------------- + COMMON HEADER: + timestamp 4 + type 1 + server_id 4 + total_size 4 + master_position 4 + flags 2 + --------------------------------- + POST HEADER: + slave_proxy_id 4 + exec_time 4 + db_len 1 ++ query_len 2 (see Note 1) + error_code 2 + status_vars_len 2 ++ tables_info_len 2 (see Note 2) + --------------------------------- + BODY: + status_vars status_vars_len +- db db_len + 1 ++ db db_len (see Note 3) + query query_len ++ tables_info + + tables_info binary format + --------------------------------- + Name Size (bytes) + --------------------------------- + db_len 1 (see Note 4) + db db_len + table_name_len 1 + table_name table_name_len + ... + db_len 1 + db db_len + table_name_len 1 + table_name table_name_len + +NOTES +1. Currently Query_log_event format doesn't include 'query_len' because + it considers the query to extent to the end of the event. +2. If tables_info is not included in the event (--binlog-with-tables-info + option), tables_info_len = 0. +3. The trailing zero is redundant since the length is already known. +4. In case of db = current db, db_len = 0 and db = empty, because + current db is already included in the current event format. + +2. Where to get tables info from? +********************************* + +2.1. Case study: CREATE TABLE +****************************** + +*** CREATE TABLE table [SELECT ...] + + bool mysql_create_table_no_lock( + THD *thd, + const char *db, + const char *table_name, ...) + { + ... + // ------------------------------------- + // WL40: To be included in tables_info: + // * db, table_name + // * thd->lex->query_tables (tables refered to in + // the select-part; empty if no select-part) + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + +*** CREATE TABLE table LIKE src-table + + bool mysql_create_like_table( + ... + TABLE_LIST *table, + TABLE_LIST *src_table, + ...) + { + ... + if (thd->current_stmt_binlog_row_based) + { // RBR: In this case we don't replicate temp tables + if (!(create_info->options & HA_LEX_CREATE_TMP_TABLE)) + { + if (src_table->table->s->tmp_table) + { // CREATE normal-table LIKE temp-table: + + // Generate new query without LIKE-part + store_create_info(thd, table, &query, create_info, FALSE); + + // ------------------------------------- + // WL40: To include to tables_info: + // * table (src_table is not included) + // ------------------------------------- + write_bin_log(thd, TRUE, query.ptr(), query.length()); + } + else + { // CREATE normal-table LIKE normal-table + + // ------------------------------------- + // WL40: To include to log_tables_info: + // * table + // * src_table + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + } + // CREATE temp-table LIKE ... + // This case is not replicated + } + else + { // SBR: + // ------------------------------------- + // WL40: To include to tables_info: + // * table + // * src_table + // ------------------------------------- + write_bin_log(thd, TRUE, thd->query, thd->query_length); + } + } + +To be continued -=-=(Alexi - Mon, 02 Nov 2009, 11:34)=-=- Worked 2 hours on option 2.5 Worked 2 hours and estimate 48 hours remain (original estimate increased by 50 hours). -=-=(Alexi - Mon, 02 Nov 2009, 11:20)=-=- High-Level Specification modified. --- /tmp/wklog.40.old.4848 2009-11-02 11:20:24.000000000 +0200 +++ /tmp/wklog.40.new.4848 2009-11-02 11:20:24.000000000 +0200 @@ -90,3 +90,25 @@ It might be useful to integrate this with the code that already handles --replicate-ignore-db and similar slave options. +2.5 Extend Query Events With Tables Info +---------------------------------------- + +We could extend query events structure with a tables info - a list of tables +which the query refers to: + + <current query event structure> + tables_info_len + dbase_len dbase + table_len table + ... + dbase_len dbase + table_len table + +Note. In case of <dbase> = current data base, we can set dbase_len = 0 + and dbase = empty because current query event structure already + includes current data base name. + +Note. Possibly it is reasonable also to add a --binlog-with-tables-info + option which defines whether tables info must be included to the + query events. + -=-=(Knielsen - Fri, 14 Aug 2009, 15:47)=-=- High-Level Specification modified. --- /tmp/wklog.40.old.10896 2009-08-14 15:47:39.000000000 +0300 +++ /tmp/wklog.40.new.10896 2009-08-14 15:47:39.000000000 +0300 @@ -72,3 +72,21 @@ /* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ... and further processing in mysqlbinlog will be trivial. + +2.4 Implement server functionality to ignore certain tables +----------------------------------------------------------- + +We could add a general facility in the server to ignore certain tables: + + SET SESSION ignored_tables = "db1.t1,db2.t2"; + +This would work similar to --replicate-ignore-table, but in a general way not +restricted to the slave SQL thread. + +It would then be trivial for mysqlbinlog to add such statements at the start +of the output, or probably the user could just do it manually with no need for +additional options for mysqlbinlog. + +It might be useful to integrate this with the code that already handles +--replicate-ignore-db and similar slave options. + ------------------------------------------------------------ -=-=(View All Progress Notes, 14 total)=-=- http://askmonty.org/worklog/index.pl?tid=40&nolimit=1 DESCRIPTION: Replication slave can be set to filter updates to certain tables with --replicate-[wild-]{do,ignore}-table options. This task is about adding similar functionality to mysqlbinlog. HIGH-LEVEL SPECIFICATION: 1. Context ---------- (See http://askmonty.org/wiki/index.php/Scratch/ReplicationOptions for global overview) At the moment, the server has these replication slave options: --replicate-do-table=db.tbl --replicate-ignore-table=db.tbl --replicate-wild-do-table=pattern.pattern --replicate-wild-ignore-table=pattern.pattern They affect both RBR and SBR events. SBR events are checked after the statement has been parsed, the server iterates over list of used tables and checks them againist --replicate instructions. What is interesting is that this scheme still allows to update the ignored table through a VIEW. 2. Table filtering in mysqlbinlog --------------------------------- Per-table filtering of RBR events is easy (as it is relatively easy to extract the name of the table that the event applies to). Per-table filtering of SBR events is hard, as generally it is not apparent which tables the statement refers to. This opens possible options: 2.1 Put the parser into mysqlbinlog ----------------------------------- Once we have a full parser in mysqlbinlog, we'll be able to check which tables are used by a statement, and will allow to show behaviour identical to those that one obtains when using --replicate-* slave options. (It is not clear how much effort is needed to put the parser into mysqlbinlog. Any guesses?) 2.2 Use dumb regexp match ------------------------- Use a really dumb approach. A query is considered to be modifying table X if it matches an expression CREATE TABLE $tablename DROP $tablename UPDATE ...$tablename ... SET // here '...' can't contain the word 'SET' DELETE ...$tablename ... WHERE // same as above ALTER TABLE $tablename .. etc (go get from the grammar) .. The advantage over doing the same in awk is that mysqlbinlog will also process RBR statements, and together with that will provide a working solution for those who are careful with their table names not mixing with string constants and such. (TODO: string constants are of particular concern as they come from [potentially hostile] users, unlike e.g. table aliases which come from [not hostile] developers. Remove also all string constants before attempting to do match?) 2.3 Have the master put annotations ----------------------------------- We could add a master option so that it injects into query a mark that tells which tables the query will affect, e.g. for the query UPDATE t1 LEFT JOIN db3.t2 ON ... WHERE ... the binlog will have /* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ... and further processing in mysqlbinlog will be trivial. 2.4 Implement server functionality to ignore certain tables ----------------------------------------------------------- We could add a general facility in the server to ignore certain tables: SET SESSION ignored_tables = "db1.t1,db2.t2"; This would work similar to --replicate-ignore-table, but in a general way not restricted to the slave SQL thread. It would then be trivial for mysqlbinlog to add such statements at the start of the output, or probably the user could just do it manually with no need for additional options for mysqlbinlog. It might be useful to integrate this with the code that already handles --replicate-ignore-db and similar slave options. 2.5 Extend Query Events With Tables Info ---------------------------------------- We could extend query events structure with a tables info - a list of tables which the query refers to: <current query event structure> tables_info_len dbase_len dbase table_len table ... dbase_len dbase table_len table Note. In case of <dbase> = current data base, we can set dbase_len = 0 and dbase = empty because current query event structure already includes current data base name. Note. Possibly it is reasonable also to add a --binlog-with-tables-info option which defines whether tables info must be included to the query events. LOW-LEVEL DESIGN: OPTION: 2.5 Extend Query Events With Tables Info ================================================ 1. Adding --binlog-with-tables-info option ****************************************** GLOBAL, read-only option. When set, Query events are to be written in the extended binary format which contains tables_info. When not set, Query events are to be written in usual format (without any changes). 2. Query event extended binary format ************************************* When --binlog-with-tables-info is set, Query events are writen to binary log in the following (extended) format. Query_log_event binary format --------------------------------- Name Size (bytes) --------------------------------- COMMON HEADER: timestamp 4 type 1 server_id 4 total_size 4 master_position 4 flags 2 --------------------------------- POST HEADER: slave_proxy_id 4 exec_time 4 db_len 1 error_code 2 status_vars_len 2 + query_len 2 (see Note 1) + tables_info_len 2 --------------------------------- BODY: status_vars status_vars_len - db db_len + 1 + db db_len (see Note 2) query query_len + tables_info tables_info binary format --------------------------------- Name Size (bytes) --------------------------------- db_len 1 (see Note 3) db db_len table_name_len 1 table_name table_name_len ... db_len 1 db db_len table_name_len 1 table_name table_name_len NOTES 1. In usual format, Query_log_event doesn't include 'query_len' because it considers the query to extent to the end of the event. 2. For 'db' (current db) the trailing zero is redundant since the length is already known. 3. In tables_info, db_len = 0 means that this is the current db. When reading Query events from binary log, we can recognize its format by its post-header length: in extended case the post-header includes 4 additional bytes. #define QUERY_HEADER_LEN (QUERY_HEADER_MINIMAL_LEN + 4) + #define QUERY_HEADER_LEN_EXT (QUERY_HEADER_LEN + 4) ... #define Q_STATUS_VARS_LEN_OFFSET 11 + #define Q_QUERY_LEN_OFFSET Q_STATUS_VARS_LEN_OFFSET + 2 + #define Q_QUERY_TABLES_INFO_LEN_OFFSET Q_QUERY_LEN_OFFSET + 2 *********************************************************************** HELP NEEDED *********************************************************************** The QUERY_HEADER_LEN is used in the definition of MAX_LOG_EVENT_HEADER: log_event.h ~~~~~~~~~~~ #define MAX_LOG_EVENT_HEADER ( /* in order of Query_log_event::write */ \ LOG_EVENT_HEADER_LEN + /* write_header */ \ QUERY_HEADER_LEN + /* write_data */ \ EXECUTE_LOAD_QUERY_EXTRA_HEADER_LEN + /*write_post_header_for_derived */ \ MAX_SIZE_LOG_EVENT_STATUS + /* status */ \ NAME_LEN + 1) which is used only for setting thd->variables.max_allowed_packet mysql->net.max_packet_size Looks like (but I am not quite sure) that QUERY_HEADER_LEN can simply (without making any other changes) be substituted in this definition by QUERY_HEADER_LEN_EXT. Below I list all places where MAX_LOG_EVENT_HEADER is used: slave.cc ~~~~~~~~ static int init_slave_thread(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN to the max_allowed_packet on all slave threads, since a replication event can become this much larger than the corresponding packet (query) sent from client to master. */ thd->variables.max_allowed_packet= global_system_variables.max_allowed_packet + MAX_LOG_EVENT_HEADER; /* note, incr over the global not session var */ ... } pthread_handler_t handle_slave_io(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN to the max_packet_size on the I/O thread, since a replication event can become this much larger than the corresponding packet (query) sent from client to master. */ mysql->net.max_packet_size= thd->net.max_packet_size+= MAX_LOG_EVENT_HEADER; ... } sql_repl.cc ~~~~~~~~~~~ void mysql_binlog_send(...) { ... /* Adding MAX_LOG_EVENT_HEADER_LEN, since a binlog event can become this larger than the corresponding packet (query) sent from client to master. */ thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; ... } bool mysql_show_binlog_events(...) { ... /* to account binlog event header size */ thd->variables.max_allowed_packet+= MAX_LOG_EVENT_HEADER; ... } 3. Changes in log events ************************ 3.1. Format description event ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Changes needed here concern setting post-header length for Query events. This setting is done in Format description event constructor which creates the event for writing to binary log: if (opt_binlog_with_tables_info) post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN_EXT; else post_header_len[QUERY_EVENT - 1] = QUERY_HEADER_LEN; This change is to be done only for case binlog_ver = 4. NOTE. The refered above constructor is allowed to be invoked in a client context for creating "artificial" Format description events in case of MySQL < 5.0 (e.g. see mysqlbinlog code). To avoid compilation problems (because of 'binlog_with_tables_info') and taking into account the "MySQL < 5.0" restriction, we have to #ifdef out the above code in following manner: switch (binlog_ver) { + #ifndef MYSQL_CLIENT case 4: /* MySQL 5.0 and higher */ ... break; + #endif case 1: case 3: ... } 3.2. Query event ~~~~~~~~~~~~~~~~ Changes needed here include adding tables_info and tables_info_len members (member for query length already exists) and modifying the following function-members: Query_log_event(buf) constructor -------------------------------- [Parses binary format written to the 'buf'] Getting post-header length from the Format description event (passed to the constructor as an argument), define whether buf contains an extended or usual Query event and parse the buf contents accordingly. NOTE. Defining Query event format here should be done with taking into account that this constructor can be called within a Query-derived event with the event_type argument != QUERY_EVENT. Query_log_event(thd) constructor -------------------------------- [Creates the event for binlogging] In case of opt_binlog_with_tables_info = TRUE, set additionally query_len, tables_info_len, and tables_info members (the constructor is to have an additional 'tables_info' argument). write() function ---------------- [Writes the event to binlog] In case of opt_binlog_with_tables_info = TRUE, write additional members (query_len, tables_info_len, and tables_info) to binary log. Also write corresponding whole event length to the common-header. <To be continued> 4. Where to get tables info from? ********************************* 4.1. Case study: CREATE TABLE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** CREATE TABLE table [SELECT ...] bool mysql_create_table_no_lock( THD *thd, const char *db, const char *table_name, ...) { ... // ------------------------------------- // WL40: To be included in tables_info: // * db, table_name // * thd->lex->query_tables (tables refered to in // the select-part; empty if no select-part) // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } *** CREATE TABLE table LIKE src-table bool mysql_create_like_table( ... TABLE_LIST *table, TABLE_LIST *src_table, ...) { ... if (thd->current_stmt_binlog_row_based) { // RBR: In this case we don't replicate temp tables if (!(create_info->options & HA_LEX_CREATE_TMP_TABLE)) { if (src_table->table->s->tmp_table) { // CREATE normal-table LIKE temp-table: // Generate new query without LIKE-part store_create_info(thd, table, &query, create_info, FALSE); // ------------------------------------- // WL40: To include to tables_info: // * table (src_table is not included) // ------------------------------------- write_bin_log(thd, TRUE, query.ptr(), query.length()); } else { // CREATE normal-table LIKE normal-table // ------------------------------------- // WL40: To include to log_tables_info: // * table // * src_table // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } } // CREATE temp-table LIKE ... // This case is not replicated } else { // SBR: // ------------------------------------- // WL40: To include to tables_info: // * table // * src_table // ------------------------------------- write_bin_log(thd, TRUE, thd->query, thd->query_length); } } <To be continued> ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v3.5.9)

1 0

[Maria-developers] Sketch of CRETEA TABLE options
by Oleksandr Byelkin 17 Nov '09

17 Nov '09

Hi! Here is try to implement CRETE TABLE options. Here is only for TABLE, and no real support from the engine side, but the result looks like: create table t1 (a int) ttt = xxx; Warnings: Warning 1639 Unused option 'ttt' (value 'xxx') of table 'test'.'t1' I publish it to let other to catch problems on early stage.

1 0

[Maria-developers] Using 5.1 tag on launchpad bugs
by Henrik Ingo 17 Nov '09

17 Nov '09

Hi all I've started using the tag "5.1" for launchpad bugs that should be fixed before 5.1 final release. This is so that one can easily get an overview of release activity with this list: https://bugs.launchpad.net/maria/+bugs?field.tag=5.1 I'm aware that there is also a "Nominate for release" feature, and would be happy to use that if someone else sets it up. For now it seems that using tags is the simplest solution. I don't seem to have rights to set importance of bugs. Of the four "cosmetics" bugs I've filed last week: These 2 are MUST have for 5.1 RC: 481311 Anything needed to build MariaDB binaries and packages must be in the maria launchpad repository Undecided New 481320 Remove "brand/tag" in the official MariaDB 5.1 rpm/deb/tar binaries (ourdelta, or otherwise) Undecided New While these 2 can be discussed (ie nice to have): 481298 MariaDB deb/rpm/tar binaries should be called mariadb-*, not mysql-* Undecided New 481326 Remove "-maria" string from MariaDB tar packages Undecided New As a general comment, while lp bugtracker may be a bit simplistic, I think we are not fully using all of its capabilities either. We can talk about this today on the MP call at least, how to become better users of it. Thanks. henrik -- email: henrik.ingo(a)avoinelama.fi tel: +358-40-5697354 www: www.avoinelama.fi/~hingo book: www.openlife.cc

1 0