- developers - lists.mariadb.org

[Maria-developers] MariaDB and MySQL 5.5
by MARK CALLAGHAN 11 Feb '11

11 Feb '11

When will there be a MariaDB branch based on MySQL 5.5? I want to use that as the basis for the 5.5 version of the Facebook patch. -- Mark Callaghan mdcallag(a)gmail.com

2 1

[Maria-developers] WL#175 New (by Mdcallag): Unique ID in binlog event metadata
by worklog-noreply＠askmonty.org 11 Feb '11

11 Feb '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Unique ID in binlog event metadata CREATION DATE..: Fri, 11 Feb 2011, 05:03 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 175 (http://askmonty.org/worklog/?tid=175) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: While I would love to have global transaction IDs for MySQL replication (http://askmonty.org/worklog/Server-RawIdeaBin/?tid=31) that is a huge project. I think we can automate slave failover if binlog events have a metadata attribute that is a unique ID and that metadata is sticky like server ID when a slave is run with --log-slave-updates. Assuming this metadata is added, when it is time to do slave failover, first determine the serverID + uniqueID for the last binlog event group executed on the slave, then find that filename/offset for that event on the new master, then CHANGE MASTER on the slave to start replication on the new master just past that offset. I mention that serverID can be used in addition to the uniqueID, so maybe uniqueID only needs to be unique within the binlog events generated for one serverID. The search on the new master is more efficient when the uniqueID are generated in increasing order. But I am not sure whether that is required. We might need a new server command that maps (serverID, uniqueID) to (filename, offset), but I am not sure that is required. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 08 Feb '11

08 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa > Ichikawa > Sent: Dienstag, 8. Februar 2011 04:46 > To: Vladislav Vaintroub > Cc: Philip Stoev; maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Hello, Vladislav. > All previous 5.2.5 installer created its data directory even when > upgrading existing instance. Therefore, if datadir was not set > explicity in the inherited my.ini, it was using the new data directory > (in other words, it was not reusing the previous database files, I'm > afraid). Yes. Somewhat unusual case my.ini without datadir was not handled correctly though (case without my.ini at all was handled). Fixed that, MSIs are in the usual location. Thanks for input! > - Kazuhisa

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 05 Feb '11

05 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa > Ichikawa > Sent: Samstag, 5. Februar 2011 04:34 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. Hello Kazuhisa, > I BROWSED through it but did not read it carefully line by line before > reporting bugs. Sorry about that. Yet, all of your reports were valid . > > By the way, after finished installing 5.2.5 and upgrading the previous > (5.1.53) service instance, I have 2 service definitions for 5.2.5, one > that was newly created and the other that was successfully upgraded > from 5.1.53. > > I think this is a human (that is my) error rather than a bug, because > I checked "Install as service" in the "Defaults settings" dialog > before the installer starts installing 5.2.5 files and then chose to > upgrade the existing service in the Upgrade Wizard that was started > after installation. If you think installing instance was a mistake on you part (you only wanted to upgrade what you have), you can correct that easily. In maintenance mode install (Add/Remove Programs=>MariaDB5.2=>Change/Uninstall), select "Change" button. Deselect "Database Instance". Choose "Remove data" afterwards. Now the mistakenly installed instance is gone. > My expectation was that the installer and upgrade > wizard will install a new service instance based on the information > from the existing service definition. Hmm, have you see that in any of MySQL installers ;)? > If I could (mutually) exclusively select between "installing a new > service" and "upgrading from existing service" in the same dialog, I > could have avoided this mis-configuration. I understand, and agree from usability point of view this would be ideal. Currently, "Database Instance" feature can only create new databases, while "Upgrade Wizard" (runs outside of the installer) upgrades existing instances. Their functionality is non-overlapping. This allows for painless (for user) and simple to implement installer rollback in case of errors. So it will stay this way for the version 1 of the new installer. One thing that is possible to do to handle your case, is to deselect "database instance" by default, if upgradable services are detected. I'm not sure this is the best way to handle it , and whether it would confuse the user thus I'll consult our QA for his opinion. Philip, what do you think? Vladislav

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 04 Feb '11

04 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Freitag, 4. Februar 2011 03:17 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Hello Vladislav, Hello Kazuhisa, > First of all, thanks for all the information on "MSI v.s. NSIS" issues. > > > On a bright side, this is a valid bug that was simple to fix forcing per-machine install, so installer does not need to guess. I > > did that and refreshed the MSI on the download page. > > I confirmed the fix with the same machine/user with which I had the > original problem. Thanks. > > However, I'm afraid I hit a new one. If I have MariaDB 5.1.53 > installed (by the 5.1 installer) on the same machine, the new > installer doesn't allow me to install 5.2.5 saying: > "Previous version of MariaDB was found, that used incompatible > installer. Please remove "MariaDB 5.1.53" before you proceed with this > installation." Ok, valid bug. Thanks for reporting. There will *still* be this message if you're trying to install 5.2.5 msi with existing 5.2.(<5) NSIS . For me personally ,main reason NOT to offer 5.2.4=>5.2.5 autoupgrade is the fact that MariaDBs NSIS was not designed to run in silent mode, its popups "Directory not deleted" during upgrade would only alienate users . > I think this is not an intended behavior, because this kills the big > benefit (of your new installer) of automatically upgrading older > MariaDB instance. The upgrade is not always automatic, sometimes it is semiautomatic (supported by the UI tool launched at the end of installation). Did you look in the WL on how upgrades are handled? Here is link just in case http://askmonty.org/worklog/Server-Sprint/?tid=55 . Feedback on it is welcome. I have fixed 5.2.5 over 5.1 NSIS installation ,the update is on usual place. http://cid-ff0c950417b4f8a4.office.live.com/browse.aspx/share Thanks for your efforts, Vladislav

2 3

Re: [Maria-developers] [Commits] Rev 2892: BUG#711565 Index Condition Pushdown can make a thread hold MyISAM locks as well as be unKILLable for long time in file:///home/psergey/dev2/5.3/
by Michael Widenius 03 Feb '11

03 Feb '11

Hi! >>>>> "Sergey" == Sergey Petrunya <psergey(a)askmonty.org> writes: Sergey> At file:///home/psergey/dev2/5.3/ Sergey> ------------------------------------------------------------ Sergey> revno: 2892 Sergey> revision-id: psergey(a)askmonty.org-20110201233314-6j8yc7jrx9425f3b Sergey> parent: sergii(a)pisem.net-20110114205155-i8poq0jti8o8mob1 Sergey> committer: Sergey Petrunya <psergey(a)askmonty.org> Sergey> branch nick: 5.3 Sergey> timestamp: Wed 2011-02-02 02:33:14 +0300 Sergey> message: Sergey> BUG#711565 Index Condition Pushdown can make a thread hold MyISAM locks as well as be unKILLable for long time <cut> Sergey> === modified file 'mysql-test/t/myisam_icp.test' Sergey> --- a/mysql-test/t/myisam_icp.test 2011-01-14 13:38:41 +0000 Sergey> +++ b/mysql-test/t/myisam_icp.test 2011-02-01 23:33:14 +0000 Sergey> @@ -4,3 +4,183 @@ Sergey> --source include/icp_tests.inc <cut> Instead of using # for each row, please use --disable_parsing --enable_parsing This makes it easier to enable the test temporarly when testing. <cut> Sergey> === modified file 'sql/multi_range_read.cc' Sergey> --- a/sql/multi_range_read.cc 2010-12-21 23:26:35 +0000 Sergey> +++ b/sql/multi_range_read.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -902,7 +902,7 @@ Sergey> close_second_handler(); Sergey> /* Safety, not really needed but: */ Sergey> strategy= NULL; Sergey> - DBUG_RETURN(1); Sergey> + DBUG_RETURN(res); Sergey> use_default_impl: Sergey> DBUG_ASSERT(primary_file->inited == handler::INDEX); Sergey> === modified file 'sql/sql_base.cc' Sergey> --- a/sql/sql_base.cc 2010-12-06 08:25:44 +0000 Sergey> +++ b/sql/sql_base.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -8283,11 +8283,9 @@ Sergey> List_iterator_fast<Item> f(fields),v(values); Sergey> Item *value, *fld; Sergey> Item_field *field; Sergey> - TABLE *table= 0; Sergey> - List<TABLE> tbl_list; Sergey> + TABLE *table= 0, *vcol_table= 0; Sergey> bool abort_on_warning_saved= thd->abort_on_warning; Sergey> DBUG_ENTER("fill_record"); Sergey> - tbl_list.empty(); Sergey> /* Sergey> Reset the table->auto_increment_field_not_null as it is valid for Sergey> @@ -8310,7 +8308,7 @@ Sergey> f.rewind(); Sergey> } Sergey> else if (thd->lex->unit.insert_table_with_stored_vcol) Sergey> - tbl_list.push_back(thd->lex->unit.insert_table_with_stored_vcol); Sergey> + vcol_table= thd->lex->unit.insert_table_with_stored_vcol; Sergey> while ((fld= f++)) Sergey> { Sergey> if (!(field= fld->filed_for_view_update())) Sergey> @@ -8340,31 +8338,17 @@ Sergey> my_message(ER_UNKNOWN_ERROR, ER(ER_UNKNOWN_ERROR), MYF(0)); Sergey> goto err; Sergey> } Sergey> - tbl_list.push_back(table); Sergey> + DBUG_ASSERT(vcol_table == 0 || vcol_table == table); Sergey> + vcol_table= table; Sergey> } Sergey> /* Update virtual fields*/ Sergey> thd->abort_on_warning= FALSE; Sergey> - if (tbl_list.head()) Sergey> + if (vcol_table) Sergey> { Sergey> - List_iterator_fast<TABLE> it(tbl_list); Sergey> - TABLE *prev_table= 0; Sergey> - while ((table= it++)) Sergey> + if (vcol_table->vfield) Sergey> { Sergey> - /* Sergey> - Do simple optimization to prevent unnecessary re-generating Sergey> - values for virtual fields Sergey> - */ Sergey> - if (table != prev_table) Sergey> - { Sergey> - prev_table= table; Sergey> - if (table->vfield) Sergey> - { Sergey> - if (update_virtual_fields(thd, table, TRUE)) Sergey> - { Sergey> - goto err; Sergey> - } Sergey> - } Sergey> - } Sergey> + if (update_virtual_fields(thd, vcol_table, TRUE)) Sergey> + goto err; Sergey> } Sergey> } Sergey> thd->abort_on_warning= abort_on_warning_saved; What has the above to do with the IPC patch? In any case, how will the above work when there is more than one table involved in fill_tables ? (By the way, agree that the old code as stupid; There is no reason to add all involved tables to tbl_list (trivial to avoid). Sergey> === modified file 'sql/sql_select.cc' Sergey> --- a/sql/sql_select.cc 2011-01-14 11:03:41 +0000 Sergey> +++ b/sql/sql_select.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -13618,13 +13618,17 @@ Sergey> " cond: %p error: %d", join, join_tab, select_cond, error)); Sergey> if (error > 0 || (join->thd->is_error())) // Fatal error Sergey> DBUG_RETURN(NESTED_LOOP_ERROR); Sergey> + /* Sergey> + Check if we're killed before checking for EOF. This is important because Sergey> + if we're killed while reading a record, the handler will return EOF. Sergey> + */ Please update the comment; We should not send EOF from the handler if the query is killed; There is too many things that can go wrong if we send EOF on killed from the handler. Sergey> + if (join->thd->killed) // Aborted by user Sergey> + { Sergey> + join->thd->send_kill_message(); Sergey> + DBUG_RETURN(NESTED_LOOP_KILLED); /* purecov: inspected */ Sergey> + } <cut> Sergey> === modified file 'storage/maria/ma_rkey.c' Sergey> --- a/storage/maria/ma_rkey.c 2011-01-14 11:05:46 +0000 Sergey> +++ b/storage/maria/ma_rkey.c 2011-02-01 23:33:14 +0000 Sergey> @@ -34,7 +34,7 @@ Sergey> HA_KEYSEG *last_used_keyseg; Sergey> uint32 nextflag; Sergey> MARIA_KEY key; Sergey> - int icp_res= 1; Sergey> + ICP_RESULT icp_res= ICP_MATCH; Sergey> DBUG_ENTER("maria_rkey"); Sergey> DBUG_PRINT("enter", ("base: 0x%lx buf: 0x%lx inx: %d search_flag: %d", Sergey> (long) info, (long) buf, inx, search_flag)); Sergey> @@ -115,7 +115,7 @@ Sergey> not satisfied with an out-of-range condition. Sergey> */ Sergey> if ((*share->row_is_visible)(info) && Sergey> - ((icp_res= ma_check_index_cond(info, inx, buf)) != 0)) Sergey> + ((icp_res= ma_check_index_cond(info, inx, buf)) != ICP_NO_MATCH)) Sergey> break; Sergey> /* The key references a concurrently inserted record. */ Sergey> @@ -145,6 +145,19 @@ Sergey> if (_ma_search_next(info, &lastkey, maria_readnext_vec[search_flag], Sergey> info->s->state.key_root[inx])) Sergey> break; /* purecov: inspected */ Sergey> + Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + buf= 0; /* Fast abort */ Sergey> + icp_res= ICP_ERROR; We should not return an IPC_ERROR here as this test has nothing to do with IPC. It can also happens when we are scanning for a visible row. Better to just threat this as a normal handler error. See later comments... Sergey> + break; Sergey> + } Sergey> + Sergey> /* Sergey> Check that the found key does still match the search. Sergey> _ma_search_next() delivers the next key regardless of its Sergey> @@ -168,9 +181,9 @@ Sergey> if (share->lock_key_trees) Sergey> rw_unlock(&keyinfo->root_lock); Sergey> - if (info->cur_row.lastpos == HA_OFFSET_ERROR || (icp_res != 1)) Sergey> + if (info->cur_row.lastpos == HA_OFFSET_ERROR || (icp_res != ICP_MATCH)) Sergey> { Sergey> - if (icp_res == 2) Sergey> + if (icp_res == ICP_OUT_OF_RANGE || icp_res == ICP_ERROR) Sergey> { Sergey> info->cur_row.lastpos= HA_OFFSET_ERROR; Sergey> my_errno= HA_ERR_KEY_NOT_FOUND; We should not return HA_ERR_KEY_NOT_FOUND in the case of a real error. (Too big risk we loose the reason that something went wrong). Sergey> +/* Sergey> + Yield to possible other writers during a index scan. Sergey> + Check also if we got killed by the user and if yes, return Sergey> + HA_ERR_LOCK_WAIT_TIMEOUT Sergey> + Sergey> + return 0 ok Sergey> + return 1 Query has been requested to be killed Sergey> +*/ Sergey> + Sergey> +my_bool ma_yield_and_check_if_killed(MARIA_HA *info, int inx) Sergey> +{ Sergey> + MARIA_SHARE *share; Sergey> + if (ma_killed(info)) Sergey> + { Sergey> + //info->lastpos= HA_OFFSET_ERROR; Sergey> + my_errno= HA_ERR_KEY_NOT_FOUND; Sergey> + return 1; The above is not safe. Please use: info->cur_row.lastpos= HA_OFFSET_ERROR; my_errno= HA_ERR_ABORTED_BY_USER; In this case we can not return HA_ERR_KEY_NOT_FOUND as it's too big risk that the calling code may think that there was really no key and send a result to the client instead of aborting. By sending a fatal error message to the user, we can be sure that the calling code will not ignore it. That is why we probably should introduce HA_ERR_ABORTED_BY_USER. If we do this, then we have to fix that handler::print_error() ignores this error if thd->killed is set. dispatch_command will generate the necessary error message. Sergey> + } Sergey> + Sergey> + if ((share= info->s)->lock_key_trees) Sergey> + { Sergey> + /* Give writers a chance to access index */ Sergey> + rw_unlock(&share->keyinfo[inx].root_lock); Sergey> + rw_rdlock(&share->keyinfo[inx].root_lock); Sergey> + } Sergey> + return 0; Sergey> +} Sergey> + Sergey> === modified file 'storage/maria/ma_rnext.c' Sergey> --- a/storage/maria/ma_rnext.c 2009-12-15 07:16:46 +0000 Sergey> +++ b/storage/maria/ma_rnext.c 2011-02-01 23:33:14 +0000 <cut> Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Same thing as above, don't use IPC_ERROR, treat this as a real error! Sergey> + Sergey> /* Skip rows inserted by other threads since we got a lock */ Sergey> if ((error= _ma_search_next(info, &info->last_key, Sergey> SEARCH_BIGGER, Sergey> @@ -108,10 +120,10 @@ Sergey> info->update&= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED); Sergey> info->update|= HA_STATE_NEXT_FOUND; Sergey> - if (icp_res == 2) Sergey> + if (icp_res == ICP_OUT_OF_RANGE) Sergey> my_errno=HA_ERR_END_OF_FILE; /* got beyond the end of scanned range */ Sergey> - if (error || icp_res != 1) Sergey> + if (error || icp_res != ICP_MATCH) Sergey> { Sergey> if (my_errno == HA_ERR_KEY_NOT_FOUND) Sergey> my_errno=HA_ERR_END_OF_FILE; Don't return HA_ERR_END_OF_FILE or HA_ERR_KEY_NOT_FOUND for IPC_ERROR Sergey> === modified file 'storage/maria/ma_rnext_same.c' Sergey> --- a/storage/maria/ma_rnext_same.c 2010-10-30 01:59:39 +0000 Sergey> +++ b/storage/maria/ma_rnext_same.c 2011-02-01 23:33:14 +0000 Sergey> @@ -30,7 +30,7 @@ Sergey> int error; Sergey> uint inx,not_used[2]; Sergey> MARIA_KEYDEF *keyinfo; Sergey> - int icp_res= 1; Sergey> + ICP_RESULT icp_res= ICP_MATCH; Sergey> DBUG_ENTER("maria_rnext_same"); Sergey> if ((int) (inx= info->lastinx) < 0 || Sergey> @@ -80,9 +80,20 @@ Sergey> info->cur_row.lastpos= HA_OFFSET_ERROR; Sergey> break; Sergey> } Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Same thing as above, don't use IPC_ERROR, treat this as a real error! <cut> Sergey> - if (error || icp_res != 1) Sergey> + if (error || icp_res != ICP_MATCH) Sergey> { Sergey> if (my_errno == HA_ERR_KEY_NOT_FOUND) Sergey> my_errno=HA_ERR_END_OF_FILE; Don't return HA_ERR_END_OF_FILE or HA_ERR_KEY_NOT_FOUND for IPC_ERROR Sergey> === modified file 'storage/maria/ma_rprev.c' <cut> Sergey> + if (ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Return a real error! Sergey> + } Sergey> + Sergey> /* Skip rows that are inserted by other threads since we got a lock */ Sergey> if ((error= _ma_search_next(info, &info->last_key, Sergey> SEARCH_SMALLER, Sergey> @@ -66,6 +84,10 @@ Sergey> } Sergey> if (share->lock_key_trees) Sergey> rw_unlock(&keyinfo->root_lock); Sergey> + if (icp_res == ICP_ERROR || icp_res == ICP_OUT_OF_RANGE) Sergey> + { Sergey> + DBUG_RETURN(my_errno= HA_ERR_END_OF_FILE); Don't return HA_ERR_END_OF_FILE in case of IPC_ERROR Sergey> === modified file 'storage/maria/maria_def.h' Sergey> --- a/storage/maria/maria_def.h 2011-01-14 11:03:41 +0000 Sergey> +++ b/storage/maria/maria_def.h 2011-02-01 23:33:14 +0000 Sergey> @@ -505,6 +505,7 @@ Sergey> DYNAMIC_ARRAY pinned_pages; Sergey> /* accumulate indexfile changes between write's */ Sergey> TREE *bulk_insert; Sergey> + void *__external_ref; /* For MariaDB TABLE */ /* psergey-todo: need this?*/ No, we don't need this as we have external_ptr already in Aria that we can use. Sergey> --- a/storage/myisam/mi_rkey.c 2009-12-22 12:33:21 +0000 Sergey> @@ -168,26 +176,37 @@ Sergey> if (share->concurrent_insert) Sergey> rw_unlock(&share->key_root_lock[inx]); Sergey> - /* Calculate length of the found key; Used by mi_rnext_same */ Sergey> - if ((keyinfo->flag & HA_VAR_LENGTH_KEY) && last_used_keyseg && Sergey> - info->lastpos != HA_OFFSET_ERROR) Sergey> - info->last_rkey_length= _mi_keylength_part(keyinfo, info->lastkey, Sergey> - last_used_keyseg); Sergey> + info->last_rkey_length= pack_key_length; Sergey> + Sergey> + if (info->lastpos == HA_OFFSET_ERROR) /* No such record */ Sergey> + { Sergey> + /* Sergey> + psergey: what doe the next line do:vv initially there were no writeinfo Sergey> + calls! Sergey> + */ Sergey> + fast_mi_writeinfo(info); The original code had a bug. At start of mi_rkey.c we do fast_mi_readinfo(), which must be followed by a fast_mi_writeinfo() before return, which didn't happen if buf was 0. In the normal case 'read_record()' will do the fast_mi_writeinfo() call for the other cases. Sergey> + if (!buf) Sergey> + DBUG_RETURN(my_errno); Sergey> + } Sergey> else Sergey> + /* Calculate length of the found key; Used by mi_rnext_same */ Sergey> + if ((keyinfo->flag & HA_VAR_LENGTH_KEY) && last_used_keyseg) Sergey> + info->last_rkey_length= _mi_keylength_part(keyinfo, info->lastkey, Sergey> + last_used_keyseg); Sergey> + Sergey> + /* Check if we don't want to have record back, only error message */ Sergey> + if (!buf) Sergey> + DBUG_RETURN(0); We have to add fast_mi_writeinfo(info) before the DBUG_RETURN above. Sergey> + if (!(*info->read_record)(info,info->lastpos,buf)) Sergey> + { Sergey> + info->update|= HA_STATE_AKTIV; /* Record is read */ Sergey> + DBUG_RETURN(0); Sergey> + } <cut> Sergey> +my_bool mi_yield_and_check_if_killed(MI_INFO *info, int inx) Sergey> +{ Sergey> + MYISAM_SHARE *share; Sergey> + if (mi_killed(info)) Sergey> + { Sergey> + info->lastpos= HA_OFFSET_ERROR; Sergey> + my_errno= HA_ERR_KEY_NOT_FOUND; HA_ERR_KEY_NOT_FOUND -> HA_ERR_ABORTED_BY_USER Sergey> + return 1; Sergey> + } Sergey> + Sergey> + if ((share= info->s)->concurrent_insert) Sergey> + { Sergey> + /* Give writers a chance to access index */ Sergey> + rw_unlock(&share->key_root_lock[inx]); Sergey> + rw_rdlock(&share->key_root_lock[inx]); Sergey> + } Sergey> + return 0; Sergey> +} Sergey> === modified file 'storage/myisam/mi_rnext.c' Sergey> --- a/storage/myisam/mi_rnext.c 2010-06-26 10:05:41 +0000 Sergey> +++ b/storage/myisam/mi_rnext.c 2011-02-01 23:33:14 +0000 Sergey> @@ -104,6 +104,18 @@ Sergey> (info->index_cond_func && Sergey> (res= mi_check_index_cond(info, inx, buf)) == ICP_NO_MATCH)) Sergey> { Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + res= ICP_ERROR; Return a real error! Sergey> + break; Sergey> + } Sergey> + Sergey> /* Sergey> Skip rows that are either inserted by other threads since Sergey> we got a lock or do not match pushed index conditions Sergey> @@ -115,7 +127,8 @@ Sergey> info->s->state.key_root[inx]))) Sergey> break; Sergey> } Sergey> - if (!error && res == ICP_OUT_OF_RANGE) Sergey> + Sergey> + if (!error && (res == ICP_OUT_OF_RANGE || res == ICP_ERROR)) Sergey> { Sergey> if (info->s->concurrent_insert) Sergey> rw_unlock(&info->s->key_root_lock[inx]); Return error code in case of ICP_ERROR Sergey> === modified file 'storage/myisam/mi_rnext_same.c' Sergey> --- a/storage/myisam/mi_rnext_same.c 2009-12-22 12:33:21 +0000 Sergey> +++ b/storage/myisam/mi_rnext_same.c 2011-02-01 23:33:14 +0000 Sergey> @@ -63,6 +63,18 @@ Sergey> } Sergey> for (;;) Sergey> { Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error=1; Sergey> + buf= 0; /* Fast abort */ Sergey> + break; Return real error Sergey> + } Sergey> + Sergey> if ((error=_mi_search_next(info,keyinfo,info->lastkey, Sergey> info->lastkey_length,SEARCH_BIGGER, Sergey> info->s->state.key_root[inx]))) Sergey> @@ -87,6 +99,10 @@ Sergey> } Sergey> if (info->s->concurrent_insert) Sergey> rw_unlock(&info->s->key_root_lock[inx]); Sergey> + Sergey> + if (!buf) Sergey> + DBUG_RETURN(my_errno); Sergey> + Remove above; We have to update info->update even if buf is 0 Sergey> /* Don't clear if database-changed */ Sergey> info->update&= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED); Sergey> info->update|= HA_STATE_NEXT_FOUND | HA_STATE_RNEXT_SAME; Sergey> === modified file 'storage/myisam/mi_rprev.c' Sergey> --- a/storage/myisam/mi_rprev.c 2009-12-15 07:16:46 +0000 Sergey> +++ b/storage/myisam/mi_rprev.c 2011-02-01 23:33:14 +0000 Sergey> @@ -53,12 +53,28 @@ Sergey> if (!error) Sergey> { Sergey> - int res= 0; Sergey> + ICP_RESULT res= 0; Sergey> + my_off_t cur_keypage= info->last_keypage; Sergey> while ((share->concurrent_insert && Sergey> info->lastpos >= info->state->data_file_length) || Sergey> (info->index_cond_func && Sergey> !(res= mi_check_index_cond(info, inx, buf)))) Sergey> { Sergey> + /* Sergey> + If we are at the last (i.e. first?) key on the key page, Sergey> + allow writers to access the index. Sergey> + */ Sergey> + if (info->last_keypage != cur_keypage) Sergey> + { Sergey> + cur_keypage= info->last_keypage; Sergey> + if (mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + res= ICP_ERROR; Sergey> + break; Return error Sergey> + } Sergey> + } Sergey> + Sergey> /* Sergey> Skip rows that are either inserted by other threads since Sergey> we got a lock or do not match pushed index conditions Sergey> @@ -69,7 +85,7 @@ Sergey> share->state.key_root[inx]))) Sergey> break; Sergey> } Sergey> - if (!error && res == 2) Sergey> + if (!error && (res == ICP_OUT_OF_RANGE || res == ICP_ERROR)) Sergey> { Sergey> if (share->concurrent_insert) Sergey> rw_unlock(&share->key_root_lock[inx]); Return error code in case of ICP_ERROR <cut> Sergey> --- a/storage/xtradb/handler/ha_innodb.cc 2011-01-12 12:00:10 +0000 Sergey> +++ b/storage/xtradb/handler/ha_innodb.cc 2011-02-01 23:33:14 +0000 <cut> Sergey> @@ -12094,15 +12102,18 @@ Sergey> See note on ICP_RESULT for return values description. Sergey> */ Sergey> -static int index_cond_func_innodb(void *arg) Sergey> +static xtradb_icp_result_t index_cond_func_innodb(void *arg) Sergey> { Sergey> ha_innobase *h= (ha_innobase*)arg; Sergey> + if (h->is_thd_killed()) Sergey> + return XTRADB_ICP_ERROR; Sergey> + rename to XTRADB_ABORTED_BY_USER; Sergey> if (h->end_range) Sergey> { Sergey> if (h->compare_key2(h->end_range) > 0) Sergey> - return ICP_OUT_OF_RANGE; /* caller should return HA_ERR_END_OF_FILE already */ Sergey> + return XTRADB_ICP_OUT_OF_RANGE; /* caller should return HA_ERR_END_OF_FILE already */ Sergey> } Sergey> - return h->pushed_idx_cond->val_int()? ICP_MATCH : ICP_NO_MATCH; Sergey> + return h->pushed_idx_cond->val_int()? XTRADB_ICP_MATCH : XTRADB_ICP_NO_MATCH; Sergey> } Sergey> C_MODE_END <cut> Sergey> --- a/storage/xtradb/include/row0mysql.h 2010-12-06 08:25:44 +0000 Sergey> +++ b/storage/xtradb/include/row0mysql.h 2011-02-01 23:33:14 +0000 Sergey> @@ -577,7 +577,15 @@ Sergey> #define ROW_PREBUILT_ALLOCATED 78540783 Sergey> #define ROW_PREBUILT_FREED 26423527 Sergey> -typedef int (*index_cond_func_t)(void *param); Sergey> + Sergey> +typedef enum xtradb_icp_result { Sergey> + XTRADB_ICP_ERROR=-1, rename to XTRADB_ABORTED_BY_USER Sergey> + XTRADB_ICP_NO_MATCH=0, Sergey> + XTRADB_ICP_MATCH=1, Sergey> + XTRADB_ICP_OUT_OF_RANGE=2 Sergey> +} xtradb_icp_result_t; Sergey> + Sergey> +typedef xtradb_icp_result_t (*index_cond_func_t)(void *param); Sergey> /** A struct for (sometimes lazily) prebuilt structures in an Innobase table Sergey> handle used within MySQL; these are used to save CPU time. */ Sergey> === modified file 'storage/xtradb/row/row0sel.c' Sergey> --- a/storage/xtradb/row/row0sel.c 2010-12-06 17:44:17 +0000 Sergey> +++ b/storage/xtradb/row/row0sel.c 2011-02-01 23:33:14 +0000 Sergey> @@ -4396,12 +4396,13 @@ Sergey> */ Sergey> ut_ad(ib_res); Sergey> res= prebuilt->idx_cond_func(prebuilt->idx_cond_func_arg); Sergey> - if (res == 0) Sergey> + if (res == XTRADB_ICP_NO_MATCH) Sergey> goto next_rec; Sergey> - if (res == 2) { Sergey> + if (res == XTRADB_ICP_OUT_OF_RANGE || res == XTRADB_ICP_ERROR) { Sergey> err = DB_RECORD_NOT_FOUND; Sergey> goto idx_cond_failed; Don't return DB_RECORD_NOT_FOUND in case of XTRADB_ICP_ERROR. Return an actual error like DB_ABORTED_BY_USER which can be translated to HA_ERR_ABORTED_BY_USER. Regards, Monty

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 03 Feb '11

03 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Donnerstag, 3. Februar 2011 02:00 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > Hello Kazuhisa, > > But to reiterate, this package is not really thought to be run by non-admin users. It installs into program files, and it writes to > > the HKLM registry hive, and it creates a service. All this stuff is not possible without being an administrator. I'm quite surprised > > Well, if that is how it is designed, that's fine with me. I just > thought that there might be many users who are new to MariaDB, not > sure about its quality and want to install/run it in a sandbox > environment. If switching to HKCU registry hive and disabling the > option to run it as a service when the installing user is non-admin > can satisfy the needs, implementing them might not be a bad idea. But > again, I do not have a strong opinion on that. (The user can choose to > install from the zipped package as long as zip version is also > distributed). I this the per-user install is a feature in NSIS which you discovered, but which is was not planned from the start on. From the start on, it was planned as an installation where you get a running service at the end, and that was not implemented in NSIS. if it was, it would not work in this sandboxed environment. With MSI I checked, and the rules for per-user install appear to be relatively complicated. Officially dual package authoring (single package that can be installed either per-user or per-machine) is supported with MSI 5.0 and Win7, so there is no hope to get that working in the same for unprivileged XP user. I might do some experimentation with it on Win7, however the benefit it provide compared to ZIP is low: is a menu shortcut with PATH including bin directory of the installation and Add/Remove Program entry. On a bright side, this is a valid bug that was simple to fix forcing per-machine install, so installer does not need to guess. I did that and refreshed the MSI on the download page. > One more thing that I noticed is, when installing 5.1 all the messages > in the dialogs are in Japanese (with the exception of GPL license > statement), while they are all in English for 5.2. Maybe this is a > bonus that came from NSIS. Maybe. NSIS guys deserve a compliment for that effort. With MSI dealing with multiple languages inside single MSI is not that simple. But, localization is definitely possible in Wix or generally MSI, I so far ignored that topic altogether;) But back to NSIS package .I believe, just like with per-user installation, you discovered hidden and untested feature, an Easter egg. It has more localization bugs that just GPL text. The feature tree is not localized, neither uninstall popup box. If that NSIS package had more functionality than plain unpack, and had own dialogs, there would be more localization bugs.

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 03 Feb '11

03 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Mittwoch, 2. Februar 2011 23:01 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Vladislav, > > I tried the new 5.2.5 installer in Japanese Windows XP (32bit) > environment using a user account which does NOT have administrator > privilege. I wonder why it has started at all. It definitely needs administrator and installer on later Windows brings all the necessary UAC popups. > During the installation I encountered a lot of warning > dialogs (see below for an example) about not being able to write into > the registry, although I did not have any noticeable problem startnig > mysqld.exe after the installation is finished (No, the installation > was not aborted and it said "Completed the MariaDB 5.2 Setup WIzard.") > > (Example warning message) > "Could not write value InstallLocation to key > \Software\Microsoft\Windows\CurrenVersion\Uninstall\MariaDB 5.2. > Verify that you have sufficient access to that key, or contact your > support personnel." If you do not have that key, you do not have an entry in "Add/remove programs" > I think what this error message means is that the user account used > for the installation needs to have an administrator privilege, but the > current (older) installer for MariaDB 5.1.53 did not have this > problem. This installer should be run as administrator, there is no per-user option. > The other (related) issue is that I cannot uninstall mariadb 5.2 now > because it is not listed in the installed program list in the control > panel (most likely because the necessary information is not written > into the registry). You will need to run msiexec /x mariadb-5.2.5-win32.msi from the command line. This uninstalls installed package (I'm not really sure it was installed, but it does not hurt to try, it nothing is installed it will tell) > Again, the installer for 5.1 did not have this problem. Well, it is not installer for 5.1, the old one had more or less the same functionality as downloading and unpacking a zip file. It might even be that NSIS guys automatically switched to HKCU registry keys or so for a weak user. > Also, unlike the current installer for 5.1, the new installer > does not seem to install a standalone uninstaller (Uninstall.exe) on > the disk. So, I'm stuck and wondering how to uninstall 5.2 (Well, not > really, because if nothing is written into the registry, deleting all > the files and directories should 'cleanly' uninstall everything, > right?.). I would stick to documented stuff. msiexec /x mariadb-5.2.5-win32.msi But to reiterate, this package is not really thought to be run by non-admin users. It installs into program files, and it writes to the HKLM registry hive, and it creates a service. All this stuff is not possible without being an administrator. I'm quite surprised that MSI let you go that far actually, and that it would not rollback and cleanup after the error. On the other hand, thanks for an interesting error report, makes sense for me to have a test for anomalies like that. Can you tell service pack of your XP and MSI version (launch winver and msiexec /?) Thanks, Vladislav > - Kazuhisa >

1 0

[Maria-developers] Benchmarking mrr_sort_keys=on|off.
by Sergey Petrunya 02 Feb '11

02 Feb '11

Hi! Following the discussion of benchmarks of mrr_sort_keys=on|off functionality, I've done a re-run. Please find the results below. ## Dataset: DBT-3, scale=10, xtradb, ## innodb_file_per_table=1 ## innodb_file_format=barracuda ## innodb_log_file_size=256M ## innodb_buffer_pool_size=4000M ## ## Platform: work.askmonty.org host ## Cold-cache is achieved by: ## - stopping the server ## - echo 1 > /proc/sys/vm/drop_caches ## - starting the server ## - running EXPLAIN {query} ## ## MariaDB: 5.3-current, release build (compile-pentium-max) ## Result summary: * mrr_sort_keys=on provides speedup (2.25 min vs 13.5 min) on cold cache. * on hot cache, there is no big difference (~1.3 sec in both cases). * I was able to repeat Igor's August results with the current 5.3. It is not clear why Igor can't repeat them anymore. (An evasive bug? or just mistake in experiments?) set join_buffer_size=1024*1024*32; set join_buffer_space_limit=1024*1024*32; set join_cache_level=6; ## ## Cold cache, mrr_sort_keys=on: ## [22:12] root@dbt3sf10>explain select avg(l_extendedprice) from orders, lineitem where l_orderkey=o_orderkey and o_orderdate between date '1992-07-01' and date '1992-07-31'; +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+------------------------------------+ | 1 | SIMPLE | orders | range | PRIMARY,i_o_orderdate | i_o_orderdate | 4 | NULL | 142680 | Using where; Using index | | 1 | SIMPLE | lineitem | ref | PRIMARY,i_l_orderkey,i_l_orderkey_quantity | PRIMARY | 4 | dbt3sf10.orders.o_orderkey | 1 | Using join buffer (flat, BKA join) | +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+------------------------------------+ 2 rows in set (3.04 sec) [22:06] root@dbt3sf10>select avg(l_extendedprice) from orders, lineitem where l_orderkey=o_orderkey and o_orderdate between date '1992-07-01' and date '1992-07-31'; +----------------------+ | avg(l_extendedprice) | +----------------------+ | 38229.0683265894 | +----------------------+ 1 row in set (2 min 13.40 sec) [22:09] root@dbt3sf10> show status like "%handler%"; +----------------------------+--------+ | Variable_name | Value | +----------------------------+--------+ | Handler_commit | 2 | | Handler_read_key | 193396 | | Handler_read_next | 966535 | +----------------------------+--------+ 15 rows in set (0.29 sec) # # mrr_sort_keys=off, cold cache: # [22:20] root@dbt3sf10>explain select avg(l_extendedprice) from orders, lineitem where l_orderkey=o_orderkey and o_orderdate between date '1992-07-01' and date '1992-07-31'; +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+--------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+--------------------------+ | 1 | SIMPLE | orders | range | PRIMARY,i_o_orderdate | i_o_orderdate | 4 | NULL | 142680 | Using where; Using index | | 1 | SIMPLE | lineitem | ref | PRIMARY,i_l_orderkey,i_l_orderkey_quantity | PRIMARY | 4 | dbt3sf10.orders.o_orderkey | 2 | | +----+-------------+----------+-------+--------------------------------------------+---------------+---------+----------------------------+--------+--------------------------+ [22:20] root@dbt3sf10>select avg(l_extendedprice) from orders, lineitem where l_orderkey=o_orderkey and o_orderdate between date '1992-07-01' and date '1992-07-31'; +----------------------+ | avg(l_extendedprice) | +----------------------+ | 38229.0683265898 | +----------------------+ 1 row in set (13 min 23.68 sec) [22:09] root@dbt3sf10> show status like "%handler%"; +----------------------------+--------+ | Variable_name | Value | +----------------------------+--------+ | Handler_commit | 2 | | Handler_read_key | 193396 | | Handler_read_next | 966535 | +----------------------------+--------+ ## ## Hot caches (re-run the query until time stabilizes): ## mrr_sort_keys=off: 1.34 sec mrr_sort_keys=on: 1.38 sec BR Sergey -- Sergey Petrunia, Software Developer Monty Program AB, http://askmonty.org Blog: http://s.petrunia.net/blog

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 02 Feb '11

02 Feb '11

From: Peter Laursen [mailto:peter_laursen@webyog.com] Sent: Mittwoch, 2. Februar 2011 17:14 To: Vladislav Vaintroub Cc: maria-developers(a)lists.launchpad.net Subject: Re: [Maria-developers] Windows installer MWL#55 finished. Hi Peter, >Your default install path is \program files [(x86)]. You copy the /datadir as a subfolder of /basedirr. >This will cause problems on Vista+ with UAC enabled if the server is ever started from commandline as a non-service (also if >the user doing so is an administrator user). For the user who installed MySQL, it won't cause problems. After you stop the service you can start mysqld --defaults-file=path\to\datadir\my.ini --console from the non-elevated command line, with/without UAC. No file system virtualization will happen. That bug http://bugs.mysql.com/bug.php?id=38373 (user can not from mysqld from command line after installation) was filed by myself 2.5 years ago , I could not leave that unfixed now. I know, writing to ProgramFiles might not be kosher, but it works if ACLs on our tree are set. (And Microsoft should not tell people that writing to program files is bad, they are doing it with SQLServer, and also document it http://msdn.microsoft.com/en-us/library/ms143547.aspx , so we are again as good/bad as SQLServer here) >2) There is no my.ini generated with a fresh install as far as I can see - so the server probably runs (it does!) with very low default buffer settings etc. Also latin1 charset and MyISAM will be default. I think a my.ini with settings fit for a 1GB-2GB RAM machine should be generated - and with a few configuration options. And a my.ini will be required to solve 1) anyway. There is a my.ini, it is in database directory. It is spartanic, it has port and datadir, nothing else. You can edit it using non-elevated notepad (with or without UAC) and there is a shortcut for it created in start menu, MariaDB 5.2 folder (rsp MariaDB 5.2 (x64) if for 64 bit package). There are zero performance related options, which means in fact it won't run on full speed. I have no strong opinion on it, nor I feel qualified for configuring "optimal performance out of the box" . If there is a strong demand on specific parameters, they should probably go there. I would like to hear more opinions on this, one thing that will definitely not be in the first incarnation of this installer is fully-featured ConfigWizard like stuff. Where I see differences between this my.ini and MySQL installer's default is * sql_mode (MySQL's default is "strict transactional tables") * default storage engine (MySQL default is innodb) * innodb buffer pool size (IIRC, MySQL's default is something like 12% of RAM) Those are trivial to change modifying my.ini Stuff like innodb log file size (calculated from innodb buffer pool size) not trivial to change with simple my.ini modification, there must be a clean shutdown and exiting files need to be removed. > But still it is a good installer (and there is a 64 bit one as well), so this is an important step in the right direction. So thanks for your effort! Thanks! -- Peter On Wed, Feb 2, 2011 at 14:37, Peter Laursen <peter_laursen(a)webyog.com> wrote: I am a little bit busy now, but will confirm tonight. -- Peter On Wed, Feb 2, 2011 at 14:12, Vladislav Vaintroub <wlad(a)montyprogram.com> wrote: From: Peter Laursen [mailto:peter_laursen@webyog.com] Sent: Dienstag, 1. Februar 2011 20:31 To: Vladislav Vaintroub Cc: maria-developers(a)lists.launchpad.net Subject: Re: [Maria-developers] Windows instller MWL#55 finished. >It is a Danish Windows (Home Premium 64 bit). Hi Peter, As it turns out, installer had the same "localization" problem registering service on German Windows. Fixed now, I uploaded the msis to the same place http://cid-ff0c950417b4f8a4.office.live.com/browse.aspx/share Thanks, Wlad

2 1