February 2011 - developers - lists.mariadb.org

[Maria-developers] WL#176 New (by Dreas): protocl typo in replication error
by worklog-noreply＠askmonty.org 17 Feb '11

17 Feb '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: protocl typo in replication error CREATION DATE..: Thu, 17 Feb 2011, 16:58 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Maria-RawIdeaBin TASK ID........: 176 (http://askmonty.org/worklog/?tid=176) VERSION........: WorkLog-4.0 STATUS.........: Un-Assigned PRIORITY.......: 10 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: Using 5.1.50-mariadb84: ===== Last_IO_Error: error connecting to master 'repl2@mysql1.*****.com:3306' - retry- time: 60 retries: 86400 message: SSL error: protocl version mismatch ==== "protocl" seems to be a typo. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

[Maria-developers] work in progress for the Facebook patch
by MARK CALLAGHAN 16 Feb '11

16 Feb '11

I am writing this to describe work in progress for the Facebook patch. Comments are welcome. In the future we hope to get parts of the Facebook patch into public forks. We have two big projects in progress: * admission control * InnoDB compression for OLTP --- Admission control enforces max concurrent queries per account. MySQL already limits max concurrent connections per account, but we want to set the connection limit to a much larger value than the query limit. The limit will be enforced at statement start and will allow queries to timeout in the queue while waiting to run. By "descheduled" I mean that a connection decrements the count of concurrent queries for a particular account. Connections will be descheduled when: * a statement ends * a connection is about to send results back to a client -- only when the socket write is to be done, not when data is buffered * srv_suspend_mysql_thread is called on a row lock wait * possibly when a disk read is done within innodb * when enter_cond is called By "scheduled" I mean that a connection increments the count of concurrent queries for an account. If done with "noforce" then the caller will wait until this doesn't exceed concurrency limits for the account. If done with "force" then this doesn't wait and can exceed the concurrency limits. Schedule with noforce is done: * at statement start * after a socket write Schedule with force is done: * when exit_cond is called * when a thread wakes after calling srv_suspend_mysql_thread This change will allow us to run with higher values for max concurrent connections and we assume that will significantly increase mutex contention on kernel_mutex so we have begun to evaluate: * using the pthread mutex implementation that does not spin in more cases * limit the max number of threads that can do concurrent busy-wait loops for InnoDB's mutex and rw-lock * removing use of the sync array within InnoDB * changing Innodb to wake one rather than all waiters when a mutex is unlocked * somehow making read_view_open_now faster --- InnoDB compression for OLTP tries to make compression faster for OLTP workloads. We have work in progress to change InnoDB to not log page images for many but not all insert/update/delete operations. When InnoDB compression is used, InnoDB can log page images. That was not done before and for my production workloads this doubles the rate at which IO is done to the transaction log and increases the dirty page write rate by at least 20% because checkpoint IO must be done sooner. We previously added an option that allows the LZ compression level to be set globally. I think the default is level 6. We will evaluate: * using QuickLZ which provides faster compression and decompression than zlib * adding compress_ops and compress_ops_ok per table to information_schema.table_statistics * adding an option to limit compression to the clustered index and not compress secondary indexes -- Mark Callaghan mdcallag(a)gmail.com

1 0

[Maria-developers] MariaDB and MySQL 5.5
by MARK CALLAGHAN 11 Feb '11

11 Feb '11

When will there be a MariaDB branch based on MySQL 5.5? I want to use that as the basis for the 5.5 version of the Facebook patch. -- Mark Callaghan mdcallag(a)gmail.com

2 1

[Maria-developers] WL#175 New (by Mdcallag): Unique ID in binlog event metadata
by worklog-noreply＠askmonty.org 11 Feb '11

11 Feb '11

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: Unique ID in binlog event metadata CREATION DATE..: Fri, 11 Feb 2011, 05:03 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Server-RawIdeaBin TASK ID........: 175 (http://askmonty.org/worklog/?tid=175) VERSION........: Server-9.x STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: DESCRIPTION: While I would love to have global transaction IDs for MySQL replication (http://askmonty.org/worklog/Server-RawIdeaBin/?tid=31) that is a huge project. I think we can automate slave failover if binlog events have a metadata attribute that is a unique ID and that metadata is sticky like server ID when a slave is run with --log-slave-updates. Assuming this metadata is added, when it is time to do slave failover, first determine the serverID + uniqueID for the last binlog event group executed on the slave, then find that filename/offset for that event on the new master, then CHANGE MASTER on the slave to start replication on the new master just past that offset. I mention that serverID can be used in addition to the uniqueID, so maybe uniqueID only needs to be unique within the binlog events generated for one serverID. The search on the new master is more efficient when the uniqueID are generated in increasing order. But I am not sure whether that is required. We might need a new server command that maps (serverID, uniqueID) to (filename, offset), but I am not sure that is required. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 08 Feb '11

08 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa > Ichikawa > Sent: Dienstag, 8. Februar 2011 04:46 > To: Vladislav Vaintroub > Cc: Philip Stoev; maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Hello, Vladislav. > All previous 5.2.5 installer created its data directory even when > upgrading existing instance. Therefore, if datadir was not set > explicity in the inherited my.ini, it was using the new data directory > (in other words, it was not reusing the previous database files, I'm > afraid). Yes. Somewhat unusual case my.ini without datadir was not handled correctly though (case without my.ini at all was handled). Fixed that, MSIs are in the usual location. Thanks for input! > - Kazuhisa

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 05 Feb '11

05 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa > Ichikawa > Sent: Samstag, 5. Februar 2011 04:34 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. Hello Kazuhisa, > I BROWSED through it but did not read it carefully line by line before > reporting bugs. Sorry about that. Yet, all of your reports were valid . > > By the way, after finished installing 5.2.5 and upgrading the previous > (5.1.53) service instance, I have 2 service definitions for 5.2.5, one > that was newly created and the other that was successfully upgraded > from 5.1.53. > > I think this is a human (that is my) error rather than a bug, because > I checked "Install as service" in the "Defaults settings" dialog > before the installer starts installing 5.2.5 files and then chose to > upgrade the existing service in the Upgrade Wizard that was started > after installation. If you think installing instance was a mistake on you part (you only wanted to upgrade what you have), you can correct that easily. In maintenance mode install (Add/Remove Programs=>MariaDB5.2=>Change/Uninstall), select "Change" button. Deselect "Database Instance". Choose "Remove data" afterwards. Now the mistakenly installed instance is gone. > My expectation was that the installer and upgrade > wizard will install a new service instance based on the information > from the existing service definition. Hmm, have you see that in any of MySQL installers ;)? > If I could (mutually) exclusively select between "installing a new > service" and "upgrading from existing service" in the same dialog, I > could have avoided this mis-configuration. I understand, and agree from usability point of view this would be ideal. Currently, "Database Instance" feature can only create new databases, while "Upgrade Wizard" (runs outside of the installer) upgrades existing instances. Their functionality is non-overlapping. This allows for painless (for user) and simple to implement installer rollback in case of errors. So it will stay this way for the version 1 of the new installer. One thing that is possible to do to handle your case, is to deselect "database instance" by default, if upgradable services are detected. I'm not sure this is the best way to handle it , and whether it would confuse the user thus I'll consult our QA for his opinion. Philip, what do you think? Vladislav

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 04 Feb '11

04 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Freitag, 4. Februar 2011 03:17 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Hello Vladislav, Hello Kazuhisa, > First of all, thanks for all the information on "MSI v.s. NSIS" issues. > > > On a bright side, this is a valid bug that was simple to fix forcing per-machine install, so installer does not need to guess. I > > did that and refreshed the MSI on the download page. > > I confirmed the fix with the same machine/user with which I had the > original problem. Thanks. > > However, I'm afraid I hit a new one. If I have MariaDB 5.1.53 > installed (by the 5.1 installer) on the same machine, the new > installer doesn't allow me to install 5.2.5 saying: > "Previous version of MariaDB was found, that used incompatible > installer. Please remove "MariaDB 5.1.53" before you proceed with this > installation." Ok, valid bug. Thanks for reporting. There will *still* be this message if you're trying to install 5.2.5 msi with existing 5.2.(<5) NSIS . For me personally ,main reason NOT to offer 5.2.4=>5.2.5 autoupgrade is the fact that MariaDBs NSIS was not designed to run in silent mode, its popups "Directory not deleted" during upgrade would only alienate users . > I think this is not an intended behavior, because this kills the big > benefit (of your new installer) of automatically upgrading older > MariaDB instance. The upgrade is not always automatic, sometimes it is semiautomatic (supported by the UI tool launched at the end of installation). Did you look in the WL on how upgrades are handled? Here is link just in case http://askmonty.org/worklog/Server-Sprint/?tid=55 . Feedback on it is welcome. I have fixed 5.2.5 over 5.1 NSIS installation ,the update is on usual place. http://cid-ff0c950417b4f8a4.office.live.com/browse.aspx/share Thanks for your efforts, Vladislav

2 3

Re: [Maria-developers] [Commits] Rev 2892: BUG#711565 Index Condition Pushdown can make a thread hold MyISAM locks as well as be unKILLable for long time in file:///home/psergey/dev2/5.3/
by Michael Widenius 03 Feb '11

03 Feb '11

Hi! >>>>> "Sergey" == Sergey Petrunya <psergey(a)askmonty.org> writes: Sergey> At file:///home/psergey/dev2/5.3/ Sergey> ------------------------------------------------------------ Sergey> revno: 2892 Sergey> revision-id: psergey(a)askmonty.org-20110201233314-6j8yc7jrx9425f3b Sergey> parent: sergii(a)pisem.net-20110114205155-i8poq0jti8o8mob1 Sergey> committer: Sergey Petrunya <psergey(a)askmonty.org> Sergey> branch nick: 5.3 Sergey> timestamp: Wed 2011-02-02 02:33:14 +0300 Sergey> message: Sergey> BUG#711565 Index Condition Pushdown can make a thread hold MyISAM locks as well as be unKILLable for long time <cut> Sergey> === modified file 'mysql-test/t/myisam_icp.test' Sergey> --- a/mysql-test/t/myisam_icp.test 2011-01-14 13:38:41 +0000 Sergey> +++ b/mysql-test/t/myisam_icp.test 2011-02-01 23:33:14 +0000 Sergey> @@ -4,3 +4,183 @@ Sergey> --source include/icp_tests.inc <cut> Instead of using # for each row, please use --disable_parsing --enable_parsing This makes it easier to enable the test temporarly when testing. <cut> Sergey> === modified file 'sql/multi_range_read.cc' Sergey> --- a/sql/multi_range_read.cc 2010-12-21 23:26:35 +0000 Sergey> +++ b/sql/multi_range_read.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -902,7 +902,7 @@ Sergey> close_second_handler(); Sergey> /* Safety, not really needed but: */ Sergey> strategy= NULL; Sergey> - DBUG_RETURN(1); Sergey> + DBUG_RETURN(res); Sergey> use_default_impl: Sergey> DBUG_ASSERT(primary_file->inited == handler::INDEX); Sergey> === modified file 'sql/sql_base.cc' Sergey> --- a/sql/sql_base.cc 2010-12-06 08:25:44 +0000 Sergey> +++ b/sql/sql_base.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -8283,11 +8283,9 @@ Sergey> List_iterator_fast<Item> f(fields),v(values); Sergey> Item *value, *fld; Sergey> Item_field *field; Sergey> - TABLE *table= 0; Sergey> - List<TABLE> tbl_list; Sergey> + TABLE *table= 0, *vcol_table= 0; Sergey> bool abort_on_warning_saved= thd->abort_on_warning; Sergey> DBUG_ENTER("fill_record"); Sergey> - tbl_list.empty(); Sergey> /* Sergey> Reset the table->auto_increment_field_not_null as it is valid for Sergey> @@ -8310,7 +8308,7 @@ Sergey> f.rewind(); Sergey> } Sergey> else if (thd->lex->unit.insert_table_with_stored_vcol) Sergey> - tbl_list.push_back(thd->lex->unit.insert_table_with_stored_vcol); Sergey> + vcol_table= thd->lex->unit.insert_table_with_stored_vcol; Sergey> while ((fld= f++)) Sergey> { Sergey> if (!(field= fld->filed_for_view_update())) Sergey> @@ -8340,31 +8338,17 @@ Sergey> my_message(ER_UNKNOWN_ERROR, ER(ER_UNKNOWN_ERROR), MYF(0)); Sergey> goto err; Sergey> } Sergey> - tbl_list.push_back(table); Sergey> + DBUG_ASSERT(vcol_table == 0 || vcol_table == table); Sergey> + vcol_table= table; Sergey> } Sergey> /* Update virtual fields*/ Sergey> thd->abort_on_warning= FALSE; Sergey> - if (tbl_list.head()) Sergey> + if (vcol_table) Sergey> { Sergey> - List_iterator_fast<TABLE> it(tbl_list); Sergey> - TABLE *prev_table= 0; Sergey> - while ((table= it++)) Sergey> + if (vcol_table->vfield) Sergey> { Sergey> - /* Sergey> - Do simple optimization to prevent unnecessary re-generating Sergey> - values for virtual fields Sergey> - */ Sergey> - if (table != prev_table) Sergey> - { Sergey> - prev_table= table; Sergey> - if (table->vfield) Sergey> - { Sergey> - if (update_virtual_fields(thd, table, TRUE)) Sergey> - { Sergey> - goto err; Sergey> - } Sergey> - } Sergey> - } Sergey> + if (update_virtual_fields(thd, vcol_table, TRUE)) Sergey> + goto err; Sergey> } Sergey> } Sergey> thd->abort_on_warning= abort_on_warning_saved; What has the above to do with the IPC patch? In any case, how will the above work when there is more than one table involved in fill_tables ? (By the way, agree that the old code as stupid; There is no reason to add all involved tables to tbl_list (trivial to avoid). Sergey> === modified file 'sql/sql_select.cc' Sergey> --- a/sql/sql_select.cc 2011-01-14 11:03:41 +0000 Sergey> +++ b/sql/sql_select.cc 2011-02-01 23:33:14 +0000 Sergey> @@ -13618,13 +13618,17 @@ Sergey> " cond: %p error: %d", join, join_tab, select_cond, error)); Sergey> if (error > 0 || (join->thd->is_error())) // Fatal error Sergey> DBUG_RETURN(NESTED_LOOP_ERROR); Sergey> + /* Sergey> + Check if we're killed before checking for EOF. This is important because Sergey> + if we're killed while reading a record, the handler will return EOF. Sergey> + */ Please update the comment; We should not send EOF from the handler if the query is killed; There is too many things that can go wrong if we send EOF on killed from the handler. Sergey> + if (join->thd->killed) // Aborted by user Sergey> + { Sergey> + join->thd->send_kill_message(); Sergey> + DBUG_RETURN(NESTED_LOOP_KILLED); /* purecov: inspected */ Sergey> + } <cut> Sergey> === modified file 'storage/maria/ma_rkey.c' Sergey> --- a/storage/maria/ma_rkey.c 2011-01-14 11:05:46 +0000 Sergey> +++ b/storage/maria/ma_rkey.c 2011-02-01 23:33:14 +0000 Sergey> @@ -34,7 +34,7 @@ Sergey> HA_KEYSEG *last_used_keyseg; Sergey> uint32 nextflag; Sergey> MARIA_KEY key; Sergey> - int icp_res= 1; Sergey> + ICP_RESULT icp_res= ICP_MATCH; Sergey> DBUG_ENTER("maria_rkey"); Sergey> DBUG_PRINT("enter", ("base: 0x%lx buf: 0x%lx inx: %d search_flag: %d", Sergey> (long) info, (long) buf, inx, search_flag)); Sergey> @@ -115,7 +115,7 @@ Sergey> not satisfied with an out-of-range condition. Sergey> */ Sergey> if ((*share->row_is_visible)(info) && Sergey> - ((icp_res= ma_check_index_cond(info, inx, buf)) != 0)) Sergey> + ((icp_res= ma_check_index_cond(info, inx, buf)) != ICP_NO_MATCH)) Sergey> break; Sergey> /* The key references a concurrently inserted record. */ Sergey> @@ -145,6 +145,19 @@ Sergey> if (_ma_search_next(info, &lastkey, maria_readnext_vec[search_flag], Sergey> info->s->state.key_root[inx])) Sergey> break; /* purecov: inspected */ Sergey> + Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + buf= 0; /* Fast abort */ Sergey> + icp_res= ICP_ERROR; We should not return an IPC_ERROR here as this test has nothing to do with IPC. It can also happens when we are scanning for a visible row. Better to just threat this as a normal handler error. See later comments... Sergey> + break; Sergey> + } Sergey> + Sergey> /* Sergey> Check that the found key does still match the search. Sergey> _ma_search_next() delivers the next key regardless of its Sergey> @@ -168,9 +181,9 @@ Sergey> if (share->lock_key_trees) Sergey> rw_unlock(&keyinfo->root_lock); Sergey> - if (info->cur_row.lastpos == HA_OFFSET_ERROR || (icp_res != 1)) Sergey> + if (info->cur_row.lastpos == HA_OFFSET_ERROR || (icp_res != ICP_MATCH)) Sergey> { Sergey> - if (icp_res == 2) Sergey> + if (icp_res == ICP_OUT_OF_RANGE || icp_res == ICP_ERROR) Sergey> { Sergey> info->cur_row.lastpos= HA_OFFSET_ERROR; Sergey> my_errno= HA_ERR_KEY_NOT_FOUND; We should not return HA_ERR_KEY_NOT_FOUND in the case of a real error. (Too big risk we loose the reason that something went wrong). Sergey> +/* Sergey> + Yield to possible other writers during a index scan. Sergey> + Check also if we got killed by the user and if yes, return Sergey> + HA_ERR_LOCK_WAIT_TIMEOUT Sergey> + Sergey> + return 0 ok Sergey> + return 1 Query has been requested to be killed Sergey> +*/ Sergey> + Sergey> +my_bool ma_yield_and_check_if_killed(MARIA_HA *info, int inx) Sergey> +{ Sergey> + MARIA_SHARE *share; Sergey> + if (ma_killed(info)) Sergey> + { Sergey> + //info->lastpos= HA_OFFSET_ERROR; Sergey> + my_errno= HA_ERR_KEY_NOT_FOUND; Sergey> + return 1; The above is not safe. Please use: info->cur_row.lastpos= HA_OFFSET_ERROR; my_errno= HA_ERR_ABORTED_BY_USER; In this case we can not return HA_ERR_KEY_NOT_FOUND as it's too big risk that the calling code may think that there was really no key and send a result to the client instead of aborting. By sending a fatal error message to the user, we can be sure that the calling code will not ignore it. That is why we probably should introduce HA_ERR_ABORTED_BY_USER. If we do this, then we have to fix that handler::print_error() ignores this error if thd->killed is set. dispatch_command will generate the necessary error message. Sergey> + } Sergey> + Sergey> + if ((share= info->s)->lock_key_trees) Sergey> + { Sergey> + /* Give writers a chance to access index */ Sergey> + rw_unlock(&share->keyinfo[inx].root_lock); Sergey> + rw_rdlock(&share->keyinfo[inx].root_lock); Sergey> + } Sergey> + return 0; Sergey> +} Sergey> + Sergey> === modified file 'storage/maria/ma_rnext.c' Sergey> --- a/storage/maria/ma_rnext.c 2009-12-15 07:16:46 +0000 Sergey> +++ b/storage/maria/ma_rnext.c 2011-02-01 23:33:14 +0000 <cut> Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Same thing as above, don't use IPC_ERROR, treat this as a real error! Sergey> + Sergey> /* Skip rows inserted by other threads since we got a lock */ Sergey> if ((error= _ma_search_next(info, &info->last_key, Sergey> SEARCH_BIGGER, Sergey> @@ -108,10 +120,10 @@ Sergey> info->update&= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED); Sergey> info->update|= HA_STATE_NEXT_FOUND; Sergey> - if (icp_res == 2) Sergey> + if (icp_res == ICP_OUT_OF_RANGE) Sergey> my_errno=HA_ERR_END_OF_FILE; /* got beyond the end of scanned range */ Sergey> - if (error || icp_res != 1) Sergey> + if (error || icp_res != ICP_MATCH) Sergey> { Sergey> if (my_errno == HA_ERR_KEY_NOT_FOUND) Sergey> my_errno=HA_ERR_END_OF_FILE; Don't return HA_ERR_END_OF_FILE or HA_ERR_KEY_NOT_FOUND for IPC_ERROR Sergey> === modified file 'storage/maria/ma_rnext_same.c' Sergey> --- a/storage/maria/ma_rnext_same.c 2010-10-30 01:59:39 +0000 Sergey> +++ b/storage/maria/ma_rnext_same.c 2011-02-01 23:33:14 +0000 Sergey> @@ -30,7 +30,7 @@ Sergey> int error; Sergey> uint inx,not_used[2]; Sergey> MARIA_KEYDEF *keyinfo; Sergey> - int icp_res= 1; Sergey> + ICP_RESULT icp_res= ICP_MATCH; Sergey> DBUG_ENTER("maria_rnext_same"); Sergey> if ((int) (inx= info->lastinx) < 0 || Sergey> @@ -80,9 +80,20 @@ Sergey> info->cur_row.lastpos= HA_OFFSET_ERROR; Sergey> break; Sergey> } Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Same thing as above, don't use IPC_ERROR, treat this as a real error! <cut> Sergey> - if (error || icp_res != 1) Sergey> + if (error || icp_res != ICP_MATCH) Sergey> { Sergey> if (my_errno == HA_ERR_KEY_NOT_FOUND) Sergey> my_errno=HA_ERR_END_OF_FILE; Don't return HA_ERR_END_OF_FILE or HA_ERR_KEY_NOT_FOUND for IPC_ERROR Sergey> === modified file 'storage/maria/ma_rprev.c' <cut> Sergey> + if (ma_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + icp_res= ICP_ERROR; Sergey> + break; Sergey> + } Return a real error! Sergey> + } Sergey> + Sergey> /* Skip rows that are inserted by other threads since we got a lock */ Sergey> if ((error= _ma_search_next(info, &info->last_key, Sergey> SEARCH_SMALLER, Sergey> @@ -66,6 +84,10 @@ Sergey> } Sergey> if (share->lock_key_trees) Sergey> rw_unlock(&keyinfo->root_lock); Sergey> + if (icp_res == ICP_ERROR || icp_res == ICP_OUT_OF_RANGE) Sergey> + { Sergey> + DBUG_RETURN(my_errno= HA_ERR_END_OF_FILE); Don't return HA_ERR_END_OF_FILE in case of IPC_ERROR Sergey> === modified file 'storage/maria/maria_def.h' Sergey> --- a/storage/maria/maria_def.h 2011-01-14 11:03:41 +0000 Sergey> +++ b/storage/maria/maria_def.h 2011-02-01 23:33:14 +0000 Sergey> @@ -505,6 +505,7 @@ Sergey> DYNAMIC_ARRAY pinned_pages; Sergey> /* accumulate indexfile changes between write's */ Sergey> TREE *bulk_insert; Sergey> + void *__external_ref; /* For MariaDB TABLE */ /* psergey-todo: need this?*/ No, we don't need this as we have external_ptr already in Aria that we can use. Sergey> --- a/storage/myisam/mi_rkey.c 2009-12-22 12:33:21 +0000 Sergey> @@ -168,26 +176,37 @@ Sergey> if (share->concurrent_insert) Sergey> rw_unlock(&share->key_root_lock[inx]); Sergey> - /* Calculate length of the found key; Used by mi_rnext_same */ Sergey> - if ((keyinfo->flag & HA_VAR_LENGTH_KEY) && last_used_keyseg && Sergey> - info->lastpos != HA_OFFSET_ERROR) Sergey> - info->last_rkey_length= _mi_keylength_part(keyinfo, info->lastkey, Sergey> - last_used_keyseg); Sergey> + info->last_rkey_length= pack_key_length; Sergey> + Sergey> + if (info->lastpos == HA_OFFSET_ERROR) /* No such record */ Sergey> + { Sergey> + /* Sergey> + psergey: what doe the next line do:vv initially there were no writeinfo Sergey> + calls! Sergey> + */ Sergey> + fast_mi_writeinfo(info); The original code had a bug. At start of mi_rkey.c we do fast_mi_readinfo(), which must be followed by a fast_mi_writeinfo() before return, which didn't happen if buf was 0. In the normal case 'read_record()' will do the fast_mi_writeinfo() call for the other cases. Sergey> + if (!buf) Sergey> + DBUG_RETURN(my_errno); Sergey> + } Sergey> else Sergey> + /* Calculate length of the found key; Used by mi_rnext_same */ Sergey> + if ((keyinfo->flag & HA_VAR_LENGTH_KEY) && last_used_keyseg) Sergey> + info->last_rkey_length= _mi_keylength_part(keyinfo, info->lastkey, Sergey> + last_used_keyseg); Sergey> + Sergey> + /* Check if we don't want to have record back, only error message */ Sergey> + if (!buf) Sergey> + DBUG_RETURN(0); We have to add fast_mi_writeinfo(info) before the DBUG_RETURN above. Sergey> + if (!(*info->read_record)(info,info->lastpos,buf)) Sergey> + { Sergey> + info->update|= HA_STATE_AKTIV; /* Record is read */ Sergey> + DBUG_RETURN(0); Sergey> + } <cut> Sergey> +my_bool mi_yield_and_check_if_killed(MI_INFO *info, int inx) Sergey> +{ Sergey> + MYISAM_SHARE *share; Sergey> + if (mi_killed(info)) Sergey> + { Sergey> + info->lastpos= HA_OFFSET_ERROR; Sergey> + my_errno= HA_ERR_KEY_NOT_FOUND; HA_ERR_KEY_NOT_FOUND -> HA_ERR_ABORTED_BY_USER Sergey> + return 1; Sergey> + } Sergey> + Sergey> + if ((share= info->s)->concurrent_insert) Sergey> + { Sergey> + /* Give writers a chance to access index */ Sergey> + rw_unlock(&share->key_root_lock[inx]); Sergey> + rw_rdlock(&share->key_root_lock[inx]); Sergey> + } Sergey> + return 0; Sergey> +} Sergey> === modified file 'storage/myisam/mi_rnext.c' Sergey> --- a/storage/myisam/mi_rnext.c 2010-06-26 10:05:41 +0000 Sergey> +++ b/storage/myisam/mi_rnext.c 2011-02-01 23:33:14 +0000 Sergey> @@ -104,6 +104,18 @@ Sergey> (info->index_cond_func && Sergey> (res= mi_check_index_cond(info, inx, buf)) == ICP_NO_MATCH)) Sergey> { Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + res= ICP_ERROR; Return a real error! Sergey> + break; Sergey> + } Sergey> + Sergey> /* Sergey> Skip rows that are either inserted by other threads since Sergey> we got a lock or do not match pushed index conditions Sergey> @@ -115,7 +127,8 @@ Sergey> info->s->state.key_root[inx]))) Sergey> break; Sergey> } Sergey> - if (!error && res == ICP_OUT_OF_RANGE) Sergey> + Sergey> + if (!error && (res == ICP_OUT_OF_RANGE || res == ICP_ERROR)) Sergey> { Sergey> if (info->s->concurrent_insert) Sergey> rw_unlock(&info->s->key_root_lock[inx]); Return error code in case of ICP_ERROR Sergey> === modified file 'storage/myisam/mi_rnext_same.c' Sergey> --- a/storage/myisam/mi_rnext_same.c 2009-12-22 12:33:21 +0000 Sergey> +++ b/storage/myisam/mi_rnext_same.c 2011-02-01 23:33:14 +0000 Sergey> @@ -63,6 +63,18 @@ Sergey> } Sergey> for (;;) Sergey> { Sergey> + /* Sergey> + If we are at the last key on the key page, allow writers to Sergey> + access the index. Sergey> + */ Sergey> + if (info->int_keypos >= info->int_maxpos && Sergey> + mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error=1; Sergey> + buf= 0; /* Fast abort */ Sergey> + break; Return real error Sergey> + } Sergey> + Sergey> if ((error=_mi_search_next(info,keyinfo,info->lastkey, Sergey> info->lastkey_length,SEARCH_BIGGER, Sergey> info->s->state.key_root[inx]))) Sergey> @@ -87,6 +99,10 @@ Sergey> } Sergey> if (info->s->concurrent_insert) Sergey> rw_unlock(&info->s->key_root_lock[inx]); Sergey> + Sergey> + if (!buf) Sergey> + DBUG_RETURN(my_errno); Sergey> + Remove above; We have to update info->update even if buf is 0 Sergey> /* Don't clear if database-changed */ Sergey> info->update&= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED); Sergey> info->update|= HA_STATE_NEXT_FOUND | HA_STATE_RNEXT_SAME; Sergey> === modified file 'storage/myisam/mi_rprev.c' Sergey> --- a/storage/myisam/mi_rprev.c 2009-12-15 07:16:46 +0000 Sergey> +++ b/storage/myisam/mi_rprev.c 2011-02-01 23:33:14 +0000 Sergey> @@ -53,12 +53,28 @@ Sergey> if (!error) Sergey> { Sergey> - int res= 0; Sergey> + ICP_RESULT res= 0; Sergey> + my_off_t cur_keypage= info->last_keypage; Sergey> while ((share->concurrent_insert && Sergey> info->lastpos >= info->state->data_file_length) || Sergey> (info->index_cond_func && Sergey> !(res= mi_check_index_cond(info, inx, buf)))) Sergey> { Sergey> + /* Sergey> + If we are at the last (i.e. first?) key on the key page, Sergey> + allow writers to access the index. Sergey> + */ Sergey> + if (info->last_keypage != cur_keypage) Sergey> + { Sergey> + cur_keypage= info->last_keypage; Sergey> + if (mi_yield_and_check_if_killed(info, inx)) Sergey> + { Sergey> + error= 0; Sergey> + res= ICP_ERROR; Sergey> + break; Return error Sergey> + } Sergey> + } Sergey> + Sergey> /* Sergey> Skip rows that are either inserted by other threads since Sergey> we got a lock or do not match pushed index conditions Sergey> @@ -69,7 +85,7 @@ Sergey> share->state.key_root[inx]))) Sergey> break; Sergey> } Sergey> - if (!error && res == 2) Sergey> + if (!error && (res == ICP_OUT_OF_RANGE || res == ICP_ERROR)) Sergey> { Sergey> if (share->concurrent_insert) Sergey> rw_unlock(&share->key_root_lock[inx]); Return error code in case of ICP_ERROR <cut> Sergey> --- a/storage/xtradb/handler/ha_innodb.cc 2011-01-12 12:00:10 +0000 Sergey> +++ b/storage/xtradb/handler/ha_innodb.cc 2011-02-01 23:33:14 +0000 <cut> Sergey> @@ -12094,15 +12102,18 @@ Sergey> See note on ICP_RESULT for return values description. Sergey> */ Sergey> -static int index_cond_func_innodb(void *arg) Sergey> +static xtradb_icp_result_t index_cond_func_innodb(void *arg) Sergey> { Sergey> ha_innobase *h= (ha_innobase*)arg; Sergey> + if (h->is_thd_killed()) Sergey> + return XTRADB_ICP_ERROR; Sergey> + rename to XTRADB_ABORTED_BY_USER; Sergey> if (h->end_range) Sergey> { Sergey> if (h->compare_key2(h->end_range) > 0) Sergey> - return ICP_OUT_OF_RANGE; /* caller should return HA_ERR_END_OF_FILE already */ Sergey> + return XTRADB_ICP_OUT_OF_RANGE; /* caller should return HA_ERR_END_OF_FILE already */ Sergey> } Sergey> - return h->pushed_idx_cond->val_int()? ICP_MATCH : ICP_NO_MATCH; Sergey> + return h->pushed_idx_cond->val_int()? XTRADB_ICP_MATCH : XTRADB_ICP_NO_MATCH; Sergey> } Sergey> C_MODE_END <cut> Sergey> --- a/storage/xtradb/include/row0mysql.h 2010-12-06 08:25:44 +0000 Sergey> +++ b/storage/xtradb/include/row0mysql.h 2011-02-01 23:33:14 +0000 Sergey> @@ -577,7 +577,15 @@ Sergey> #define ROW_PREBUILT_ALLOCATED 78540783 Sergey> #define ROW_PREBUILT_FREED 26423527 Sergey> -typedef int (*index_cond_func_t)(void *param); Sergey> + Sergey> +typedef enum xtradb_icp_result { Sergey> + XTRADB_ICP_ERROR=-1, rename to XTRADB_ABORTED_BY_USER Sergey> + XTRADB_ICP_NO_MATCH=0, Sergey> + XTRADB_ICP_MATCH=1, Sergey> + XTRADB_ICP_OUT_OF_RANGE=2 Sergey> +} xtradb_icp_result_t; Sergey> + Sergey> +typedef xtradb_icp_result_t (*index_cond_func_t)(void *param); Sergey> /** A struct for (sometimes lazily) prebuilt structures in an Innobase table Sergey> handle used within MySQL; these are used to save CPU time. */ Sergey> === modified file 'storage/xtradb/row/row0sel.c' Sergey> --- a/storage/xtradb/row/row0sel.c 2010-12-06 17:44:17 +0000 Sergey> +++ b/storage/xtradb/row/row0sel.c 2011-02-01 23:33:14 +0000 Sergey> @@ -4396,12 +4396,13 @@ Sergey> */ Sergey> ut_ad(ib_res); Sergey> res= prebuilt->idx_cond_func(prebuilt->idx_cond_func_arg); Sergey> - if (res == 0) Sergey> + if (res == XTRADB_ICP_NO_MATCH) Sergey> goto next_rec; Sergey> - if (res == 2) { Sergey> + if (res == XTRADB_ICP_OUT_OF_RANGE || res == XTRADB_ICP_ERROR) { Sergey> err = DB_RECORD_NOT_FOUND; Sergey> goto idx_cond_failed; Don't return DB_RECORD_NOT_FOUND in case of XTRADB_ICP_ERROR. Return an actual error like DB_ABORTED_BY_USER which can be translated to HA_ERR_ABORTED_BY_USER. Regards, Monty

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 03 Feb '11

03 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Donnerstag, 3. Februar 2011 02:00 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > Hello Kazuhisa, > > But to reiterate, this package is not really thought to be run by non-admin users. It installs into program files, and it writes to > > the HKLM registry hive, and it creates a service. All this stuff is not possible without being an administrator. I'm quite surprised > > Well, if that is how it is designed, that's fine with me. I just > thought that there might be many users who are new to MariaDB, not > sure about its quality and want to install/run it in a sandbox > environment. If switching to HKCU registry hive and disabling the > option to run it as a service when the installing user is non-admin > can satisfy the needs, implementing them might not be a bad idea. But > again, I do not have a strong opinion on that. (The user can choose to > install from the zipped package as long as zip version is also > distributed). I this the per-user install is a feature in NSIS which you discovered, but which is was not planned from the start on. From the start on, it was planned as an installation where you get a running service at the end, and that was not implemented in NSIS. if it was, it would not work in this sandboxed environment. With MSI I checked, and the rules for per-user install appear to be relatively complicated. Officially dual package authoring (single package that can be installed either per-user or per-machine) is supported with MSI 5.0 and Win7, so there is no hope to get that working in the same for unprivileged XP user. I might do some experimentation with it on Win7, however the benefit it provide compared to ZIP is low: is a menu shortcut with PATH including bin directory of the installation and Add/Remove Program entry. On a bright side, this is a valid bug that was simple to fix forcing per-machine install, so installer does not need to guess. I did that and refreshed the MSI on the download page. > One more thing that I noticed is, when installing 5.1 all the messages > in the dialogs are in Japanese (with the exception of GPL license > statement), while they are all in English for 5.2. Maybe this is a > bonus that came from NSIS. Maybe. NSIS guys deserve a compliment for that effort. With MSI dealing with multiple languages inside single MSI is not that simple. But, localization is definitely possible in Wix or generally MSI, I so far ignored that topic altogether;) But back to NSIS package .I believe, just like with per-user installation, you discovered hidden and untested feature, an Easter egg. It has more localization bugs that just GPL text. The feature tree is not localized, neither uninstall popup box. If that NSIS package had more functionality than plain unpack, and had own dialogs, there would be more localization bugs.

1 0

Re: [Maria-developers] Windows installer MWL#55 finished.
by Vladislav Vaintroub 02 Feb '11

02 Feb '11

> -----Original Message----- > From: kahzoo(a)gmail.com [mailto:kahzoo@gmail.com] On Behalf Of Kazuhisa Ichikawa > Sent: Mittwoch, 2. Februar 2011 23:01 > To: Vladislav Vaintroub > Cc: maria-developers(a)lists.launchpad.net > Subject: Re: [Maria-developers] Windows installer MWL#55 finished. > > Vladislav, > > I tried the new 5.2.5 installer in Japanese Windows XP (32bit) > environment using a user account which does NOT have administrator > privilege. I wonder why it has started at all. It definitely needs administrator and installer on later Windows brings all the necessary UAC popups. > During the installation I encountered a lot of warning > dialogs (see below for an example) about not being able to write into > the registry, although I did not have any noticeable problem startnig > mysqld.exe after the installation is finished (No, the installation > was not aborted and it said "Completed the MariaDB 5.2 Setup WIzard.") > > (Example warning message) > "Could not write value InstallLocation to key > \Software\Microsoft\Windows\CurrenVersion\Uninstall\MariaDB 5.2. > Verify that you have sufficient access to that key, or contact your > support personnel." If you do not have that key, you do not have an entry in "Add/remove programs" > I think what this error message means is that the user account used > for the installation needs to have an administrator privilege, but the > current (older) installer for MariaDB 5.1.53 did not have this > problem. This installer should be run as administrator, there is no per-user option. > The other (related) issue is that I cannot uninstall mariadb 5.2 now > because it is not listed in the installed program list in the control > panel (most likely because the necessary information is not written > into the registry). You will need to run msiexec /x mariadb-5.2.5-win32.msi from the command line. This uninstalls installed package (I'm not really sure it was installed, but it does not hurt to try, it nothing is installed it will tell) > Again, the installer for 5.1 did not have this problem. Well, it is not installer for 5.1, the old one had more or less the same functionality as downloading and unpacking a zip file. It might even be that NSIS guys automatically switched to HKCU registry keys or so for a weak user. > Also, unlike the current installer for 5.1, the new installer > does not seem to install a standalone uninstaller (Uninstall.exe) on > the disk. So, I'm stuck and wondering how to uninstall 5.2 (Well, not > really, because if nothing is written into the registry, deleting all > the files and directories should 'cleanly' uninstall everything, > right?.). I would stick to documented stuff. msiexec /x mariadb-5.2.5-win32.msi But to reiterate, this package is not really thought to be run by non-admin users. It installs into program files, and it writes to the HKLM registry hive, and it creates a service. All this stuff is not possible without being an administrator. I'm quite surprised that MSI let you go that far actually, and that it would not rollback and cleanup after the error. On the other hand, thanks for an interesting error report, makes sense for me to have a test for anomalies like that. Can you tell service pack of your XP and MSI version (launch winver and msiexec /?) Thanks, Vladislav > - Kazuhisa >

1 0