[Maria-discuss] Spider 3.0 is merged against MariaDB 10.0
Hi!
Kentoku Shiba has merged the latest Spider 3.0 against MariaDB 10.0: https://code.launchpad.net/~kentokushiba/maria/10.0-spider-3.0
I would like to propose this for merging
There is also a complete test suite now, and Kentoku has written some instructions:
== testing Spider I pushed Spider into "lp:~kentokushiba/maria/10.0-spider-3.0". It's tested on "CentOS 6.3 x86_64" and "Windows 7 professional 32bit". Test command is the followings.
cd mysql-test ./mysql-test-run.pl --mysqld=--innodb --suite=spider
The following test command can't done yet, because it needs some change for MariaDB.
./mysql-test-run.pl --mysqld=--innodb --suite=spider/bg
Spider's background/parallel search feature works allocate and free result-set from data node on different thread, but MariaDB 10.0's "per thread memory usage" feature can't understand it.
./mysql-test-run.pl --mysqld=--innodb --suite=spider/handler
Table partition feature need to support "handler" command. I have a patch file.
In addition, it is awesome to note that Kentoku & his team behind the development of Groonga storage engine have also won an award (Japan OSS Incentive Award): http://ossforum.jp/en/node/1277
Thanks
cheers, -colin
-- Colin Charles, http://bytebot.net/blog/ | twitter: @bytebot | skype: colincharles MariaDB: Community developed. Feature enhanced. Backward compatible. Download it at: http://www.mariadb.org/ Open MariaDB/MySQL documentation at the Knowledgebase: http://kb.askmonty.org/
Hi!
Could you please bundle Spider into MariaDB?
./mysql-test-run.pl --mysqld=--innodb --suite=spider/bg
Spider's background/parallel search feature works allocate and free
result-set from data node on different thread, but MariaDB 10.0's "per thread memory usage" feature can't understand it.
Could you please change sql-common/client.c for choosing "MY_THREAD_SPECIFIC" (or not) by caller? This memory allocation is not always thread specific.
Thanks, Kentoku
2013/4/12 Colin Charles colin@montyprogram.com
Hi!
Kentoku Shiba has merged the latest Spider 3.0 against MariaDB 10.0: https://code.launchpad.net/~kentokushiba/maria/10.0-spider-3.0
I would like to propose this for merging
There is also a complete test suite now, and Kentoku has written some instructions:
== testing Spider I pushed Spider into "lp:~kentokushiba/maria/10.0-spider-3.0". It's tested on "CentOS 6.3 x86_64" and "Windows 7 professional 32bit". Test command is the followings.
cd mysql-test ./mysql-test-run.pl --mysqld=--innodb --suite=spider
The following test command can't done yet, because it needs some change for MariaDB.
./mysql-test-run.pl --mysqld=--innodb --suite=spider/bg
Spider's background/parallel search feature works allocate and free result-set from data node on different thread, but MariaDB 10.0's "per thread memory usage" feature can't understand it.
./mysql-test-run.pl --mysqld=--innodb --suite=spider/handler
Table partition feature need to support "handler" command. I have a patch file.
In addition, it is awesome to note that Kentoku & his team behind the development of Groonga storage engine have also won an award (Japan OSS Incentive Award): http://ossforum.jp/en/node/1277
Thanks
cheers, -colin
-- Colin Charles, http://bytebot.net/blog/ | twitter: @bytebot | skype: colincharles MariaDB: Community developed. Feature enhanced. Backward compatible. Download it at: http://www.mariadb.org/ Open MariaDB/MySQL documentation at the Knowledgebase: http://kb.askmonty.org/
Hi!
"kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
./mysql-test-run.pl --mysqld=--innodb --suite=spider/bg
Spider's background/parallel search feature works allocate and free
kentoku> result-set from data node on different thread, but MariaDB 10.0's "per kentoku> thread memory usage" feature can't understand it.
kentoku> Could you please change sql-common/client.c for choosing kentoku> "MY_THREAD_SPECIFIC" (or not) by caller? This memory allocation is not kentoku> always thread specific.
I will fix that during this week. It's hard to do in the caller, as there is no parameter that can be used for this.
I can fix this by adding a new 'mysql_options()' where one says that the all memory in this connection will be thread specific and use this in the replication threads and in the federated engine (where this is needed).
Would this solve your problem?
Regards, Monty
Hi, Michael!
On Apr 17, Michael Widenius wrote:
"kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
I was going to do it. And to solve MY_THREAD_SPECIFIC problem too.
Regards, Sergei
Hi Monty and Sergei!
Thank you for your response and reviewing.
I will fix that during this week. It's hard to do in the caller, as there is no parameter that can be used for this.
I can fix this by adding a new 'mysql_options()' where one says that the all memory in this connection will be thread specific and use this in the replication threads and in the federated engine (where this is needed).
Would this solve your problem?
Yes, this solves my problem. Additionally, if 'mysql_options()' can be used dynamically before calling 'mysql_store_result()' and 'cli_use_result()' for switching "MY_THREAD_SPECIFIC" (or not), it is better for me.
Thanks, Kentoku
2013/4/17 Sergei Golubchik serg@askmonty.org
Hi, Michael!
On Apr 17, Michael Widenius wrote:
> "kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
I was going to do it. And to solve MY_THREAD_SPECIFIC problem too.
Regards, Sergei
Hi!
"kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi Monty and Sergei! kentoku> Thank you for your response and reviewing.
I will fix that during this week. It's hard to do in the caller, as there is no parameter that can be used for this.
I can fix this by adding a new 'mysql_options()' where one says that the all memory in this connection will be thread specific and use this in the replication threads and in the federated engine (where this is needed).
Would this solve your problem?
kentoku> Yes, this solves my problem. Additionally, if 'mysql_options()' can be used kentoku> dynamically before calling 'mysql_store_result()' and 'cli_use_result()' kentoku> for switching "MY_THREAD_SPECIFIC" (or not), it is better for me.
For now, one can only call mysql_options() before the connection happens. I prefer to not change this for this release.
Regards, Monty
Hi Monty!
For now, one can only call mysql_options() before the connection happens. I prefer to not change this for this release.
O.K. It's no problem.
Thanks, Kentoku
2013/4/17 Michael Widenius monty@askmonty.org
Hi!
"kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi Monty and Sergei! kentoku> Thank you for your response and reviewing.
I will fix that during this week. It's hard to do in the caller, as there is no parameter that can be used for this.
I can fix this by adding a new 'mysql_options()' where one says that the all memory in this connection will be thread specific and use this in the replication threads and in the federated engine (where this is needed).
Would this solve your problem?
kentoku> Yes, this solves my problem. Additionally, if 'mysql_options()' can be used kentoku> dynamically before calling 'mysql_store_result()' and 'cli_use_result()' kentoku> for switching "MY_THREAD_SPECIFIC" (or not), it is better for me.
For now, one can only call mysql_options() before the connection happens. I prefer to not change this for this release.
Regards, Monty
Hi!
"Sergei" == Sergei Golubchik serg@askmonty.org writes:
Sergei> Hi, Michael! Sergei> On Apr 17, Michael Widenius wrote:
> "kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
Sergei> I was going to do it.
good!
Sergei> And to solve MY_THREAD_SPECIFIC problem too.
The MY_THREAD_SPECIFIC issue is already solved and pushed in 10.0-base.
Regards, Monty
Hi, Monty!
Sergei> And to solve MY_THREAD_SPECIFIC problem too.
The MY_THREAD_SPECIFIC issue is already solved and pushed in 10.0-base.
Thank you for solving! Currently, "spider/bg" test is success.
Thanks, Kentoku
2013/4/19 Michael Widenius monty@askmonty.org
Hi!
"Sergei" == Sergei Golubchik serg@askmonty.org writes:
Sergei> Hi, Michael! Sergei> On Apr 17, Michael Widenius wrote:
>> "kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
Sergei> I was going to do it.
good!
Sergei> And to solve MY_THREAD_SPECIFIC problem too.
The MY_THREAD_SPECIFIC issue is already solved and pushed in 10.0-base.
Regards, Monty
Hi!
I merged again Spider 3.0b into MariaDB 10.0.2. https://code.launchpad.net/~kentokushiba/maria/10.0.2-spider-3.0 This version supports "assisted discovery" feature without partitioning.(Currently this support is only for MariaDB/MySQL connection)
Thanks, Kentoku
2013/4/20 kentoku kentokushiba@gmail.com
Hi, Monty!
Sergei> And to solve MY_THREAD_SPECIFIC problem too.
The MY_THREAD_SPECIFIC issue is already solved and pushed in 10.0-base.
Thank you for solving! Currently, "spider/bg" test is success.
Thanks, Kentoku
2013/4/19 Michael Widenius monty@askmonty.org
Hi!
> "Sergei" == Sergei Golubchik serg@askmonty.org writes:
Sergei> Hi, Michael! Sergei> On Apr 17, Michael Widenius wrote:
>>> "kentoku" == kentoku kentokushiba@gmail.com writes:
kentoku> Hi! kentoku> Could you please bundle Spider into MariaDB?
Sergei Golubchik or Sergei Petrunia, could either of you do a quick review of this?
Sergei> I was going to do it.
good!
Sergei> And to solve MY_THREAD_SPECIFIC problem too.
The MY_THREAD_SPECIFIC issue is already solved and pushed in 10.0-base.
Regards, Monty
participants (4)
-
Colin Charles -
kentoku -
Michael Widenius -
Sergei Golubchik