Hi all, I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities. Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list. R: Jan -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.
"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table" it is also mandatory for compressed tables for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority
Hi, I understand your point, however changing defaults is always a dangerous thing to do because change has always a effect on many users. R: Jan
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list. "innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"
it is also mandatory for compressed tables
for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority
-- -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
default of innodb_log_file_size still at 5M on a fresh install. packaging scripts could detect the actual value in use or default to at least 50M for a fresh install. is a "detect whats there" in the mysqld applicable since there seems to be enough log sequence number and tables space number checks on them anyway? ----- Original Message -----
Hi,
I understand your point, however changing defaults is always a dangerous thing to do because change has always a effect on many users.
detect values for existing installations and provide a sane default for new installations.
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list. "innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"
Good call -- -- Daniel Black, Engineer @ Open Query (http://openquery.com) Remote expertise & maintenance for MySQL/MariaDB server environments.
do you have time to test toku engine? i'm trying it and it's very nice, i'm considering changing my tables from innodb to toku really soon 2014-01-28 Reindl Harald <h.reindl@thelounge.net>:
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.
"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"
it is also mandatory for compressed tables
for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
no, i use InnoDB only for DBMail and most other things are and will be MyISAM in the foreseeable future because i do not appreciate global namespaces instead having a folder per database which is in doubt the easiest restore having 6000 mostly small tables in 250 databases is perfectly suiteable with MyISAM Am 29.01.2014 16:14, schrieb Roberto Spadim:
do you have time to test toku engine? i'm trying it and it's very nice, i'm considering changing my tables from innodb to toku really soon
2014-01-28 Reindl Harald <h.reindl@thelounge.net>:
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.
"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"
it is also mandatory for compressed tables
for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority
yes, for myisam with important data, aria engine is nice too, i changed some important tables from myisam to aria some years ago, and had no problem , some bugs repaired by mariadb team, but works nice :) 2014-01-29 Reindl Harald <h.reindl@thelounge.net>:
no, i use InnoDB only for DBMail and most other things are and will be MyISAM in the foreseeable future because i do not appreciate global namespaces instead having a folder per database which is in doubt the easiest restore
having 6000 mostly small tables in 250 databases is perfectly suiteable with MyISAM
Am 29.01.2014 16:14, schrieb Roberto Spadim:
do you have time to test toku engine? i'm trying it and it's very nice, i'm considering changing my tables from innodb to toku really soon
2014-01-28 Reindl Harald <h.reindl@thelounge.net>:
Am 28.01.2014 13:38, schrieb Jan Lindström:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.
"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"
it is also mandatory for compressed tables
for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
Hi Jan, (nice name btw... ;-) ) I'd add the following feature requests to the list: - Being able to have different partitions of a table on different disks: We have tables that have 4 partitions for each day (partitioned by time) and hold 10 days of data. We have a (almost) constantly increasing time, but during a day the "active" (=todays) partitons get random inserts (time is only the third party of the PK). Each partition is 10-20GB. We have these on SSDs, but already had to decrease the number of days from 14 to 10 (we drop/create the partitions with a cronjob) as we run out of space. I have more SAS hardisk diskspace available in the servers but no more SSDs, so instead of deleting the old partitions I'd like to move them *on the fly* to another mountpoint for another 2 weeks... - we found a "alter table ... rebuild partition ..." blocks the whole table, although we are rebuilding (=defragmentation) old partitions that is not written to. - I haven't had this problem for a long time, but I remember having had some servers with a big useless ibdata file although all tables were moved to single files (file_per_table). In that case, some simple possibility to replace the ibdata by a small version would be helpful. restarting the mysqld is OK, but import/export tablespaces and such is a pain with big databases - Some tool/functionality to reimport/repair tablespaces of partitioned tables. There are ways to do that, you find tips and information when you google a little bit, but from my impression most of the steps could be automated: i.e. the user should never have to deal with tablespace IDs and such. I'd like to be able to copy all the partitions ( TABLENAME#P#<XXX>#SP#<XXX><YYY>.ibd) from one server (or a backup) to the other one (while mysqld is running on the receiving server) and then import all tablespaces at once with one command. done. thanks for reading this ;-) Jan Am 28.01.2014 13:38, schrieb Jan Lindström:
Hi all,
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.
Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.
R: Jan
--
Jan Lindström Principal Engineer
MariaDB | MaxScale | skype: jan_p_lindstrom
www.skysql.com <http://www.skysql.com/>
Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
On 01/28/2014 01:38 PM, Jan Lindström wrote:
I would like to know the most affecting InnoDB pain points the customers/potential customers first refer.
some random first thoughts: * main table space defrag / shrink was mentioned already, even with file-per-table this can still be an issue with large transactions able to blow up undo logs ... even an offline defrag by an external tool that requires mysqld to be shut down first may already be worth it * a more personal pain point: now that FULLTEXT index support is finally there SPATIAL indexes are the last missing MyISAM feature that may keep some tables back from being converted to InnoDB * DATA DIRECTORY / INDEX DIRECTORY support for InnoDB tables (implicitly enabling file-per-table, too) ... right now this CREATE option is just silently ignored, doesn't even throw a warning ... * better insight into undo log utilization, esp. with separate undo tablespace ... * ... -- Hartmut Holzgraefe, Principal Support Engineer (EMEA) SkySQL | http://www.skysql.com/
Hi! I just read this news from SkySQL: http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh... "XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin" This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB? Regards Federico
Am 05.02.2014 16:03, schrieb Federico Razzoli:
I just read this news from SkySQL:
http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh... "XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin" This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB?
that would be really bad servers with xtradb-configs would no loger start and it is not really funny have to care on each update how defaults have changed - nobody is changing a storage engine for fun the following line is for good reasons in my MariaDB RPM-SPEC beause i do not need any plugins and XtraDB is fine and tested rm -rf %{buildroot}%{_libdir}/mysql/plugin/
Hi, Federico! On Feb 05, Federico Razzoli wrote:
I just read this news from SkySQL:
http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh...
"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"
This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB?
It is not about advantages and disadvantages at the moment. But XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB). I hope we'll have XtraDB as the default in 10.0. Regards, Sergei
On 5 Feb 2014, at 23:44, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Federico!
On Feb 05, Federico Razzoli wrote:
I just read this news from SkySQL:
http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh...
"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"
This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB?
It is not about advantages and disadvantages at the moment.
But XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream? Do we plan on fixing this ourselves if upstream doesn't?
I hope we'll have XtraDB as the default in 10.0.
Regards, Sergei
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Colin Charles, Chief Evangelist MariaDB | t: +6-012-204-3201 | Skype: colincharles
Hi, Colin! On Feb 06, Colin Charles wrote:
"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"
XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream?
https://bugs.launchpad.net/bugs/1276963
Do we plan on fixing this ourselves if upstream doesn't?
Not at the moment - that code is quite complex. Regards, Sergei
Sergei, Colin, MariaDB -
"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"
XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream?
https://bugs.launchpad.net/bugs/1276963
Do we plan on fixing this ourselves if upstream doesn't?
Not at the moment - that code is quite complex.
In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you? We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you? -- Laurynas
Laurynas Biveinis <laurynas.biveinis@gmail.com> writes:
We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is
Hm, this is interesting, I was not aware of the __thread feature. This looks to be much more efficient than pthread_getspecific(), which is one of the top functions in the profiles for sysbench readonly: static __thread long tcount = 0; long bump(long del) { return (tcount += del); } 0000000000000000 <bump>: 0: 48 89 f8 mov %rdi,%rax 3: 64 48 03 04 25 00 00 add %fs:0x0,%rax a: 00 00 c: 64 48 89 04 25 00 00 mov %rax,%fs:0x0 13: 00 00 15: c3 retq It is just a single instruction using %fs: addressing to access the thread_local storage. Apparently the linker resolves the %fs: offset in the resulting binary: 0000000000400580 <bump>: 400580: 48 89 f8 mov %rdi,%rax 400583: 64 48 03 04 25 e0 ff add %fs:0xffffffffffffffe0,%rax 40058a: ff ff 40058c: 64 48 89 04 25 e0 ff mov %rax,%fs:0xffffffffffffffe0 400593: ff ff 400595: c3 retq 00000000004005a0 <bump2>: 4005a0: 48 89 f8 mov %rdi,%rax 4005a3: 64 48 03 04 25 f0 ff add %fs:0xfffffffffffffff0,%rax 4005aa: ff ff 4005ac: 64 48 89 04 25 f0 ff mov %rax,%fs:0xfffffffffffffff0 4005b3: ff ff 4005b5: c3 retq So that could be very promising for an easy way to tweak out a little more performance. Something to look into at some point. - Kristian.
Hi, Laurynas! On Feb 07, Laurynas Biveinis wrote:
Sergei, Colin, MariaDB -
"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"
XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream?
https://bugs.launchpad.net/bugs/1276963
Do we plan on fixing this ourselves if upstream doesn't?
Not at the moment - that code is quite complex.
In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you?
That kind of worked. XtraDB compiled. But query_response_time failed with "unable to find a register to spill in class 'GENERAL_REGS'". Still better than before. In the worst case, I'll enable XtraDB and disable query_response_time plugin.
We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is
Yes, I've noticed. It caused lots of valgrind warnings because __thread implementation uses __tls_get_addr() in glibc and that allocated memory which valgrind didn't see being freed. I've added a suppression for it.
used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you?
Besides this valgrind warning there were no issues so far. Or do you mean a different behavior of innodb_sched_priority_* options? I think it's ok for us, if it's ok for you :) Regards, Sergei
Sergei -
XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream?
https://bugs.launchpad.net/bugs/1276963
Do we plan on fixing this ourselves if upstream doesn't?
Not at the moment - that code is quite complex.
In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you?
That kind of worked. XtraDB compiled. But query_response_time failed with "unable to find a register to spill in class 'GENERAL_REGS'".
Still better than before. In the worst case, I'll enable XtraDB and disable query_response_time plugin.
Pushing down -march=i686 to xtradb dir only would work around this, wouldn't it? (our proper fix for 1276963 is likely to contain that. It would add the flag to the top level CMake and to the innodb subdir. The former is for us, the latter is for folks who take innodb only, i.e. you. This would work because core and InnoDB atomic builtin tests are independent in CMake currently). I tried to reproduce the compiler failure on our Jenkins, but it didn't show up: http://jenkins.percona.com/job/mariadb-10.0-trunk/3/, although the CentOS 5 slaves failed due to missing dependencies, and our build flags used are probably different from yours.
We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is
Yes, I've noticed. It caused lots of valgrind warnings because __thread implementation uses __tls_get_addr() in glibc and that allocated memory which valgrind didn't see being freed. I've added a suppression for it.
Interesting, I haven't noticed this before, will check.
used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you?
Or do you mean a different behavior of innodb_sched_priority_* options? I think it's ok for us, if it's ok for you :)
Yes, I mean the different behavior. It's OK for us because the behavior is correct on all the platforms we support, but it wouldn't be OK on some of the platforms you support, would it? -- Laurynas
Hi, Laurynas! On Feb 10, Laurynas Biveinis wrote:
Sergei -
XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).
Have we reported bugs upstream?
https://bugs.launchpad.net/bugs/1276963
Do we plan on fixing this ourselves if upstream doesn't?
Not at the moment - that code is quite complex.
In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you?
That kind of worked. XtraDB compiled. But query_response_time failed with "unable to find a register to spill in class 'GENERAL_REGS'".
Still better than before. In the worst case, I'll enable XtraDB and disable query_response_time plugin.
Pushing down -march=i686 to xtradb dir only would work around this, wouldn't it?
Yes. I wasn't quite sure whether it's a good idea to build different parts of the server with different -mtune settings. But ok, I'll try that.
used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you?
Or do you mean a different behavior of innodb_sched_priority_* options? I think it's ok for us, if it's ok for you :)
Yes, I mean the different behavior. It's OK for us because the behavior is correct on all the platforms we support, but it wouldn't be OK on some of the platforms you support, would it?
I believe it's ok anyway. MySQL always used to have platform-specific features, like thread priorities, connections that use shared memory, and so on. Regards, Sergei
Hi all, After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB (1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834 (2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834 Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links. R: -- Jan Lindström, Principal Engineer SkySQL - The MariaDB Company MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
Hi all, Based on discussions on Percona Live I have added two more projects of interest: (1) Support > 16K pages on InnoDB: https://mariadb.atlassian.net/browse/MDEV-6075 (2) Persistent auto increment on InnoDB: https://mariadb.atlassian.net/browse/MDEV-6076 This to let you vote for your favorite.
Hi all,
After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB
(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834
(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834
Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.
R:
R: -- Jan Lindström, Principal Engineer SkySQL - The MariaDB Company MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com <http://www.skysql.com/> Twitter <http://twitter.com/skysql> Blog <http://www.skysql.com/blog/> Facebook <http://www.facebook.com/skysql> LinkedIn <http://www.linkedin.com/company/1214250> Google+ <https://plus.google.com/117544963211695643458/posts>
Second link is the same as the first one: MDEV-5834 On Wed, Mar 12, 2014 at 5:59 AM, Jan Lindström <jplindst@mariadb.org> wrote:
Hi all,
After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB
(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834
(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834
Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.
R:
--
Jan Lindström, Principal Engineer SkySQL - The MariaDB Company
MariaDB | MaxScale | skype: jan_p_lindstrom
www.skysql.com
[image: Twitter] <http://twitter.com/skysql> [image: Blog]<http://www.skysql.com/blog/> [image: Facebook] <http://www.facebook.com/skysql> [image: LinkedIn]<http://www.linkedin.com/company/1214250> [image: Google+] <https://plus.google.com/117544963211695643458/posts>
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Adam, you are right, bulk load project is 5835: (2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5835 On Fri, Apr 11, 2014 at 5:28 PM, Adam Scott <adam.c.scott@gmail.com> wrote:
Second link is the same as the first one: MDEV-5834
On Wed, Mar 12, 2014 at 5:59 AM, Jan Lindström <jplindst@mariadb.org>wrote:
Hi all,
After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB
(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834
(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834
Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.
R:
--
Jan Lindström, Principal Engineer SkySQL - The MariaDB Company
MariaDB | MaxScale | skype: jan_p_lindstrom
www.skysql.com
[image: Twitter] <http://twitter.com/skysql> [image: Blog]<http://www.skysql.com/blog/> [image: Facebook] <http://www.facebook.com/skysql> [image: LinkedIn]<http://www.linkedin.com/company/1214250> [image: Google+] <https://plus.google.com/117544963211695643458/posts>
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Jan, I think "InnoDB file space defragmentation" could use significantly more explanation. By my understanding, delete-marked rows are in fact "physically deleted" (re-linked to the garbage record list) during purge. However, the garbage records in the list are not reclaimed unless: a. the first record in the list has enough usable space for a newly inserted records (potentially leaving some of the record's space unused); or b. the page becomes full and gets re-organized, freeing all garbage space implicitly. Of course then even in the best case the page is still "used" and not free. Freeing unused file space is an entirely different problem, involving potentially relocating used pages to be more contiguous, and then reducing the free limit and then the page count for the space. They seem to be entirely different problems and without much overlap at all. They are both potentially useful, and interesting. (Although I am not sure why to make it an external tool as opposed to internal functionality, that seems unnecessary.) Can you explain your ideas/goals for this? Regards, Jeremy On Fri, Apr 11, 2014 at 9:41 AM, Pantelis Theodosiou <ypercube@gmail.com>wrote:
Adam, you are right, bulk load project is 5835:
(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5835
On Fri, Apr 11, 2014 at 5:28 PM, Adam Scott <adam.c.scott@gmail.com>wrote:
Second link is the same as the first one: MDEV-5834
On Wed, Mar 12, 2014 at 5:59 AM, Jan Lindström <jplindst@mariadb.org>wrote:
Hi all,
After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB
(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834
(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834
Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.
R:
--
Jan Lindström, Principal Engineer SkySQL - The MariaDB Company
MariaDB | MaxScale | skype: jan_p_lindstrom
www.skysql.com
[image: Twitter] <http://twitter.com/skysql> [image: Blog]<http://www.skysql.com/blog/> [image: Facebook] <http://www.facebook.com/skysql> [image: LinkedIn]<http://www.linkedin.com/company/1214250> [image: Google+] <https://plus.google.com/117544963211695643458/posts>
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Hi,
Jan,
I think "InnoDB file space defragmentation" could use significantly more explanation.
By my understanding, delete-marked rows are in fact "physically deleted" (re-linked to the garbage record list) during purge. However, the garbage records in the list are not reclaimed unless: a. the first record in the list has enough usable space for a newly inserted records (potentially leaving some of the record's space unused); or b. the page becomes full and gets re-organized, freeing all garbage space implicitly. Of course then even in the best case the page is still "used" and not free.
This all is true.
Freeing unused file space is an entirely different problem, involving potentially relocating used pages to be more contiguous, and then reducing the free limit and then the page count for the space.
This is what I was proposing with defragmentation, either physically freeing unused pages or relocating used pages to be more contiguous, reducing the free limit and page count.
They seem to be entirely different problems and without much overlap at all. They are both potentially useful, and interesting. (Although I am not sure why to make it an external tool as opposed to internal functionality, that seems unnecessary.)
Both are useful, but you need to start from somewhere. I could use internal functionality if there is suitable syntax that can be used to do this already, I would not like to extend SQL-again. R: Jan -- -- Jan Lindström, Principal Engineer SkySQL - The MariaDB Company
participants (14)
-
Adam Scott -
Colin Charles -
Daniel Black -
Federico Razzoli -
Hartmut Holzgraefe -
Jan Kirchhoff -
Jan Lindström -
Jeremy Cole -
Kristian Nielsen -
Laurynas Biveinis -
Pantelis Theodosiou -
Reindl Harald -
Roberto Spadim -
Sergei Golubchik