Hi, I understand your point, however changing defaults is always a dangerous thing to do because change has always a effect on many users. R: Jan

Am 28.01.2014 13:38, schrieb Jan Lindström:

...

I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.

Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list. "innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"

it is also mandatory for compressed tables

for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority

-- -- Jan Lindström Principal Engineer MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com http://www.skysql.com/ Twitter http://twitter.com/skysql Blog http://www.skysql.com/blog/ Facebook http://www.facebook.com/skysql LinkedIn http://www.linkedin.com/company/1214250 Google+ https://plus.google.com/117544963211695643458/posts

do you have time to test toku engine? i'm trying it and it's very nice, i'm considering changing my tables from innodb to toku really soon 2014-01-28 Reindl Harald :

Am 28.01.2014 13:38, schrieb Jan Lindström:

...

I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.

Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.

"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"

it is also mandatory for compressed tables

for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

yes, for myisam with important data, aria engine is nice too, i changed some important tables from myisam to aria some years ago, and had no problem , some bugs repaired by mariadb team, but works nice :) 2014-01-29 Reindl Harald :

no, i use InnoDB only for DBMail and most other things are and will be MyISAM in the foreseeable future because i do not appreciate global namespaces instead having a folder per database which is in doubt the easiest restore

having 6000 mostly small tables in 250 databases is perfectly suiteable with MyISAM

Am 29.01.2014 16:14, schrieb Roberto Spadim:

...

do you have time to test toku engine? i'm trying it and it's very nice, i'm considering changing my tables from innodb to toku really soon

2014-01-28 Reindl Harald :

...

Am 28.01.2014 13:38, schrieb Jan Lindström:

...

I would like to know the most affecting InnoDB pain points the customers/potential customers first refer. By pain point, I mean anything that causes concerns, restricts sales etc, pain point can be bug, missing feature, usage problem, performance etc. My goal on asking this question is to find out what are the real issues customers or potential customer are struggling or issues that block customers using MariaDB. After that I will see what I can do about the situation and plan forward to address the raised issues with proper priorities.

Dean, if this list does to reach the support engineers and technical sales teams please forward to more suitable list.

"innodb_file_per_table" should be clearly default because doing it later is really a pain as well as if you remove large amount of data that the space never comes back while with file_per_table you can do "optimize table"

it is also mandatory for compressed tables

for sure there are setups where the potential fragmentation may be a issue, but i am sure they are not the majority

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

Am 05.02.2014 16:03, schrieb Federico Razzoli:

I just read this news from SkySQL:

http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh... "XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin" This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB?

that would be really bad servers with xtradb-configs would no loger start and it is not really funny have to care on each update how defaults have changed - nobody is changing a storage engine for fun the following line is for good reasons in my MariaDB RPM-SPEC beause i do not need any plugins and XtraDB is fine and tested rm -rf %{buildroot}%{_libdir}/mysql/plugin/

On 5 Feb 2014, at 23:44, Sergei Golubchik wrote:

Hi, Federico!

On Feb 05, Federico Razzoli wrote:

...

I just read this news from SkySQL:

http://www.skysql.com/blogs/chriscalender/mariadb-1007-overview-and-highligh...

"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"

This is confusing to me. When 10.0 is stable, will XtraDB be the default again? If not, can someone summarize the advantages/disadvantages of using XtraDB?

It is not about advantages and disadvantages at the moment.

But XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).

Have we reported bugs upstream? Do we plan on fixing this ourselves if upstream doesn't?

I hope we'll have XtraDB as the default in 10.0.

Regards, Sergei

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

-- Colin Charles, Chief Evangelist MariaDB | t: +6-012-204-3201 | Skype: colincharles

Sergei, Colin, MariaDB -

...

...

...

"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"

XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).

Have we reported bugs upstream?

https://bugs.launchpad.net/bugs/1276963

...

Do we plan on fixing this ourselves if upstream doesn't?

Not at the moment - that code is quite complex.

In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you? We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you? -- Laurynas

Hi, Laurynas! On Feb 07, Laurynas Biveinis wrote:

Sergei, Colin, MariaDB -

...

...

...

...

"XtraDB storage engine was upgraded to the 5.6 version. Now one can use XtraDB with MariaDB 10.0. Unlike MariaDB 5.5, in 10.0 XtraDB is not the default engine, the default is InnoDB, and XtraDB is available as a dynamic plugin"

XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).

Have we reported bugs upstream?

https://bugs.launchpad.net/bugs/1276963

...

Do we plan on fixing this ourselves if upstream doesn't?

Not at the moment - that code is quite complex.

In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you?

That kind of worked. XtraDB compiled. But query_response_time failed with "unable to find a register to spill in class 'GENERAL_REGS'". Still better than before. In the worst case, I'll enable XtraDB and disable query_response_time plugin.

We have another platform-specific addition: thread-local storage implemented by __thread GCC keyword, which is GNU specific. This is

Yes, I've noticed. It caused lots of valgrind warnings because __thread implementation uses __tls_get_addr() in glibc and that allocated memory which valgrind didn't see being freed. I've added a suppression for it.

used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you?

Besides this valgrind warning there were no issues so far. Or do you mean a different behavior of innodb_sched_priority_* options? I think it's ok for us, if it's ok for you :) Regards, Sergei

Hi, Laurynas! On Feb 10, Laurynas Biveinis wrote:

Sergei -

...

...

...

...

...

XtraDB simply does not compile on all our builders - Percona has introduced patches that use CPU atomic ops and didn't implement a fallback for setups where they are not available (like all the rest of the code does, including InnoDB).

Have we reported bugs upstream?

https://bugs.launchpad.net/bugs/1276963

...

Do we plan on fixing this ourselves if upstream doesn't?

Not at the moment - that code is quite complex.

In the bug report you mention RHEL 5 with GCC 4.1.2 (presumably on 32 bits). This is one of our supported platforms too, and we solve this by adding -march=i686 which makes the necessary builtins available. Would that work for you?

That kind of worked. XtraDB compiled. But query_response_time failed with "unable to find a register to spill in class 'GENERAL_REGS'".

Still better than before. In the worst case, I'll enable XtraDB and disable query_response_time plugin.

Pushing down -march=i686 to xtradb dir only would work around this, wouldn't it?

Yes. I wasn't quite sure whether it's a good idea to build different parts of the server with different -mtune settings. But ok, I'll try that.

...

...

used to implement the relative XtraDB thread priorities: http://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance.... On non-Linux plaftorms the feature should compile but should be silently disabled, that is, setting innodb_sched_priority_* options are no-ops. How big is this an issue for you?

Or do you mean a different behavior of innodb_sched_priority_* options? I think it's ok for us, if it's ok for you :)

Yes, I mean the different behavior. It's OK for us because the behavior is correct on all the platforms we support, but it wouldn't be OK on some of the platforms you support, would it?

I believe it's ok anyway. MySQL always used to have platform-specific features, like thread priorities, connections that use shared memory, and so on. Regards, Sergei

Hi all, Based on discussions on Percona Live I have added two more projects of interest: (1) Support > 16K pages on InnoDB: https://mariadb.atlassian.net/browse/MDEV-6075 (2) Persistent auto increment on InnoDB: https://mariadb.atlassian.net/browse/MDEV-6076 This to let you vote for your favorite.

Hi all,

After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB

(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834

(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834

Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.

R:

R: -- Jan Lindström, Principal Engineer SkySQL - The MariaDB Company MariaDB | MaxScale | skype: jan_p_lindstrom www.skysql.com http://www.skysql.com/ Twitter http://twitter.com/skysql Blog http://www.skysql.com/blog/ Facebook http://www.facebook.com/skysql LinkedIn http://www.linkedin.com/company/1214250 Google+ https://plus.google.com/117544963211695643458/posts

Adam, you are right, bulk load project is 5835: (2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5835 On Fri, Apr 11, 2014 at 5:28 PM, Adam Scott wrote:

Second link is the same as the first one: MDEV-5834

On Wed, Mar 12, 2014 at 5:59 AM, Jan Lindström wrote:

...

Hi all,

After careful weighting and selection process, I have selected following two projects as a starting point to improve InnoDB

(1) InnoDB file space defragmentation Description: External tool to physically delete delete marked rows from InnoDB file space and freeing unused file space. https://mariadb.atlassian.net/browse/MDEV-5834

(2) InnoDB fast bulk load Description: External tool to load CVS file directly to InnoDB file space (innodb_file_per_table = 1). https://mariadb.atlassian.net/browse/MDEV-5834

Now, I hope these projects mostly represent the most frequently requested features and are most usable to current customers and/or potential new customers. Next, I would hope some indication which one of the selected project you would like to see first. Votes can be added above links.

R:

--

Jan Lindström, Principal Engineer SkySQL - The MariaDB Company

MariaDB | MaxScale | skype: jan_p_lindstrom

www.skysql.com

[image: Twitter] http://twitter.com/skysql [image: Blog]http://www.skysql.com/blog/ [image: Facebook] http://www.facebook.com/skysql [image: LinkedIn]http://www.linkedin.com/company/1214250 [image: Google+] https://plus.google.com/117544963211695643458/posts

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

Hi,

Jan,

I think "InnoDB file space defragmentation" could use significantly more explanation.

By my understanding, delete-marked rows are in fact "physically deleted" (re-linked to the garbage record list) during purge. However, the garbage records in the list are not reclaimed unless: a. the first record in the list has enough usable space for a newly inserted records (potentially leaving some of the record's space unused); or b. the page becomes full and gets re-organized, freeing all garbage space implicitly. Of course then even in the best case the page is still "used" and not free.

This all is true.

Freeing unused file space is an entirely different problem, involving potentially relocating used pages to be more contiguous, and then reducing the free limit and then the page count for the space.

This is what I was proposing with defragmentation, either physically freeing unused pages or relocating used pages to be more contiguous, reducing the free limit and page count.

They seem to be entirely different problems and without much overlap at all. They are both potentially useful, and interesting. (Although I am not sure why to make it an external tool as opposed to internal functionality, that seems unnecessary.)

Both are useful, but you need to start from somewhere. I could use internal functionality if there is suitable syntax that can be used to do this already, I would not like to extend SQL-again. R: Jan -- -- Jan Lindström, Principal Engineer SkySQL - The MariaDB Company