[Maria-discuss] MariaDB Server 10.3 notes
Hi, Sorry this is a little later than expected (who couldn’t resist walking around Amsterdam on that late beautiful Saturday afternoon?)! This session was led by Rasmus whom while employed by MariaDB Corporation, was also the MariaDB Foundation Board member leading it. This was the last session on the formal schedule before lunch (more was added to it as well later on) I notice he’s also blogged [part 1](https://mariadb.org/thoughts-mariadb-server-10-3-mariadb-developers-meeting-...), and maybe these notes help for part 2 (or maybe they don’t). Scroll down for what is still coming in MariaDB Server 10.2 of course (post-GA even)! # MariaDB Server 10.3 * MDEV-10177 * MDEV-371 * MDEV-7773 - create aggregate functions in SQL * User defined types (pluggable data type API) * CJK language support - MDEV-10267, MDEV-10268 * SPIDER storage engine - MDEV-7698 * Further MyRocks development (10.2 will have it as experimental; MariaDB Corporation is looking to hire a dedicated engineer to work on MyRocks) * InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says there's next to no changes in InnoDB 8... Instant add column. New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, check). Lock wait policy (contribution) * Pluggable parser - MDEV-10142, PL/SQL parser, Oracle syntax support, INTERSECT, EXCLUDE * Sequences (like PostgreSQL implementation) * more for window functions (user defined window functions, MDEV-10855) * more for CTEs (CTE sharing isn't in now, and MySQL 8 has it, so needs to be done; Writable CTEs) * replace yaSSL with GnuTLS (SChannel in Windows) * "always encrypt" on the client side - Booking says there is no particular interest in this at the moment. * socket authentication * X Protocol/Document store - only if people have time or money will it be done * Peter asks if there is any plans to support other languages like V8? Supporting legacy database like Oracle PL/SQL is nice, but what about JavaScript and stuff? Eric mentions his old work on Perl and Java external language stored procedures. Peter says, if JSON and JavaScript are your lingua franca, PL/SQL isn't so interesting - but there was no interest in this amongst the folk. * No slave left behind patch - MDEV-8112 (Booking wants a bit more than the current patch, some extensions) - maybe in 10.2 * Compressed binary log (from Tencent) * For ALTER TABLE show the corresponding CREATE TABLE statement as a comment in the binary log. This is an annotation. * Counters for unsafe statements in replication (maybe in 10.2) * Extended table map in RBR is missing * Galera 4, encryption of gcache * Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 * Indexes on expressions (this is part of virtual columns, will it not go into 10.2? Check with Serg) * Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only works with row based replication. Talk about what to name it. There's already a MySQL time machine on github. Many like the name Rewind (but not Monty). Let's do a poll on the mailing list * Additional GIS functions to stay compatible. Also it would be good to have a standalone GIS library (Georg suggests; wlad isn't too happy with the suggestion). Georg suggests that calculations should use the reference systems (Unflatten the world - MariaDB Server GIS the world looks flat) * Optimizer trace - MDEV-6111 * Query rewriting - MDEV-5561 * Replace libmysqlclient with Connector/C - MDEV-9055 - embedded/replication/ storage engine (connect, spider, federatedX) - work continues from 10.2 * Invisible indexes from MySQL 8 - this is something Eric wants to see * With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better * ORDER BY LIMIT optimizer bugs (MDEV-8306) # Still in MariaDB Server 10.2 * Flashback DML - MDEV-10570 * Time delayed replication - MDEV-7145 * MyRocks (10.2.4 - maturity experimental) * GeoJSON (in 10.2 after beta) and ST_Geohash * sha256_password plugin (after GA) - MDEV-9804 - Serg will do it in a MySQL compatible way * sys schema (after GA) * Server system default values MDEV-7635 * MySQL deprecated `DES_ENCRYPT`, `DES_DECRYPT` - Monty says he knows a high level customer using it, a bank. * Column compression from Tencent? -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi
Hi, all! Thanks, Colin! Just for the sake of everyone who wasn't present there, I want to remind that these were discussions and plans, not promises. Nothing of this (written below) is guaranteed to happen, although certain items have a high probability of being done as planned. On Oct 12, Colin Charles wrote:
Hi,
Sorry this is a little later than expected (who couldn’t resist walking around Amsterdam on that late beautiful Saturday afternoon?)! This session was led by Rasmus whom while employed by MariaDB Corporation, was also the MariaDB Foundation Board member leading it. This was the last session on the formal schedule before lunch (more was added to it as well later on)
I notice he’s also blogged [part 1](https://mariadb.org/thoughts-mariadb-server-10-3-mariadb-developers-meeting-...), and maybe these notes help for part 2 (or maybe they don’t). Scroll down for what is still coming in MariaDB Server 10.2 of course (post-GA even)!
# MariaDB Server 10.3 * MDEV-10177 * MDEV-371 * MDEV-7773 - create aggregate functions in SQL * User defined types (pluggable data type API) * CJK language support - MDEV-10267, MDEV-10268 * SPIDER storage engine - MDEV-7698 * Further MyRocks development (10.2 will have it as experimental; MariaDB Corporation is looking to hire a dedicated engineer to work on MyRocks) * InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says there's next to no changes in InnoDB 8... Instant add column. New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, check). Lock wait policy (contribution) * Pluggable parser - MDEV-10142, PL/SQL parser, Oracle syntax support, INTERSECT, EXCLUDE * Sequences (like PostgreSQL implementation) * more for window functions (user defined window functions, MDEV-10855) * more for CTEs (CTE sharing isn't in now, and MySQL 8 has it, so needs to be done; Writable CTEs) * replace yaSSL with GnuTLS (SChannel in Windows) * "always encrypt" on the client side - Booking says there is no particular interest in this at the moment. * socket authentication * X Protocol/Document store - only if people have time or money will it be done * Peter asks if there is any plans to support other languages like V8? Supporting legacy database like Oracle PL/SQL is nice, but what about JavaScript and stuff? Eric mentions his old work on Perl and Java external language stored procedures. Peter says, if JSON and JavaScript are your lingua franca, PL/SQL isn't so interesting - but there was no interest in this amongst the folk. * No slave left behind patch - MDEV-8112 (Booking wants a bit more than the current patch, some extensions) - maybe in 10.2 * Compressed binary log (from Tencent) * For ALTER TABLE show the corresponding CREATE TABLE statement as a comment in the binary log. This is an annotation. * Counters for unsafe statements in replication (maybe in 10.2) * Extended table map in RBR is missing * Galera 4, encryption of gcache * Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 * Indexes on expressions (this is part of virtual columns, will it not go into 10.2? Check with Serg) * Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only works with row based replication. Talk about what to name it. There's already a MySQL time machine on github. Many like the name Rewind (but not Monty). Let's do a poll on the mailing list * Additional GIS functions to stay compatible. Also it would be good to have a standalone GIS library (Georg suggests; wlad isn't too happy with the suggestion). Georg suggests that calculations should use the reference systems (Unflatten the world - MariaDB Server GIS the world looks flat) * Optimizer trace - MDEV-6111 * Query rewriting - MDEV-5561 * Replace libmysqlclient with Connector/C - MDEV-9055 - embedded/replication/ storage engine (connect, spider, federatedX) - work continues from 10.2 * Invisible indexes from MySQL 8 - this is something Eric wants to see * With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better * ORDER BY LIMIT optimizer bugs (MDEV-8306)
# Still in MariaDB Server 10.2 * Flashback DML - MDEV-10570 * Time delayed replication - MDEV-7145 * MyRocks (10.2.4 - maturity experimental) * GeoJSON (in 10.2 after beta) and ST_Geohash * sha256_password plugin (after GA) - MDEV-9804 - Serg will do it in a MySQL compatible way * sys schema (after GA) * Server system default values MDEV-7635 * MySQL deprecated `DES_ENCRYPT`, `DES_DECRYPT` - Monty says he knows a high level customer using it, a bank. * Column compression from Tencent?
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
* InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says
Questions comments are inline: there's next to no changes in InnoDB 8... Instant add column. New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, check). Lock wait policy (contribution) Monty is *notorious* for low balling estimates. His famous phrase is "it is trivial". Everybody knows that if Monty says it is trivial, you can add 10x the work to get it done, or more. I am loathe to bring up CS, but it is a perfect example. You have to look no further than the time estimates for the CS project. The CS team was forced to accept an unacceptable timeline based on poor estimates by non-CS team members, and when the team couldn't meet arbitrary deadlines an alpha was shipped that should never have touched customer systems, and could have damaged them. InnoDB 8 also includes transportable tablespaces for partitioned tables, and will likely support native partitioning by the time 8 rolls. Native partitioning also entails implementing the changes to the SE. See: http://mysqlserverteam.com/innodb-native-partitioning-early-access/
* Pluggable parser - MDEV-10142, PL/SQL parser, Oracle syntax support, INTERSECT, EXCLUDE FULL OUTER JOIN? Table functions?
* more for window functions (user defined window functions, MDEV-10855) What other databases have user definable window functions? Will the syntax be similar? I can't really think of a reason to need custom window functions personally, as the existing set is very comprehensive. What kind of custom window function do you think people would need to write? In most implementations any aggregate functions can be used in a window frame context, thus any aggregate SQL functions should satisfy requirements for user defined window funcs.
* more for CTEs (CTE sharing isn't in now, and MySQL 8 has it, so needs to be done; Writable CTEs) I assume writable CTE will have the same restrictions as updateable views?
* "always encrypt" on the client side - Booking says there is no particular interest in this at the moment. Other RDBMS are rolling this out though. Microsoft has a new ADO.NET layer that supports end-to-end encryption from data at rest, to where it is accessed in the client.
* socket authentication Can you explain this. Is there an MDEV?
* X Protocol/Document store - only if people have time or money will it be done If you want people to pay for it, perhaps you should implement it in MaxScale instead of the server :)
* Peter asks if there is any plans to support other languages like V8? Supporting legacy database like Oracle PL/SQL is nice, but what about JavaScript and stuff? Eric mentions his old work on Perl and Java external language stored procedures. Peter says, if JSON and JavaScript are your lingua franca, PL/SQL isn't so interesting - but there was no interest in this amongst the folk.
* Compressed binary log (from Tencent) Compressing the binary log saves on space on the master, but it makes seeking into binary logs much more difficult, and searching backwards
PLEASE PLEASE PLEASE PLEASE IMPLEMENT WL-820. External stored procs and table functions. Been sitting there for the taking for a long time. Antony has tried to get you to get it into the server, but alas, it has never happened, and though there continues to be wide requests for this feature from the community, they fall on deaf ears. ALSO: PL/SQL support requires support for ARRAYS and other contructs that MySQL doesn't have. Without greatly improving the SQL language support, even basic Oracle 7 PL/SQL blocks won't work. MySQL cursor support is kind of lacking, and UPDATE WHERE CURRENT OF, etc, could be quite hard to implement. PLEASE PLEASE PLEASE add proper array support. PLEASE. through them becomes much more difficult (which has implications for query 'rewind')
* For ALTER TABLE show the corresponding CREATE TABLE statement as a comment in the binary log. This is an annotation. I guess this is to make streaming CDC easier, so that you can just stream changes without having a copy of the DDL downstream in an idempotent manner (updates that match no rows are inserts,etc?)
* Counters for unsafe statements in replication (maybe in 10.2) MySQL and MariaDB are notorious for marking statements unsafe that are actually safe. I'm not sure a counter would be of much use. I think SBR should be deprecated and ROW mode should be made the default. Any CDC related feature (like rewind) needs full RBR.
* Extended table map in RBR is missing Please explain. Is there an MDEV?
* Galera 4, encryption of gcache This is a pretty big bullet point in a small space. Adding support for Galera 4 while also making all the changes for InnoDB needs to be coordinated carefully.
* Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 Please actually complete XA support, don't just half fix it like Oracle did. Add full support for XA SUSPEND and XA RESUME and allow more than one thread to participate in a distributed transaction in the server. This will allow multiple transactions to share the same exact point-in-time so that multi-threaded applications can easily maintain consistent snapshots! This support could be exclusive to read-only transactions.
* Indexes on expressions (this is part of virtual columns, will it not go into 10.2? Check with Serg) Indexes on expressions requires parser support. create index expr_idx on some_table(a + b); select * from some_table where a+b > 30 and a+b <= 50 -- (uses range over expr_idx)
* Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only works with row based replication. Talk about what to name it. There's already a MySQL time machine on github. Many like the name Rewind (but not Monty). Let's do a poll on the mailing list Okay, Flashback query is VERY complicated. A flashback query is a materialized view. There are two ways generally to achieve flashback: a) materialize the query as is, and instead of rolling the query forward incrementally, you roll it backwards. Flexviews achieves this by computing the query as it is now, then computing the delta from a prior point in time until the transaction in which that computation happened. Then the delta is played back against the query, but the MONUS of each operation is applied which changes insertions to deletions and vice versa. This still presents a problem for OUTER JOIN. No documented asynchronous refresh algorithms exist for outer join that I'm aware of, that would work with the concept of reading row history from serialized changes. Flexviews uses the "rolling join propagation" algorithm, which only works with inner joins.
* Additional GIS functions to stay compatible. Also it would be good to have a standalone GIS library (Georg suggests; wlad isn't too happy with
* Query rewriting - MDEV-5561 I would like you to provide a SQL->DOM function call instead of just
* With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better It is certainly disheartening that Percona isn't responsive to MariaDB bugs, but I'm sure you understand that it is hard for a competitor to fix bugs for another competitor. MariaDB maintains a forked version of TokuDB. It isn't fair to expect the upstream vendor to fix bugs that your customers are paying you to support, does it? Perhaps you should pay a
b) provide a point-in-time snapshot of every table used in the query at the desired point in time, and run the query over those tables (this is a synchronous mechanism which supports OUTER JOIN). The problem here is how to provide such a table. The most straight-forward way is to copy the table, and then do the backwards replay for each (ideally in parallel) but this is obviously undesirable if the query is "select * from some_big_table join another_big_table on (...)" because you have to fully copy each table before you can undo changes. Otherwise you need to have the SE display old versions, just like it does for MVCC, but it has to display versions from binary logs, not undo logs, and this requires a lot of SE and engine changes! Mysql-time-machine on github is just a MySQL -> HBASE replicator. It relies on HBASE for flashback. I don't think you plan to embed HBASE in MariaDB, so adding DML flashback is a VERY BIG job. I've been working on Flexviews for 8 years and I just recently added flashback support to it! If you are going to have generic flashback, you might as well commit to incrementally refreshable materialized views. the suggestion). Georg suggests that calculations should use the reference systems (Unflatten the world - MariaDB Server GIS the world looks flat) GIS functions should use gdal, just like postgresql does. Then you can also add support for rasters. Here is an example of the awesomeness of postgresql and postgis. It uses SQL aggregate functions, table functions, CTE, GIS raster functions, sequences, etc: https://github.com/greenlion/osmvox/blob/master/postgresql/combined_schema.s... providing a DOM to plugins. This function could be exposed as a regular item function as well so that anything in the server can parse SQL. I suggest you implement my SQL "shim" interface and just provide some way to get a easy to iterate over parsed data structure from the SQL, such as a nested JSON array of objects. A nested array is generated by PHP-SQL-Parser and would provide a good template for such a JSON object. https://github.com/greenlion/PHP-SQL-Parser percentage of your support fees for TokuDB issues to Percona, or come to some other support agreement. Perhaps YOU should make the bug fixes and submit them to Percona. You could have just hired somebody to work on it at MariaDB like you plan to do so for MyRocks, but you didn't. Don't blame Percona for your failure to properly be a steward of an open source fork!
* ORDER BY LIMIT optimizer bugs (MDEV-8306) Oh god, this is a downward spiral every time somebody touches it. Fixing it correctly requires rewriting the parser, something that Oracle is undertaking.
What about other new MySQL 8 features? Are you getting rid of the .FRM nightmare? Are you going to support SET PERSIST? etc? -- Greenlion
* With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better
It is certainly disheartening that Percona isn't responsive to MariaDB bugs, but I'm sure you understand that it is hard for a competitor to fix bugs for another competitor. MariaDB maintains a forked version of TokuDB. It isn't fair to expect the upstream vendor to fix bugs that your customers are paying you to support, does it? Perhaps you should pay a percentage of your support fees for TokuDB issues to Percona, or come to some other support agreement. Perhaps YOU should make the bug fixes and submit them to Percona. You could have just hired somebody to work on it at MariaDB like you plan to do so for MyRocks, but you didn't. Don't blame Percona for your failure to properly be a steward of an open source fork!
This would be sad if that happened. Of course I can't say anything about internal issues but those I have followed on Jira regarding TokuDB have been fixed in upstream fairly quickly (even more the code has been updated to comply Marias standards/needs (like in case of memory allocation)). Also looking at MyRocks - in the current state it supports even less than toku ("No support for Partitions, Online DDL, Transportable Tablespace, Foreign Key, Spatial Index, and Fulltext Index") .. A major drawback (atleast until compressed binlogs are introduced) is the requirement for RBR as it in my case would skyrocket the used space. rr
Hi, Reinis! On Oct 12, Reinis Rozitis wrote:
* With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed.
This would be sad if that happened.
Of course I can't say anything about internal issues but those I have followed on Jira regarding TokuDB have been fixed in upstream fairly quickly (even more the code has been updated to comply Marias standards/needs (like in case of memory allocation)).
As far as I know, bugs get fixed, indeed. The memory allocation issue was a genuine bug in TokuDB, that caused failures in Percona Server 5.7. I did found it on MariaDB 10.0 first, indeed, but if it wouldn't affect Percona Server I wouldn't have reported it upstream at all :)
Also looking at MyRocks - in the current state it supports even less than toku ("No support for Partitions, Online DDL, Transportable Tablespace, Foreign Key, Spatial Index, and Fulltext Index") .. A major drawback (atleast until compressed binlogs are introduced) is the requirement for RBR as it in my case would skyrocket the used space.
The main reason to drop TokuDB would be it being redundant. If MyRocks and TokuDB would cover, basically, the same use case and MyRocks would've done it better - then nobody would need TokuDB, right? But you're saying that they have different use cases and even on the common use case, MyRocks is not necessarily better - do I understand you correctly? In that case we should be interesed in keeping TokuDB in MariaDB Server, not removing it. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
The main reason to drop TokuDB would be it being redundant. If MyRocks and TokuDB would cover, basically, the same use case and MyRocks would've done it better - then nobody would need TokuDB, right?
But you're saying that they have different use cases and even on the common use case, MyRocks is not necessarily better - do I understand you correctly? In that case we should be interesed in keeping TokuDB in MariaDB Server, not removing it.
It's true that in general it shouldn't matter for the end-user what the storage engine is called as far it has more or less the same (needed) features and delivers comparable performance (in the end it would be great if we could have only one for all purposes and workloads :)). But my concern is that at least currently publicly available Facebook version has several limitations ( https://github.com/facebook/mysql-5.6/wiki/MyRocks-limitations ) which (for me) would be problematic (for example no partitions, no online ddl and only row based replication (the locking is also interesting)) and seems strange since I would guess people (would) use theese engines mainly for compressing large tables (at least in some cases I can't even go back to compressed Innodb by how much the database size would inflate). Myself have tested Rocks only as mongo-rocks in MongoDB environment but for me as a user the features the engine has for Mysql seem subpar. Obviously there are bunch of issues with Toku and in the future RocksDB might get all the options people need - wanted to just share my opinion. rr
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA. Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut? On Wed, Oct 12, 2016 at 9:43 AM, Reinis Rozitis <r@roze.lv> wrote:
The main reason to drop TokuDB would be it being redundant. If MyRocks
and TokuDB would cover, basically, the same use case and MyRocks would've done it better - then nobody would need TokuDB, right?
But you're saying that they have different use cases and even on the
common use case, MyRocks is not necessarily better - do I understand you correctly? In that case we should be interesed in keeping TokuDB in MariaDB Server, not removing it.
It's true that in general it shouldn't matter for the end-user what the storage engine is called as far it has more or less the same (needed) features and delivers comparable performance (in the end it would be great if we could have only one for all purposes and workloads :)).
But my concern is that at least currently publicly available Facebook version has several limitations ( https://github.com/facebook/my sql-5.6/wiki/MyRocks-limitations ) which (for me) would be problematic (for example no partitions, no online ddl and only row based replication (the locking is also interesting)) and seems strange since I would guess people (would) use theese engines mainly for compressing large tables (at least in some cases I can't even go back to compressed Innodb by how much the database size would inflate).
Myself have tested Rocks only as mongo-rocks in MongoDB environment but for me as a user the features the engine has for Mysql seem subpar.
Obviously there are bunch of issues with Toku and in the future RocksDB might get all the options people need - wanted to just share my opinion.
rr
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts? as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
Hi, When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck. --Greenlion On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or
even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Am 12.10.2016 um 17:34 schrieb Justin Swanhart:
When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net <mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
0) Sergei and Justin - I prefer to avoid the "Monty says" game either for or against. We are trying to establish a community here. He is one of many. His reputation (good or bad) shouldn't be a stick that is deployed to win arguments. 1) RocksDB has persistent snapshots. That would make flashback trivial to implement for MyRocks assuming you took snapshots ahead of time. So maybe that isn't flashing back. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.h... 2) I expect almost everyone to choose MyRocks given a choice between MyRocks and TokuDB. I have evaluated TokuDB over several years. I have many performance and efficiency results from this year that I have yet to publish. Results are almost always much better with MyRocks, we have more resources dedicated to MyRocks so it will improve quickly. MariaDB is a community project, so if someone maintains TokuDB I assume it will be included in MariaDB. But I think time is better spent on MyRocks. Our docs need to be updated and our communication needs to be better. But we move fast and Yoshi has been updating the limitations wiki page as I write this (https://github.com/facebook/mysql-5.6/wiki/MyRocks-limitations ). I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts. 3) Partitioning via the partition storage engine is supported for MyRocks. My employer doesn't need it for the initial deployment. It might be needed elsewhere and I am sure the community needs it. I assume we also need native partitioning in MyRocks when we move up to MySQL 8. The limitations page is getting updated to explain this. 4) By default the FB MySQL branch doesn't allow mysqld to start if MyRocks and InnoDB are enabled. This was misinterpreted to mean it wasn't possible to use MyRocks and InnoDB at the same time. It is possible now. And MariaDB/Percona are free to always allow both to be enabled. It was previously prevented in the FB MySQL repo because of fear that internal users might try to do a transaction that spans engines. I think that is a bad idea and XA scares me. I assume evaluations for MariaDB Server and Percona Server require that both InnoDB and MyRocks are enabled concurrently. See https://github.com/facebook/mysql-5.6/issues/106 5) We have work in progress to support XA for MyRocks. Code has been merged. Perf tests are in progress. 6) We have work in progress to support online DDL. 7) MyRocks supports read-committed (RC) and repeatable-read (RR). RR in MyRocks is Postgres-style and I assume that it won't get much use, just like i assume it doesn't in Postgres. If we add gap-locking to MyRocks RR then it will match InnoDB. For details see https://github.com/mdcallag/mytools/wiki/Cursor-Isolation 8) I am a long time user of SBR and think that RBR is the future. MyRocks needs RBR because we expect most deployments to use it with read-committed and just like InnoDB, that needs RBR. If gap locking is added to MyRocks RR then SBR is feasible, but you give up all of the great new replication features that will be RBR only. http://dba.stackexchange.com/questions/101122/mysql-why-statement-based-binl... 9) MyRocks could do transportable column family for all indexes/tables in a column family. Transportable is a nice feature but I don't think the lack of it is a big deal. It is part of being storage efficient, but the overall storage efficiency story is much stronger for MyRocks than InnoDB. 10) I hope there is a plan for foreign key and spatial in MyRocks. There isn't one yet. I don't think full text in any MySQL engine is a good use of time given how infrequently it is used. On Wed, Oct 12, 2016 at 8:39 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 17:34 schrieb Justin Swanhart:
When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net
<mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
"Monty says it is trivial" is the oldest running joke in the community. I heard it in my first talk I attended, about adding data types to MySQL, a talk given by Brian Aker. :) But ok. I'll try to be nicer. I really think community is important, and nobody should be a pariah, or a punching bag. Sent from my iPhone
On Oct 12, 2016, at 12:25 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
0) Sergei and Justin - I prefer to avoid the "Monty says" game either for or against. We are trying to establish a community here. He is one of many. His reputation (good or bad) shouldn't be a stick that is deployed to win arguments.
1) RocksDB has persistent snapshots. That would make flashback trivial to implement for MyRocks assuming you took snapshots ahead of time. So maybe that isn't flashing back. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.h...
2) I expect almost everyone to choose MyRocks given a choice between MyRocks and TokuDB. I have evaluated TokuDB over several years. I have many performance and efficiency results from this year that I have yet to publish. Results are almost always much better with MyRocks, we have more resources dedicated to MyRocks so it will improve quickly.
MariaDB is a community project, so if someone maintains TokuDB I assume it will be included in MariaDB. But I think time is better spent on MyRocks. Our docs need to be updated and our communication needs to be better. But we move fast and Yoshi has been updating the limitations wiki page as I write this (https://github.com/facebook/mysql-5.6/wiki/MyRocks-limitations).
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
3) Partitioning via the partition storage engine is supported for MyRocks. My employer doesn't need it for the initial deployment. It might be needed elsewhere and I am sure the community needs it. I assume we also need native partitioning in MyRocks when we move up to MySQL 8. The limitations page is getting updated to explain this.
4) By default the FB MySQL branch doesn't allow mysqld to start if MyRocks and InnoDB are enabled. This was misinterpreted to mean it wasn't possible to use MyRocks and InnoDB at the same time. It is possible now. And MariaDB/Percona are free to always allow both to be enabled. It was previously prevented in the FB MySQL repo because of fear that internal users might try to do a transaction that spans engines. I think that is a bad idea and XA scares me. I assume evaluations for MariaDB Server and Percona Server require that both InnoDB and MyRocks are enabled concurrently. See https://github.com/facebook/mysql-5.6/issues/106
5) We have work in progress to support XA for MyRocks. Code has been merged. Perf tests are in progress.
6) We have work in progress to support online DDL.
7) MyRocks supports read-committed (RC) and repeatable-read (RR). RR in MyRocks is Postgres-style and I assume that it won't get much use, just like i assume it doesn't in Postgres. If we add gap-locking to MyRocks RR then it will match InnoDB. For details see https://github.com/mdcallag/mytools/wiki/Cursor-Isolation
8) I am a long time user of SBR and think that RBR is the future. MyRocks needs RBR because we expect most deployments to use it with read-committed and just like InnoDB, that needs RBR. If gap locking is added to MyRocks RR then SBR is feasible, but you give up all of the great new replication features that will be RBR only. http://dba.stackexchange.com/questions/101122/mysql-why-statement-based-binl...
9) MyRocks could do transportable column family for all indexes/tables in a column family. Transportable is a nice feature but I don't think the lack of it is a big deal. It is part of being storage efficient, but the overall storage efficiency story is much stronger for MyRocks than InnoDB.
10) I hope there is a plan for foreign key and spatial in MyRocks. There isn't one yet. I don't think full text in any MySQL engine is a good use of time given how infrequently it is used.
On Wed, Oct 12, 2016 at 8:39 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 17:34 schrieb Justin Swanhart: When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net <mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
MyRocks now supports group commit and XA - https://github.com/facebook/mysql-5.6/issues/25 On Wed, Oct 12, 2016 at 9:25 AM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
0) Sergei and Justin - I prefer to avoid the "Monty says" game either for or against. We are trying to establish a community here. He is one of many. His reputation (good or bad) shouldn't be a stick that is deployed to win arguments.
1) RocksDB has persistent snapshots. That would make flashback trivial to implement for MyRocks assuming you took snapshots ahead of time. So maybe that isn't flashing back. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for- efficient-snapshots.html
2) I expect almost everyone to choose MyRocks given a choice between MyRocks and TokuDB. I have evaluated TokuDB over several years. I have many performance and efficiency results from this year that I have yet to publish. Results are almost always much better with MyRocks, we have more resources dedicated to MyRocks so it will improve quickly.
MariaDB is a community project, so if someone maintains TokuDB I assume it will be included in MariaDB. But I think time is better spent on MyRocks. Our docs need to be updated and our communication needs to be better. But we move fast and Yoshi has been updating the limitations wiki page as I write this (https://github.com/facebook/mysql-5.6/wiki/MyRocks-limitations ).
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
3) Partitioning via the partition storage engine is supported for MyRocks. My employer doesn't need it for the initial deployment. It might be needed elsewhere and I am sure the community needs it. I assume we also need native partitioning in MyRocks when we move up to MySQL 8. The limitations page is getting updated to explain this.
4) By default the FB MySQL branch doesn't allow mysqld to start if MyRocks and InnoDB are enabled. This was misinterpreted to mean it wasn't possible to use MyRocks and InnoDB at the same time. It is possible now. And MariaDB/Percona are free to always allow both to be enabled. It was previously prevented in the FB MySQL repo because of fear that internal users might try to do a transaction that spans engines. I think that is a bad idea and XA scares me. I assume evaluations for MariaDB Server and Percona Server require that both InnoDB and MyRocks are enabled concurrently. See https://github.com/facebook/mysql-5.6/issues/106
5) We have work in progress to support XA for MyRocks. Code has been merged. Perf tests are in progress.
6) We have work in progress to support online DDL.
7) MyRocks supports read-committed (RC) and repeatable-read (RR). RR in MyRocks is Postgres-style and I assume that it won't get much use, just like i assume it doesn't in Postgres. If we add gap-locking to MyRocks RR then it will match InnoDB. For details see https://github.com/ mdcallag/mytools/wiki/Cursor-Isolation
8) I am a long time user of SBR and think that RBR is the future. MyRocks needs RBR because we expect most deployments to use it with read-committed and just like InnoDB, that needs RBR. If gap locking is added to MyRocks RR then SBR is feasible, but you give up all of the great new replication features that will be RBR only. http://dba.stackexchange.com/questions/ 101122/mysql-why-statement-based-binlog-format-cannot- work-with-innodb-read-uncommitte
9) MyRocks could do transportable column family for all indexes/tables in a column family. Transportable is a nice feature but I don't think the lack of it is a big deal. It is part of being storage efficient, but the overall storage efficiency story is much stronger for MyRocks than InnoDB.
10) I hope there is a plan for foreign key and spatial in MyRocks. There isn't one yet. I don't think full text in any MySQL engine is a good use of time given how infrequently it is used.
On Wed, Oct 12, 2016 at 8:39 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 17:34 schrieb Justin Swanhart:
When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net
<mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
-- Mark Callaghan mdcallag@gmail.com
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
True but one can see also in your own comment ( https://github.com/facebook/mysql-5.6/issues/25#issuecomment-218494710 ) that some tests in different circumstances can have rather skewed results which people base their decisions on. Glad it has been fixed (would be nice to see some current bench tests). rr
If your workload is insert-only and you are happy with TokuDB then please don't try to migrate. But most workloads have reads concurrent with writes. Add that to insert benchmark and TokuDB becomes unhappy. Why do we always get results for insert-only iibench but not for insert+query iibench? My client supports it - https://github.com/mdcallag/mytools/blob/master/bench/ibench/iibench.py I think insert-only iibench is a silly workload with results that are usually taken out of context. MyRocks is great on it, with and without concurrent queries. So I guess I am happy about that. On Wed, Oct 12, 2016 at 12:07 PM, Reinis Rozitis <r@roze.lv> wrote:
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
True but one can see also in your own comment ( https://github.com/facebook/mysql-5.6/issues/25#issuecomment-218494710 ) that some tests in different circumstances can have rather skewed results which people base their decisions on.
Glad it has been fixed (would be nice to see some current bench tests).
rr
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
And if I sound annoyed when talking about TokuDB performance, I am a bit. I wish someone else documented the performance problems it has. The problems are known but not much has been shared to explain them. That has been left to me as I am promoting the other write-optimized storage engine for MySQL. A lot of extremely talented people worked on TokuDB. A few (hello Rich) continue to help with it. One of them is related to me (hello Tim). One of them works for my employer (Zardosht). I wish more of them worked for my employer (anon, anon, anon). Now I get to be the jerk. On Wed, Oct 12, 2016 at 2:51 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
If your workload is insert-only and you are happy with TokuDB then please don't try to migrate. But most workloads have reads concurrent with writes. Add that to insert benchmark and TokuDB becomes unhappy. Why do we always get results for insert-only iibench but not for insert+query iibench? My client supports it - https://github.com/mdcallag/ mytools/blob/master/bench/ibench/iibench.py
I think insert-only iibench is a silly workload with results that are usually taken out of context. MyRocks is great on it, with and without concurrent queries. So I guess I am happy about that.
On Wed, Oct 12, 2016 at 12:07 PM, Reinis Rozitis <r@roze.lv> wrote:
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
True but one can see also in your own comment ( https://github.com/facebook/mysql-5.6/issues/25#issuecomment-218494710 ) that some tests in different circumstances can have rather skewed results which people base their decisions on.
Glad it has been fixed (would be nice to see some current bench tests).
rr
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
-- Mark Callaghan mdcallag@gmail.com
Mariadb should understand the comparison between Myrocks and Tokudb, and let their current users decide what should be in the product. It is their product and their responsibility. Unfortunately, I am no longer working on database software as my primary focus, so I can not make any significant changes to the Tokudb software; it is what it is. On Wed, Oct 12, 2016 at 6:32 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
And if I sound annoyed when talking about TokuDB performance, I am a bit. I wish someone else documented the performance problems it has. The problems are known but not much has been shared to explain them. That has been left to me as I am promoting the other write-optimized storage engine for MySQL. A lot of extremely talented people worked on TokuDB. A few (hello Rich) continue to help with it. One of them is related to me (hello Tim). One of them works for my employer (Zardosht). I wish more of them worked for my employer (anon, anon, anon). Now I get to be the jerk.
On Wed, Oct 12, 2016 at 2:51 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
If your workload is insert-only and you are happy with TokuDB then please don't try to migrate. But most workloads have reads concurrent with writes. Add that to insert benchmark and TokuDB becomes unhappy. Why do we always get results for insert-only iibench but not for insert+query iibench? My client supports it - https://github.com/mdcallag/ mytools/blob/master/bench/ibench/iibench.py
I think insert-only iibench is a silly workload with results that are usually taken out of context. MyRocks is great on it, with and without concurrent queries. So I guess I am happy about that.
On Wed, Oct 12, 2016 at 12:07 PM, Reinis Rozitis <r@roze.lv> wrote:
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
True but one can see also in your own comment ( https://github.com/facebook/mysql-5.6/issues/25#issuecomment-218494710 ) that some tests in different circumstances can have rather skewed results which people base their decisions on.
Glad it has been fixed (would be nice to see some current bench tests).
rr
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
-- Mark Callaghan mdcallag@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
just some ideas... -- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache) a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
I have a few questions 1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think). 2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options? -- Peter -- Webyog On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Now I remember that the name of the storage engine, I referred, was PBXT. -- Peter On Thu, Oct 13, 2016 at 12:00 PM, Peter Laursen <peter_laursen@webyog.com> wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
On 13.10.2016 12:00, Peter Laursen wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think). Personally, I plan to make this happen . At least RocksDB works on Windows, the port seems to be done by competent people (MS Bing folks) so that part compiles, a runs and was benchmarked as well. https://github.com/facebook/rocksdb/blob/4cab5ebecdb42ceff7030a7606b5b5c31ba...
I expect this port to be solid. As for TokuDB, if I remember correctly, its original team declared it to be compilable and runnable only on Linux x64 with "modern" C++ (back than it could have been gcc's idea of C++11) and absolutely necessary jemalloc , so I figured it was not worth looking at making a port. I even would speculate nobody has tried to compile TokuDB on Windows that far.
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br <mailto:roberto@spadim.com.br>> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss <https://launchpad.net/%7Emaria-discuss> Post to : maria-discuss@lists.launchpad.net <mailto:maria-discuss@lists.launchpad.net> Unsubscribe : https://launchpad.net/~maria-discuss <https://launchpad.net/%7Emaria-discuss> More help : https://help.launchpad.net/ListHelp <https://help.launchpad.net/ListHelp>
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
I think Sergei G. told me once that TokuDB would not compile with Visual Studio and that MariaDB would only try to fix it in MariaDB if Toku people wanted to collaborate on this (what they did not). it probably was before TokuDB was swalloed by Percona . -- Peter On Thu, Oct 13, 2016 at 1:36 PM, Vladislav Vaintroub <vvaintroub@gmail.com> wrote:
On 13.10.2016 12:00, Peter Laursen wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
Personally, I plan to make this happen . At least RocksDB works on Windows, the port seems to be done by competent people (MS Bing folks) so that part compiles, a runs and was benchmarked as well. https://github.com/facebook/rocksdb/blob/4cab5ebecdb42ceff7030a7606b5b5 c31ba702ef/WINDOWS_PORT.md I expect this port to be solid.
As for TokuDB, if I remember correctly, its original team declared it to be compilable and runnable only on Linux x64 with "modern" C++ (back than it could have been gcc's idea of C++11) and absolutely necessary jemalloc , so I figured it was not worth looking at making a port. I even would speculate nobody has tried to compile TokuDB on Windows that far.
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
.. so +1 for Rocks as compared to Toku: cross-platform support. -- Peter On Thu, Oct 13, 2016 at 1:49 PM, Peter Laursen <peter_laursen@webyog.com> wrote:
I think Sergei G. told me once that TokuDB would not compile with Visual Studio and that MariaDB would only try to fix it in MariaDB if Toku people wanted to collaborate on this (what they did not). it probably was before TokuDB was swalloed by Percona .
-- Peter
On Thu, Oct 13, 2016 at 1:36 PM, Vladislav Vaintroub <vvaintroub@gmail.com
wrote:
On 13.10.2016 12:00, Peter Laursen wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
Personally, I plan to make this happen . At least RocksDB works on Windows, the port seems to be done by competent people (MS Bing folks) so that part compiles, a runs and was benchmarked as well. https://github.com/facebook/rocksdb/blob/4cab5ebecdb42ceff70 30a7606b5b5c31ba702ef/WINDOWS_PORT.md I expect this port to be solid.
As for TokuDB, if I remember correctly, its original team declared it to be compilable and runnable only on Linux x64 with "modern" C++ (back than it could have been gcc's idea of C++11) and absolutely necessary jemalloc , so I figured it was not worth looking at making a port. I even would speculate nobody has tried to compile TokuDB on Windows that far.
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :) -Tim On Thu, Oct 13, 2016 at 7:49 AM, Peter Laursen <peter_laursen@webyog.com> wrote:
I think Sergei G. told me once that TokuDB would not compile with Visual Studio and that MariaDB would only try to fix it in MariaDB if Toku people wanted to collaborate on this (what they did not). it probably was before TokuDB was swalloed by Percona .
-- Peter
On Thu, Oct 13, 2016 at 1:36 PM, Vladislav Vaintroub <vvaintroub@gmail.com
wrote:
On 13.10.2016 12:00, Peter Laursen wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
Personally, I plan to make this happen . At least RocksDB works on Windows, the port seems to be done by competent people (MS Bing folks) so that part compiles, a runs and was benchmarked as well. https://github.com/facebook/rocksdb/blob/4cab5ebecdb42ceff70 30a7606b5b5c31ba702ef/WINDOWS_PORT.md I expect this port to be solid.
As for TokuDB, if I remember correctly, its original team declared it to be compilable and runnable only on Linux x64 with "modern" C++ (back than it could have been gcc's idea of C++11) and absolutely necessary jemalloc , so I figured it was not worth looking at making a port. I even would speculate nobody has tried to compile TokuDB on Windows that far.
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production? for development? seriously? develop a application on Windows which finally runs on a different OS? not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
On 10/15/2016 10:10 AM, Reindl Harald wrote:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
Probably just a few if any. Yet millions of students use MySQL/MariaDB on Windows. Regards, Igor.
for development? seriously? develop a application on Windows which finally runs on a different OS?
not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Am 15.10.2016 um 23:18 schrieb Igor Babaev:
On 10/15/2016 10:10 AM, Reindl Harald wrote:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
Probably just a few if any. Yet millions of students use MySQL/MariaDB on Windows
and they all did not hear about virtualization in the past 10 years?
There're financial software running windows without problems Em sábado, 15 de outubro de 2016, Reindl Harald <h.reindl@thelounge.net> escreveu:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
for development? seriously? develop a application on Windows which finally runs on a different OS?
not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
could you get rid of * top posting * convert a plaintext thread to html * reply all? Am 16.10.2016 um 04:09 schrieb Roberto Spadim:
There're financial software running windows withoutproblems
that is all nice - but the whole point of a database server which is reachable over network is that it can run on a dedicated machine and the client can by anything anywhere consider the development ressources which could be spent better like fixing bugs as "mysqld: 161016 5:16:22 [Note] /usr/libexec/mysqld (mysqld 10.0.27-MariaDB) starting as process 19795 ..." after a year still spit in the syslog instead in the mysql general log....
Em sábado, 15 de outubro de 2016, Reindl Harald <h.reindl@thelounge.net <mailto:h.reindl@thelounge.net>> escreveu:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
for development? seriously? develop a application on Windows which finally runs on a different OS?
not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
I was just adding context. I agree that Windows is very uninteresting in a production context, but not having a version of TokuDB for Windows certainly excluded people (academic and professional) from trying it out on their laptops. On Sat, Oct 15, 2016 at 1:10 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
for development? seriously? develop a application on Windows which finally runs on a different OS?
not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Doesn't the Bing team port rocks? Obviously there is interest in support for Windows in Redmond, but apparently not for TokuDB. The market decides. It has been demonstrated that TokuDB isn't precisely general purpose, so the desire to port it is lesser. Plenty of people run Windows and MySQL in production. They just aren't members of our community, because we tend to ostracize them for their OS choice. Sent from my iPhone
On Oct 16, 2016, at 8:45 AM, Tim Callaghan <tmcallaghan@gmail.com> wrote:
I was just adding context. I agree that Windows is very uninteresting in a production context, but not having a version of TokuDB for Windows certainly excluded people (academic and professional) from trying it out on their laptops.
On Sat, Oct 15, 2016 at 1:10 PM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 15.10.2016 um 17:22 schrieb Tim Callaghan: Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production?
for development? seriously? develop a application on Windows which finally runs on a different OS?
not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine)
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
2016-10-16 11:38 GMT-02:00 Justin Swanhart <greenlion@gmail.com>: > Doesn't the Bing team port rocks? Obviously there is interest in support for Windows in Redmond, but apparently not for TokuDB. > The market decides. It has been demonstrated that TokuDB isn't precisely general purpose, so the desire to port it is lesser. i think market use more innodb/myisam/ariadb than tokudb, and when someone need 'enterprise db' he/she try postgresql or mssql or oracle, at least that's what i see here, working with oracle give more money than mysql... i tested tokudb cause i was asking myself whats the performace of tokudb vs innodb, and with time i will test rocksdb too i tested column store with scada system and it worked really nice, many queries are faster, but i'm not sure if the software is stable for heavy use (turnoff computer with more than 100gb of data), for now i'm running innodb and testing column in some smaller production servers > Plenty of people run Windows and MySQL in production. They just aren't members of our community, because we tend to ostracize them for their OS choice. there're many, many guys using mysql and windows, brazil goverment produce software to track invoices using mysql and windows running in many (1mi+) machines, just an example, it's innodb based (why? i don't know, maybe "it just works" and it's not default engine?!) --- >could you get rid of * top posting (it was top post) * convert a plaintext thread to html (sorry i'm using gmail) * reply all? (i'm using reply all with gmail, cc to maria discuss list) -- Roberto Spadim
I was a software developer at Tokutek working on TokuDB. If I wanted to continue working on TokuDB, Tokutek had to make money to pay for the expenses of running a company. Supporting TokuDB on Windows led to absolutely no business, so why would I want to waste time supporting it? We would jump through hoops to support Windows (or any other platform), but the money to pay for it was not there. On Sat, Oct 15, 2016 at 11:22 AM, Tim Callaghan <tmcallaghan@gmail.com> wrote:
Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :)
-Tim
On Thu, Oct 13, 2016 at 7:49 AM, Peter Laursen <peter_laursen@webyog.com> wrote:
I think Sergei G. told me once that TokuDB would not compile with Visual Studio and that MariaDB would only try to fix it in MariaDB if Toku people wanted to collaborate on this (what they did not). it probably was before TokuDB was swalloed by Percona .
-- Peter
On Thu, Oct 13, 2016 at 1:36 PM, Vladislav Vaintroub < vvaintroub@gmail.com> wrote:
On 13.10.2016 12:00, Peter Laursen wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
Personally, I plan to make this happen . At least RocksDB works on Windows, the port seems to be done by competent people (MS Bing folks) so that part compiles, a runs and was benchmarked as well. https://github.com/facebook/rocksdb/blob/4cab5ebecdb42ceff70 30a7606b5b5c31ba702ef/WINDOWS_PORT.md I expect this port to be solid.
As for TokuDB, if I remember correctly, its original team declared it to be compilable and runnable only on Linux x64 with "modern" C++ (back than it could have been gcc's idea of C++11) and absolutely necessary jemalloc , so I figured it was not worth looking at making a port. I even would speculate nobody has tried to compile TokuDB on Windows that far.
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
With hyper-v becoming free, the arrival of the Linux subsystem, and docker for windows 10; there isn't much of an excuse to proclaim you are unable to use the Linux binaries. The Redis and Postgres devs are already rejoicing.
RocksDB runs on Windows. An external contributor keeps the build from breaking and uses it on Windows. On Thu, Oct 13, 2016 at 3:00 AM, Peter Laursen <peter_laursen@webyog.com> wrote:
I have a few questions
1) will RocksDB be available on Windows? ToduDB isn't (because it does not compile on Visual Studio I think).
2) How large a set of configuraton options will RocksDB have? InnoDB now has100+ I guess, and that is a mess IMO. I liked the simplicity of the now dead Primebase (if I remember the name right) storage engine in this respect. Also is there an overview somehere of current and planned configuration options?
-- Peter -- Webyog
On Thu, Oct 13, 2016 at 6:50 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
just some ideas...
-- at first time i read plugable parser, and i was thinking that it could allow nosql (handler socket/memcahce) protocol inside mysql "session" (connect + auth + allow communicate with server), removing the sql parser/optimizer without loosing mysql_connect() authentication/security
a "plugable" parser, could remove a lot of nosql client side/server side libs, maybe a nosql parser could be nice to be implemented like"mysql_crud_get()", "mysql_crud_set()", etc .. without installing a daemon plugin (handlersocket/memcache)
a plugable parser could allow a python/R/julia/anything to run a analytics algorithm direct to storage engine? i don't know if this could help at allow v8/others languages at server side but the idea sounds ok to me, something like mysql_v8_eval("v8 script"), just ideas...
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
Hello All, Back in the day, I was one of the Tokudb developers @ Tokutek. Two of the features that I worked on were read free replication (RFR) and online schema change for Tokudb. RFR has some nice slave performance benefits by the replacing read modify write algorithm with a write algorithm. This works because the replication stream includes the before and after rows. IMO, this would be nice to have with Myrocks. There is a Mariadb push request for RFR that works with Tokudb that may be used as the API for this feature for other storage engines. Online schema change in Tokudb only injects 'schema change messages' into the top of the fractal tree. This is very fast. The actual change in the encoded row format occurs much later when these messages are executed on the leaf nodes. This seems nice to me, but it can be replaced with other online schema change packages. It would be nice to compare these various method WRT to performance. Tokudb also supports large transactions defined as transactions that effect a large number of rows such that the memory footprint of the undo log for these operations spills to external storage. Personally, I don't like large transactions, and believe that they should be avoided. However, some people do use them, and the mysql server should be able to support them without crashing, perhaps with degraded performance. On Wed, Oct 12, 2016 at 12:25 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
0) Sergei and Justin - I prefer to avoid the "Monty says" game either for or against. We are trying to establish a community here. He is one of many. His reputation (good or bad) shouldn't be a stick that is deployed to win arguments.
1) RocksDB has persistent snapshots. That would make flashback trivial to implement for MyRocks assuming you took snapshots ahead of time. So maybe that isn't flashing back. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for- efficient-snapshots.html
2) I expect almost everyone to choose MyRocks given a choice between MyRocks and TokuDB. I have evaluated TokuDB over several years. I have many performance and efficiency results from this year that I have yet to publish. Results are almost always much better with MyRocks, we have more resources dedicated to MyRocks so it will improve quickly.
MariaDB is a community project, so if someone maintains TokuDB I assume it will be included in MariaDB. But I think time is better spent on MyRocks. Our docs need to be updated and our communication needs to be better. But we move fast and Yoshi has been updating the limitations wiki page as I write this (https://github.com/facebook/mysql-5.6/wiki/MyRocks-limitations ).
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
3) Partitioning via the partition storage engine is supported for MyRocks. My employer doesn't need it for the initial deployment. It might be needed elsewhere and I am sure the community needs it. I assume we also need native partitioning in MyRocks when we move up to MySQL 8. The limitations page is getting updated to explain this.
4) By default the FB MySQL branch doesn't allow mysqld to start if MyRocks and InnoDB are enabled. This was misinterpreted to mean it wasn't possible to use MyRocks and InnoDB at the same time. It is possible now. And MariaDB/Percona are free to always allow both to be enabled. It was previously prevented in the FB MySQL repo because of fear that internal users might try to do a transaction that spans engines. I think that is a bad idea and XA scares me. I assume evaluations for MariaDB Server and Percona Server require that both InnoDB and MyRocks are enabled concurrently. See https://github.com/facebook/mysql-5.6/issues/106
5) We have work in progress to support XA for MyRocks. Code has been merged. Perf tests are in progress.
6) We have work in progress to support online DDL.
7) MyRocks supports read-committed (RC) and repeatable-read (RR). RR in MyRocks is Postgres-style and I assume that it won't get much use, just like i assume it doesn't in Postgres. If we add gap-locking to MyRocks RR then it will match InnoDB. For details see https://github.com/ mdcallag/mytools/wiki/Cursor-Isolation
8) I am a long time user of SBR and think that RBR is the future. MyRocks needs RBR because we expect most deployments to use it with read-committed and just like InnoDB, that needs RBR. If gap locking is added to MyRocks RR then SBR is feasible, but you give up all of the great new replication features that will be RBR only. http://dba.stackexchange.com/questions/ 101122/mysql-why-statement-based-binlog-format-cannot- work-with-innodb-read-uncommitte
9) MyRocks could do transportable column family for all indexes/tables in a column family. Transportable is a nice feature but I don't think the lack of it is a big deal. It is part of being storage efficient, but the overall storage efficiency story is much stronger for MyRocks than InnoDB.
10) I hope there is a plan for foreign key and spatial in MyRocks. There isn't one yet. I don't think full text in any MySQL engine is a good use of time given how infrequently it is used.
On Wed, Oct 12, 2016 at 8:39 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 17:34 schrieb Justin Swanhart:
When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net
<mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Rich - we have read-free replication although I am not sure on the status of it -- whether it is done or work-in-progress. https://github.com/facebook/mysql-5.6/wiki/Read-Free-Replication On Wed, Oct 12, 2016 at 1:18 PM, Rich Prohaska <prohaska7@gmail.com> wrote:
Hello All, Back in the day, I was one of the Tokudb developers @ Tokutek. Two of the features that I worked on were read free replication (RFR) and online schema change for Tokudb.
RFR has some nice slave performance benefits by the replacing read modify write algorithm with a write algorithm. This works because the replication stream includes the before and after rows. IMO, this would be nice to have with Myrocks. There is a Mariadb push request for RFR that works with Tokudb that may be used as the API for this feature for other storage engines.
Online schema change in Tokudb only injects 'schema change messages' into the top of the fractal tree. This is very fast. The actual change in the encoded row format occurs much later when these messages are executed on the leaf nodes. This seems nice to me, but it can be replaced with other online schema change packages. It would be nice to compare these various method WRT to performance.
Tokudb also supports large transactions defined as transactions that effect a large number of rows such that the memory footprint of the undo log for these operations spills to external storage. Personally, I don't like large transactions, and believe that they should be avoided. However, some people do use them, and the mysql server should be able to support them without crashing, perhaps with degraded performance.
On Wed, Oct 12, 2016 at 12:25 PM, MARK CALLAGHAN <mdcallag@gmail.com> wrote:
0) Sergei and Justin - I prefer to avoid the "Monty says" game either for or against. We are trying to establish a community here. He is one of many. His reputation (good or bad) shouldn't be a stick that is deployed to win arguments.
1) RocksDB has persistent snapshots. That would make flashback trivial to implement for MyRocks assuming you took snapshots ahead of time. So maybe that isn't flashing back. http://rocksdb.org/blog/2015/11/10/use-checkpoints-for-effic ient-snapshots.html
2) I expect almost everyone to choose MyRocks given a choice between MyRocks and TokuDB. I have evaluated TokuDB over several years. I have many performance and efficiency results from this year that I have yet to publish. Results are almost always much better with MyRocks, we have more resources dedicated to MyRocks so it will improve quickly.
MariaDB is a community project, so if someone maintains TokuDB I assume it will be included in MariaDB. But I think time is better spent on MyRocks. Our docs need to be updated and our communication needs to be better. But we move fast and Yoshi has been updating the limitations wiki page as I write this (https://github.com/facebook/m ysql-5.6/wiki/MyRocks-limitations).
I am not surprised that there are so few performance results published for TokuDB. And when they are published they are almost always for trivial workloads -- like insert-only insert benchmark. Try adding queries concurrent with the inserts.
3) Partitioning via the partition storage engine is supported for MyRocks. My employer doesn't need it for the initial deployment. It might be needed elsewhere and I am sure the community needs it. I assume we also need native partitioning in MyRocks when we move up to MySQL 8. The limitations page is getting updated to explain this.
4) By default the FB MySQL branch doesn't allow mysqld to start if MyRocks and InnoDB are enabled. This was misinterpreted to mean it wasn't possible to use MyRocks and InnoDB at the same time. It is possible now. And MariaDB/Percona are free to always allow both to be enabled. It was previously prevented in the FB MySQL repo because of fear that internal users might try to do a transaction that spans engines. I think that is a bad idea and XA scares me. I assume evaluations for MariaDB Server and Percona Server require that both InnoDB and MyRocks are enabled concurrently. See https://github.com/facebook/mysql-5.6/issues/106
5) We have work in progress to support XA for MyRocks. Code has been merged. Perf tests are in progress.
6) We have work in progress to support online DDL.
7) MyRocks supports read-committed (RC) and repeatable-read (RR). RR in MyRocks is Postgres-style and I assume that it won't get much use, just like i assume it doesn't in Postgres. If we add gap-locking to MyRocks RR then it will match InnoDB. For details see https://github.com/mdcalla g/mytools/wiki/Cursor-Isolation
8) I am a long time user of SBR and think that RBR is the future. MyRocks needs RBR because we expect most deployments to use it with read-committed and just like InnoDB, that needs RBR. If gap locking is added to MyRocks RR then SBR is feasible, but you give up all of the great new replication features that will be RBR only. http://dba.stackexchange .com/questions/101122/mysql-why-statement-based-binlog- format-cannot-work-with-innodb-read-uncommitte
9) MyRocks could do transportable column family for all indexes/tables in a column family. Transportable is a nice feature but I don't think the lack of it is a big deal. It is part of being storage efficient, but the overall storage efficiency story is much stronger for MyRocks than InnoDB.
10) I hope there is a plan for foreign key and spatial in MyRocks. There isn't one yet. I don't think full text in any MySQL engine is a good use of time given how infrequently it is used.
On Wed, Oct 12, 2016 at 8:39 AM, Reindl Harald <h.reindl@thelounge.net> wrote:
Am 12.10.2016 um 17:34 schrieb Justin Swanhart:
When upgrading from 5.0 -> 5.1 the sort order of some german characters was modified, which meant indexes had to be rebuilt (they are in sorted order after all). The mysql_upgrade would suggest a table rebuild (ALTER TABLE xyz ENGINE=INNODB) or dump/restore for every affected table. I was a consultant during this time, and I can't tell you how many tables I had to rebuild. Thousands to say the least. Yuck.
well, a five-liner in PHP using a root account while UTF8_GENERAL_CI was the wrong charset/collation anyways for german umlauts :-)
On Wed, Oct 12, 2016 at 10:01 AM, Reindl Harald <h.reindl@thelounge.net
<mailto:h.reindl@thelounge.net>> wrote:
Am 12.10.2016 um 15:53 schrieb Justin Swanhart:
Also having to dump/restore to go from 10.2 to 10.3 tables is a PITA
it's a simple no-go and the answer to "should i use postgre or mysql/mariadb" is always "stay away from postgre, while it got better you need to dump your data and import them again still way too often while our mysql tables where created 2002 and now running on MariaDB 10 without doing such a bullshit a single time"
Remember how awful it was to migrate all those BDB tables to InnoDB? Or even just how frustrating it was when UTF8_GENERAL_CI changed the umlaut?
when did something change umlauts?
as charsets where introdcued with MySQL 5.0 it was a simple script just to add the correct charset information that it is *not* UTF8 to 5000 tables with not a single char damaged and since we are located in austria üöäß are very common
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Mark Callaghan mdcallag@gmail.com
Hi, Justin! Very good questions, thanks! Some answers below: On Oct 12, Justin Swanhart wrote:
* InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says there's next to no changes in InnoDB 8... Instant add column. New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, check). Lock wait policy (contribution)
Monty is *notorious* for low balling estimates. His famous phrase is "it is trivial". Everybody knows that if Monty says it is trivial, you can add 10x the work to get it done, or more.
Yes, I tend to agree with that. But Monty estimates are not that far off when applied to *Monty*. They're often too low when applied to others. https://www.google.de/search?q=10x+developer
includes transportable tablespaces for partitioned tables, and will likely support native partitioning by the time 8 rolls. Native partitioning also entails implementing the changes to the SE. See: http://mysqlserverteam.com/innodb-native-partitioning-early-access/
InnoDB native partitioning is in 5.7.6: https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-6.html Is it better than upper-layer partitioning? Why?
* more for window functions (user defined window functions, MDEV-10855) What other databases have user definable window functions? Will the syntax be similar? I can't really think of a reason to need custom window functions personally, as the existing set is very comprehensive. What kind of custom window function do you think people would need to write? In most implementations any aggregate functions can be used in a window frame context, thus any aggregate SQL functions should satisfy requirements for user defined window funcs.
Yes, that was a confusing description. What it really means - we will have aggregate SQL functions (MDEV-7773) and preferrably, it should be possible to make them window-aware. Using an aggregate function as a window function, means applying it to every possible position of a sliding frame, and it has O(N*n) complexity (where N is total number of rows, n is the number of rows in the frame). If the aggregate function can remove rows from a group, it will only be O(N).
* socket authentication Can you explain this. Is there an MDEV?
That's using unix_socket authentication by default for the root user. It improves security and maintainability, but is not really new - Debian is already doing that for months. This change could be confusing and break exising user scripts, so it needs to be done with care.
* X Protocol/Document store - only if people have time or money will it be done If you want people to pay for it, perhaps you should implement it in MaxScale instead of the server :)
Interesting idea :) But the MariaDB Meetup is a community event organized by the MariaDB Foundation. This particular session was about MariaDB Server planning. MariaDB Foundation has nothing to do with MaxScale, so we could not have planned anything for it.
* Peter asks if there is any plans to support other languages like V8? PLEASE PLEASE PLEASE PLEASE IMPLEMENT WL-820. External stored procs and table functions. Been sitting there for the taking for a long time. Antony has tried to get you to get it into the server, but alas, it has never happened, and though there continues to be wide requests for this feature from the community, they fall on deaf ears.
Frankly speaking, I'd love it. But this feature never got enough priority, not in MySQL times nor in MariaDB. Unfortunately. Perhaps we'll be able to sneak it into 10.3, but no promises here.
* Compressed binary log (from Tencent) Compressing the binary log saves on space on the master, but it makes seeking into binary logs much more difficult, and searching backwards through them becomes much more difficult (which has implications for query 'rewind')
Tencent compresses individual events (think Compressed_query_log_event type). So seeking works as before, events are not decompressed on reading, they are sent compressed to slaves, etc. But the compression ratio is worse, of course.
* Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 Please actually complete XA support, don't just half fix it like Oracle did. Add full support for XA SUSPEND and XA RESUME and allow more than one thread to participate in a distributed transaction in the server.
Right. Still, MDEV-7974 is an important step, we cannot have proper XA without it. XA SUSPEND and XA RESUME should be the next one, I agree.
* Indexes on expressions (this is part of virtual columns, will it not go into 10.2? Check with Serg) Indexes on expressions requires parser support. create index expr_idx on some_table(a + b); select * from some_table where a+b > 30 and a+b <= 50 -- (uses range over expr_idx)
Right, but the parser support is the least of my worries. I don't want to low ball estmates :) but the *parser* can be fixed in a few hours. The most tricky part will be to fix the optimizer.
* Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only works with row based replication. Talk about what to name it. There's already a MySQL time machine on github. Many like the name Rewind (but not Monty). Let's do a poll on the mailing list Okay, Flashback query is VERY complicated. A flashback query is a materialized view. There are two ways generally to achieve flashback: a) materialize the query as is, and instead of rolling the query forward incrementally, you roll it backwards. Flexviews achieves this by computing the query as it is now, then computing the delta from a prior point in time until the transaction in which that computation happened. Then the delta is played back against the query, but the MONUS of each operation is applied which changes insertions to deletions and vice versa. This still presents a problem for OUTER JOIN. No documented asynchronous refresh algorithms exist for outer join that I'm aware of, that would work with the concept of reading row history from serialized changes. Flexviews uses the "rolling join propagation" algorithm, which only works with inner joins.
Interesting... I'll continue reading on that, thanks. But see below
b) provide a point-in-time snapshot of every table used in the query at the desired point in time, and run the query over those tables (this is a synchronous mechanism which supports OUTER JOIN). The problem here is how to provide such a table. The most straight-forward way is to copy the table, and then do the backwards replay for each (ideally in parallel) but this is obviously undesirable if the query is "select * from some_big_table join another_big_table on (...)" because you have to fully copy each table before you can undo changes. Otherwise you need to have the SE display old versions, just like it does for MVCC, but it has to display versions from binary logs, not undo logs, and this requires a lot of SE and engine changes!
This is about correct. There are different applications for "flashback" with different use cases and different implementation trade-offs. One is, for example, to see historical sales numbers, for different months or years. That is, basically, for some kind of data analytics. Lots of SELECT queries, ad-hoc queries, at different historical time points. Another one is "damn, I've made a typo in a WHERE clause and updated too much". In this case one doesn't need joins or materialized views, one needs to see the data before the erroneous statement. This is not used often. For the second use case one can afford to copy tables and backward-replay the binary log. For the first use case one would need a completely different approach, I agree. May be something like what Flexviews does.
If you are going to have generic flashback, you might as well commit to incrementally refreshable materialized views.
Right, when we'll have materialized views, than we could think about incrementally updating them using rolling join propagation.
* Additional GIS functions to stay compatible. Also it would be good to have a standalone GIS library (Georg suggests; wlad isn't too happy with the suggestion). Georg suggests that calculations should use the reference systems (Unflatten the world - MariaDB Server GIS the world looks flat)
GIS functions should use gdal, just like postgresql does. Then you can also add support for rasters. Here is an example of the awesomeness of postgresql and postgis. It uses SQL aggregate functions, table functions, CTE, GIS raster functions, sequences, etc: https://github.com/greenlion/osmvox/blob/master/postgresql/combined_schema.s...
May be. On the other hand, we're more precise that postgis, because our implementation uses fixed-point math, not doubles. And let's not forget that MySQL uses Boost::Geometry.
* Query rewriting - MDEV-5561 I would like you to provide a SQL->DOM function call instead of just providing a DOM to plugins. This function could be exposed as a regular item function as well so that anything in the server can parse SQL. I suggest you implement my SQL "shim" interface and just provide some way to get a easy to iterate over parsed data structure from the SQL, such as a nested JSON array of objects. A nested array is generated by PHP-SQL-Parser and would provide a good template for such a JSON object. https://github.com/greenlion/PHP-SQL-Parser
We actually have an MDEV for that :)
* With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better It is certainly disheartening that Percona isn't responsive to MariaDB bugs, but I'm sure you understand that it is hard for a competitor to fix bugs for another competitor. MariaDB maintains a forked version of TokuDB. It isn't fair to expect the upstream vendor to fix bugs that your customers are paying you to support, does it? Perhaps you should pay a percentage of your support fees for TokuDB issues to Percona, or come to some other support agreement. Perhaps YOU should make the bug fixes and submit them to Percona.
And we do, look for bug reports I've reported on TokuDB to Percona - with patches :) But, really, Percona *is* fixing TokuDB bugs, it's just that they did a bit of refactoring after getting TokuDB and it took time for it to stabilize. And MariaDB Server has a vanilla TokuDB with almost no changes, we have no plans to fork it.
* ORDER BY LIMIT optimizer bugs (MDEV-8306) Oh god, this is a downward spiral every time somebody touches it. Fixing it correctly requires rewriting the parser, something that Oracle is undertaking.
This didn't have much to do with the parser, and we've refactored that part of the parser in 10.2, so now this has nothing to do with the parser, it's purely the optimizer issue.
What about other new MySQL 8 features? Are you getting rid of the .FRM nightmare? Are you going to support SET PERSIST? etc?
FRM was made purely optional in 10.0 - every storage engine decides for itself whether it uses FRMs or not. May be InnoDB will use FRMs in 10.3, may be it will not. MyISAM most probably will continue use them. SET PERSIST - and this is my personal opinion - this needs to be thought over *very* carefully. There have been quite a few security vulnerabilities with my.cnf stored in the datadir, and that's why since 2005 the server no longer reads my.cnf in the datadir. But mysql_safe still does - and there have been new security vulnerabilities *last month*, caused by my.cnf in the datadir. And finally both MySQL (in 5.7?) and MariaDB (in 10.2) stopped reading my.cnf in the datadir for real. So, having my security@mariadb.org hat on, I look at SET PERSIST with a lot of suspicion, because it's nothing else than another attempt of storing server configuration information in the datadir and have it writable by the server itself. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Thanks for the write up Colin. I had known about TokuDB potentially being dropped due to lack of developers and support. Understandable, but a real shame. At least rocksdb has a backup tool and some fun snapshot features. What I really want to know is, what's the future of transactional DDL in MariaDB (MDEV-4259) with the impending changes in MySQL 8? How much of that is going to be merged with MariaDB? Cheers, Richard
Hi, Ricky! On Oct 13, Ricky B wrote:
Thanks for the write up Colin.
I had known about TokuDB potentially being dropped due to lack of developers and support. Understandable, but a real shame. At least rocksdb has a backup tool and some fun snapshot features.
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty).
What I really want to know is, what's the future of transactional DDL in MariaDB (MDEV-4259) with the impending changes in MySQL 8? How much of that is going to be merged with MariaDB?
Current understanding is - not much of it. We seem to be going different path. MySQL is moving everything into InnoDB - one engine to rule them all. System tables are in InnoDB, data dictionary is is InnoDB, temporary tables - too. And so on. New storage engine API extensions work only for InnoDB. The manual has a chapter on InnoDB and a chapter on "alternative storage engines". MariaDB - currently - tries to treat all engines equally. More or less. New storage engine API extensions work for all capable engines - almost always for more than one. New features are added in the engine independent way. And so on. For example, in MariaDB one can use InnoDB for system tables too. Always could, even in 5.5, it's not a new feature. There is no code that limits system tables to a specific storage engine. FRM files in MariaDB are optional since 10.0. But it depends on the engine, and InnoDB relies on FRM files in MariaDB. May be we will change it in 10.3, then InnoDB tables won't use FRM files anymore and DDLs will be atomic and may be, in some later version, even transactional. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
For example, in MariaDB one can use InnoDB for system tables too. Always could, even in 5.5, it's not a new feature. There is no code that limits system tables to a specific storage engine.
This is not true even in MariaDB 10.2: https://github.com/MariaDB/server/blob/10.2/storage/innobase/row/row0mysql.c.... And that's only the explicit restriction. Even when you remove that you still need a lot of other code changes to make MariaDB work with system tables using InnoDB. There are many places assuming that system tables are not transactional. On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Ricky!
On Oct 13, Ricky B wrote:
Thanks for the write up Colin.
I had known about TokuDB potentially being dropped due to lack of developers and support. Understandable, but a real shame. At least rocksdb has a backup tool and some fun snapshot features.
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty).
What I really want to know is, what's the future of transactional DDL in MariaDB (MDEV-4259) with the impending changes in MySQL 8? How much of that is going to be merged with MariaDB?
Current understanding is - not much of it. We seem to be going different path. MySQL is moving everything into InnoDB - one engine to rule them all. System tables are in InnoDB, data dictionary is is InnoDB, temporary tables - too. And so on. New storage engine API extensions work only for InnoDB. The manual has a chapter on InnoDB and a chapter on "alternative storage engines".
MariaDB - currently - tries to treat all engines equally. More or less. New storage engine API extensions work for all capable engines - almost always for more than one. New features are added in the engine independent way. And so on. For example, in MariaDB one can use InnoDB for system tables too. Always could, even in 5.5, it's not a new feature. There is no code that limits system tables to a specific storage engine. FRM files in MariaDB are optional since 10.0. But it depends on the engine, and InnoDB relies on FRM files in MariaDB. May be we will change it in 10.3, then InnoDB tables won't use FRM files anymore and DDLs will be atomic and may be, in some later version, even transactional.
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Hi, Pavel! On Oct 13, Pavel Ivanov wrote:
For example, in MariaDB one can use InnoDB for system tables too. Always could, even in 5.5, it's not a new feature. There is no code that limits system tables to a specific storage engine.
This is not true even in MariaDB 10.2: https://github.com/MariaDB/server/blob/10.2/storage/innobase/row/row0mysql.c.... And that's only the explicit restriction. Even when you remove that you still need a lot of other code changes to make MariaDB work with system tables using InnoDB. There are many places assuming that system tables are not transactional.
That's basically what I said - the server doesn't limit system tables to a specific engine. Every engine can decide for itself. And InnoDB, apparently, doesn't want to create mysql.host, mysql.user, and mysql.db tables. This test: https://github.com/MariaDB/server/blob/5.5/mysql-test/t/plugin_innodb.test verifies that mysql.plugin works when altered to InnoDB This test: https://github.com/MariaDB/server/blob/10.1/mysql-test/suite/innodb/t/system... verifies that mysql.time_zone_name works when altered to InnoDB This is not a very popular use case, we had no bug reports about mysql.host, mysql.user, and mysql.db in InnoDB. Otherwise this would've been fixed too. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Ricky!
On Oct 13, Ricky B wrote:
Thanks for the write up Colin.
I had known about TokuDB potentially being dropped due to lack of developers and support. Understandable, but a real shame. At least rocksdb has a backup tool and some fun snapshot features.
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty).
Is MariaDB.com providing the development for TokuDB or are you speculating about the commitment to it provided by someone else? If the latter, then it would be better to hear directly from them. If the former, can you be more specific about what MariaDB.com plans to do for TokuDB?
There is development and support for TokuDB by Percona. 2016-10-14 6:43 GMT+03:00 MARK CALLAGHAN <mdcallag@gmail.com>:
On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Ricky!
On Oct 13, Ricky B wrote:
Thanks for the write up Colin.
I had known about TokuDB potentially being dropped due to lack of developers and support. Understandable, but a real shame. At least rocksdb has a backup tool and some fun snapshot features.
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty).
Is MariaDB.com providing the development for TokuDB or are you speculating about the commitment to it provided by someone else? If the latter, then it would be better to hear directly from them. If the former, can you be more specific about what MariaDB.com plans to do for TokuDB?
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Laurynas
Hi, MARK! On Oct 13, MARK CALLAGHAN wrote:
On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote:
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty).
Is MariaDB.com providing the development for TokuDB or are you speculating about the commitment to it provided by someone else? If the latter, then it would be better to hear directly from them. If the former, can you be more specific about what MariaDB.com plans to do for TokuDB?
I am speculating about the commitment to it provided by Percona. Based on what (I believe) Peter said, and it's in the Colin email too:
Peter says that bugs are fixed for customers... and there is ongoing development to make it better
but mainly based on the changes I see in TokuDB codebase inside Percona Server: 5.6.32-78.1 - 73 files changed, 4192 insertions(+), 3715 deletions(-) 5.6.31-77.0 - 35 files changed, 153 insertions(+), 79 deletions(-) 5.6.30-76.3 - 77 files changed, 862 insertions(+), 309 deletions(-) 5.6.29-76.2 - 25 files changed, 487 insertions(+), 153 deletions(-) 5.6.28-76.1 - 14 files changed, 997 insertions(+), 182 deletions(-) 5.6.27-76.0 - 206 files changed, 15619 insertions(+), 5969 deletions(-) 5.6.26-74.0 - initial checkin (into mergetrees/tree/merge-tokudb-5.6 repo) I agree that it doesn't prove that they will continue to do so in the future. If MariaDB.com has any plans about TokuDB - I don't know about them. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Hi! Regarding the TokuDB bugs, they are so far fixed quite quickly by Percona. For example https://bugs.launchpad.net/percona-server/+bug/1607300 was investigated and fixed in less than 10 days, and George answered the same day of the bug report. And hopefully, the latest remaining patch from Rich needed to enable optimistic parallel replication will be merged quickly as well ( https://github.com/percona/PerconaFT/pull/360 ). The same is true for the RFR patchs on the MariaDB side, ( https://github.com/MariaDB/server/pull/213 + https://github.com/percona/percona-server/pull/1070 ) and the tokubackup script ( https://jira.mariadb.org/browse/MDEV-8843 ) ;) As for the performance of TokuDB with concurrent read/write queries, from my experience it depends if you try to read immediately the rows which have just been inserted/updated or not. Is this case the messages in the fractal tree have to be propagated and you loose the TokuDB speed insert/update advantage, but they are still a few workload where TokuDB is performing quite well. Jocelyn Fournier Founder M : +33 6 51 21 54 10 https://www.softizy.com Softizy - At your side to Optimize your PHP / MySQL applications Le 14/10/2016 à 09:11, Sergei Golubchik a écrit :
Hi, MARK!
On Oct 13, MARK CALLAGHAN wrote:
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty). Is MariaDB.com providing the development for TokuDB or are you speculating about the commitment to it provided by someone else? If
On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote: the latter, then it would be better to hear directly from them. If the former, can you be more specific about what MariaDB.com plans to do for TokuDB? I am speculating about the commitment to it provided by Percona. Based on what (I believe) Peter said, and it's in the Colin email too:
Peter says that bugs are fixed for customers... and there is ongoing development to make it better but mainly based on the changes I see in TokuDB codebase inside Percona Server:
5.6.32-78.1 - 73 files changed, 4192 insertions(+), 3715 deletions(-) 5.6.31-77.0 - 35 files changed, 153 insertions(+), 79 deletions(-) 5.6.30-76.3 - 77 files changed, 862 insertions(+), 309 deletions(-) 5.6.29-76.2 - 25 files changed, 487 insertions(+), 153 deletions(-) 5.6.28-76.1 - 14 files changed, 997 insertions(+), 182 deletions(-) 5.6.27-76.0 - 206 files changed, 15619 insertions(+), 5969 deletions(-) 5.6.26-74.0 - initial checkin (into mergetrees/tree/merge-tokudb-5.6 repo)
I agree that it doesn't prove that they will continue to do so in the future.
If MariaDB.com has any plans about TokuDB - I don't know about them.
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Hi all! On 10/14/2016 11:10 AM, jocelyn fournier wrote:
Hi!
Regarding the TokuDB bugs, they are so far fixed quite quickly by Percona.
I want to clarify the source of the question (FUD) about Percona not fixing TokuDB bugs, and hopefully bring it to an end. It apparently comes from me, saying so during the discussion at the meetup. My impression was based on interaction with community bug reporters, which is one of my chores. We ask them to re-file TokuDB bugs to Percona, and then after a while they would tell me, via JIRA or otherwise, that there is no progress on the re-filed ticket. It happened several times at least, and given that TokuDB doesn't come up in our JIRA all that often, it did not seem to be negligible. But then again, it happens to everyone that bugs get stalled, MariaDB is certainly no saint in this regard, either. Besides, I must admit that my statistics is biased, because I only hear from TokuDB reporters again when things don't go well, while success stories might well remain unnoticed by me. So, since involved parties confirm that there is in fact good progress, I readily retrieve my previous statement with due apologies to everyone who I unwillingly upset by it. Regards, Elena
https://bugs.launchpad.net/percona-server/+bug/1607300 was investigated and fixed in less than 10 days, and George answered the same day of the bug report.
And hopefully, the latest remaining patch from Rich needed to enable optimistic parallel replication will be merged quickly as well ( https://github.com/percona/PerconaFT/pull/360 ). The same is true for the RFR patchs on the MariaDB side, ( https://github.com/MariaDB/server/pull/213 + https://github.com/percona/percona-server/pull/1070 ) and the tokubackup script ( https://jira.mariadb.org/browse/MDEV-8843 ) ;)
As for the performance of TokuDB with concurrent read/write queries, from my experience it depends if you try to read immediately the rows which have just been inserted/updated or not. Is this case the messages in the fractal tree have to be propagated and you loose the TokuDB speed insert/update advantage, but they are still a few workload where TokuDB is performing quite well.
Jocelyn Fournier Founder M : +33 6 51 21 54 10 https://www.softizy.com Softizy - At your side to Optimize your PHP / MySQL applications
Le 14/10/2016 à 09:11, Sergei Golubchik a écrit :
Hi, MARK!
On Oct 13, MARK CALLAGHAN wrote:
There is development and support for TokuDB. The main reason for dropping it could be RocksDB based engine covering all TokuDB use cases, but better. I mean, *if* RocksDB will be able to do that, then TokuDB won't be needed (it's *if*, not a certainty). Is MariaDB.com providing the development for TokuDB or are you speculating about the commitment to it provided by someone else? If
On Thu, Oct 13, 2016 at 4:00 PM, Sergei Golubchik <serg@mariadb.org> wrote: the latter, then it would be better to hear directly from them. If the former, can you be more specific about what MariaDB.com plans to do for TokuDB? I am speculating about the commitment to it provided by Percona. Based on what (I believe) Peter said, and it's in the Colin email too:
Peter says that bugs are fixed for customers... and there is ongoing development to make it better but mainly based on the changes I see in TokuDB codebase inside Percona Server:
5.6.32-78.1 - 73 files changed, 4192 insertions(+), 3715 deletions(-) 5.6.31-77.0 - 35 files changed, 153 insertions(+), 79 deletions(-) 5.6.30-76.3 - 77 files changed, 862 insertions(+), 309 deletions(-) 5.6.29-76.2 - 25 files changed, 487 insertions(+), 153 deletions(-) 5.6.28-76.1 - 14 files changed, 997 insertions(+), 182 deletions(-) 5.6.27-76.0 - 206 files changed, 15619 insertions(+), 5969 deletions(-) 5.6.26-74.0 - initial checkin (into mergetrees/tree/merge-tokudb-5.6 repo)
I agree that it doesn't prove that they will continue to do so in the future.
If MariaDB.com has any plans about TokuDB - I don't know about them.
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Is anyone surprised by the lack of TokuDB support? Percona is primarily an Oracle MySQL fork . The InnoDB changes in MySQL 8 represent something entirely different. Good, but different. I don't think I am being too speculative when I say, I am sure Oracle are in no hurry to make MySQL a serious contender to Oracle just yet! But MariaDB could. Me and many others had hope that TokuDB would be the death of InnoDB in MariaDB, and the catalyst to a truly amazing open source database. RocksDB can still come into play later, but it's not going to help users migrate from other RDBM systems, fix existing "missing" features from MariaDB, nor is it anything I don't think, existing TokuDB users care about. Richard
I don't think (most) guys must migrate from other RDBM, normally they select mysql/mariadb/webscale/etcetc before starting the project, a migration "in the middle" of a project is costly. I agree, there's many open issue at jira/mysql bug report/percona/etc/etc and others issue tracker systems that being completed make users migrate to mysql 2016-10-14 10:21 GMT-03:00 Richard Bensley <richardbensley@gmail.com>:
Is anyone surprised by the lack of TokuDB support? Percona is primarily an Oracle MySQL fork . The InnoDB changes in MySQL 8 represent something entirely different. Good, but different.
I don't think I am being too speculative when I say, I am sure Oracle are in no hurry to make MySQL a serious contender to Oracle just yet! But MariaDB could.
Me and many others had hope that TokuDB would be the death of InnoDB in MariaDB, and the catalyst to a truly amazing open source database.
RocksDB can still come into play later, but it's not going to help users migrate from other RDBM systems, fix existing "missing" features from MariaDB, nor is it anything I don't think, existing TokuDB users care about.
Richard
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
On Friday, 14 October 2016, Roberto Spadim <roberto@spadim.com.br> wrote:
I don't think (most) guys must migrate from other RDBM, normally they select mysql/mariadb/webscale/etcetc before starting the project, a migration "in the middle" of a project is costly. I agree, there's many open issue at jira/mysql bug report/percona/etc/etc and others issue tracker systems that being completed make users migrate to mysql
For years Roberto, that's all I did. From oracle, MySQL, MSSQL, access and sybase to mariadb for ecommerce and finance. Connect, gtid, toku, and multi source rep are killer features.
On Friday, 14 October 2016, Roberto Spadim <roberto@spadim.com.br> wrote:
I don't think (most) guys must migrate from other RDBM, normally they select mysql/mariadb/webscale/etcetc before starting the project, a migration "in the middle" of a project is costly. I agree, there's many open issue at jira/mysql bug report/percona/etc/etc and others issue tracker systems that being completed make users migrate to mysql
For years Roberto, that's all I did. From oracle, MySQL, MSSQL, access and sybase to mariadb for ecommerce and finance.
nice, but are these companies big? mid? small? normally what i see (i'm in brazil) is only big/mid companies doing this
Connect, gtid, toku, and multi source rep are killer features.
this i must agree with you, reduce the costs aggressively, i used many times -- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
Am 14.10.2016 um 15:21 schrieb Richard Bensley:
Me and many others had hope that TokuDB would be the death of InnoDB in MariaDB, and the catalyst to a truly amazing open source database.
please fix that "many others" above by saying "some others" - if we would like a "truly amazing open source database" with is not a mysql dropin replacement we would just use postgresql - the "dead of innodb and myisam in mariadb" would be a clear reason to migrate away at all
i read opnions about why choose mysql, and the common point is 'mysql runs simple queries really fast' i think that's the main feature of mysql 2016-10-14 11:25 GMT-03:00 Reindl Harald <h.reindl@thelounge.net>:
Am 14.10.2016 um 15:21 schrieb Richard Bensley:
Me and many others had hope that TokuDB would be the death of InnoDB in MariaDB, and the catalyst to a truly amazing open source database.
please fix that "many others" above by saying "some others" - if we would like a "truly amazing open source database" with is not a mysql dropin replacement we would just use postgresql - the "dead of innodb and myisam in mariadb" would be a clear reason to migrate away at all
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
Making Percona/MariaDB/MySQL better at sharding, and supporting complex queries (with MPP capability, window functions, CTE,etc.) would really compete with an enterprise DB, with RedShift, and with Fabric. Shard-Query can greatly outperform both the enterprise MySQL Infobright (and MC ColumnStore) because it parallelizes operations differently, and supports horizontal scale-out over more than one node (which IB does, and CS does differently). So what about group replication? I'm very much interested in group replication plugins. Using the GR plugin interface in combination with a rewrite layer that sits outside of the parser, it should provide notifications for transaction events and row changes which a sharding layer could act on appropriately to replicate or distribute data. It would coordinate distributed transactions. This would provide a layer to compete with Fabric. If I were to contribute such a layer it would of course have OLAP support. This would work with all storage engines and push down operations in a more optimal way than other distributed MySQL solutions like MC ColumnStore. Making it so you can run SQL in the server should be a priority, in my opinion, as well as implementing a rewrite interface that is outside of the parser. I can commit to contributing this new rewrite interface to Percona/MySQL/MariaDB (whomever wants to implement it) through a relatively simple patch to MariaDB and MySQL (Percona Server is similar enough to MySQL such that the MySQL patch should apply cleanly or close to it). With those capabilities I can implement Shard-Query inside of the server. The company that I am in the process of joining will support these changes, and will be responsive to bug reports, etc. I want to stress that operating in the free market to support extensions and plugins to open source software does not represent "competition". MariaDB can't use their non-compete clause to prevent me from operating in the ecosystem, as I would be unable to obtain any job in the marketplace. That represents unfair competition. You can't prevent me from practicing my profession. Shard-Query (WarpSQL) is engine and server agnostic as long as the server conforms to the pluggable API which WarpSQL uses. In order to prevent any possible perceived conflict of interest, or perceived competition, I won't provide any patches for columnstore, which is a separate MySQL fork maintained by MariaDB Corporation, than MariaDB itself, which is overseen by the Foundation. Let's have a peaceful and community, and let's make MySQL much better at complex queries. On Fri, Oct 14, 2016 at 10:32 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
i read opnions about why choose mysql, and the common point is 'mysql runs simple queries really fast' i think that's the main feature of mysql
2016-10-14 11:25 GMT-03:00 Reindl Harald <h.reindl@thelounge.net>:
Am 14.10.2016 um 15:21 schrieb Richard Bensley:
Me and many others had hope that TokuDB would be the death of InnoDB in MariaDB, and the catalyst to a truly amazing open source database.
please fix that "many others" above by saying "some others" - if we would like a "truly amazing open source database" with is not a mysql dropin replacement we would just use postgresql - the "dead of innodb and myisam in mariadb" would be a clear reason to migrate away at all
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
-- Roberto Spadim SPAEmpresarial - Software ERP Eng. Automação e Controle
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (19)
-
Colin Charles -
Elena Stepanova -
Igor Babaev -
jocelyn fournier -
Justin Swanhart -
Laurynas Biveinis -
MARK CALLAGHAN -
Pavel Ivanov -
Peter Laursen -
Reindl Harald -
Reinis Rozitis -
Rich Prohaska -
Richard Bensley -
Ricky B
-
Roberto Spadim -
Sergei Golubchik -
Tim Callaghan -
Tom Worster -
Vladislav Vaintroub