October 2016 - discuss - lists.mariadb.org

[Maria-discuss] MariaDB Server 10.3 notes
by Colin Charles 17 Oct '16

17 Oct '16

Hi, Sorry this is a little later than expected (who couldn’t resist walking around Amsterdam on that late beautiful Saturday afternoon?)! This session was led by Rasmus whom while employed by MariaDB Corporation, was also the MariaDB Foundation Board member leading it. This was the last session on the formal schedule before lunch (more was added to it as well later on) I notice he’s also blogged [part 1](https://mariadb.org/thoughts-mariadb-server-10-3-mariadb-developers-meet…, and maybe these notes help for part 2 (or maybe they don’t). Scroll down for what is still coming in MariaDB Server 10.2 of course (post-GA even)! # MariaDB Server 10.3 * MDEV-10177 * MDEV-371 * MDEV-7773 - create aggregate functions in SQL * User defined types (pluggable data type API) * CJK language support - MDEV-10267, MDEV-10268 * SPIDER storage engine - MDEV-7698 * Further MyRocks development (10.2 will have it as experimental; MariaDB Corporation is looking to hire a dedicated engineer to work on MyRocks) * InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty says there's next to no changes in InnoDB 8... Instant add column. New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, check). Lock wait policy (contribution) * Pluggable parser - MDEV-10142, PL/SQL parser, Oracle syntax support, INTERSECT, EXCLUDE * Sequences (like PostgreSQL implementation) * more for window functions (user defined window functions, MDEV-10855) * more for CTEs (CTE sharing isn't in now, and MySQL 8 has it, so needs to be done; Writable CTEs) * replace yaSSL with GnuTLS (SChannel in Windows) * "always encrypt" on the client side - Booking says there is no particular interest in this at the moment. * socket authentication * X Protocol/Document store - only if people have time or money will it be done * Peter asks if there is any plans to support other languages like V8? Supporting legacy database like Oracle PL/SQL is nice, but what about JavaScript and stuff? Eric mentions his old work on Perl and Java external language stored procedures. Peter says, if JSON and JavaScript are your lingua franca, PL/SQL isn't so interesting - but there was no interest in this amongst the folk. * No slave left behind patch - MDEV-8112 (Booking wants a bit more than the current patch, some extensions) - maybe in 10.2 * Compressed binary log (from Tencent) * For ALTER TABLE show the corresponding CREATE TABLE statement as a comment in the binary log. This is an annotation. * Counters for unsafe statements in replication (maybe in 10.2) * Extended table map in RBR is missing * Galera 4, encryption of gcache * Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 * Indexes on expressions (this is part of virtual columns, will it not go into 10.2? Check with Serg) * Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only works with row based replication. Talk about what to name it. There's already a MySQL time machine on github. Many like the name Rewind (but not Monty). Let's do a poll on the mailing list * Additional GIS functions to stay compatible. Also it would be good to have a standalone GIS library (Georg suggests; wlad isn't too happy with the suggestion). Georg suggests that calculations should use the reference systems (Unflatten the world - MariaDB Server GIS the world looks flat) * Optimizer trace - MDEV-6111 * Query rewriting - MDEV-5561 * Replace libmysqlclient with Connector/C - MDEV-9055 - embedded/replication/ storage engine (connect, spider, federatedX) - work continues from 10.2 * Invisible indexes from MySQL 8 - this is something Eric wants to see * With MyRocks coming, should we drop TokuDB (and maybe even deprecate in 10.2?) - bugs that MariaDB Corporation reports to Percona don't seem to get fixed. Peter says that bugs are fixed for customers... and there is ongoing development to make it better * ORDER BY LIMIT optimizer bugs (MDEV-8306) # Still in MariaDB Server 10.2 * Flashback DML - MDEV-10570 * Time delayed replication - MDEV-7145 * MyRocks (10.2.4 - maturity experimental) * GeoJSON (in 10.2 after beta) and ST_Geohash * sha256_password plugin (after GA) - MDEV-9804 - Serg will do it in a MySQL compatible way * sys schema (after GA) * Server system default values MDEV-7635 * MySQL deprecated `DES_ENCRYPT`, `DES_DECRYPT` - Monty says he knows a high level customer using it, a bank. * Column compression from Tencent? -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

19 54

[Maria-discuss] New MariaDB 10.1.18 Galera setup question
by Alex Evonosky 17 Oct '16

17 Oct '16

Hello team- Quick question- I recently just installed mariaDB 10.1.18 (via apt-get) and all went well, no issues.. Here is my galera.cnf file: cat galera.cnf [mysqld] binlog_format = ROW default_storage_engine = InnoDB innodb_autoinc_lock_mode = 2 innodb_locks_unsafe_for_binlog = 1 wsrep_on = ON wsrep_causal_reads = ON wsrep_cluster_address = gcomm://10.10.10.104,10.10.13.2 wsrep_cluster_name = soho_cluster wsrep_node_address = 10.10.10.104 wsrep_provider = /usr/lib/galera/libgalera_smm.so wsrep_provider_options = "gcache.size=512M" wsrep_slave_threads = 4 # Should be equal to the number of cpu-cores. wsrep_sst_auth = "sstuser:r3pl1c@t3" #wsrep_sst_method = xtrabackup-v2 wsrep_sst_method = rsync I started the server as: galera_new_cluster and see no errors. Syslog reports mariaDB has started. However, I never see any messages with WSREP. the database stat shows: mysql -u root -p -e "show status like 'wsrep%'" Enter password: +--------------------------+----------------------+ | Variable_name | Value | +--------------------------+----------------------+ | wsrep_cluster_conf_id | 18446744073709551615 | | wsrep_cluster_size | 0 | | wsrep_cluster_state_uuid | | | wsrep_cluster_status | Disconnected | | wsrep_connected | OFF | | wsrep_local_bf_aborts | 0 | | wsrep_local_index | 4294967295 | | wsrep_provider_name | | | wsrep_provider_vendor | | | wsrep_provider_version | | | wsrep_ready | OFF | | wsrep_thread_count | 0 | +--------------------------+----------------------+ The wsrep always shows OFF. I have removed and purged any and all mysql, mysql-server etc three times and get the same result. Is there something I am missing? Thank you!

2 2

[Maria-discuss] Best way to scale writes
by Jon Foster 17 Oct '16

17 Oct '16

I have a DB scenario that is very write intensive. Essentially its a large scale hit counter of sorts. Currently we're running on a single 12core server with 6 SSDs in a RAID6 array. But we're looking for a way to scale out write volume by adding more servers, hopefully as conveniently as I might add Apache servers to a website. We see two servers laboring so we add a third to the mix and so on. I'm still looking at the various technologies available to me and was wondering if someone out there had some suggestions on this front. Although Galera Cluster says it scales for write loads too it also says that all servers write the same data and make sure its committed by the time the transaction is completed. So it seems like write performance would never scale because all servers are doing the same write. Maybe I'm missing something. Anyhow some pointers would be appreciated. TIA - Jon -- Sent from my Debian Linux workstation -- http://www.debian.org/intro/about Jon Foster JF Possibilities, Inc. jon(a)jfpossibilities.com 541-410-2760 Making computers work for you!

4 6

[Maria-discuss] Virtual CHECK constraint using Virtual columns
by Peter Laursen 16 Oct '16

16 Oct '16

As described in this Blog http://mablomy.blogspot.dk/2016/04/check-constraint-for-mysql-not-null-on.h…. A very nice hack/trick IMO. However it is not working with MariaDB as VC's cannot be declared NOT NULL. What prevents that? -- Peter -- Webyog

3 7

Re: [Maria-discuss] Virtual CHECK constraint using Virtual columns
by Federico Razzoli 16 Oct '16

16 Oct '16

As Sergei said, MariaDB 10.2 has CHECKs. In old versions, honestly I don't like your solution. Take a look at what locks will be set: https://www.percona.com/blog/2006/12/12/innodb-locking-and-foreign-keys/ You can use trigger instead. When you only want to execute a check like this, triggers are not slow. Just SIGNAL an error is the value of i_must_be_between_7_and_12 is not valid. Federico -------------------------------------------- Dom 16/10/16, Pantelis Theodosiou <ypercube(a)gmail.com> ha scritto: Oggetto: Re: [Maria-discuss] Virtual CHECK constraint using Virtual columns A: "Maria Discuss" <maria-discuss(a)lists.launchpad.net> Data: Domenica 16 ottobre 2016, 16:45 Would this be good to be added in the documentation of VIRTUAL columns? Or as a separate page, as a way to enforce/emulate arbitrary CHECK constraints? It can be slightly simplified (IF is not needed) and the BOOLEAN could be BIT (not sure if that adds any complication): CREATE TABLE truth (t BIT PRIMARY KEY) ; INSERT INTO truth (t) VALUES (TRUE) ; -- and remove all write permissions to the table CREATE TABLE checker ( i float, i_must_be_between_7_and_12 BIT AS (i BETWEEN 7 AND 12) -- whatever CHECK constraint we want here PERSISTENT, CONSTRAINT check_i_must_be_between_7_and_ 12 FOREIGN KEY (i_must_be_between_7_and_12) REFERENCES truth (t) ); On Wed, Apr 6, 2016 at 6:46 PM, Pantelis Theodosiou <ypercube(a)gmail.com> wrote: On Mon, Apr 4, 2016 at 2:10 PM, Peter Laursen <peter_laursen(a)webyog.com> wrote: As described in this Blog http://mablomy.blogspot. dk/2016/04/check-constraint- for-mysql-not-null-on.html. A very nice hack/trick IMO. However it is not working with MariaDB as VC's cannot be declared NOT NULL. What prevents that? (Peter, sorry fro the previous private reply, not sure how I got the reply buttons wrong.) I can't answer that, but there's another workaround for (some) CHECK constraints, described here: http://dba.stackexchange.com/ questions/9662/check- constraint-does-not-work/ 22019#22019 Unfortunately, it works only for smallish (int or date) ranges. We can't use for floats or decimals (as it would require a very big reference table). But it could be combined with the hack you link, using something like: CREATE TABLE truth (t BOOLEAN PRIMARY KEY) ; INSERT INTO truth (t) VALUES (TRUE) ; -- and remove all write permissions to the table CREATE TABLE checker ( i int, i_must_be_between_7_and_12 BOOLEAN AS (IF(i BETWEEN 7 AND 12, TRUE, FALSE)) PERSISTENT, CONSTRAINT check_i_must_be_between_7_and_ 12 FOREIGN KEY (i_must_be_between_7_and_12) REFERENCES truth (t) ); Haven't tested it but should work for more complex constraints as well. Pantelis -----Segue allegato----- _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

1 0

Re: [Maria-discuss] MariaDB Server 10.3 notes
by Federico Razzoli 16 Oct '16

16 Oct '16

But those people can't use Galera, pt, Xtrabackup... If we care about Windows, that software should be ported first, because it's general purpose. Federico -------------------------------------------- Dom 16/10/16, Tim Callaghan <tmcallaghan(a)gmail.com> ha scritto: Oggetto: Re: [Maria-discuss] MariaDB Server 10.3 notes A: "Reindl Harald" <h.reindl(a)thelounge.net> Cc: "Maria Discuss" <maria-discuss(a)lists.launchpad.net> Data: Domenica 16 ottobre 2016, 14:45 I was just adding context. I agree that Windows is very uninteresting in a production context, but not having a version of TokuDB for Windows certainly excluded people (academic and professional) from trying it out on their laptops. On Sat, Oct 15, 2016 at 1:10 PM, Reindl Harald <h.reindl(a)thelounge.net> wrote: Am 15.10.2016 um 17:22 schrieb Tim Callaghan: Every time the topic of getting TokuDB to run on Windows the developers could not have run out of the room any faster. :) frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production? for development? seriously? develop a application on Windows which finally runs on a different OS? not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine) ______________________________ _________________ Mailing list: https://launchpad.net/~maria-d iscuss Post to : maria-discuss(a)lists.launchpad. net Unsubscribe : https://launchpad.net/~maria-d iscuss More help : https://help.launchpad.net/Lis tHelp -----Segue allegato----- _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

1 0

Re: [Maria-discuss] MariaDB Server 10.3 notes
by Federico Razzoli 15 Oct '16

15 Oct '16

A lot of things don't run on Windows: Galera, Percona Toolkit, Xtrabackup... Is this really a problem for MySQL users? Federico -------------------------------------------- Sab 15/10/16, Reindl Harald <h.reindl(a)thelounge.net> ha scritto: Oggetto: Re: [Maria-discuss] MariaDB Server 10.3 notes A: maria-discuss(a)lists.launchpad.net Data: Sabato 15 ottobre 2016, 19:10 Am 15.10.2016 um 17:22 schrieb Tim Callaghan: > Every time the topic of getting TokuDB to run on Windows the developers > could not have run out of the room any faster. :) frankly what is the purpose of running MySQL/MariaDB on Windows? who does that in production? for development? seriously? develop a application on Windows which finally runs on a different OS? not in 2016 when anybody sane in his mind would setup a virtual machine so that his testing servers are really reflecting the later production environment (for the poor guys which are running Windows on their bare metal development machine) _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

2 1

[Maria-discuss] Reg replication and commit
by Karthick Subramanian 14 Oct '16

14 Oct '16

Hi All, I have the below topology: Master -> Slave (using GTID replication) Master -> Maxscale (Binlog Router) Maxscale -> DR Site (via binlog router) my.cnf is same across all sites: lower-case-table-names=1 autocommit=0 #replication variables log-bin=binlog binlog-format=ROW replicate_annotate_row_events=OFF binlog_annotate_row_events=OFF I have created a table and I could see on slave DB as well as at DR Site DB. When I INSERT records, this is not shown up in Slave DB as well DR Site DB. I have checked replication status and both are sync with master server binlog pos. But the rows reflected in slave and DR site DB only when we COMMIT on both Slave as well DR site DB. Below when I try at Slave DB: MariaDB [dr_repl]> select * from test_dr_repl; Empty set (0.00 sec) MariaDB [dr_repl]> commit; Query OK, 0 rows affected (0.00 sec) MariaDB [dr_repl]> select * from test_dr_repl; +----+------+ | id | val | +----+------+ | 1 | 1 | | 2 | 2 | | 3 | 3 | +----+------+ 3 rows in set (0.00 sec) I am wondering whether I did any weird mistake. Could anyone please help me to understand and a solution for this. Regards, Karthick

3 3

Re: [Maria-discuss] MariaDB Server 10.3 notes
by Federico Razzoli 14 Oct '16

14 Oct '16

Hi all, So basically everyone would love, love, love to have external languages for stored procedures, but no one is working on it... so bad. Please consider something: 1- Some features could be implemented as stored procedures, it's much easier. This has been done in the past (Flexviews, Securich...) but SQL is too limited. 2- I am sure that a lot of people would implement procedures libraries if they could use something like JavaScript or PHP. If we could use Python, stuff like NumPy and SciPy could be used. 3- SQL limitations could be lifted (namespaces for global objects, arrays, argv, cursors based on a prepared statement...) but will you ever do it? Probably not. But if we have external languages, who cares about procedural SQL limitations. I believe that next releases features are selected based on their cost (not only that of course). But please consider points 2 and 3, and try to estimate the cost of features that the community can develop for you, plus the cost of features that won't be really needed anymore if external languages are available (arrays in SQL). Cheers Federico -------------------------------------------- Mer 12/10/16, Sergei Golubchik <serg(a)mariadb.org> ha scritto: Oggetto: Re: [Maria-discuss] MariaDB Server 10.3 notes A: "Justin Swanhart" <greenlion(a)gmail.com> Cc: "Maria Discuss" <maria-discuss(a)lists.launchpad.net> Data: Mercoledì 12 ottobre 2016, 14:15 Hi, Justin! Very good questions, thanks! Some answers below: On Oct 12, Justin Swanhart wrote: > > > * InnoDB: InnoDB native partitioning - so MySQL 8 InnoDB? But Monty > > says there's next to no changes in InnoDB 8... Instant add column. > > New InnoDB deadlock detection (8.0). New INFORMATION_SCHEMA table > > (8.0). Dedicated tablespace for temporary tables (in 5.7 and merged, > > check). Lock wait policy (contribution) > > Monty is *notorious* for low balling estimates. His famous phrase is "it > is trivial". Everybody knows that if Monty says it is trivial, you can add > 10x the work to get it done, or more. Yes, I tend to agree with that. But Monty estimates are not that far off when applied to *Monty*. They're often too low when applied to others. https://www.google.de/search?q=10x+developer > includes transportable tablespaces for partitioned tables, and will likely > support native partitioning by the time 8 rolls. Native partitioning also > entails implementing the changes to the SE. See: > http://mysqlserverteam.com/innodb-native-partitioning-early-access/ InnoDB native partitioning is in 5.7.6: https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-6.html Is it better than upper-layer partitioning? Why? > > * more for window functions (user defined window functions, MDEV-10855) > What other databases have user definable window functions? Will the syntax > be similar? I can't really think of a reason to need custom window > functions personally, as the existing set is very comprehensive. What kind > of custom window function do you think people would need to write? In most > implementations any aggregate functions can be used in a window frame > context, thus any aggregate SQL functions should satisfy requirements for > user defined window funcs. Yes, that was a confusing description. What it really means - we will have aggregate SQL functions (MDEV-7773) and preferrably, it should be possible to make them window-aware. Using an aggregate function as a window function, means applying it to every possible position of a sliding frame, and it has O(N*n) complexity (where N is total number of rows, n is the number of rows in the frame). If the aggregate function can remove rows from a group, it will only be O(N). > > * socket authentication > Can you explain this. Is there an MDEV? That's using unix_socket authentication by default for the root user. It improves security and maintainability, but is not really new - Debian is already doing that for months. This change could be confusing and break exising user scripts, so it needs to be done with care. > > * X Protocol/Document store - only if people have time or money will > > it be done > If you want people to pay for it, perhaps you should implement it in > MaxScale instead of the server :) Interesting idea :) But the MariaDB Meetup is a community event organized by the MariaDB Foundation. This particular session was about MariaDB Server planning. MariaDB Foundation has nothing to do with MaxScale, so we could not have planned anything for it. > >* Peter asks if there is any plans to support other languages like V8? > PLEASE PLEASE PLEASE PLEASE IMPLEMENT WL-820. External stored procs and > table functions. Been sitting there for the taking for a long time. > Antony has tried to get you to get it into the server, but alas, it has > never happened, and though there continues to be wide requests for this > feature from the community, they fall on deaf ears. Frankly speaking, I'd love it. But this feature never got enough priority, not in MySQL times nor in MariaDB. Unfortunately. Perhaps we'll be able to sneak it into 10.3, but no promises here. > > * Compressed binary log (from Tencent) > Compressing the binary log saves on space on the master, but it makes > seeking into binary logs much more difficult, and searching backwards > through them becomes much more difficult (which has implications for query > 'rewind') Tencent compresses individual events (think Compressed_query_log_event type). So seeking works as before, events are not decompressed on reading, they are sent compressed to slaves, etc. But the compression ratio is worse, of course. > > * Fix the XA transaction bug ( MySQL has fixed it already ) - MDEV-7974 > Please actually complete XA support, don't just half fix it like Oracle > did. Add full support for XA SUSPEND and XA RESUME and allow more than one > thread to participate in a distributed transaction in the server. Right. Still, MDEV-7974 is an important step, we cannot have proper XA without it. XA SUSPEND and XA RESUME should be the next one, I agree. > > * Indexes on expressions (this is part of virtual columns, will it not > > go into 10.2? Check with Serg) > Indexes on expressions requires parser support. > create index expr_idx on some_table(a + b); > select * from some_table where a+b > 30 and a+b <= 50 -- (uses range over > expr_idx) Right, but the parser support is the least of my worries. I don't want to low ball estmates :) but the *parser* can be fixed in a few hours. The most tricky part will be to fix the optimizer. > > * Flashback DDL MDEV-10571 (flashback DML will come in 10.2). It only > > works with row based replication. Talk about what to name it. There's > > already a MySQL time machine on github. Many like the name Rewind (but > > not Monty). Let's do a poll on the mailing list > Okay, Flashback query is VERY complicated. A flashback query is a > materialized view. There are two ways generally to achieve flashback: > a) materialize the query as is, and instead of rolling the query forward > incrementally, you roll it backwards. Flexviews achieves this by computing > the query as it is now, then computing the delta from a prior point in time > until the transaction in which that computation happened. Then the delta is > played back against the query, but the MONUS of each operation is applied > which changes insertions to deletions and vice versa. This still presents > a problem for OUTER JOIN. No documented asynchronous refresh algorithms > exist for outer join that I'm aware of, that would work with the concept of > reading row history from serialized changes. Flexviews uses the "rolling > join propagation" algorithm, which only works with inner joins. Interesting... I'll continue reading on that, thanks. But see below > b) provide a point-in-time snapshot of every table used in the query at the > desired point in time, and run the query over those tables (this is a > synchronous mechanism which supports OUTER JOIN). The problem here is how > to provide such a table. The most straight-forward way is to copy the > table, and then do the backwards replay for each (ideally in parallel) but > this is obviously undesirable if the query is "select * from some_big_table > join another_big_table on (...)" because you have to fully copy each table > before you can undo changes. Otherwise you need to have the SE display old > versions, just like it does for MVCC, but it has to display versions from > binary logs, not undo logs, and this requires a lot of SE and engine > changes! This is about correct. There are different applications for "flashback" with different use cases and different implementation trade-offs. One is, for example, to see historical sales numbers, for different months or years. That is, basically, for some kind of data analytics. Lots of SELECT queries, ad-hoc queries, at different historical time points. Another one is "damn, I've made a typo in a WHERE clause and updated too much". In this case one doesn't need joins or materialized views, one needs to see the data before the erroneous statement. This is not used often. For the second use case one can afford to copy tables and backward-replay the binary log. For the first use case one would need a completely different approach, I agree. May be something like what Flexviews does. > If you are going to have generic flashback, you might as well commit to > incrementally refreshable materialized views. Right, when we'll have materialized views, than we could think about incrementally updating them using rolling join propagation. > > * Additional GIS functions to stay compatible. Also it would be good > > to have a standalone GIS library (Georg suggests; wlad isn't too happy > > with the suggestion). Georg suggests that calculations should use the > > reference systems (Unflatten the world - MariaDB Server GIS the world > > looks flat) > > GIS functions should use gdal, just like postgresql does. Then you can > also add support for rasters. Here is an example of the awesomeness of > postgresql and postgis. It uses SQL aggregate functions, table functions, > CTE, GIS raster functions, sequences, etc: > https://github.com/greenlion/osmvox/blob/master/postgresql/combined_schema.… May be. On the other hand, we're more precise that postgis, because our implementation uses fixed-point math, not doubles. And let's not forget that MySQL uses Boost::Geometry. > > * Query rewriting - MDEV-5561 > I would like you to provide a SQL->DOM function call instead of just > providing a DOM to plugins. This function could be exposed as a regular > item function as well so that anything in the server can parse SQL. I > suggest you implement my SQL "shim" interface and just provide some way to > get a easy to iterate over parsed data structure from the SQL, such as a > nested JSON array of objects. A nested array is generated by > PHP-SQL-Parser and would provide a good template for such a JSON object. > https://github.com/greenlion/PHP-SQL-Parser We actually have an MDEV for that :) > > * With MyRocks coming, should we drop TokuDB (and maybe even deprecate > > in 10.2?) - bugs that MariaDB Corporation reports to Percona don't > > seem to get fixed. Peter says that bugs are fixed for customers... and > > there is ongoing development to make it better > It is certainly disheartening that Percona isn't responsive to MariaDB > bugs, but I'm sure you understand that it is hard for a competitor to fix > bugs for another competitor. MariaDB maintains a forked version of > TokuDB. It isn't fair to expect the upstream vendor to fix bugs that your > customers are paying you to support, does it? Perhaps you should pay a > percentage of your support fees for TokuDB issues to Percona, or come to > some other support agreement. Perhaps YOU should make the bug fixes and > submit them to Percona. And we do, look for bug reports I've reported on TokuDB to Percona - with patches :) But, really, Percona *is* fixing TokuDB bugs, it's just that they did a bit of refactoring after getting TokuDB and it took time for it to stabilize. And MariaDB Server has a vanilla TokuDB with almost no changes, we have no plans to fork it. > > * ORDER BY LIMIT optimizer bugs (MDEV-8306) > Oh god, this is a downward spiral every time somebody touches it. > Fixing it correctly requires rewriting the parser, something that > Oracle is undertaking. This didn't have much to do with the parser, and we've refactored that part of the parser in 10.2, so now this has nothing to do with the parser, it's purely the optimizer issue. > What about other new MySQL 8 features? Are you getting rid of the .FRM > nightmare? Are you going to support SET PERSIST? etc? FRM was made purely optional in 10.0 - every storage engine decides for itself whether it uses FRMs or not. May be InnoDB will use FRMs in 10.3, may be it will not. MyISAM most probably will continue use them. SET PERSIST - and this is my personal opinion - this needs to be thought over *very* carefully. There have been quite a few security vulnerabilities with my.cnf stored in the datadir, and that's why since 2005 the server no longer reads my.cnf in the datadir. But mysql_safe still does - and there have been new security vulnerabilities *last month*, caused by my.cnf in the datadir. And finally both MySQL (in 5.7?) and MariaDB (in 10.2) stopped reading my.cnf in the datadir for real. So, having my security(a)mariadb.org hat on, I look at SET PERSIST with a lot of suspicion, because it's nothing else than another attempt of storing server configuration information in the datadir and have it writable by the server itself. Regards, Sergei Chief Architect MariaDB and security(a)mariadb.org _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss(a)lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

6 11

Re: [Maria-discuss] MariaDB Server 10.3 notes
by Justin Swanhart 12 Oct '16

12 Oct '16

Hi Sergei! I moved this back to the mailing list, because of your .FRM explanation, and also because I'd like to explain rasters. A GIS raster is an array of data represented as a raster image. Rasters have "layers" and each layer can have a different measurement in it. It is most commonly used for height maps. The pgsql schema that I sent you is for a personal project of mine that uses Voxel.js (http://voxeljs.com/) to turn real world height maps from the space shuttle (SRTM v3 data) into interpolated 1 meter resolution voxel (minecraft-like, as 1 block = 1 sq meter) landscapes. I intersect the raster data with the GIS data from OSM (open street map) so that I can render buildings, roads, forest areas, rivers, etc, somewhat correctly (overpasses, underhangs, and missing building heights cause significant problems.). The ST_PixelsAsCentroids or ST_PixelsAsPoints can be used to get the specific value for a band in the raster, such as the height in a specific 30sq meter area (for the 1 arcsecond resolution of the SRTM data). I could never do such a thing in MySQL, and that makes me sad :( On Wed, Oct 12, 2016 at 9:34 AM, Sergei Golubchik <serg(a)mariadb.org> wrote: > Hi, Justin! > > On Oct 12, Justin Swanhart wrote: > > > > Regarding SET PERSIST, yes, it has dangers. One of them would be to > > set the buffer pool too large, so I filed a MySQL FR to set the BP to > > the default size if the malloc fails. It can be increased dynamically > > later. But there are many changes that could cause problems. This > > feature has to be managed carefully. Then again, Oracle RDBMS has had > > pfile configuration for well over a decade, and it is functionally > > equivalent to a binary my.cnf in the data dir. > > Right. Personally I fully expect that we will need to suport SET PERSIST > eventually. It's just so convenient. I only said it needs some very > careful thinking - and I was responsible for security(a)mysql.com for ten > years and now for security(a)mariadb.org for six, so I am a bit paranoid > about this :) > > > re: GIS, well, gdal and proj are used by almost every application > > that deals with GIS. What does the extra accuracy get you? Anyway, > > rasters really aren't useful without table functions, as you can't get > > the data from each point without them. > > I don't really know what rasters are. > > And I do not actually *use* GIS, so may be extra accuracy is not useful. > I'd think it might be particularly important for functions like > ST_TOUCHES(). > > I really want MariaDB to have table functions - perhaps we will have > them in 10.3. > > > Some more questions: > > Even though .FRM are optional, that still doesn't necessarily make DDL > > changes atomic, does it? > > If the engine declares FRM optional, then the whole DDL is happening > inside the engine, and it's up to the engine to make it atomic. I > expect that when we'll make FRMs optional for InnoDB, InnoDB DDLs > will be atomic. > > Regards, > Sergei > Chief Architect MariaDB > and security(a)mariadb.org >

1 0