- developers - lists.mariadb.org

[Maria-developers] Additions to GIS in 10.1.
by Alexey Botchkov 23 Jan '15

23 Jan '15

Hi all. What's new about GIS in 10.1. Related tasks in JIRA are: MDEV-4045 Missing OGC Spatial functions. MDEV-60 Support for Spatial Reference systems for the GIS data. MDEV-12 OpenGIS: create required tables: GeometryColumns, related views. Speaking shortly, the MariaDB GIS part is now OpenGIS compliant, and passes all the OpenGIS conformance tests To get that the next things were done: Missing functions were added: IsRing(ln) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; return TRUE if c is a ring, i.e., if c is closed and simple. A simple Curve does not pass through the same Point more than once. PointOnSurface(s) The return type is POINT, returns a Point guaranteed to lie on the Surface. arguments: POLYGON or MYLTIPOLYGON. NumInteriorRing(p) The return type is Integer, returns the number of interiorRings arguments: POLYGON. Relate(g1 Geometry, g2 Geometry, patternMatrix String) The return type is Integer, with a return value of 1 for TRUE, 0 for FALSE, and –1 for UNKNOWN corresponding to a function invocation on NULL arguments; returns TRUE if the spatial relationship specified by the pattern Matrix holds. ConvexHull(g1 Geometry) The return is Geometry, returna a geometric object that is the convex hull of g1 arguments: GEOMETRY. The information about the GEOMETRY column's reference system (SRID) is now stored in the field's metadata. Syntax of the CREATE and ALTER was modified to specify the SRID for the geometry columns. With this information some views required by OpenGIS, were added: GEOMETRY_COLUMNS, SPATIAL_REF_SYS: These are links to the INFORMATION_SCHEMA.GEOMETRY_COLUMNS and INFORMATION_SCHEMA.SPATIAL_REF_SYS respectively. INFORMATION_SCHEMA.GEOMETRY_COLUMNS: describes the available feature tables and their Geometry properties. Fields: F_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, F_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, F_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the fully qualified name of the feature table containing the geometry column; F_GEOMETRY_COLUMN CHARACTER VARYING(256) NOT NULL, the name of the column in the feature table that is the Geometry Column. G_TABLE_CATALOG CHARACTER VARYING(256) NOT NULL, G_TABLE_SCHEMA CHARACTER VARYING(256) NOT NULL, G_TABLE_NAME CHARACTER VARYING(256) NOT NULL, the above 3 fields are the name of the geometry table and its schema and catalog. The geometry table implements the geometry column; STORAGE_TYPE INTEGER, always 1 in MariaDB (means binary geometry implementation) GEOMETRY_TYPE INTEGER, the type of geometry values stored in this column. 0 = GEOMETRY 1 = POINT 3 = LINESTRING 5 = POLYGON 7 = MULTIPOINT 9 = MULTILINESTRING 11 = MULTIPOLYGON COORD_DIMENSION INTEGER, the number of dimensions in the spatial reference system. Always 2 in MariaDB. MAX_PPR INTEGER, Always 0 in MariaDB. SRID INTEGER the ID of the Spatial Reference System used for the coordinate geometry in this table. It is a foreign key reference to the SPATIAL_REF_SYS table. INFORMATION_SCHEMA.SPATIAL_REF_SYS: stores information on each spatial reference system used in the database. Fields: SRID INTEGER NOT NULL PRIMARY KEY, an integer value that uniquely identifies each Spatial Reference System within a database. AUTH_NAME CHARACTER VARYING(256), the name of the standard or standards body that is being cited for this reference system. In most cases the value is "EPSG". AUTH_SRID INTEGER TEXT the Well-known Text Representation of the Spatial Reference System. Required stored procedures added: AddGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME, SRID) adds new column of spatial type to the specified table DropGeometryColumn(FEATURE_TABLE_CATALOG, FEATURE_TABLE_SCHEMA, FEATURE_TABLE_NAME, GEOMETRY_COLUMN_NAME) removes the spatial column from the table. That's it for now. Best regards. HF

1 0

[Maria-developers] Renaming Virtual Columns to Computed Columns
by Ian Gilfillan 19 Jan '15

19 Jan '15

There was a suggestion on the docs mailing list that we rename Virtual Columns to Computed Columns. I think it makes sense, but before we implement I want to make sure everyone is happy with the change, so am repeating it here as not everyone follows docs. Currently, we use the term Virtual Columns (see https://mariadb.com/kb/en/mariadb/documentation/sql-structure-and-commands/…) Oracle uses this term, while MS SQL and Firebird use the term Computed Columns to describe similar functionality. MariaDB's virtual column implementation has both persistent and virtual attributes. Talking of "virtual virtual" and "persistent virtual" can be confusing. Oracle's implementation, to my knowledge, only has virtual virtual, while MS SQL's Computed Columns offers both persistent ("persisted" in their case) and virtual. I think "Computed" is a slightly more descriptive term, and avoids the ambiguity of "persistent virtual" and "virtual virtual". The MariaDB error messages already use the term computed, for example "Constant expression in computed column function is not allowed". Unless anyone has a good reason to prefer the term virtual, we're going to rather use the term "computed column" in the docs. The article will still mention virtual column as an alternative etc, but computed will be the preferred term. ian

3 3

Re: [Maria-developers] [Commits] Rev 4562: MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list in lp:~maria-captains/maria/10.0
by Kristian Nielsen 19 Jan '15

19 Jan '15

Sergei Golubchik <serg(a)mariadb.org> writes: > ------------------------------------------------------------ > revno: 4562 > revision-id: sergii(a)pisem.net-20150118130747-rs67yzzk8a7ebwdd > parent: sergii(a)pisem.net-20150118002113-snztyzwvzc500c2m > fixes bug: https://mariadb.atlassian.net/browse/MDEV-7402 > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 10.0 > timestamp: Sun 2015-01-18 14:07:47 +0100 > message: > MDEV-7402 'reset master' hangs, waits for signalled COND_xid_list > > Using a boolean flag for 'there is a RESET MASTER in progress' doesn't > work very well for multiple concurrent RESET MASTER statements. Ok to push with a testcase like the below, as we discussed. - Kristian. ----------------------------------------------------------------------- --echo *** MDEV-7402: 'reset master' hangs, waits for signalled COND_xid_list *** --source include/wait_for_binlog_checkpoint.inc connect(con3,localhost,root,,); # Make the binlog background thread wait before clearing the pending checkpoint. # The bug was that one RESET MASTER would clear the reset_master_pending # flag set by another RESET MASTER; this could cause the wakeup from the # binlog background thread not to be sent, and thus the second RESET MASTER # to wait infinitely. SET debug_sync="reset_logs_after_set_reset_master_pending SIGNAL reset_master_ready WAIT_FOR reset_master_cont"; send RESET MASTER; --connection default SET @old_dbug= @@global.DEBUG_DBUG; SET GLOBAL debug_dbug="+d,inject_binlog_background_thread_before_mark_xid_done"; SET debug_sync="now WAIT_FOR reset_master_ready"; RESET MASTER; SET debug_sync="now WAIT_FOR injected_binlog_background_thread"; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync="now SIGNAL reset_master_cont"; --connection con3 REAP; --connection default SET debug_sync = 'reset';

1 0

Re: [Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Sergei Golubchik 17 Jan '15

17 Jan '15

Hi, Vicențiu! On Jan 17, Vicențiu Ciorbaru wrote: > > Here's the new diff, let me know if you have any more comments. Shall I > push this as is? Yes, just one small detail (file name and the comment), see below. Then please push, no need to ask for another review :) But... 1. Please don't paste patches into the email anymore. At least not with your mail client that wraps long lines around and sends everything as html. See https://mariadb.com/kb/en/using-git/#commit-emails 2. Why git? This is the patch for 10.0 bug, at the moment 10.0 is still in bzr. 3. This looks like a reverse diff, how did you manage to do that? :) > diff --git a/mysql-test/t/statistics_index_crash.test b/mysql-test/t/statistics_index_crash.test > deleted file mode 100644 > index dbbdca3..0000000 > --- a/mysql-test/t/statistics_index_crash.test if you create a new test file for one bug only (as opposite to adding to an existing test file), mention the bug number in the file name. Like statistics_index_crash-7362.test > +++ /dev/null > @@ -1,26 +0,0 @@ > ---source include/have_stat_tables.inc > ---source include/have_innodb.inc Start a test case for a bug with a comment that says the bug number, the synopsys, and optionally more information, if needed. The bare minimum looks like # # MDEV-7362 ANALYZE TABLES crash with table-independent-statistics gathering # MDEV-7380 engine-independent stats SEGV on ANALYZE TABLE (#2) # > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=InnoDB; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; > -INSERT INTO t1 VALUES (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); > -ANALYZE TABLE t1 PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='a' AND table_name='t1'; > -DROP TABLE t1; > - > -CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; > -INSERT INTO geom VALUES > - (MultiPolygonFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))')), > - (MPolyFromWKB(AsWKB(MultiPolygon(Polygon(LineString(Point(0, 3), Point(3, 3), Point(3, 0), Point(0, 3))))))); > -ANALYZE TABLE geom PERSISTENT FOR ALL; > ---sorted_result > -SELECT * FROM mysql.index_stats WHERE index_name='g' AND table_name='geom'; > -DROP TABLE geom; Regards, Sergei

1 0

[Maria-developers] [MDEV-7362] ANALYZE TABLES crash with table-independent-statistics gathering
by Vicențiu Ciorbaru 17 Jan '15

17 Jan '15

Hi Sergei! I've implemented a fix for the bug described in MDEV-7362, that also fixes MDEV-7380. The test case also covers MDEV-7380. With the following fix,the I've spent a bit of time trying to figure out the architecture of things there. I am not confident my fix is complete, or if this is the way we want to go about fixing it. The fix does what we discussed over IRC, that we are not interested in computing statistics for a FULLTEXT index. What I think would be the root cause of the problem is calling ha_index_init() on an index that does not support this. By calling table->file->ha_index_init(), the table->field array gets corrupted and that ultimately leads to a crash in the function using it afterwards. Then again, this module is completely new to me, so any comments are appreciated. Here is the diff of the first changes. diff --git a/mysql-test/t/mdev-7362.test b/mysql-test/t/mdev-7362.test new file mode 100644 index 0000000..6551893 --- /dev/null +++ b/mysql-test/t/mdev-7362.test @@ -0,0 +1,14 @@ +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541726D')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (a longtext, FULLTEXT KEY (`a`)) ENGINE=MyISAM; +insert into t1 values (unhex('3E0D0A4141414142334E7A6143317963324541414141424977414141674541')); +analyze table t1 persistent for all; +drop table t1; + +CREATE TABLE t1 (f longtext NOT NULL, FULLTEXT KEY f (f)) ENGINE=MyISAM; +INSERT INTO t1 VALUES (REPEAT('a',27)),('foo'); +ANALYZE TABLE t1 PERSISTENT FOR ALL; +drop table t1; diff --git a/sql/sql_statistics.cc b/sql/sql_statistics.cc index d368145..2c1cccf 100644 --- a/sql/sql_statistics.cc +++ b/sql/sql_statistics.cc @@ -2367,6 +2367,9 @@ int collect_statistics_for_index(THD *thd, TABLE *table, uint index) DBUG_RETURN(rc); } + if (key_info->flags & HA_FULLTEXT) // skip if FULLTEXT index + DBUG_RETURN(rc); + table->key_read= 1; table->file->extra(HA_EXTRA_KEYREAD); Regards, Vicențiu

3 5

[Maria-developers] MDEV-7440 Build fails in libgroonga on Solaris
by Alexander Barkov 16 Jan '15

16 Jan '15

Hello Kentoku, Can you please have a look into this problem? https://mariadb.atlassian.net/browse/MDEV-7440 Can you fix please? Thanks.

1 0

Re: [Maria-developers] [Commits] Rev 4398: MDEV-7368 : SLES: Failed to start mysql.service: Unit
by Sergei Golubchik 15 Jan '15

15 Jan '15

Hi, Nirbhay! On Dec 23, Nirbhay Choubey wrote: > At lp:~maria-captains/maria/5.5 > > ------------------------------------------------------------ > revno: 4398 > revision-id: nirbhay(a)mariadb.com-20141224033207-8lh5i8d84qsjey79 > parent: elenst(a)montyprogram.com-20141223172123-pkfhjnv7oy1suxpa > committer: Nirbhay Choubey <nirbhay(a)mariadb.com> > branch nick: b7368-5.5 > timestamp: Tue 2014-12-23 22:32:07 -0500 > message: > MDEV-7368 : SLES: Failed to start mysql.service: Unit > mysql.service failed to load > > Added 'systemctl daemon-reload' command in postin and > postun rpm scripts. > === modified file 'support-files/rpm/server-postin.sh' > --- a/support-files/rpm/server-postin.sh 2013-11-20 11:18:46 +0000 > +++ b/support-files/rpm/server-postin.sh 2014-12-24 03:32:07 +0000 > @@ -1,6 +1,10 @@ > > # Make MySQL start/shutdown automatically when the machine does it. > if [ $1 = 1 ] ; then > + if [ -x /usr/bin/systemctl ] ; then > + /usr/bin/systemctl daemon-reload >/dev/null 2>&1 > + fi > + looks ok > if [ -x /sbin/chkconfig ] ; then > /sbin/chkconfig --add mysql > fi > @@ -51,30 +55,31 @@ > SETARGETDIR=/etc/selinux/targeted/src/policy > SEDOMPROG=$SETARGETDIR/domains/program > SECONPROG=$SETARGETDIR/file_contexts/program > +if [ -f /etc/redhat-release ] ; then > + if grep -q "Red Hat Enterprise Linux .. release 4" /etc/redhat-release \ > + || grep -q "CentOS release 4" /etc/redhat-release ; then I'd rather written if grep -Eqs '(Red Hat Enterprise Linux ..|CentOS) release 4' /etc/redhat-release; then or (which is more portable, but perhaps unnecessary so for an rpm postin script) if grep '$Red Hat Enterprise Linux ..\|CentOS$ release 4' /etc/redhat-release >/dev/null 2>&1; then but if you'd like you can push your variant. Regards, Sergei

2 1

Re: [Maria-developers] [Commits] Rev 4401: MDEV-7448 - mtr may leave stale mysqld in lp:maria/5.5
by Sergei Golubchik 14 Jan '15

14 Jan '15

Hi, Sergey! On Jan 14, Sergey Vojtovich wrote: > ------------------------------------------------------------ > revno: 4401 > revision-id: svoj(a)mariadb.org-20150114135038-jss2vvpkhihysfmj > parent: sergii(a)pisem.net-20150110130746-xlrofqddt4j8xkw6 > committer: Sergey Vojtovich <svoj(a)mariadb.org> > branch nick: 5.5 > timestamp: Wed 2015-01-14 17:50:38 +0400 > message: > MDEV-7448 - mtr may leave stale mysqld > > Let mtr handle SIGHUP the same way as SIGINT. This solves stale mysqld processes > left after broken/closed ssh connection. Looks ok. But please make sure that all tests pass before pushing it. mysqld uses SIGHUP to print various information to the error log. If there's a test that actually tests that by sending SIGHUP - your change might've broken it. Regards, Sergei

1 0

[Maria-developers] MySQL bug #68557 "sjis changes in 5.5 break date parsing" fixed in MariaDB >= 5.5.38
by Hartmut Holzgraefe 13 Jan '15

13 Jan '15

Any idea how that happened, as I can't find anything that seems to be related in the 5.5.38 changelog? http://bugs.mysql.com/bug.php?id=68557 -- Hartmut Holzgraefe, Principal Support Engineer (EMEA) MariaDB Corporation | http://www.mariadb.com/

2 1

[Maria-developers] Buildbot slow upload
by Kristian Nielsen 13 Jan '15

13 Jan '15

Hi Serg, Daniel, Some time ago there was discussion of slow Buildbot upload speed (sorry, don't have the original mail thread to reply to). I found some new information on this. Yesterday I had to do a custom build, but was not able to because my tarbake upload kept getting stuck. So I had to investigate. Turns out the real problem was that hasky:/media/backup/archive/pack was full. And apparently, the error handling in Buildbot upload step is broken, so the error is not caught properly but just causes the upload to hang. I have noticed this before, that upload gets stuck. I wonder if full disk on hasky has caused this before. Daniel: Do we have something that removes old builds from /archive/pack? I did not see anything in root or buildbot crontabs? With respect to slow uploads, while trying various things, one thing I did was increase the upload blocksize in Buildbot config. Now I see tarbake upload (50MB) taking just 1 minute, which seems fair (probably it can be more if the network or buildbot is busy). So I'm wondering if the slow upload that was seen was in fact a combination of too low default blocksize, and uploads occasionally getting _completely_ stuck? It's of course still fine to move to using rsync, as Serg proposed earlier; just wanted to mention what I found. And in either case, we still need a solution to prevent /archive/pack from running full in the future... - Kristian.

2 1