- discuss - lists.mariadb.org

Spatial indexes, innodb, myisam and update lock cannot be acquired during a READ UNCOMMITTED transaction.
by Pascal Boulesteix 27 Mar '24

27 Mar '24

Hello everyone I am currently testing the migration of my web server to PHP 8.2.13 and mariadb 11.2.2. During my tests, I got stuck on a SELECT problem on tables using spatial indexes. During a complex page chaining several CREATE TABLEs on temporary tables with spatial indexes, a SELECT planted with the message "update lock cannot be acquired during a READ UNCOMMITTED transaction". After consulting Google, I first unblocked the situation by removing the spatial indexes on temporary tables but then the performance of intermediate queries collapsed. In the end, after doing a test package, I definitely found a solution: the default database engine is innodb. By forcing the myisam engine on the temporary tables the page executes in a few seconds! Questions: - what to think of this use case of spatial indexes with innodb vs myisa? - Is this a bug? Below are the DMLs for queries with myisam explicitly indicated. With innodb, the last query in the list crashes, either timeout if I execute via php, or "update lock cannot be acquired during a READ UNCOMMITTED transaction" if I execute the query via phpmyadmin 1 CREATE TABLE TEMPTABLE_localites_du_site(Localite_unik_commune int, Localite_dept varchar(15), Localite_unik int, GPS_unik int, Type_Localite varchar(1), Localite_p_coordonnees_spacial point not null) ENGINE = myisam ; 0 0 2 ALTER TABLE TEMPTABLE_localites_du_site ADD SPATIAL INDEX IDX_temptable_localites_du_site_Localite_p_coordonnees_spacial(Localite_p_coordonnees_spacial); 0 0 1 CREATE TABLE WKWNat_2A2B50F4_415A_443F_A959_38AB2F0EC1C6 (Localite_unik_commune int, Localite_dept varchar(15), Localite_unik int, GPS_unik int, Type_Localite varchar(1), Localite_p_coordonnees_spacial point, Maille_unik int, Maille_unik_site int, Maille_maille varchar(80), Maille_taille int, Maille_g_coordonnees_spacial geometry NOT NULL, Maille_couleur_bordure int, Maille_couleur_aire int, Maille_transparence_aire int, Maille_Exclusion bit NOT NULL, Longitude_Centre double(10,6), Latitude_Centre double(10,6), in_use_by_leudit bit NOT NULL, myTaille varchar(3)) ENGINE = myisam; 0 0 5 insert INTO TEMPTABLE_localites_du_site (SELECT DISTINCT wnat_lieuxdits.unik_commune, wnat_communes.Dept, wnat_lieuxdits.unik AS unik_lieudit, NULL AS unik_gps, 'L' AS Type_Localite, wnat_lieuxdits.p_coordonnees_spacial FROM wnat_lieuxdits INNER JOIN wnat_inventaires ON wnat_inventaires.unik_lieudit = wnat_lieuxdits.unik INNER JOIN wnat_communes ON wnat_communes.unik = wnat_lieuxdits.unik_commune WHERE wnat_inventaires.unik_organisme=10 AND ST_CONTAINS(ST_PolygonFromText('POLYGON((41.36475929466526 8.54315203804477,43.0275854970405 8.54315203804477,43.0275854970405 9.555702267296203,41.36475929466526 9.555702267296203,41.36475929466526 8.54315203804477))'),wnat_lieuxdits.p_coordonnees_spacial) ) 0 788 6 insert INTO TEMPTABLE_localites_du_site (SELECT DISTINCT wnat_lieuxdits.unik_commune, wnat_communes.Dept, wnat_lieuxdits.unik AS unik_lieudit, wnat_gps.unik AS unik_gps, 'G' AS Type_Localite, wnat_gps.p_coordonnees_spacial FROM wnat_lieuxdits INNER JOIN wnat_gps ON wnat_lieuxdits.unik = wnat_gps.unik_lieudit INNER JOIN wnat_inventaires ON wnat_gps.unik = wnat_inventaires.unik_gps INNER JOIN wnat_communes ON wnat_communes.unik = wnat_lieuxdits.unik_commune WHERE wnat_inventaires.unik_organisme=10 AND ST_CONTAINS(ST_PolygonFromText('POLYGON((41.36475929466526 8.54315203804477,43.0275854970405 8.54315203804477,43.0275854970405 9.555702267296203,41.36475929466526 9.555702267296203,41.36475929466526 8.54315203804477))'),wnat_gps.p_coordonnees_spacial) ) 0 29566 7 DELETE TEMPTABLE_localites_du_site FROM TEMPTABLE_localites_du_site INNER JOIN wnat_lieuxdits ON ( TEMPTABLE_localites_du_site.Localite_unik = wnat_lieuxdits.unik) INNER JOIN wnat_communes ON (wnat_lieuxdits.unik_commune = wnat_communes.unik) where wnat_communes.unik_organisme<>10 0 0 765 CREATE TABLE TEMPTABLE_W1 ENGINE = myisam AS SELECT MAX(wnat_geometries_mailles.taille) AS max_taille FROM TEMPTABLE_localites_du_site INNER JOIN wnat_geometries_mailles ON ST_CONTAINS(wnat_geometries_mailles.g_coordonnees_spacial, TEMPTABLE_localites_du_site.Localite_p_coordonnees_spacial) WHERE wnat_geometries_mailles.unik_geometrie_objet IN (121332,121333,121334,121335,121419,121385,121376,121383,121449,121450,121345,121373,121370,121371,121317,121379,121355,121362,121363,121367,121372,121346,121347,121348,121349,121365,121366,121378,121442,121443,121444,121448,121411,121404,121325,121406,121315,121392,121393,121394,121395,121396,121391,121343,121344,121407,121388,121389,121412,121413,121414,121415,121420,121421,121422,121423,121424,121322,121318,121319,121320,121321,121369,121360,121361,121426,121427,121428,121429,121430,121431,121432,121327,121356,121357,121405,121316,121408,121409,121425,121375,121351,121350,121323,121384,121390,121364,121324,121416,121417,121418,121368,121386,121387,121352,121447,121440,121441,121433,121434,121435,121328,121329,121330,121331,121446,121374,121451,121397,121377,121445,121382,121336,121337,121338,121339,121340,121341,121342,121436,121437,121438,121353,121354,121380,121381,121439,121326,121398,121399,121400,121401,121402,121403,121410,121358,121359,-1) AND wnat_geometries_mailles.Exclusion = FALSE; 0 1 query_Recordset1 insert INTO WKWNat_2A2B50F4_415A_443F_A959_38AB2F0EC1C6 (SELECT TEMPTABLE_localites_du_site.Localite_unik_commune, TEMPTABLE_localites_du_site.Localite_dept, TEMPTABLE_localites_du_site.Localite_unik, TEMPTABLE_localites_du_site.GPS_unik, TEMPTABLE_localites_du_site.Type_Localite, TEMPTABLE_localites_du_site.Localite_p_coordonnees_spacial, wnat_geometries_mailles.unik, wnat_geometries_mailles.unik_geometrie_objet, wnat_geometries_mailles.maille, wnat_geometries_mailles.taille, wnat_geometries_mailles.g_coordonnees_spacial, wnat_geometries_mailles.couleur_bordure, wnat_geometries_mailles.couleur_aire, wnat_geometries_mailles.transparence_aire, wnat_geometries_mailles.Exclusion, wnat_geometries_mailles.longitude_centre, wnat_geometries_mailles.latitude_centre, 1, MAX_TAILLE.max_taille AS myTaille FROM TEMPTABLE_localites_du_site INNER JOIN wnat_geometries_mailles ON ST_CONTAINS(wnat_geometries_mailles.g_coordonnees_spacial, TEMPTABLE_localites_du_site.Localite_p_coordonnees_spacial) INNER JOIN TEMPTABLE_W1 as MAX_TAILLE ON wnat_geometries_mailles.taille = MAX_TAILLE.max_taille WHERE wnat_geometries_mailles.unik_geometrie_objet IN (121332,121333,121334,121335,121419,121385,121376,121383,121449,121450,121345,121373,121370,121371,121317,121379,121355,121362,121363,121367,121372,121346,121347,121348,121349,121365,121366,121378,121442,121443,121444,121448,121411,121404,121325,121406,121315,121392,121393,121394,121395,121396,121391,121343,121344,121407,121388,121389,121412,121413,121414,121415,121420,121421,121422,121423,121424,121322,121318,121319,121320,121321,121369,121360,121361,121426,121427,121428,121429,121430,121431,121432,121327,121356,121357,121405,121316,121408,121409,121425,121375,121351,121350,121323,121384,121390,121364,121324,121416,121417,121418,121368,121386,121387,121352,121447,121440,121441,121433,121434,121435,121328,121329,121330,121331,121446,121374,121451,121397,121377,121445,121382,121336,121337,121338,121339,121340,121341,121342,121436,121437,121438,121353,121354,121380,121381,121439,121326,121398,121399,121400,121401,121402,121403,121410,121358,121359,-1) AND wnat_geometries_mailles.Exclusion = FALSE and in_use_by_lieudit<>0); 8 insert INTO WKWNat_2A2B50F4_415A_443F_A959_38AB2F0EC1C6 (SELECT TEMPTABLE_localites_du_site.Localite_unik_commune, TEMPTABLE_localites_du_site.Localite_dept, TEMPTABLE_localites_du_site.Localite_unik, TEMPTABLE_localites_du_site.GPS_unik, TEMPTABLE_localites_du_site.Type_Localite, TEMPTABLE_localites_du_site.Localite_p_coordonnees_spacial, wnat_geometries_mailles.unik, wnat_geometries_mailles.unik_geometrie_objet, wnat_geometries_mailles.maille, wnat_geometries_mailles.taille, wnat_geometries_mailles.g_coordonnees_spacial, wnat_geometries_mailles.couleur_bordure, wnat_geometries_mailles.couleur_aire, wnat_geometries_mailles.transparence_aire, wnat_geometries_mailles.Exclusion, wnat_geometries_mailles.longitude_centre, wnat_geometries_mailles.latitude_centre, 1, MAX_TAILLE.max_taille AS myTaille FROM TEMPTABLE_localites_du_site INNER JOIN wnat_geometries_mailles ON ST_CONTAINS(wnat_geometries_mailles.g_coordonnees_spacial, TEMPTABLE_localites_du_site.Localite_p_coordonnees_spacial) INNER JOIN TEMPTABLE_W1 as MAX_TAILLE ON wnat_geometries_mailles.taille = MAX_TAILLE.max_taille WHERE wnat_geometries_mailles.unik_geometrie_objet IN (121332,121333,121334,121335,121419,121385,121376,121383,121449,121450,121345,121373,121370,121371,121317,121379,121355,121362,121363,121367,121372,121346,121347,121348,121349,121365,121366,121378,121442,121443,121444,121448,121411,121404,121325,121406,121315,121392,121393,121394,121395,121396,121391,121343,121344,121407,121388,121389,121412,121413,121414,121415,121420,121421,121422,121423,121424,121322,121318,121319,121320,121321,121369,121360,121361,121426,121427,121428,121429,121430,121431,121432,121327,121356,121357,121405,121316,121408,121409,121425,121375,121351,121350,121323,121384,121390,121364,121324,121416,121417,121418,121368,121386,121387,121352,121447,121440,121441,121433,121434,121435,121328,121329,121330,121331,121446,121374,121451,121397,121377,121445,121382,121336,121337,121338,121339,121340,121341,121342,121436,121437,121438,121353,121354,121380,121381,121439,121326,121398,121399,121400,121401,121402,121403,121410,121358,121359,-1) AND wnat_geometries_mailles.Exclusion = FALSE and in_use_by_lieudit<>0); 0 17672

1 0

Advice concerning inconsistent query results
by Jef Van Loon 24 Mar '24

24 Mar '24

Hello, I am looking for advice concering inconsistent query results. Problem: We are observing queries that return inconsistent query results. The query in question selects records within a period of time (e.g., month) based on a datetime column. The result set is missing random records within the queried time period. It appears that the inconsistent result is produced when the query uses an index containing the datetime column. When the index is not used, the result seems to contain all records. Dropping and recreating the index caused the result to be complete again. However, this has NOT solved the problem permanently as the wrong behaviour has reappeared afterwards. On second appearance, we have used OPTIMIZE table on the table in question which has again resolved the problem, we will have to wait to see whether the problem occurs again. CHECK TABLE returned no erros for the affected table. The problem does not seem to be limited to one installation, we are seing it on different MariaDB installations. Questions: - Has anybody seen such behaviour in the past? - How can we diagnose the problem? Environment: - MariaDB Version: MariaDB 10.5.13 - MariaDB Storage engine: InnoDB - MariaDB Collation: utf8mb4_unicode_ci - OS: Windows Server Kind regards, Jef

4 7

Possible Security Bug
by Gordan Bobic 24 Mar '24

24 Mar '24

Hi, What is an appropriate forum for discussing what seems to be a security bug that I ran into?

3 3

Re: Setting up a jira self hosted instance
by Daniel Black 20 Mar '24

20 Mar '24

Query obviously from connector. Works in 11.3.3 MariaDB [test]> SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@collation_connection AS collation_connection, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS net_write_timeout, @@performance_schema AS performance_schema, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout\G *************************** 1. row *************************** auto_increment_increment: 1 character_set_client: utf8mb3 character_set_connection: utf8mb3 character_set_results: utf8mb3 character_set_server: latin1 collation_server: latin1_swedish_ci collation_connection: utf8mb3_general_ci init_connect: interactive_timeout: 28800 license: GPL lower_case_table_names: 0 max_allowed_packet: 16777216 net_write_timeout: 60 performance_schema: 0 sql_mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION system_time_zone: AEDT time_zone: SYSTEM transaction_isolation: REPEATABLE-READ wait_timeout: 28800 1 row in set (0.001 sec) MariaDB [test]> select version(); +----------------+ | version() | +----------------+ | 11.3.3-MariaDB | +----------------+ 1 row in set (0.000 sec) Obviously fails in earlier versions: MariaDB [test]> SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@collation_connection AS collation_connection, -> @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS net_write_timeout, @@performance_schema AS performance_schema, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout\G ERROR 1193 (HY000): Unknown system variable 'transaction_isolation' MariaDB [test]> select version(); +-----------------+ | version() | +-----------------+ | 10.11.8-MariaDB | +-----------------+ 1 row in set (0.000 sec) Are you sure you have the right version of MariaDB that you described? Otherwise upgrade/use MariaDB Connector/J. On Thu, 21 Mar 2024 at 04:45, <juanhernandez98(a)gmail.com> wrote: > > Just as a FYI, I enabled the MariaDB general logs: > > And I'm getting this when the jira instance tries and connect to my mriadb server: > > 240319 14:28:23 239 Connect jira@localhost on jira using TCP/IP > 239 Query SET SESSION default_storage_engine=InnoDB > 239 Query /* mysql-connector-j-8.3.0 (Revision: 805f872a57875f311cb82487efcfb070411a3fa0) */SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@collation_connection AS collation_connection, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS net_write_timeout, @@performance_schema AS performance_schema, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout > > > -----Original Message----- > From: Daniel Black via discuss <discuss(a)lists.mariadb.org> > Sent: Tuesday, March 19, 2024 9:57 PM > To: discuss(a)lists.mariadb.org > Subject: [MariaDB discuss] Re: Setting up a jira self hosted instance > > On Wed, 20 Mar 2024 at 07:20, Juan Hernandez via discuss <discuss(a)lists.mariadb.org> wrote: > > > > Hi, > > > > So I’ve been running MariaDB 11.3 for a while. I have various applications working great with this server. > > > > I want to setup a Jira 9.14 self hosted instance on one of my servers. I see mariadb.com uses Jira. > > > > So I put the mysql java connector v8.3.0, and the mariadb java connector v3.3.3 installed in my /opt/Atlassian/jira/lib directory. > > > > I tried connecting jira via the setup web interface to my maria db, using the mysql 8.0 driver, and I get the error that I’m not running the right version. > > > > Is that from the driver or from JIRA? > > > I then added under [mysqld] in my.cnf, version = “8.0.25-0” and now get the below error. > > > Error connecting to database > > > > Unknown system variable 'transaction_isolation' > > https://jira.mariadb.org/browse/MDEV-21921 - added this to 11.1, so you should be right in 11.3. Its not 10.11.3 right? > > Can JIRA use the MariaDB Java Connector v3.3.3? > > A last resort is to specify the version=5.7.44 (any 5.7) and/or use a > 5.7 compatible mysql java connector. > > > > I am now getting this error when testing the connection in jira: > > > > I don’t want to setup a postgresql instance or a mysql 8 instance, can I get this to work? Any help would be greatly appreciated. > > Us too. We're still battling against JIRA supported databases a little after all these years. > _______________________________________________ > discuss mailing list -- discuss(a)lists.mariadb.org To unsubscribe send an email to discuss-leave(a)lists.mariadb.org >

1 0

Setting up a jira self hosted instance
by juanhernandez98＠gmail.com 20 Mar '24

20 Mar '24

Hi, So I've been running MariaDB 11.3 for a while. I have various applications working great with this server. I want to setup a Jira 9.14 self hosted instance on one of my servers. I see mariadb.com uses Jira. So I put the mysql java connector v8.3.0, and the mariadb java connector v3.3.3 installed in my /opt/Atlassian/jira/lib directory. I tried connecting jira via the setup web interface to my maria db, using the mysql 8.0 driver, and I get the error that I'm not running the right version. I then added under [mysqld] in my.cnf, version = "8.0.25-0" and now get the below error. Error connecting to database Unknown system variable 'transaction_isolation' I am now getting this error when testing the connection in jira: I don't want to setup a postgresql instance or a mysql 8 instance, can I get this to work? Any help would be greatly appreciated. Best Juan

2 1

Trigger Fails
by steffan＠hldns.com 15 Mar '24

15 Mar '24

I just converted to MariaDB from MySQL where this trigger worked fine. DELIMITER ;; CREATE TRIGGER `email_uuid_before_insert` BEFORE INSERT ON `email` FOR EACH ROW BEGIN IF new.id_email IS NULL THEN SET new.id_email = uuid(); END IF; END;; DELIMITER ;; In MariaDB, it creates with no errors but when adding a row, I get an error 1062 : Duplicate entry '' for key 'PRIMARY' It’s caused because the last row that inserted wasn’t populated with that UUID – meaning the trigger failed. What would cause this to fail? Thank you, Steffan Cline steffan(a)hldns.com<mailto:steffan@hldns.com> 602-793-0014

2 2

Support for MariaDB Community Repo after EL7 EOL
by james.manley＠jadu.co.uk 14 Mar '24

14 Mar '24

Hi, We currently use MariaDB 10.5 from the MariaDB Community Repo on EL7. EL7 is end of life at the end of June 2024. Will MariaDB 10.5 continue to receive support and security updates until its end of life, or will support and updates stop on EL7 once the OS reaches end of life? Thanks!

2 1

MariaDB's Percona Tookit Fork Source Code Repository?
by Gordan Bobic 13 Mar '24

13 Mar '24

Hi, I noticed references to these tools: https://mariadb.com/kb/en/mariadb-tools/ but the github links are all 404. Where can all of the sources for these be downloaded from?

3 4

next and previous words
by ChipsRunner 24 Feb '24

24 Feb '24

Hi I've noticed that between debian 10 and 11 (mariadb-client-core-10.3 and mariadb-client-core-10.5) the client console tool (CLI) treats words in a different way: alt+f and alt+b will skip the dot on the new version. Example if I have something like 'test.domain.com' using alt+f on the old version would treat 'test', 'domain' and 'com' as three separate words while the new version would skip at the end of '.com' Is there a way to restore the old behaviour ? Thank you, PM

2 2

Back again for more help
by Ken Wright 21 Feb '24

21 Feb '24

Now that you kind people have pointed me in the right direction as far as log-bin was concerned, I've moved on to trying to replicate the databases from the primary to the secondary. I've created the connection profile and started the secondary connection, but when I ran show slave 'master01' status\G I got the following error: Last_Errno: 1062 Last_Error: Could not execute Write_rows_v1 event on table databasename.oc_filecache; Duplicate entry '110' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log master-bin.00000#, end_log_pos ###### How do I resolve this duplicate entry error? And do I need to do the same resolution on the primary as well? Ubuntu Server 23.10, mariadb version 15.1 Distrib 10.11.6-MariaDB Ken -- If you ever think international affairs make sense, remember this: because a Serb shot an Austrian in Bosnia, Germany invaded Belgium.

3 2

Setting up primary-secondary replication
by Ken Wright 17 Feb '24

17 Feb '24

Greetings! I'm trying to set up a backup email server using Ubuntu 23.10 Server. I want to replicate my MariaDB databases from the primary email server to the secondary, but whenever I add the line log-bin = /var/log/mysql/master-bin to /etc/mysql/mariadb.conf.d/50-server.cnf MariaDB fails to restart. I've looked for help online, but haven't had any luck. Can anyone help? Further details available on request. Ken -- If you ever think international affairs make sense, remember this: because a Serb shot an Austrian in Bosnia, Germany invaded Belgium.

3 4

Windows downloads
by madhon＠madhon.co.uk 15 Feb '24

15 Feb '24

Have the windows downloads for versions 10.6.17 and 10.11.7 been pulled as the only option i see are source and linux ?

3 2

Slow log and processlist.memory_used
by Gordan Bobic 13 Feb '24

13 Feb '24

In processlist, there is a column called memory_used. Is there a way to make the peak value for a query get logged into slow log? Failing that, does the performance_schema provide any such info for previously run statements to go with statement digests? Or is the only place where this can be found in the information_schema.processlist table while the query is running?

1 0

Upgrading MariaDB 10.6.15 to 10.6.17 on Ubuntu 18.04 (unsupported)
by stephan.vos＠metacom.net 11 Feb '24

11 Feb '24

We are currently unable to upgrade our server os from Ubuntu 18.04 to 22.04. So we are stuck on MariaDB 10.6.15 and cannot apply any subsequent minor/major releases. Is it possible to somehow use installation binary package for Focal/Jammy or worse case recompiling from source for 18.04?

4 10

Fwd: REGEXP_REPLACE behaviour
by Richard Ellerbrock 07 Feb '24

07 Feb '24

Hi All, I am trying to understand the behaviour of REGEXP_REPLACE when there is a backslash in the subject string. It appears that the subject needs to have \'s escaped which is not desirable (see example 1 below). I can understand having to escape \'s in the regex, but not the subject string. Any suggestions? Thanks. Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 397751 Server version: 10.11.5-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> SELECT REGEXP_REPLACE('Et12\4', '^Et', 'Ethernet'); +---------------------------------------------+ | REGEXP_REPLACE('Et12\4', '^Et', 'Ethernet') | +---------------------------------------------+ | Ethernet124 | +---------------------------------------------+ 1 row in set (0.000 sec) MariaDB [(none)]> SELECT REGEXP_REPLACE('Et12\\4', '^Et', 'Ethernet'); +----------------------------------------------+ | REGEXP_REPLACE('Et12\\4', '^Et', 'Ethernet') | +----------------------------------------------+ | Ethernet12\4 | +----------------------------------------------+ 1 row in set (0.000 sec) MariaDB [(none)]> SELECT REGEXP_REPLACE('Et12/4', '^Et', 'Ethernet'); +---------------------------------------------+ | REGEXP_REPLACE('Et12/4', '^Et', 'Ethernet') | +---------------------------------------------+ | Ethernet12/4 | +---------------------------------------------+ 1 row in set (0.000 sec) MariaDB [(none)]> SELECT REGEXP_REPLACE('Et12 4', '^Et', 'Ethernet'); +---------------------------------------------+ | REGEXP_REPLACE('Et12 4', '^Et', 'Ethernet') | +---------------------------------------------+ | Ethernet12 4 | +---------------------------------------------+ 1 row in set (0.000 sec) -- Richard Ellerbrock

2 3

Question regarding AMD EPYC servers and MariaDB
by highclass99 06 Feb '24

06 Feb '24

Hello, In the past when 2nd gen EPYC's were launching and beating Intel servers on most benchmarks, I visited a MariaDB conference and when I asked a senior MariaDB engineer, and asked whether he recommended AMD servers. At that time, he recommended Intel for MariaDB, because MariaDB had much more Intel testing equipment and was optimized for Intel. I am curious whether that is still the case, and whether MariaDB should be more optimized for Intel? For example, would AMD EPYC 7763 vs Intel Platinum 8380, would one be recommended over the other for MariaDB? (of course cores, and workload differ, but on a general perspective) Thank you.

3 3

Replication corrupts index on InnoDB
by Oliver Welter 05 Feb '24

05 Feb '24

Hi All, I have a very strange problem which I am not able to debug any further, so I kindly ask here for any advise to find the root cause of this issue. I run a master-slave replication setup, both servers run on Debian Bookworm with the packages from the distro (10.11.4-MariaDB-1~deb12u1-log). There are two tables in two databases with the same setup (same shop software) and both suffer from recurring index corruption on the slave side. While the master server works without any issues, the slave machine stops replicating from time to time with the error: Last_Errno: 1712 Last_Error: Error 'Index s_order_details is corrupted' on query. Default database: 'foobar_shop'. Query: 'DELETE FROM s_order_details WHERE orderID='40007'' After rebuilding the index by running "alter table foobar_shop.s_order_details engine=innodb" I can restart the slave and it works again. What I did so far: Rebuild the whole slave from fresh SQL Dumps (twice!) Changed replication to "Using_Gtid: Slave_Pos" Reduced slave parallel threads Settings as different from debian defaults binlog_format = mixed expire_logs_days = 1 max_binlog_size = 256M slave_parallel_max_queued=524288 slave_parallel_threads=5 sync_binlog=0 sync_relay_log=0 innodb_flush_log_at_trx_commit=0 best regards Oliver -- Protect your environment - close windows and adopt a penguin!

2 4

terminology: Oracle and non-Oracle?
by Federico Razzoli 04 Feb '24

04 Feb '24

Peter Gulutzan has a very good point that the terms "Oracle" and "non-Oracle" for procedural dialects are not ideal: http://ocelot.ca/blog/blog/2024/01/31/packages-in-mariadb-default-mode/ I'd suggest you choose a name for your native procedural language, and consistently use it in the docs. A lot of DBMSs did that. Postgres uses PL/pgSQL, so you could go for PL/mdbSQL? Cheers, Federico -- Federico Razzoli Vettabase Ltd director & founder, Database Consultant https://vettabase.com federico.razzoli(a)vettabase.com +44 7739 427279 Telegram: federico_razzoli Skype: fede.razzoli The information in this e-mail and any attachments may be confidential. If you received it by mistake, please inform the sender. Unless you are the intended recipient or his/her representative you are not authorised to, and must not, read, copy, distribute, use or retain this message or any part of it.

1 0

Repeatable read anomalies
by Kyle Kingsbury 16 Jan '24

16 Jan '24

Hi y'all! I've just completed an analysis of isolation levels in MySQL. The report focuses on MySQL, but the test suite runs against MariaDB as well. MariaDB exhibits all the same behaviors--"Repeatable Read" allows a whole bunch of anomalies that should be prohibited under Repeatable Read, including G2-item, G-single (even without predicates), non-repeatable reads, violations of Monotonic Atomic View, etc. Figured this might be of interest to the list. https://jepsen.io/analyses/mysql-8.0.34 I'm also curious whether this is widely-understood behavior in the MariaDB community, or if it's news to most people. The extant literature doesn't really seem to discuss these behaviors much, but maybe there's more tacit knowledge among experts? Sincerely, --Kyle

7 19

Mariadb with hibernate-6
by Rajesh Jose 14 Jan '24

14 Jan '24

Hello everyone, In our WildFly based application, we are using mariadb-10.5.8 with mysql-connector as the driver. From the compatibility matrix, it appears that mariadb-10.2 and later are compatible with mysql-5.7 ORM we use is hibernate-6.2 and the documentation says MySQL-5.7 or higher and MariaDB-10.3 or higher are supported with out-of-box dialects. . So the mariadb we use should be compliant to hibernate-6 if we go by mariadb version or corresponding mysql version. However, when we start our app, we get a WARNING message - "HHH000511: The 5.5.0 version for [org.hibernate.dialect.MySQLDialect] is no longer supported" When we look at the connection meta-data returned from DB, it returns "MySQL" as the product-name and "5.5.5-10.5.8-MariaDB" as the product version. That seems to be the reason, hibernate is complaining. But if we connect with the mariadb-driver, it return "MariaDB" as the product name and "10.5.8-MariaDB" as the version. - What does the 5.5.0 prefix signify in the version? Is it the compatible MySQL version? If so, why its not 5.7 as it appears from documentation that versions above mariadb-10.2 are compatible with mysql-5.7? - Is this WARNING a problem that warrant a DB migration or is there a community supported dialect as explained here - https://docs.jboss.org/hibernate/orm/6.2/migration-guide/migration-guide.ht… -Rajesh

2 2

Replication Speed / Lag 10.3.31 vs 10.3.39
by Michael Caplan 14 Jan '24

14 Jan '24

Hello, I'm trying to troubleshoot a performance a replication issue between two version of MariaDB. I know that we are using two EOL versions, but I want to rule out all issues before prioritizing migrating to a new version. I have built a new dedicated replication server (B) to replace the current dedicated replica (A). A and B are both running identical OSes and versions and near identical hardware in a RAID 10 configuration. The main difference is that B has newer disks that slightly outperform A. I benchmarked both A and B to validate this. Both A and B are configured identically (other then server-id). Replica A (and primary) is running MariaDB 10.3.31. Replica B is running 10.3.39. A few other things to note: * B is not feeding off of the primary, but A -- that is until we sunset A. * A is handing moderate read only traffic * B is just running as a replica with no other responsibilities I have external monitoring to alert when either hits 500+ Seconds_Behind_Master. Since standing B up, I am seeing it fall behind a few times a day. The interesting thing is that A is not falling behind from the primary. Upon further examination, a scheduled job that executes hundreds of similar delete operations off the same table (that has approx. 3 million records) is where B is largely getting hung up. Something like this: DELETE FROM table WHERE x = 800000 AND y = 40000 The table has an index on just x. Running ANALYZE FORMAT=JSON SELECT * FROM table WHERE x = 800000 AND y = 40000 from B show a striking difference in total time spent: REPLICA B: { "query_block": { "select_id": 1, "r_loops": 1, "r_total_time_ms": 490.13, "table": { "table_name": "table", "access_type": "ref", "possible_keys": ["table_index"], "key": "table_index", "key_length": "4", "used_key_parts": ["x"], "ref": ["const"], "r_loops": 1, "rows": 795555, "r_rows": 31, "r_total_time_ms": 490.08, "filtered": 100, "r_filtered": 100, "index_condition": "table.y = '40000'" } } } vs: REPLICA A: { "query_block": { "select_id": 1, "r_loops": 1, "r_total_time_ms": 106.37, "table": { "table_name": "table", "access_type": "ref", "possible_keys": ["table_index"], "key": "table_index", "key_length": "4", "used_key_parts": ["x"], "ref": ["const"], "r_loops": 1, "rows": 1146482, "r_rows": 31, "r_total_time_ms": 106.34, "filtered": 100, "r_filtered": 100, "index_condition": "table.y = '40000'" } } } That is a significant difference performance wise. From a hardware point of view, B should have a slight edge over A. But the contrary is true by a large margin. As a short term "fix", I added a secondary index: create index table_index2 on table(x, y); This significantly speeds up this operation, and, for now, seems to deal with much of the experienced replication lab in B: SERVER B: { "query_block": { "select_id": 1, "r_loops": 1, "r_total_time_ms": 0.253, "table": { "table_name": "table", "access_type": "ref", "possible_keys": [ "table_index", "table_index2" ], "key": "table_index2, "key_length": "8", "used_key_parts": ["x", "y"], "ref": ["const", "const"], "r_loops": 1, "rows": 31, "r_rows": 31, "r_total_time_ms": 0.2037, "filtered": 100, "r_filtered": 100 } } } This seems reasonable as a general optimization, but provides little comfort in understanding the root cause of my performance issue with B. I'm hesitant to retire A in favor of B until I can get to the bottom of this. The question I have for the community is: what would you recommend that I do to get further perspective on the issue to determine a course of action? Thanks everyone, Mike Confidentiality Notice: The information contained in this electronic message and any attachments to this message are intended only for the individual(s) addressed in the message and may contain proprietary and confidential information. If you are not the intended recipient, you should not disseminate, distribute, or copy this e-mail. Please notify the sender and destroy this message. WARNING: Computer viruses can be transmitted via email. The recipient should scan this email before opening it. The company accepts no liability for any damage caused by any virus transmitted by this email.

3 2

Data truncated using C API
by Jogchum Reitsma 08 Jan '24

08 Jan '24

Hi, Using server version: 11.2.2, I try to extract data from a database with this C-code: #include <stdlib.h> #include <stdio.h> #include <string.h> #include <mysql.h> define MAXFLEN 66 char *bo_src = "SELECT FILENAMME, FILENUM FROM BOEK WHERE TOKENS IS NULL ORDER BY UDC_KOADE, FILENUM"; char filename[MAXFLEN], stavering[3]; long filenum; int fetch_result; MYSQL *bo_src_sql; MYSQL_ST MT *bo_src_stmt; MYSQL_RES *bo_src_res; MYSQL_BIND bo_src_bind[2], bo_src_bind_result[2]; int main (argc, argv) int argc; char *argv[]; { bo_src_sql = mysql_init(bo_src_sql) ; if ((bo_src_stmt = mysql_stmt_init(bo_src_sql))==NULL) printf ("Can't initialize %s:\n %s/n", bo_src, mysql_error(bo_src_sql)) ; if (!mysql_real_connect(bo_src_sql,"localhost","<user>","<passwd>","tdb",0,NULL,0)) fprintf(stderr, "Failed to connect to book table for select: Error: %s\n", mysql_error(bo_src_sql)) ; if (mysql_stmt_prepare(bo_src_stmt, bo_src, strlen(bo_src))) printf ("Can't prepare %s:\n %s\n", bo_src, mysql_error(bo_src_sql)) ; bo_src_bind_result[0].buffer_type = MYSQL_TYPE_STRING ; bo_src_bind_result[0].buffer = filename ; bo_src_bind_result[0].buffer_length = 100 ; bo_src_bind_result[1].buffer_type = MYSQL_TYPE_LONG ; bo_src_bind_result[1].buffer = (long *)&filenum ; bo_src_bind_result[1].buffer_length = sizeof (filenum) ; memset (bo_src_bind, 0, sizeof (bo_src_bind)) ; memset (bo_src_bind_result, 0, sizeof (bo_src_bind_result)) ; if (mysql_stmt_bind_param (bo_src_stmt, bo_src_bind)) fprintf(stderr, "Failed to bind parameters in %s: \n Error: %s\n", bo_src, mysql_error(bo_src_sql)) ; if (mysql_stmt_bind_result(bo_src_stmt, bo_src_bind_result)) fprintf(stderr, "Failed to bind result in %s: \n Error: %s\n", bo_src, mysql_error(bo_src_sql)) ; if (!mysql_stmt_execute (bo_src_stmt)) { fetch_result = mysql_stmt_fetch(bo_src_stmt) ; if (fetch_result == 0) printf ("%ld %s %s\n", filenum, filename, stavering) ; else if (fetch_result == MYSQL_NO_DATA) printf ("No data found\n") ; else if (fetch_result == MYSQL_DATA_TRUNCATED) printf ("Data truncated.\n%d/n", fetch_result) ; else printf ("Something else! %d \n%s/n", fetch_result, mysql_error(bo_src_sql)); } ; mysql_stmt_close(bo_src_stmt) ; mysql_close(bo_src_sql); } The definition of the table is describe BOEK; +-----------+-------------------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-----------+-------------------------------+------+-----+---------+----------------+ | FILENUM | smallint(4) | NO | PRI | NULL | auto_increment | | TITEL | varchar(50) | YES | | NULL | | | UDC_KOADE | varchar(10) | YES | | NULL | | | ISBN | bigint(13) | YES | | NULL | | | FILENAMME | varchar(20) | YES | | NULL | | | SKR_NUM | smallint(5) | YES | | NULL | | | UTJ_NUM | smallint(5) | YES | | NULL | | | UTJ_DATUM | date | YES | | NULL | | | OERSET | tinyint(1) | YES | | NULL | | | PRINTINGE | smallint(3) | YES | | NULL | | | YNL_DATUM | date | YES | | NULL | | | STAVERING | enum('ald','mid','nij','int') | YES | | NULL | | | TOKENS | smallint(6) | YES | | NULL | | | TYPES | smallint(5) | YES | | NULL | | | MAXTOKENS | smallint(6) | YES | | NULL | | | MAXTYPES | smallint(5) | YES | | NULL | | | TIID | time | YES | | NULL | | +-----------+-------------------------------+------+-----+---------+----------------+ So the size of FILENAMME, varchar20, is much smaller than the buffer length specified in the code: ; bo_src_bind_result[0].buffer_type = MYSQL_TYPE_STRING ; bo_src_bind_result[0].buffer = filename ; bo_src_bind_result[0].buffer_length = 100 Yet I get a MYSQL_DATA_TRUNCATED fetching the result of the query. Debugging, only the integer filenum gets its value. The statement (SELECT FILENAMME, FILENUM FROM BOEK WHERE TOKENS IS NULL ORDER BY UDC_KOADE, FILENUM;) gives the expected result when entered in a terminal session. I obviously do something wrong, but I can't find out what. Googling gets me no answer. Anyone who can spot the error? Many thanks in advance, best regards, Jogchum

1 1

Re: "Access denied for user" on AWS RDS MariaDB Instance
by serg＠mariadb.org 04 Jan '24

04 Jan '24

Hi, Rhys.Campbell, First, please reply to (or cc:) the list, if you don't mind. On Jan 04, Rhys.Campbell(a)swisscom.com wrote: > Hello Sergei, > > SHOW PLUGINS; - List three authentication plugins... > > +----------------------------------+--------+--------------------+---------------+-------------+ > | Name | Status | Type | Library | License | > +----------------------------------+--------+--------------------+---------------+-------------+ > | mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL | > | mysql_old_password | ACTIVE | AUTHENTICATION | NULL | GPL | > | unix_socket | ACTIVE | AUTHENTICATION | NULL | GPL | > +----------------------------------+--------+--------------------+---------------+-------------+ Okay. It shows what your user cannot don't use. But what they do use? Can you check whether they all - or at least those that get errors - only use IDENTIFIED WITH mysql_old_password USING 'xxxxx' ? I'm not asking to send SHOW CREATE USER to the list, you can just take a look yourself and reply "yes" /Sergei > -----Ursprüngliche Nachricht----- > Von: Sergei Golubchik <serg(a)mariadb.org> > Gesendet: Donnerstag, 4. Januar 2024 14:03 > An: Campbell Rhys, B2C-TSP-SRE <Rhys.Campbell(a)swisscom.com> > Cc: discuss(a)lists.mariadb.org > Betreff: Re: [MariaDB discuss] "Access denied for user" on AWS RDS MariaDB Instance > > Hi, Rhys.Campbell, > > How is authentication configured server-side? > Just the conventional mysql_native_password plugin? > > Regards, > Sergei > Chief Architect, MariaDB Server > and security(a)mariadb.org > > On Jan 04, Rhys.Campbell via discuss wrote: > > Hello All, > > > > I have a funny issue that is bothering me on a few MariaDB > > instances. We're running 10.6.14 on AWS RDS with very little changed > > in the default configuration. We have eight instances of a vendor > > application running on EC2 Instances that use these MariaDB > > Instances. This application is deployed and configured via > > TF/Ansible and we don't really manually login to these instance to > > change or configure anything. Very occasionally I get an issue with > > logging into the database. The vendor application is occasionally > > logging the following errors... > > > > Connect failed to database (db): Access denied for user > > 'user'@'X.X.X.X' (using password: YES) - waiting for 125 seconds > > before retry Connect failed to database (db): Access denied for user > > 'user'@'X.X.X.X' (using password: YES) - waiting for 25 seconds before > > retry Connect failed to database (db): Access denied for user > > 'user'@'X.X.X.X' (using password: YES) - waiting for 5 seconds before > > retry Connect failed to database (db): Access denied for user > > 'user'@'X.X.X.X' (using password: YES) - waiting for 1 seconds before > > retry > > > > Of course I recognized these as the classic "wrong password" login > > error. There are naturally corresponding error log entries in the > > MariaDB Log... > > > > Access denied for user 'user'@'X.X.X.X' (using password: YES) > > > > The vendor is a bit dismissive of this, blaming the MariaDB > > Instance. I'm not certain that is the case. We see perhaps 1 > > instance of this per day although somedays it doesn't happen and a > > small number of days we see two instances. It does not appear to > > happen on a predictable schedule. The MariaDB Instances are working > > fine at all other times. CPU utilization is very low and we only > > serve a small number of connections. There is nothing else in the > > error log of interest. Credentials are set, via Ansible, in > > configuration files. Details are correct and 99.9% of the time the > > vendor application has no issues logging into the MariaDB Instance. > > I can't actually tie these errors to something not working... but > > we're running very low traffic at the moment, and I'm concerned > > about it becoming more of an issue later on. > > > > Two possibilities exist in my mind... > > > > > > 1. Bug in vendor application is occasionally munging the password > > and creating the error we see logged. > > 2. Something else as yet unknown, i.e. rogue cronjob configured to > > use false vendor app credentials that creates the above error. I > > don't believe this to be the case but I will not 100% exclude it as > > a possibility. > > > > Is anyone aware of any logging/auditing that I could activate on the > > MariaDB RDS Instances to get a little more information about this? > > I'm thinking about setting up an strace on the vendor process but > > want to see if there are any better options first. > > > > Cheers, > > Rhys

2 1

"Access denied for user" on AWS RDS MariaDB Instance
by Rhys.Campbell＠swisscom.com 04 Jan '24

04 Jan '24

Hello All, I have a funny issue that is bothering me on a few MariaDB instances. We're running 10.6.14 on AWS RDS with very little changed in the default configuration. We have eight instances of a vendor application running on EC2 Instances that use these MariaDB Instances. This application is deployed and configured via TF/Ansible and we don't really manually login to these instance to change or configure anything. Very occasionally I get an issue with logging into the database. The vendor application is occasionally logging the following errors... Connect failed to database (db): Access denied for user 'user'@'X.X.X.X' (using password: YES) - waiting for 125 seconds before retry Connect failed to database (db): Access denied for user 'user'@'X.X.X.X' (using password: YES) - waiting for 25 seconds before retry Connect failed to database (db): Access denied for user 'user'@'X.X.X.X' (using password: YES) - waiting for 5 seconds before retry Connect failed to database (db): Access denied for user 'user'@'X.X.X.X' (using password: YES) - waiting for 1 seconds before retry Of course I recognized these as the classic "wrong password" login error. There are naturally corresponding error log entries in the MariaDB Log... Access denied for user 'user'@'X.X.X.X' (using password: YES) The vendor is a bit dismissive of this, blaming the MariaDB Instance. I'm not certain that is the case. We see perhaps 1 instance of this per day although somedays it doesn't happen and a small number of days we see two instances. It does not appear to happen on a predictable schedule. The MariaDB Instances are working fine at all other times. CPU utilization is very low and we only serve a small number of connections. There is nothing else in the error log of interest. Credentials are set, via Ansible, in configuration files. Details are correct and 99.9% of the time the vendor application has no issues logging into the MariaDB Instance. I can't actually tie these errors to something not working... but we're running very low traffic at the moment, and I'm concerned about it becoming more of an issue later on. Two possibilities exist in my mind... 1. Bug in vendor application is occasionally munging the password and creating the error we see logged. 2. Something else as yet unknown, i.e. rogue cronjob configured to use false vendor app credentials that creates the above error. I don't believe this to be the case but I will not 100% exclude it as a possibility. Is anyone aware of any logging/auditing that I could activate on the MariaDB RDS Instances to get a little more information about this? I'm thinking about setting up an strace on the vendor process but want to see if there are any better options first. Cheers, Rhys

2 1

Creating Replica With Rsync (But It Fails)
by Michael Caplan 30 Dec '23

30 Dec '23

Hi there, I am attempting to create a replica from a MariaDB 10.3.31 master. The replicate is running MariaDB 10.3.39 on the same OS (Ubuntu 20.04) and like hardware with identical MariaDB configuration. The strategy I'm using to create the replica is one that I have done before using earlier version of MariaDB and MySQL. But this time it is failing. It roughly looks like this: 1. rsync /var/lib/mysql from master to replica 2. On master: FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS; 3. On master: Shutdown 4. rerun rsync from master to replica; then restart master 5. Start mariadb on replica 6. CHANGE MASTER on replica 7. Celebrate and have a nap When starting the replica which now has all the master data rsynced over, I am getting the following error repeated over and over again: 2023-12-26 22:14:54 0 [ERROR] InnoDB: Page [page id: space=43077, page number=3] log sequence number 19989139444854 is in the future! Current system log sequence number 19246539202516. 2023-12-26 22:14:54 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery. Then I get the following errors for many (but not all) tables: 2023-12-26 22:16:32 0 [ERROR] InnoDB: Expected tablespace id 3986547 but found 3986540 in the file ./ddx_practice_91804/alerts.ibd 2023-12-26 22:16:32 0 [Note] InnoDB: trying to read page [page id: space=3986547, page number=0] in the background in a non-existing or being-dropped tablespace The replica is totally useless, even though it is an exact copy, running on like hardware, same OS and similar major version of MariaDB. I've in the past used innodb_fast_shutdown = 0 on the master before final rsync, but I was told by MariaDB engineers that should not matter. I'm unclear why I'm seeing this corruption. Any ideas? Thanks, Mike Confidentiality Notice: The information contained in this electronic message and any attachments to this message are intended only for the individual(s) addressed in the message and may contain proprietary and confidential information. If you are not the intended recipient, you should not disseminate, distribute, or copy this e-mail. Please notify the sender and destroy this message. WARNING: Computer viruses can be transmitted via email. The recipient should scan this email before opening it. The company accepts no liability for any damage caused by any virus transmitted by this email.

2 2

Query Cache: Does a SELECT query tagged with SQL_NO_CACHE really touch the QC mutex?
by hellomaria-letmetrythisfirst＠mailinator.com 23 Dec '23

23 Dec '23

Hello, I found the query cache in MariaDB to be super useful and especially powerful when used in DEMAND mode. So I'd like to learn more about the performance characteristics. The documentation says that for the query cache, "the mutex is hit even for SQL_NO_CACHE": https://mariadb.com/kb/en/mariadb-memory-allocation/#query-cache Is there some more information about this somewhere? Why would a SELECT query that uses this directive dabble with the mutex at all?

2 3

Scheduled jobs recommendations
by ngtech1ltd＠gmail.com 19 Dec '23

19 Dec '23

Hey Everyone, I am currently using MariaDB 10.5.x from almalinux 9.3 and am testing latest 10.11 on Debian 12. I recently encountered an issue on 10.5 which probably didn’t run the stored procedure it should have. I have verified that the DB is configured properly ie: SET GLOBAL event_scheduler = ON; And also in the cnf files is default to ON and after a restart of the service the scheduler is on. I want to create a way to debug the stored procedure and the event scheduler. When I program a software I can log messages into somewhere but I don’t know how to do that in MariaDB and I would like to get a recommendation on how to do that this kind of logging in the when the even is being triggered and also after the call to the stored procedure ends. With this I can start to see if the event is being triggered and when. I am using it to cleanup a table and only leave the last 2 hours of data but it won’t run for at-least couple month now so I am calling the stored procedure from a cronjob. Thanks for any advice, Eliezer

2 1

Some questions restarting mariadb and importing the dumped databases
by Jogchum Reitsma 13 Dec '23

13 Dec '23

Hi all, On 3th of januari I posted the following topic, on the previous list address, maria-discuss(a)lists.launchpad.net, and gave it a reminder on the 19th of januari. I did not get an answer yet, maybe also because the list address changed in the mean time. So I take the liberty to pose the question again here, hoping this time someone is able to answer the the questions asked. Would be much appreciated! (I didn't mention it in the OP, but I've read https://mariadb.com/kb/en/mariadb-dump/, which does not seem to answer the questions) regards, Jogchum Hi, 12th of October I started the topic" Maria-db refuses to start". With the help offered in the conversation following I could access my data by manually starting an older version of Mariadb. This was on the previous list address, maria-discuss(a)lists.launchpad.net, so I then dumped my tables with mysqldump, one by one, successfully. I also dumped the mysql table. Final advice was to purge all reminiscences of the older versions, and make a fresh start by installing the version of my OS (OpenSuse Tumbleweed). In that new environment, import the dumped data. A few questions remain: - should I, after the install process of the OS version (at the moment 10.10.2-1.1) take further actions to get a fully functioning system? - should I import the dumped "mysql" table? - if yes, should I do that before or after importing the user data? Or doesn't that matter? Again, thanks for your answers! kind regards, Jogchum

2 2

10.7/10.8 removed from Debian repository: unable to upgrade Galera clusters
by William Edwards 30 Nov '23

30 Nov '23

Hi, For Debian, versions 10.7 and 10.8 were removed from the repositories. This is understandable, as they're not LTS versions, but... this makes it impossible to upgrade Galera clusters on 10.6 (LTS, supported). According to the documentation, when upgrading Galera clusters, skipping major versions is not supported. Could the project either bring these versions back, or decide that skipping major versions is supported for Galera and update the documentation? Met vriendelijke groeten, William Edwards

4 3

Text Columns encoding
by Ruben Safir 23 Nov '23

23 Nov '23

Years ago I dumped the text of emails directly into the database into a text column message | text | YES | | NULL I pulled it out and displayed it on the website with a simple select like select message from postings where id = 553671 using DBI and DBD Much of the email now ends up like binary junk like this one: http://www.mrbrklyn.com/docs/file.txt I've tried decoding it with pearl with no success. Obvously it is textwidth set someout and has \ inserted in the output. Is there a fix to this? -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

2 1

Redo log and tablespace flushing
by Gordan Bobic 20 Nov '23

20 Nov '23

I have a question that may seem somewhat obscure, but what I really want to know is how the disk flushing and crash safety work. Do tablespace commits get explicitly flushed during normal runtime operation? If we have a write that successfully commits to the redo log and to the binlog, but the tablespace loses, say, 5 seconds worth of commits in an unclean shutdown, would crash recovery deal with it? Is replaying the redo log followed by binlog based recovery sufficient to put the tablespace(s) into a consistent state even if the redo+binary logs are in terms of on-disk state a few seconds ahead of the tablespaces? On other words, provided that write ordering is preserved (ordering as guided by flush calls), can I do the equivalent of LD_PRELOAD=libeatmydata on the tablespace operations safely as long as the redo and binary logs are fsync()-ed reliably?

3 10

centos7 mariadb5.5 -> MariaDB10.3
by Marc 17 Nov '23

17 Nov '23

Ever since I switched from the default included centos mariadb to your repo. The web applications are getting ~1 once a week connection refused or so. I have already scheduled a cron job to restart your server. But still that is not enough. In the previous 10 years or so I never had such issue nor do I have such issues with postgresql. Nothing is in the log file, only the restart. Should such things not be logged by default? How to configure that issues resulting in refusing access or being unable to give access are logged?

2 7

When using mariadb-10.3.38, after upgrading Openssl from 1.1.1 to 3.1 strange compilation errors occur
by Ma Zhenhua 16 Nov '23

16 Nov '23

Hi Team, When using mariadb-10.3.38, after upgrading Openssl from 1.1.1 to 3.1 strange compilation errors occur. /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_error' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_fetch_row' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_num_rows' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_select_db' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_num_fields' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_real_connect' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_close' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_init' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_free_result' /xxxx/bin/ld: ../build/lib/libyyyy.so: undefined reference to `server_mysql_store_result' collect2: error: ld returned 1 exit status make[1]: *** [../rules.Make:461: cli] Error 1 libyyyy.so uses shared library libmysqlclient.so and both of them use mysql_xxxx. I don't know why prefix "server_" is added. I found a simliar issue where valgrind is used. But in my scenario valgrind is not used. https://jira.mariadb.org/browse/MDEV-28958?page=com.atlassian.jira.plugin.s… Any constructive suggestions are welcomed. Many thanks in advance. Regards, Allen

2 1

How do I get detailed TLS debugging information?
by Nick Lockheart 01 Nov '23

01 Nov '23

Replication master: Server version: 10.5.21-MariaDB-0+deb11u1-log Debian 11 Replication slave: Server version: 10.5.21-MariaDB-0+deb11u1 Debian 11 When `require_secure_transport = on` is enabled on the master, I get: "error reconnecting to master 'user@domain.com:3306' - retry-time: 10 maximum-retries: 86400 message: Access denied for user 'user'@'domain.com' (using password: YES)." If I turn `require_secure_transport` off, replication works. So the error in the log *looks* like an authentication failure message, but it's clearly a TLS issue. journalctl -fu mariadb.service just shows the same: "Access denied for user 'user'@'domain' (using password: YES)" How can I get verbose messages about what is happening with TLS (why is it being rejected)?

3 5

Speeding up a query: how to send 3 million rows faster?
by Ivan Krylov 01 Nov '23

01 Nov '23

Dear MariaDB users, (I've asked the same question at <https://dba.stackexchange.com/q/332557>, so feel free to ignore this one if you've already seen the other.) Our scientific application needs to store and query fundamental parameters for a number of molecules. There are 2 to 28 million rows per molecule, but the number of molecules is expected to stay small (currently 4). Here's our schema: CREATE TABLE `mol_trans` ( `species_id` int(11) DEFAULT NULL, `wl_vac` double DEFAULT NULL, `upper_id` int(11) DEFAULT NULL, `lower_id` int(11) DEFAULT NULL, `prob` double DEFAULT NULL, `flag` tinyint(4) DEFAULT NULL, KEY `spid_flag_wl` (`species_id`,`flag`,`wl_vac`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci PARTITION BY LIST (`species_id`) (PARTITION `CaO` VALUES IN (6115) ENGINE = InnoDB, PARTITION `CN3` VALUES IN (6121) ENGINE = InnoDB, PARTITION `CN2` VALUES IN (6119) ENGINE = InnoDB, PARTITION `AlO` VALUES IN (6109) ENGINE = InnoDB) (The partitions are here to make it easier to drop a whole molecule if needed, which would otherwise be a painful large DELETE.) I'm using the admittedly old MariaDB 10.3.39 from Debian Buster for tests (connecting via the UNIX domain socket using the command line client), but we've been seeing the same problem on MySQL 5.6 and MariaDB 10.11 on Windows 10. The following query currently takes 1m 7s on my computer, as measured by running `time echo "$QUERY" | mysql "$DATABASE" >/dev/null`: select mtr.prob, mtr.lower_id, mtr.upper_id from mol_trans mtr where ( mtr.species_id=6115 and mtr.wl_vac > 766.0 and mtr.wl_vac < 883.0 and mtr.flag = 1 ) order by mtr.wl_vac; It needs to return 3024559 rows. The query seems to be using the right index, judging by the EXPLAIN output: id: 1 select_type: SIMPLE table: mtr type: ref possible_keys: spid_flag_wl key: spid_flag_wl key_len: 7 ref: const,const rows: 14158123 Extra: Using where With SET PROFILING=ON and the query cache disabled, the single operation with a duration of >1 second in the 23-row profile is "Sending data": Status: Sending data Duration: 66.712277 CPU_user: 66.436978 CPU_system: 0.279932 Context_voluntary: 2156 Context_involuntary: 6964 Block_ops_out: 0 Messages_sent: 0 Messages_received: 0 Page_faults_major: 0 Page_faults_minor: 0 Swaps: 0 Source_function: <unknown> Source_file: sql_parse.cc Source_line: 6086 Is there a way to spend less time sending data? I know the query can in theory run faster because it takes less than 10 seconds on SQLite and PostgreSQL, but if we switched our application from database engine to database engine every time we started having performance problems, we would be very far from done by now. I also know that MariaDB can be very fast at transferring data because it takes me only a few minutes more to download the whole database dump. -- Best regards, Ivan

5 13

INSERT ON DUPLICATE UPDATE and primary auto_increment key behaviour
by Richard Ellerbrock 31 Oct '23

31 Oct '23

I have the following table: CREATE TABLE `events_active_test` ( `timestamp` TIMESTAMP NOT NULL, `timestamp_earliest` TIMESTAMP NOT NULL, `id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT, `source` CHAR(50) NOT NULL COLLATE 'utf8mb3_general_ci', `repeats` SMALLINT(5) UNSIGNED NOT NULL DEFAULT '1', PRIMARY KEY (`id`) USING BTREE, UNIQUE INDEX `source` (`source`) USING BTREE, INDEX `timestamp` (`timestamp`) USING BTREE, INDEX `timestamp_earliest` (`timestamp_earliest`) USING BTREE ) COLLATE='utf8mb3_general_ci' ENGINE=InnoDB ; Then I insert some test rows (do this 5 times): INSERT INTO events_active_test (timestamp,timestamp_earliest,source) VALUES (NOW(),NOW(),'test') ON DUPLICATE KEY UPDATE repeats=repeats+1,timestamp=NOW(); When I then look at the table id = 1 and repeats = 5 as expected Now I insert a non duplicate key (test2): INSERT INTO events_active_test (timestamp,timestamp_earliest,source) VALUES (NOW(),NOW(),'test2') ON DUPLICATE KEY UPDATE repeats=repeats+1,timestamp=NOW(); The new row now has id = 6 and repeats = 1. This really baffled me as it appears that when using INSERT ON DUPLICATE UPDATE with a primary auto_increment column the auto_increment is increased even if the INSERT fails and the UPDATE occurs. In my case the majority of my INSERT's are duplicates causing massive jumps and gaps in the auto_increment numbers. Is this correct behaviour? Also is there an overhead of having to update the tables auto_increment metadata counter unnecessarily for every failed insert? -- Richard

3 2

Setting a debugging environment
by Gingko 10 Oct '23

10 Oct '23

Hello. This is my first message here, and I don't know if there is any usage about self introducing. I am trying to setup a comfortable developer environment for MariaDB, and especially a debugging environment, on a Linux Ubuntu box (using gdb or ddd, or gdbgui…). This is not obvious as all of them seems to have issues. Anyway, I tried to follow the instructions from that page: https://mariadb.com/kb/en/get-build-and-test-latest-mariadb-the-lazy-way/ But there is something that I am wondering: If I want to see the contents of some variables using these debuggers, is it normal that I very often get the following message? |<optimized out>| I understand what that means, but I am actually trying to debug code compiled with the following options: |-DCMAKE_BUILD_TYPE=Debug| Shouldn't all optimizations have been disabled when compiling in debugging mode? Regards, Gingko

3 2

InnoDB durability
by Federico Razzoli 29 Sep '23

29 Sep '23

Recently Laurynas commented this bug and he brought it up on social media: https://bugs.mysql.com/bug.php?id=75519 I was surprised to read thar, from his comments, this 5.6 bug is still in MySQL 8.0. Did anyone test whether it is in MariaDB too? If so, which versions? Thanks, Federico -- Federico Razzoli Vettabase Ltd director & founder, Database Consultant https://vettabase.com federico.razzoli(a)vettabase.com +44 7739 427279 Telegram: federico_razzoli Skype: fede.razzoli The information in this e-mail and any attachments may be confidential. If you received it by mistake, please inform the sender. Unless you are the intended recipient or his/her representative you are not authorised to, and must not, read, copy, distribute, use or retain this message or any part of it.

3 5

RAM/Swap Usage
by roughnecks 14 Sep '23

14 Sep '23

Hello, first time here :) I've got a single db for roundcube and yet after every couple days my swap for mariadb reaches 200MB and over. To be noted that I'm the sole roundcube's user and that even myself prefer using thunderbird, so I only access roundcube from time to time. Is this normal? I'm running debian stable, upgraded from oldstable when it came out and I remember that in bullseye ram usage was just 50MB or so. Thanks, have a nice day. -- roughnecks

3 10

Increasing RAM usage until oom
by Marco Dickert - evolver group 07 Sep '23

07 Sep '23

Hi folks, we experience a RAM issue with MariaDB (version 10.5.19-MariaDB-0+deb11u2-log) on a standard Debian bullseye system. The problem is that over time MariaDB uses more and more RAM, until the kernel's oom-killer terminates it. The server is a VMware guest with 8G RAM, 2G swap and 4 cores. The most important parameter, innodb_buffer_pool_size, is set to 2G. I know this is quite low for this system, but I want to have proof that MariaDB consumes defininitely more RAM than allowed. According to the calculation in this comment [1], the RAM usage shouldn't exceed 4.2G. Here is the result, including the raw configuration values: ``` MariaDB [(none)]> SELECT ROUND( -> ( @@GLOBAL.key_buffer_size -> + @@GLOBAL.query_cache_size -> + @@GLOBAL.tmp_table_size -> + @@GLOBAL.innodb_buffer_pool_size -> + @@GLOBAL.innodb_log_buffer_size -> + @@GLOBAL.max_connections * ( -> @@GLOBAL.sort_buffer_size -> + @@GLOBAL.read_buffer_size -> + @@GLOBAL.read_rnd_buffer_size -> + @@GLOBAL.join_buffer_size -> + @@GLOBAL.thread_stack -> + @@GLOBAL.binlog_cache_size) -> ) / 1024 / 1024, 1) `total MB`; +----------+ | total MB | +----------+ | 4220.0 | +----------+ 1 row in set (0,001 sec) MariaDB [(none)]> SELECT @@GLOBAL.key_buffer_size, @@GLOBAL.query_cache_size , @@GLOBAL.tmp_table_size, @@GLOBAL.innodb_buffer_pool_size , @@GLOBAL.innodb_log_buffer_size, @@GLOBAL.max_connections , @@GLOBAL.sort_buffer_size, @@GLOBAL.read_buffer_size , @@GLOBAL.read_rnd_buffer_size, @@GLOBAL.join_buffer_size , @@GLOBAL.thread_stack, @@GLOBAL.binlog_cache_size \G 1 row in set (0,000 sec) *************************** 1. row *************************** @@GLOBAL.key_buffer_size: 10485760 @@GLOBAL.query_cache_size: 536870912 @@GLOBAL.tmp_table_size: 16777216 @@GLOBAL.innodb_buffer_pool_size: 2147483648 @@GLOBAL.innodb_log_buffer_size: 134217728 @@GLOBAL.max_connections: 512 @@GLOBAL.sort_buffer_size: 2097152 @@GLOBAL.read_buffer_size: 131072 @@GLOBAL.read_rnd_buffer_size: 262144 @@GLOBAL.join_buffer_size: 262144 @@GLOBAL.thread_stack: 299008 @@GLOBAL.binlog_cache_size: 32768 ``` However, MariaDB uses up to 5.4G so far. And it keeps increasing. Last time the oom-killer was invoked, MariaDB used 10G total, and 7.4G rss (given in 4k pages): ``` [Mi Jul 26 16:01:06 2023] Tasks state (memory values in pages): [Mi Jul 26 16:01:06 2023] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name ... [Mi Jul 26 16:01:06 2023] [3462912] 10000 3462912 2753196 1895256 20762624 472509 0 mariadbd ... [Mi Jul 26 16:01:06 2023] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/mariadb.service,task=mariadbd,pid=3462912,uid=10000 [Mi Jul 26 16:01:06 2023] Out of memory: Killed process 3462912(mariadbd) total-vm:11012784kB, anon-rss:7581024kB, file-rss:0kB,shmem-rss:0kB, UID:10000 pgtables:20276kB oom_score_adj:0 ``` The MariaDB knowledge base [2] says that the problem probably correlates with one of the following options: ``` +----------------------------+----------------------+----------------+ | variable_name | default_value | variable_value | +----------------------------+----------------------+----------------+ | INNODB_IO_CAPACITY_MAX | 18446744073709551615 | 2000 | | QUERY_CACHE_LIMIT | 1048576 | 67108864 | | MAX_CONNECTIONS | 151 | 512 | | TABLE_OPEN_CACHE_INSTANCES | 8 | 7 | | LONG_QUERY_TIME | 10.000000 | 5.000000 | | KEY_BUFFER_SIZE | 134217728 | 10485760 | | QUERY_CACHE_SIZE | 1048576 | 536870912 | | INNODB_LOCK_WAIT_TIMEOUT | 50 | 80 | | INNODB_LOG_BUFFER_SIZE | 16777216 | 134217728 | | HOST_CACHE_SIZE | 128 | 654 | | INNODB_FT_MIN_TOKEN_SIZE | 3 | 2 | | WAIT_TIMEOUT | 28800 | 600 | | INNODB_FAST_SHUTDOWN | 1 | 0 | | INNODB_LOG_FILE_SIZE | 100663296 | 1073741824 | | THREAD_POOL_SIZE | 8 | 4 | | INNODB_BUFFER_POOL_SIZE | 134217728 | 2147483648 | | MAX_ALLOWED_PACKET | 16777216 | 268435456 | | BACK_LOG | 150 | 254 | +----------------------------+----------------------+----------------+ ``` I can't see what setting could probably cause the issue. Some values are Debian defaults. Most of them we set intentionally. Can anyone give a hint on where to look for? Do you need further information? Thanks for any help. [1] https://serverfault.com/a/1020847 [2] https://mariadb.com/kb/en/mariadb-memory-allocation/#how-to-troubleshoot-ou… Regards, Marco

3 4

Odp: Re: Re: Rocksdb as a replacement for toku?
by pslawek83 25 Aug '23

25 Aug '23

Hi Marko, thanks for the response ... so for the DOCS part > h mariadb.com ttps://mariadb.com/kb/en/innodb-page-compression/ > When InnoDB page compression is used, InnoDB may still write the compressed page to the tablespace file with the original size of the uncompressed page, which would be equivalent to the value of the mariadb.com innodb_page_size system variable. This is done by design, because when InnoDB's I/O code needs to read the page from disk, it can only read the full page size. However, this is obviously not optimal. > This punch hole technique allows InnoDB to read the compressed page from disk as the full page size, even though the compressed page really takes up less space on the file system. So as far as i understand if compression is enabled the server will write eg. 40% utilized page which will later be decompressed when read, but this compression/decompression won't have any advangage if there's no punch hole support. The only difference will be that the block will be filled with zeros for the empty space at the end. Or is it possible for PAGE_COMPRESSION to still be used without punch hole (just not that effective for example)? Also tested this row compression, and yes page compression seems much better. Is it possible to get rid of punch hole by just re-mapping the pages inside these innodb files? I was thinking about this... if it's possible to change the data structure to divide the file into 512b blocks while the beginning of each block will mark page number (0xffffffff for blocks which are part of previous page). To increase page size we'd need to add another page with same number at the end of the file, later continuous recycled blocks can be added to a list and used to store smaller pages. To make this atomic, first whole block would be allocated and written with 0xfffffffe as a dummy id and then this placeholder id could be replaced with real block number). The problem with this could be that the file would get 100% fragmented over time (but probably will work ok with NVME). The mapping could be read back by reading whole datafile during server start and it'd need 125MB of memory for 1TB of compressed data to store the mapping in memory for fast access. What you thinik? Would it be easy to add it on top of current code (if there's any API maybe, so for example i can do this) and would it even make sense? Best, Slawomir. Dnia 21 sierpnia 2023 07:08 Marko Mäkelä < marko.makela(a)mariadb.com > napisał(a): On Sun, Aug 20, 2023 at 6:12 PM pslawek83 via discuss <discuss(a)lists.mariadb.org> wrote: Thanks everyone for the answers. Tested this PAGE_COMPRESSION a bit and make some reading. Not sure if I fully understand how that works... is that correct? 1. Impossible without punch hole as if we don't have it the server will just write partially used pages which will take full page size anyway. But in docs it seems it described its use with punch hole disabled. What's the reason to still have page compression without punch hole? Can you please point to the misleading part of the documentation so that it can be corrected? 2. After enabling compression each page needs to be trimmed using punch hole, so if we have 1 TB database and 64k blocks, the database files will have over 15 million fragments (1000000000000/64000) That is correct. So if we get like 50% compression and then we'll be able to put another 1TB of data into these holes we'll be having close to 31 million file fragments. I'm not sure if under such conditions the FS will be able to even allocate space efficiently anymore, copy these files, etc. Or im not getting something? For what it is worth, around the same time when the page_compressed was being developed for MariaDB Server 10.1, something similar was being worked on at Oracle, I think for MySQL 5.7. To my understanding, the same hardware partner worked with both companies. The obvious answer to smalldatum.blogspot.com https://smalldatum.blogspot.com/2015/10/wanted-file-system-on-which-innodb.… would have been en.wikipedia.org https://en.wikipedia.org/wiki/Fusion-io_NVMFS if it had worked without kernel panics or existed in a mainline kernel. I don’t even know if it ever was open sourced. 3. I was testing compression of ~1.5GB table with nice results on default 16kb page size. One thing i noticed is that l was always getting exactly the same compressed size using `du`. However the table read/copy times were different from 40s for uncompressed to 80-400 seconds depending on algo. Is that possible that is because page size is too low? As far as I remember, most compression algorithms that are derivatives of the LZ77 family have an input buffer size of 32 KiB. From that point of view, it would seem to make sense to use innodb_page_size=32k or innodb_page_size=64k instead of the default. Furthermore, I think that it was a mistake to implement bzip2 support at all. That algorithm would work with much larger buffer sizes. 4. Myrocks seems to be very slow compared to compressed inno. Was also reading docs related to it. For its usecase is it more suitable for storing large rows rather than many small rows? To my understanding, the LSM trees that MyRocks is built on work best in an insert-only workload. On top of LSM trees, MyRocks implements some caches and Bloom filters. If you have few updates or deletes and most reads are accessing recently inserted data, it could work. 5. I remember reading that some innodb compression is going to be obsolete. Is that row compression? Or there are no plans to obsolete any compression model? I wanted to deprecate and remove the ROW_FORMAT=COMPRESSED that I designed and implemented in the InnoDB Plugin for MySQL 5.1 based on some high level ideas of Heikki Tuuri between 2005 and 2007. jira.mariadb.org https://jira.mariadb.org/browse/MDEV-22367 was the first step towards that. Based on community feedback, this won’t be happening. Even after some data structure cleanup that took place in MariaDB Server 10.6, supporting the format incurs an overhead of some 32 to 40 bytes per buffer page descriptor, which is not insignificant. Removing both ROW_FORMAT=COMPRESSED and the adaptive hash index would shrink the block descriptor to 80 bytes from its current 10.6 size of 160 bytes. The write performance with ROW_FORMAT=COMPRESSED should have been degraded when I implemented the easier-to-parse log record format in MDEV-12353 (MariaDB Server 10.5). There are open bugs about ROW_FORMAT=COMPRESSED, such as jira.mariadb.org https://jira.mariadb.org/browse/MDEV-31574. I would say that the InnoDB ROW_FORMAT=COMPRESSED seemed like a good idea back when it was implemented, when HDD storage was the only practical option. Just like the InnoDB change buffer, which I removed in MariaDB Server 11.0 after having to deal with too many cases of corruption: fosdem.org https://fosdem.org/2023/schedule/event/innodb_change_buffer/ With best regards, Marko -- Marko Mäkelä, Lead Developer InnoDB MariaDB plc

3 3

Re: Rocksdb as a replacement for toku?
by pslawek83 22 Aug '23

22 Aug '23

Thanks everyone for the answers. Tested this PAGE_COMPRESSION a bit and make some reading. Not sure if I fully understand how that works... is that correct? 1. Impossible without punch hole as if we don't have it the server will just write partially used pages which will take full page size anyway. But in docs it seems it described its use with punch hole disabled. What's the reason to still have page compression without punch hole? 2. After enabling compression each page needs to be trimmed using punch hole, so if we have 1 TB database and 64k blocks, the database files will have over 15 million fragments ( 1000000000000/64000) So if we get like 50% compression and then we'll be able to put another 1TB of data into these holes we'll be having close to 31 million file fragments. I'm not sure if under such conditions the FS will be able to even allocate space efficiently anymore, copy these files, etc. Or im not getting something? 3. I was testing compression of ~1.5GB table with nice results on default 16kb page size. One thing i noticed is that l was always getting exactly the same compressed size using `du`. However the table read/copy times were different from 40s for uncompressed to 80-400 seconds depending on algo. Is that possible that is because page size is too low? 4. Myrocks seems to be very slow compared to compressed inno. Was also reading docs related to it. For its usecase is it more suitable for storing large rows rather than many small rows? 5. I remember reading that some innodb compression is going to be obsolete. Is that row compression? Or there are no plans to obsolete any compression model? Thanks. Dnia 16 sierpnia 2023 16:08 Gordan Bobic via discuss < discuss(a)lists.mariadb.org > napisał(a): Larger pages will yield higher compression ratios, as will zstd (slightly slower than lz4). Mileage varies, but on some of my databases, between the 64KB pages (zfs recordsize) and zstd, my compression ratios approximately doubled, from about 1.8:1 to a little under 4:1. This will all heavily depend on the type of data you are storing, though. zstd has adjustable compression, so if you have tons of CPU to spare, you could turn it up from the default of 3 (probably optimal cost/size compromise) to high teens for a compression boost. On Wed, 16 Aug 2023, 14:34 Reinis Rozitis, < r(a)roze.lv > wrote: > From: Gordan Bobic via discuss < discuss(a)lists.mariadb.org > > > One of the best compromises is to configure InnoDB with 64KB pages, put it on ZFS with recordsize=64K, and enable zstd compression on ZFS. Sorry for hijacking the thread a bit. After the discontinuation of tokudb I chose the innodb+zfs route (bad previous/early experience with rocks, also million knobs to adjust for your workload etc) but at that time didn't think of increasing the Innodb page size to 64KB - do you have any (even rough) numbers of 16 vs 64KB (on zfs)? Also, what would be the benefit of zstd vs lz4? Higher compression ratio? rr _______________________________________________ discuss mailing list -- discuss(a)lists.mariadb.org To unsubscribe send an email to discuss-leave(a)lists.mariadb.org

3 2

Running events on same time
by kpesio＠gmail.com 17 Aug '23

17 Aug '23

Hi. Could someone confirm in some way about running multiple events at same time. Documentation at https://mariadb.com/kb/en/automating-mariadb-tasks-with-events/ states: "The event scheduler runs as a single thread. If any event is supposed to start while another event is running, its execution will be skipped." I have simulated events triggering at same time, running with delays to get all kinds of overlapping schedules, writing to same table from the event and not missed a single one. The information_schema.events tables column last_executed has - depending on scheduled time - valid values and I can see in the processlist events running at same time. Version I'm running is 10.6.10 on Linux. Tried to cross check against the MySQL documentation but couldn't find anything about this there. So what situation does the documentation refer to or it this something fixed long time ago?

3 5

Rocksdb as a replacement for toku?
by pslawek83 16 Aug '23

16 Aug '23

Hi Guys, i'm looking for tokudb replacement and i'm again thinking about rocksdb. Is it mature enough to be used as an engine for statistical tables (several billions rows, around 2TB in size?). - Is it (will it be?) possible to have something like single-file-per-table (or single dir per table?) - Is it shared per all tables, so all tables are still using single set of files (can single crashed table/row lead to all tables being unusable?) - Will partitioning the table have performance benefits like with other engines? Can I do instant TRUNCATE PARTITION for easy data retention? Thanks

5 6

Query on Building Maria DB for different Compression Algorithm
by Raj, Rahul 10 Aug '23

10 Aug '23

[AMD Official Use Only - General] Hi, I am building MariaDB from source code and have queries on linking of compression algorithm. Before building the source code of MariaDB, I am exporting the path for SNAPPY or LZMA or the algorithm which is currently it supports. Then I build the code with - cmake -B . -DCMAKE_INSTALL_PREFIX=$PWD/maria_bin -DPLUGIN_MROONGA=NO -DPLUGIN_ROCKSDB=NO cmake --build . -j cmake -install . It compiled the code and installed in the specified location. When I checked the lib/plugin folder it has generated, provider_lzma.so or provider_snappy.so files and when I did ldd on that .so file, it has given the path from where it is fetching the liblzma.so.1 or libsnappy.so.1 file. Query: 1. The path which ldd shows for provider_lzma.so or provider_snappy.so is taking form default path, and it is not from the path which I am exporting the library and bin path - could you please explain me the correct steps. 2. Is there any way or any Flag we can set to switch between different compression method during build time itself, and in which log we can check it has linked properly. 3. How to benchmark the linked compression method and how we can see it is using the linked compression method from the custom path of .so files. Kindly help on above query. Thanks Rahul Raj

4 9

Reduce logging output from mariabackup
by Tom Worster 03 Aug '23

03 Aug '23

Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic? Some of the log lines are very welcome, for example I like the first three Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64) and the last two are really great, like a gift from god Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK! But there's about 500 between these every hour in the journal and most of them are quite boring like this Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done Or stuff I do not understand and suspect I probably don't need to undersand like Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().

3 5

Expected release date MariaDB 10.6.15
by William Edwards 01 Aug '23

01 Aug '23

Hi, Is MariaDB 10.6.15 expected to be released this week? According to Jira[1], it is 4 days overdue. This version will include a fix that I'm waiting for. Met vriendelijke groeten, William Edwards [1]: https://jira.mariadb.org/projects/MDEV/versions/29013

3 4

Help us redesign the mariadb.org website
by Ian Gilfillan 17 Jul '23

17 Jul '23

We are currently re-designing the mariadb.org website and you are invited to participate in this process. The interactive study shouldn't take longer than 10 to 15 minutes to complete and will close at midnight on Thursday 20 July 2023 (UTC-8). Access the study here: (no registration or login is required) https://study.uxtweak.com/cardsort/gkeYA0HPSXawp6YNsjllU The activity involves a method called Card Sorting, which will influence how we design the information architecture of the website: its structure, navigation and the findability of content and functionality. Your inclusion in this effort is important and will help to inform and assist us in designing an intuitive, usable and effective future website. Many thanks from the website redesign team MariaDB Foundation

1 0

Advice on Replication
by Andrew Gable 13 Jul '23

13 Jul '23

5 4

Re: Sandboxing mariabackup
by Tom Worster 07 Jul '23

07 Jul '23

Hi Daniel, That part is looking good for now but I gotta do fire drills validate the whole shebang. There's something almost blog worthy about this. mysqldump started as a handy import export utility and ended up being an exquisite point-in-time logical SQL snapshot tool. Now xtrabackup and mariabackup are so astonishingly fast at restoration/recovery they can't be ignored. But they are terrifying! They operate directly on the datadir. mysqldump is a nice friendly SQL client like any other. Does that make sense? An SQL client can ruin the DB as well as anything else if it has the right permissions (pun!). Hence subtracting permission to write datadir from the mariabackup process feels like a step to parity. Tom On 7/6/2023 6:52:17 PM, "Daniel Black" <daniel(a)mariadb.org> wrote: >Glad you worked it out. > >I did mean to write this onlist a while ago. Got saved a draft message >and I forgot about it. > >On Thu, Jul 6, 2023 at 10:47 PM Tom Worster <fsb(a)thefsb.org> wrote: >> >> On 7/6/2023 7:14:50 AM, "Daniel Black" <daniel(a)mariadb.org> wrote: >> >> >> I prefer to sandbox systemd services with ProtectSystem=strict. >> >> Especially so when User=root. >> > >> >Should be able to use User=mysql (or whatever the mariadb.service is >> >running as), provided the backup directory is also mysql: owned. >> > >> Thanks Daniel. >> >> Yes. I discovered that in subsequent testing. >> spin >> And I found that only target-dir and tmpdir need to be read-write. I >> forget what made me think that wasn't enough. >> >> To run as mariabackup as mysql user but with read-only access to datadir >> I separated the mariabackup part of the overall hourly backup script >> into its own service unit: >> [Unit] >> Description=mariabackup DBs >> OnFailure=unit-status-mail(a)%n.service unit-status-pushover(a)%n.service >> [Service] >> User=mysql >> Type=oneshot >> ProtectSystem=strict >> ReadWritePaths=/pool2/mariabackup /pool2/mariabackup_tmpdir >> ExecStart=/path/to/backup_maria.sh >> >> This is /path/to/backup_maria.sh. It's on a replica and the test of prev >> and new size are a check (among others) that the replica isn't stuck. >> #!/bin/bash >> prev_backup_size=`du -s /pool2/mariabackup` >> rm -rf /pool2/mariabackup/* >> mariabackup --backup --target-dir=/pool2/mariabackup/ >> --tmpdir=/pool2/mariabackup_tmpdir --user=mariabackup --password=__ >> mariabackup --prepare --target-dir=/pool2/mariabackup/ >> --tmpdir=/pool2/mariabackup_tmpdir >> new_backup_size=`du -s /pool2/mariabackup` >> if [ "$prev_backup_size" = "$new_backup_size" ] >> then >> echo WARNING: Consecutive mariabackup sizes unchanged >> # FIXME send alerts >> fi >>

2 2

Sandboxing mariabackup
by Tom Worster 06 Jul '23

06 Jul '23

I prefer to sandbox systemd services with ProtectSystem=strict. Especially so when User=root. So when ProtectSystem=strict, which dirs need to be ReadWritePaths to run `mariabackup --backup` and `mariabackup --prepare`? So far I got the --target-dir and the --tmpdir. Is that sufficient? tia Tom

2 2

stop database before truncate
by robert k Wild 03 Jul '23

03 Jul '23

hi all, looking at this guide https://www.ibm.com/support/pages/temporarily-reduce-space-taken-console-da… do i ideally have to stop the database before i truncate some tables ie /opt/aspera/common/mysql/bin/mysql -u root -p'password' aspera_console TRUNCATE* fasp_sessions fasp_files fasp_rates * *jobs job_steps* *thanks,* *rob* -- Regards, Robert K Wild.

2 5

Can we place a temporary table inside a stored function
by manpritsinghece＠gmail.com 29 Jun '23

29 Jun '23

Just need to know if we can place a temporary table inside the stored function ?

3 6

can you purge data older than 4 months old
by robert k Wild 29 Jun '23

29 Jun '23

hi all, i was just wondering i know about the "mysqlcheck -a and -o" ie to optimize and to analyse ie to cut it down in size but NOT to purge any data but is there like a purge option to delete all data older than 4 months old in the database thanks, rob -- Regards, Robert K Wild.

2 2

2023 MariaDB (Un)Conference and MariaDB Server Fest
by Ian Gilfillan 28 Jun '23

28 Jun '23

The MariaDB (Un)Conference, for MariaDB Server developers, and the MariaDB Server Fest, for anyone interested in MariaDB Server, will be taking place from 3 - 5 October as hybrid events, both in Helsinki and online. See https://mariadb.org/fest2023/ for details. The CfP for both events is available at https://mariadb.org/fest2023cfp/

1 0

mysql restore error
by robert k Wild 26 Jun '23

26 Jun '23

hi all, i successfully backed up the database /opt/aspera/common/mysql/bin/mysqldump -u root -p'password' aspera_console > /mnt/pixit/engineering/aspera_console/mysql_backups/20230623.sql then i try to restore it, first i delete the database and create the database /opt/aspera/common/mysql/bin/mysql -u root -p'password' drop database aspera_console; create database aspera_console; then run the restore command and i get the error /opt/aspera/common/mysql/bin/mysql -u root -p'password' aspera_console < /mnt/pixit/engineering/aspera_console/mysql_backups/20230623.sql ERROR 1305 (42000) at line 80504: FUNCTION aspera_console.report_user_id does not exist luckily this is not the live vm, its a clone of the live, so im messing around thanks, rob -- Regards, Robert K Wild.

2 4

Very large ibdata1 growth (335Gb) with 10.6.13, solved in 10.6.14 ?
by Simon Avery 20 Jun '23

20 Jun '23

Hi All, We run around 60 Mariadb 10.6 servers on EL in vms and varying workloads. Some time after they updated from 10.6.12 to 10.6.13, four of the busiest started showing higher disk usage than expected. Upon investigation, /var/lib/mysql/ibdata1 was exceptionally large, up to 335Gb, and was still growing at several Gb/day. Note: All servers have only run with "innodb_file_per_table" set and only have innodb engines for user DBs. I cobbled together some data collection just before I went on holiday and checked the graphs today. (Pic attached - if not stripped by the list software) [A picture containing screenshot, line, parallel Description automatically generated] Four machines growing, rest static. Two affected machines stopped growing a week ago - and on checking, Mariadb was upgraded from 10.6.13 to 10.6.14 on them and they were rebooted then. The two still growing are still running 10.6.13 My questions! 1. Am I right that this was a known bug in Mariadb and has been resolved with the .14 release? 2. The .14 release notes mention "InnoDB does not free UNDO after the fix of MDEV-30671 (MDEV-31234)" - is the failure to free UNDO logs a likely cause for ibdata1 to grow, given that it stores the undo log? (The two worst affected machines process many million queries a day) 3. To recover this space, I will need to do a full mysqldump, wipe and reimport of the data, correct? I guess I'm mostly asking for reassurance that this issue is now resolved and that once I schedule in the downtime to recover the space, that it's not going to resurface... Thanks, Simon

2 2

Fwd: DBA in Tokyo
by James Tobin 19 Jun '23

19 Jun '23

Hello, I'm working with an employer that is looking to hire a DBA in Tokyo. Consequently, I had hoped some members of this list may like to discuss. I can be reached using "JamesBTobin (at) Gmail (dot) Com". Kind regards, James

1 0

Connector/C fails to read MariaDB 10.6+ binary logs
by sruli67＠gmail.com 15 Jun '23

15 Jun '23

Hi, I'm using MariaDB Connector/C to 3.3.5 to read binary logs. It works perfectly for MySQL 8.0 and MariaDB 10.5 databases. For MariaDB 10.6 and above, mariadb_rpl_fetch() crashes (!) if the binlog contains XA transactions. Specifically, the function fails when it tries to read event type GTID with this contents: 'XA START X\'54584e31\',X\'\',1 GTID 0-1-21' Please advise Sruli Ganor

4 9

[Maria-discuss] Backup on the replication server getting affected
by ragul rangarajan 09 Jun '23

09 Jun '23

3 3

Re: [Maria-discuss] Restore from mysqldump fails with "Error in foreign key constraint"
by mark f edwards 05 Jun '23

05 Jun '23

How is providers in the `xu6gc_tj_notification_providers' table defined? Is it also varchar(100) character sat uft8mbr collate utf8mb4-general-ci ?? The parent and child columns has to match exactly. On June 3, 2023, at 7:55 AM, Alex <mysqlstudent(a)gmail.com> wrote: Hi, I'm trying to use mysqlbackup to backup a mariadb-10.5.19 database on fedora37 server using the following: # mysqldump --verbose -u root -ppass --result-file=/var/backup/mysql-backup/backup-lsalex-2023-06-03-13008.sql lsalex I usually use a more involved mysqlbackup command, but it also failed, so I figured it would be better to start out more simply as above. It fails with the following when restoring to a new, test database: # mysql -u root -p lstest < /var/backup/mysql-backup/backup-lsalex-2023-06-03-13008.sql Enter password: ERROR 1005 (HY000) at line 12884: Can't create table `lsalex`.`xu6gc_tj_notification_user_exclusions` (errno: 150 "Foreign key constraint is incorrectly formed") This is a backup/restore of a Joomla database. Am I using the wrong command to perform the backup or restore? This isn't a database that I created and I really have no idea how it works. I've also searched extensively for this error, and it appears to be a programming error, not a database error, but obviously I have no control over Joomla programming. Here's what I found from "SHOW ENGINE INNODB STATUS": LATEST FOREIGN KEY ERROR ------------------------ 2023-06-03 10:09:01 0x7f3a8d8516c0 Error in foreign key constraint of table `lstest`.`xu6gc_tj_notification_user_exclusions`: Create table `lstest`.`xu6gc_tj_notification_user_exclusions` with foreign key `xu6gc_tj_notification_user_exclusions_ibfk_1` constraint failed. Field type or character set for column 'provider' does not match referenced column 'provider'.------------ Here's the lines from the backup mentioned in the error message: 12884 CREATE TABLE `xu6gc_tj_notification_user_exclusions` ( 12885 `user_id` int(11) NOT NULL, 12886 `client` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12887 `key` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12888 `provider` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12889 KEY `client1` (`client`,`provider`(50),`key`), 12890 KEY `key` (`key`), 12891 KEY `provider` (`provider`), 12892 CONSTRAINT `xu6gc_tj_notification_user_exclusions_ibfk_1` FOREIGN KEY (`provider`) REFERENCES `xu6gc_tj_notification_providers` (`provider`) 12893 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci; Any ideas are greatly appreciated.

5 4

[Maria-discuss] Restore from mysqldump fails with "Error in foreign key constraint"
by Alex 03 Jun '23

03 Jun '23

Hi, I'm trying to use mysqlbackup to backup a mariadb-10.5.19 database on fedora37 server using the following: # mysqldump --verbose -u root -ppass --result-file=/var/backup/mysql-backup/backup-lsalex-2023-06-03-13008.sql lsalex I usually use a more involved mysqlbackup command, but it also failed, so I figured it would be better to start out more simply as above. It fails with the following when restoring to a new, test database: # mysql -u root -p lstest < /var/backup/mysql-backup/backup-lsalex-2023-06-03-13008.sql Enter password: ERROR 1005 (HY000) at line 12884: Can't create table `lsalex`.`xu6gc_tj_notification_user_exclusions` (errno: 150 "Foreign key constraint is incorrectly formed") This is a backup/restore of a Joomla database. Am I using the wrong command to perform the backup or restore? This isn't a database that I created and I really have no idea how it works. I've also searched extensively for this error, and it appears to be a programming error, not a database error, but obviously I have no control over Joomla programming. Here's what I found from "SHOW ENGINE INNODB STATUS": LATEST FOREIGN KEY ERROR ------------------------ 2023-06-03 10:09:01 0x7f3a8d8516c0 Error in foreign key constraint of table `lstest`.`xu6gc_tj_notification_user_exclusions`: Create table `lstest`.`xu6gc_tj_notification_user_exclusions` with foreign key `xu6gc_tj_notification_user_exclusions_ibfk_1` constraint failed. Field type or character set for column 'provider' does not match referenced column 'provider'.------------ Here's the lines from the backup mentioned in the error message: 12884 CREATE TABLE `xu6gc_tj_notification_user_exclusions` ( 12885 `user_id` int(11) NOT NULL, 12886 `client` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12887 `key` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12888 `provider` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL, 12889 KEY `client1` (`client`,`provider`(50),`key`), 12890 KEY `key` (`key`), 12891 KEY `provider` (`provider`), 12892 CONSTRAINT `xu6gc_tj_notification_user_exclusions_ibfk_1` FOREIGN KEY (`provider`) REFERENCES `xu6gc_tj_notification_providers` (`provider`) 12893 ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci; Any ideas are greatly appreciated.

1 0

Re: [Maria-discuss] Backup on the replication server getting affected
by andrei.elkin＠pp.inet.fi 02 Jun '23

02 Jun '23

Howdy Ragul, > Hi Andrei, > > Do we have any procedures to reproduce the issue MDEV-30780? Thanks for posting the gdb bt:s. They rule out 30780 yet not suggesting to me enough about the hang reason. This is something new to me and does deserve filing an MDEV ticket. Still I'd defer that until one has confirmed the same issue is seen on the latest 10.6. So you could run your load against the most recent slave version that'd be at least the safest (for our time). It might be (a slave worker) Thread 80 spinning inside #6 0x000055de407a0a3c in log_write_up_to (lsn=<optimized out>, lsn@entry=216757233923297, flush_to_disk=flush_to_disk@entry=false, rotate_key=rotate_key@entry=false, a goto repeat "loop". That hopefully you can confirm any next time the hang appears back. Could you please check whether #6 calls iteratively indeed `group_commit_lock::release()`? (With e.g (gdb) br thd_decrement_pending_ops thread 80 of course the number may change:-)). All the other slave worker threads may be waiting for the 80 but I can't confirm that until more data gets available. Namely I need to see the output of (gdb) thr app all get_about_worker_thread where the latter is defined as define get_about_worker_thread if $_any_caller_is ("handle_rpl_parallel_thread", 50) bt p handle_rpl_parallel_thread::rpt if (handle_rpl_parallel_thread::rpt->thd->rgi_slave) p handle_rpl_parallel_thread::rpt->thd->rgi_slave p handle_rpl_parallel_thread::rpt->thd->rgi_slave->current_gtid p handle_rpl_parallel_thread::rpt->thd->rgi_slave->gtid_sub_id p handle_rpl_parallel_thread::rpt->thd->rgi_slave->worker_error end end end > Unable to reproduce the issue locally but it occurs at random. to require some more patience from us. I belive we can resolve it while you're helping so generously! Cheers, Andrei > > Regards, > Ragul > > On Mon, May 29, 2023 at 7:06 PM ragul rangarajan <ragulrangarajan(a)gmail.com> wrote: > > Thanks Andrei, > > Hope my issue is more related to the issue MDEV-30780 optimistic parallel slave hangs after hit an error > Trying to reproduce with a minimal database.

1 0

[Maria-discuss] FYI:10.5.15-MariaDB crash und recover
by Radisson97＠gmx.de 01 Jun '23

01 Jun '23

Hello I had a crash today, there was no problem with the hardware. Maybe someone is interessted or find some useful information. It is a production machine so no core file. have fun. btw: Linux 5.10.0-17-amd64 #1 SMP Debian 5.10.136-1 == mariadb.log == 230601 8:22:09 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.5.15-MariaDB-0+deb11u1-log key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=11 max_threads=252 thread_count=11 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7faf440061d8 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7fafa439bd38 thread_stack 0x49000 ??:0(my_print_stacktrace)[0x55ebc2f0954e] ??:0(handle_fatal_signal)[0x55ebc2a08f65] sigaction.c:0(__restore_rt)[0x7fb11f670140] ??:0(_ma_state_info_write)[0x55ebc2bfdc73] ??:0(_ma_state_info_write)[0x55ebc2bff737] ??:0(_ma_state_info_write)[0x55ebc2c0746c] ??:0(handler::ha_write_row(unsigned char const*))[0x55ebc2a15aa8] ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55ebc2c420da] ??:0(handler::ha_write_row(unsigned char const*))[0x55ebc2a15aa8] ??:0(write_record(THD*, TABLE*, st_copy_info*, select_result*))[0x55ebc27d587d] ??:0(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool, select_result*))[0x55ebc 27dd8a4] ??:0(mysql_execute_command(THD*))[0x55ebc2810716] ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55ebc28145db] ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55ebc2816a5d] ??:0(do_command(THD*))[0x55ebc28182de] ??:0(do_handle_one_connection(CONNECT*, bool))[0x55ebc2909fb2] ??:0(handle_one_connection)[0x55ebc290a22d] ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55ebc2c4611b] nptl/pthread_create.c:478(start_thread)[0x7fb11f664ea7] x86_64/clone.S:97(__GI___clone)[0x7fb11f27bdef] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7faf44014860): INSERT INTO wetter_daten VALUES ( 190600010 , '2023-06-01 06:00:00' , 0 , 0 ) Connection ID (thread ID): 5569194 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,e ngine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in _to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=o ff,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=o n,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,cond ition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /home/kpc/kpc.dbs Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 127967 127967 processes Max open files 32313 32313 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127967 127967 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: core 230601 08:22:10 mysqld_safe Number of processes running now: 0 230601 08:22:10 mysqld_safe mysqld restarted 2023-06-01 8:22:11 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1-log) starting as process 24005 ... 2023-06-01 8:22:11 0 [Note] mariadbd: Aria engine: starting recovery recovered pages: 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% (0.1 seconds); tables to flush: 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 (0.4 seconds); 2023-06-01 8:22:11 0 [Note] mariadbd: Aria engine: recovery done 2023-06-01 8:22:11 0 [Note] InnoDB: Uses event mutexes 2023-06-01 8:22:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2023-06-01 8:22:11 0 [Note] InnoDB: Number of pools: 1 2023-06-01 8:22:11 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2023-06-01 8:22:11 0 [Note] InnoDB: Using Linux native AIO 2023-06-01 8:22:11 0 [Note] InnoDB: Initializing buffer pool, total size = 5368709120, chunk size = 134217728 2023-06-01 8:22:11 0 [Note] InnoDB: Completed initialization of buffer pool 2023-06-01 8:22:11 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=463015889782,463015889782 2023-06-01 8:22:13 0 [Note] InnoDB: Starting final batch to recover 2019 pages from redo log. 2023-06-01 8:22:15 0 [Note] InnoDB: 128 rollback segments are active. 2023-06-01 8:22:15 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2023-06-01 8:22:15 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-06-01 8:22:15 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2023-06-01 8:22:15 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2023-06-01 8:22:15 0 [Note] InnoDB: 10.5.15 started; log sequence number 463033688604; transaction id 19549881 2023-06-01 8:22:15 0 [Note] InnoDB: Loading buffer pool(s) from /home/kpc/kpc.dbs/ib_buffer_pool 2023-06-01 8:22:15 0 [Note] Plugin 'FEEDBACK' is disabled. 2023-06-01 8:22:15 0 [Note] Server socket created on IP: '::'. 2023-06-01 8:22:16 0 [Note] Reading of all Master_info entries succeeded 2023-06-01 8:22:16 0 [Note] Added new Master_info '' to hash table 2023-06-01 8:22:16 0 [Note] /usr/sbin/mariadbd: ready for connections. Version: '10.5.15-MariaDB-0+deb11u1-log' socket: '/run/mysqld/mysqld.sock' port: 3306 Debian 11 2023-06-01 8:22:18 0 [Note] InnoDB: Buffer pool(s) load completed at 230601 8:22:18

2 1

[Maria-discuss] jujst re-joined trial e-mail don't read
by antonio scotti 30 May '23

30 May '23

=============================================== PermaCultura-es http://www.permacultura-es.org

1 0

[Maria-discuss] trial e-mail don't read
by antonio scotti 30 May '23

30 May '23

=============================================== PermaCultura-es http://www.permacultura-es.org

1 0

[Maria-discuss] Innodb_buffer_pool_read_requests is zero
by mariadb＠Biblestuph.com 19 May '23

19 May '23

Just upgraded from 10.6.12 to 10.6.13 on a Debian Bullseye system, and seeing this oddity. Innodb_buffer_pool_read_requests is zero, and seems to remain at zero no matter what I do (I even tried selecting all the data from a small table several times in a row with no change). It's been running a couple hours now; I've restarted it a couple times. I have 10.6.13 on a few other platforms as well (including Buster), and I'm not seeing the same issue on them. What would cause this? I have no other Bullseye systems running 10.6.13 yet, and the same system has been running 10.6.12 for over a week now without showing this issue. The system _seems_ to be running fine, it's a replica system that is being fed actively from the master (data comes in to it very regularly). MariaDB [(none)]> SHOW GLOBAL STATUS LIKE 'innodb_buffer_pool_%'; +-----------------------------------------+--------------------------------------------------+ | Variable_name | Value | +-----------------------------------------+--------------------------------------------------+ | Innodb_buffer_pool_dump_status | | | Innodb_buffer_pool_load_status | Buffer pool(s) load completed at 230511 23:21:26 | | Innodb_buffer_pool_resize_status | | | Innodb_buffer_pool_load_incomplete | OFF | | Innodb_buffer_pool_pages_data | 609103 | | Innodb_buffer_pool_bytes_data | 9979543552 | | Innodb_buffer_pool_pages_dirty | 19465 | | Innodb_buffer_pool_bytes_dirty | 318914560 | | Innodb_buffer_pool_pages_flushed | 0 | | Innodb_buffer_pool_pages_free | 299441 | | Innodb_buffer_pool_pages_made_not_young | 0 | | Innodb_buffer_pool_pages_made_young | 107 | | Innodb_buffer_pool_pages_misc | 0 | | Innodb_buffer_pool_pages_old | 224824 | | Innodb_buffer_pool_pages_total | 908544 | | Innodb_buffer_pool_pages_lru_flushed | 0 | | Innodb_buffer_pool_pages_lru_freed | 0 | | Innodb_buffer_pool_pages_split | 726 | | Innodb_buffer_pool_read_ahead_rnd | 0 | | Innodb_buffer_pool_read_ahead | 3 | | Innodb_buffer_pool_read_ahead_evicted | 0 | | Innodb_buffer_pool_read_requests | 0 | | Innodb_buffer_pool_reads | 604346 | | Innodb_buffer_pool_wait_free | 0 | | Innodb_buffer_pool_write_requests | 3639341 | +-----------------------------------------+--------------------------------------------------+ 25 rows in set (0.000 sec) MariaDB [(none)]> SHOW VARIABLES LIKE 'innodb_buffer_pool%'; +-------------------------------------+----------------+ | Variable_name | Value | +-------------------------------------+----------------+ | innodb_buffer_pool_chunk_size | 134217728 | | innodb_buffer_pool_dump_at_shutdown | ON | | innodb_buffer_pool_dump_now | OFF | | innodb_buffer_pool_dump_pct | 80 | | innodb_buffer_pool_filename | ib_buffer_pool | | innodb_buffer_pool_load_abort | OFF | | innodb_buffer_pool_load_at_startup | ON | | innodb_buffer_pool_load_now | OFF | | innodb_buffer_pool_size | 15032385536 | +-------------------------------------+----------------+ 9 rows in set (0.000 sec)

2 5

[Maria-discuss] Question regarding mariadb-dump/mysqldump and rocksdb
by Jonas Krauss 16 May '23

16 May '23

Hi MariaDB, in Myrocks / Facebook MySQL there was a useful addition to mysqldump which makes queries issued by the dumper not add to the block cache of the rocksdb instance. It was added quite some time ago: https://github.com/facebook/mysql-5.6/commit/2223479fea78a246884c5a09ef4d02… The corresponding server variable used is *rocksdb_skip_fill_cache* Is this active in MariaDB's version of mysqldump as well? The server variable exists in MariaDB's MyRocks (checked for 10.6.12) but I have not found a reference in MariaDB's source code when searching for rocksdb_skip_fill_cache. If it is not active it would be great if it could be added in a future version. Best Jonas

2 2

[Maria-discuss] Undo logs aren't truncating
by mariadb＠Biblestuph.com 15 May '23

15 May '23

On a recently updated MariaDB 10.6.13 server (Debian Bullseye) I noticed that my ibdata1 file was continuously growing; it had gotten a little over 3G after running a couple days. I would rather keep this growth in check so used it as an excuse to setup undo tablespaces that could be truncated (I had wanted to try this for some time anyway). So: Dump the whole DB, delete all the InnoDB tables, stop the server, delete the innodb system tables, reset the appropriate undo variables (with innodb_max_undo_log_size set to 750M, innodb_undo_tablespaces set to 4, and innodb_undo_log_truncate set to ON), restart the server and reload the data. I now have FOUR undo files that appear to be continuously growing; as of this writing they are over 1G and still increasing. This is an active read-only replica (processing updates very frequently; at times continuously but not constantly as such). How do I get it to actually reclaim the space? Or am I misunderstanding how this is supposed to work? Below are my current relevant settings (or at least what I think are relevant). Normally I run with innodb_purge_rseg_truncate_frequency = 128; but for testing I have it now set to 1 (but with no noticeable difference). Things I have tried (no luck on any of them): Set innodb_purge_rseg_truncate_frequency = 1. Set innodb_max_purge_lag_wait=0 (it returned nearly immediately). Stop both slave threads and wait for several minutes. Set innodb_max_purge_lag and innodb_max_purge_lag_delay (see below). MariaDB [(none)]> show variables like '%trunc%'; +--------------------------------------+-------+ | Variable_name | Value | +--------------------------------------+-------+ | innodb_purge_rseg_truncate_frequency | 1 | | innodb_undo_log_truncate | ON | +--------------------------------------+-------+ 2 rows in set (0.000 sec) MariaDB [(none)]> show variables like '%purge%'; +--------------------------------------+------------+ | Variable_name | Value | +--------------------------------------+------------+ | aria_log_purge_type | immediate | | innodb_max_purge_lag | 1000 | | innodb_max_purge_lag_delay | 5000 | | innodb_max_purge_lag_wait | 4294967295 | | innodb_purge_batch_size | 300 | | innodb_purge_rseg_truncate_frequency | 1 | | innodb_purge_threads | 4 | | relay_log_purge | ON | +--------------------------------------+------------+ 8 rows in set (0.000 sec) MariaDB [(none)]> show variables like '%undo%'; +--------------------------+-----------+ | Variable_name | Value | +--------------------------+-----------+ | innodb_max_undo_log_size | 786432000 | | innodb_undo_directory | ./ | | innodb_undo_log_truncate | ON | | innodb_undo_tablespaces | 4 | +--------------------------+-----------+ 4 rows in set (0.000 sec)

2 1

[Maria-discuss] mariadb crash und recovery
by Radisson97＠gmx.de 14 May '23

14 May '23

Hello list, i had a crash at weekend with no apparend cause. The aria engine did a recovery without furture problems. Maybe that is a hint that somebody can make use of. Hints welcome. CU from log: 230416 7:20:17 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.5.15-MariaDB-0+deb11u1-log key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=86 max_threads=252 thread_count=38 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 685818 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f01e4000c58 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f02b6894d38 thread_stack 0x49000 ??:0(my_print_stacktrace)[0x5559d92bb54e] ??:0(handle_fatal_signal)[0x5559d8dbaf65] sigaction.c:0(__restore_rt)[0x7f043591a140] ??:0(_ma_state_info_write)[0x5559d8fafc73] ??:0(_ma_state_info_write)[0x5559d8fb1737] ??:0(_ma_state_info_write)[0x5559d8fb946c] ??:0(handler::ha_write_row(unsigned char const*))[0x5559d8dc7aa8] ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x5559d8ff40da] ??:0(handler::ha_write_row(unsigned char const*))[0x5559d8dc7aa8] ??:0(write_record(THD*, TABLE*, st_copy_info*, select_result*))[0x5559d8b8787d] ??:0(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool, select _result*))[0x5559d8b8f8a4] ??:0(mysql_execute_command(THD*))[0x5559d8bc2716] ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x5559d8bc65db] ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x5559d8bc8a5d] ??:0(do_command(THD*))[0x5559d8bca2de] ??:0(do_handle_one_connection(CONNECT*, bool))[0x5559d8cbbfb2] ??:0(handle_one_connection)[0x5559d8cbc22d] ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x5559d8ff811b] nptl/pthread_create.c:478(start_thread)[0x7f043590eea7] x86_64/clone.S:97(__GI___clone)[0x7f0435525def] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f01e4010440): INSERT INTO messwerte VALUES ( 83110995 , 1200 , 909 , 331 , '2023-04-16 05:20:00' , '2023-04-16 05: 20:00' , 0.0703000000000000013766765505351941101253032684326171875 , 0.001000000000000000020816681711721685132943093776702880 859375 , 0 , 0 , 0 , 0 , 60 , 0 ) Connection ID (thread ID): 5463672 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_ intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=o n,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subque ry_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_increment al=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in= on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowi d_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /home/kpc/kpc.dbs Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 127976 127976 processes Max open files 32313 32313 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127976 127976 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: core 230416 07:20:19 mysqld_safe Number of processes running now: 0 230416 07:20:19 mysqld_safe mysqld restarted 2023-04-16 7:20:19 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1-log) starting as process 415 ... 2023-04-16 7:20:20 0 [Note] mariadbd: Aria engine: starting recovery recovered pages: 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% (0.0 seconds); tables to flush: 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 (0.1 seconds); 2023-04-16 7:20:20 0 [Note] mariadbd: Aria engine: recovery done

4 3

[Maria-discuss] Checkpoint Age and Redo Log Flushing in 10.5+
by Gordan Bobic 14 May '23

14 May '23

Ever since 10.5, the redo log flushing seems to have become ... suboptimal. It looks like the database does nothing at all to flush the logs until it hits the hwm, and then it goes into overdrive and flushes everything to tablespaces fast, before it goes back to doing nothing until the log fills up. This causes a problem when the redo log is already full and a large burst of writes arrives because it overwhelms the redo log and causes a stall. Is there a bug fix for this or a configuration change available to restore the old behaviour where everything gets flushed continuously and switches up a gear when the hwm is reached? Here are the screenshots comparing 10.5+ sawtooth pattern vs 10.4 and older versions.

2 1

[Maria-discuss] howto c-connect bindings
by juerg oehler 06 May '23

06 May '23

hi I'm working with|mariadb|and|c-connect|prepared statements and run into crashes executing them once a while. i so addressed my problem to stakeoverflow <https://stackoverflow.com/questions/76170354/mariadb-howto-c-connect-bindin…> hoping to get more information for binding programs data structure to db interface. i did get an answer. but this answer does not go inline with your documentation <https://mariadb.com/kb/en/mariadb-connectorc-data-structures/#mysql_bind>. i so do have the option to follow your documentation or use the answer and run into crashes in either way. you might ask why i'm not just going with your documentation? your documentation is far to thin to get a reliable idea how bind structure is used and i do get proofed this by crashes. second i played with <my_bool * is_nulll Pointer which indicates if column is NULL (not used for parameters)> for parameters and behavior was not as documented. this is my motivation to bring my problem to this mailing list hopping someone can point me to a documentation i can rely on. i found a short and useful documentation in mysql with permissible in and out types <https://dev.mysql.com/doc/c-api/8.0/en/c-api-prepared-statement-type-codes.…> and statement data structures <https://dev.mysql.com/doc/c-api/8.0/en/c-api-prepared-statement-data-struct…>. can i use these informations to build 2 test applications one focusing on parameter (attachment) one focusing on result? currently i implemented MariaPrepUpd.c using mariadb documentation. i felt like flying blind and ended up in crashes - once a while. do you think i could use mysql documentation for improving bindings? or do you have a better option? (i studied your bulk example <https://mariadb.com/kb/en/bulk-insert-row-wise-binding/>. i even was using this example as base for first try an hit the wall by using STMT_INDICATOR_NTS. i then learned my problem is not just the example using STMT_ATTR_ARRAY_SIZE = 1.) my goal is to have a version of MariaPrepUpd.c i can use for any application for iteration 1 to n. thank you for short answer. have a great day. juerg(=george) oehler

1 0

[Maria-discuss] mysql.innodb-table-stats-and mysql.innodb-index-stats are not updating properly
by ragul rangarajan 21 Apr '23

21 Apr '23

Hello all, I am facing a weird issue with my DB table where its Index and table stats not getting updated with values automatically in MariaDB 10.6.7 The table has more than a billion records. On querying the stats of the table, I could not see any stats for the table that begin automatically calculated until we do a mysql restart or ANALYZE TABLE. Once triggered we can able to see the index values are calculated and updated in this table 127.0.0.1:3307> SELECT * FROM mysql.innodb_table_stats ;+---------------+-------------------------------------+---------------------+---------+----------------------+--------------------------+| database_name | table_name | last_update | n_rows | clustered_index_size | sum_of_other_index_sizes |+---------------+-------------------------------------+---------------------+---------+----------------------+--------------------------+| DB | TableStats#P#p2023_04_11_13_00_00 | 2023-04-11 12:00:11 | 0 | 1 | 0 || DB | TableStats#P#p2023_04_11_14_00_00 | 2023-04-11 13:00:10 | 0 | 1 | 0 || DB | TableStats#P#p2023_04_11_15_00_00 | 2023-04-11 14:00:04 | 0 | 1 | 0 || DB | TableStats#P#p2023_04_11_16_00_00 | 2023-04-11 15:00:04 | 0 | 1 | 0 |+---------------+-------------------------------------+---------------------+---------+----------------------+--------------------------+ 127.0.0.1:3307> SELECT * FROM mysql.innodb_index_stats;+---------------+-------------------------------------+------------+---------------------+--------------+------------+-------------+------------------------------------------------------------+| database_name | table_name | index_name | last_update | stat_name | stat_value | sample_size | stat_description |+---------------+-------------------------------------+------------+---------------------+--------------+------------+-------------+------------------------------------------------------------+| DB | TableStats#P#p2023_04_11_16_00_00 | PRIMARY | 2023-04-11 15:00:04 | n_diff_pfx01 | 0 | 1 | AgentId || DB | TableStats#P#p2023_04_11_16_00_00 | PRIMARY | 2023-04-11 15:00:04 | n_diff_pfx02 | 0 | 1 | AgentId,rackNr || DB | TableStats#P#p2023_04_11_16_00_00 | PRIMARY | 2023-04-11 15:00:04 | n_leaf_pages | 1 | NULL | Number of leaf pages in the index || DB | TableStats#P#p2023_04_11_16_00_00 | PRIMARY | 2023-04-11 15:00:04 | size | 1 | NULL | Number of pages in the index |+---------------+-------------------------------------+------------+---------------------+--------------+------------+-------------+------------------------------------------------------------+ As per the documentation https://mariadb.com/kb/en/innodb-persistent-statistics/, enabling *innodb_stats_auto_recalc* parameter should update the index value automatically which was not happening for me even though the changes are 10% of the row got affected within few days. The same was working for other DB table which is of lower load. Is there any way to debug why the calculation was not updated properly with values or any way to reduce the sample size? *Thanks and Regards,* Ragul R

2 2

[Maria-discuss] DELETE query hanging on Statistics (10.6.12)
by mariadb＠Biblestuph.com 14 Apr '23

14 Apr '23

With this table: CREATE TABLE t1 ( label varchar(100) NOT NULL, dt datetime NOT NULL, success tinyint(1) NOT NULL, PRIMARY KEY (label,dt), KEY dt (dt), KEY success (success,dt) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC I have a DELETE query that has been stuck in Statistics for nearly an hour now: DELETE MyDB.t1 FROM MyDB.t1 WHERE dt <= '2023-02-21 08:06:46' AND dt >= '2023-02-21 08:03:48' This statement runs several times per hour to trim the table (and has for years). It has run fine against my 10.6.12 DB (this is a downstream replica DB) for a couple weeks now(when I upgraded the replica); but suddenly today produced the hang. Any ideas why? The table has nearly 15 million rows; but the rows are more or less evenly spaced across datetimes. Also I find that I cannot kill the query using KILL 12345; the KILL statement just comes back immediately without reporting an error and the statement continues. Is there no way to kill this statement without crashing the DB? Again, this is a replica statement that is executing. I had planned to stop the query, execute an ANALYZE TABLE t1 and then try it again, but I don't seem to be able to kill the query. Thank you! Dan

3 5

[Maria-discuss] Privilege Question
by Scott Canaan 06 Apr '23

06 Apr '23

We are on MariaDB 10.5.18. There is a requirement to send all syslog data to a central syslog server. In the past, we did it using a login called ITS_READ. It has limited privs on purpose, but used to be able to execute the SET GLOBAL statements that we needed. Those statements are: SET GLOBAL server_audit_output_type=SYSLOG; SET GLOBAL server_audit_logging=1; SET GLOBAL server_audit_syslog_facility=LOG_LOCAL2; SET GLOBAL server_audit_events="connect,table,query_ddl,query_dcl"; Now, it doesn't have the privs needed to do that. In looking online, I found that there is a "new" privilege (in MySQL 8), called system_variables_admin that allows the user to execute SET GLOBAL statements. However, when I try to grant that privilege, I get an error: MariaDB [(none)]> grant system_variables_admin to 'ITS_READ'@'localhost'; ERROR 1959 (OP000): Invalid role specification `system_variables_admin` It seems that the only other option is to grant the SUPER privilege, which we don't want to do. How do I get around this, or do I just tell the sysadmins they can't have the syslog data? Scott Canaan '88 Sr Database Administrator Information & Technology Services Finance & Administration Rochester Institute of Technology o: (585) 475-7886 | f: (585) 475-7520 srcdco(a)rit.edu<mailto:srcdco@rit.edu> | c: (585) 339-8659 CONFIDENTIALITY NOTE: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy any copies of this information.

3 3

[Maria-discuss] Disable validation for `DATETIME` columns
by William Edwards 03 Apr '23

03 Apr '23

Hi, Is it possible to disable validation for `DATETIME` columns? Context: A customer uses the Europe/Amsterdam session time zone. Therefore, they cannot insert dates during the hour in which DST starts and ends. Using the UTC session time zone would solve the issue. However, they also use auto-converting `TIMESTAMP` columns. Switching to a different session time zone would cause issues with those. Relaxing constraints with `sql_mode=NO_ZERO_DATE,NO_ZERO_IN_DATE,ALLOW_INVALID_DATES` does not suffice. With kind regards, William Edwards

2 3

[Maria-discuss] Column named 'row_number' gives error in 10.11
by Keith Edmunds 02 Apr '23

02 Apr '23

I have a table that includes an int column named row_number. This worked fine until I upgraded Debian from 11 to 12 which upgraded MariaDB from 10.5 to 10.11. I now get errors when referencing that column for updates. If I put backticks around the column name the update runs without error. Is this an expected behaviour change from 10.5 → 10.11? Example: CREATE DATABASE test; USE test; CREATE TABLE `kae` ( `id` int(11) NOT NULL, `row_number` int(11) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; INSERT INTO `kae` (`id`, `row_number`) VALUES (1, 0); UPDATE kae SET row_number=12 WHERE id=1; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'row_number=12 WHERE id=1' at line 1

4 8

[Maria-discuss] Mounting tmpdir via NFS
by Simon Avery 29 Mar '23

29 Mar '23

Hello, I have a question that I've been struggling to get a clear answer to, or even any indication someone has done this before, and would appreciate any experience or thoughts. Are there any known issues with defining tmpdir on multiple servers to point to a single NFS-mounted network share? Would each instance co-habit peacefully and not try to change any files from another instance? Why do I want to do this? I have around 80 Centos/Rocky vms which run Mariadb 10.6.x. Due to the nature of the work they do, some of them require a significant amount of tmpdir (up to 75Gb) relatively briefly (once or twice a day for a few minutes) This growth is not always predictable, so generally the first I know about it is when Mariadb reports lowspace and the query stops. We then grow the /tmp partition that is defined as tmpdir and if we're quick enough, the query completes, but often the job dies. Then that extra space stays locked away forever, even if that server never tries to use that much tmpdir again. This does not feel efficient and I am pondering better ways of doing this. One such thought would be to have a single HA NFS share on our network, mounted onto each VM as something like /mariadb-tempdir, and set $tmpdir to it. In my mind, this would make a better use of one large allocation of space used by many things often, rather than lots of individual small to large sized bits of space used by one thing, rarely. The chance of two or more servers filling that space feels small, but if it happened, one would wait until space was cleared - which might actually happen in this shared scenario when it never does in a single-server situation. In my particular world, we have fast storage and a fast network. Speed tests on large and tiny files over NFS give surprisingly similar performance to writing to local vm disks, so I'm not worried about the overhead of NFS and even if it was, slow might still be preferred to full. Thank you, Simon

3 5

[Maria-discuss] MariaDB Online-DDL
by Rhys.Campbell＠swisscom.com 17 Mar '23

17 Mar '23

Hello All, I'm back working with MariaDB again after a break of some years. While InnoDB Online-DDL had appeared when I was working with it, it was still sometimes a challenge to manage schema changes for very large tables. https://mariadb.com/kb/en/innodb-online-ddl-overview/ Can anyone summarize how InnoDB stacks up for Online-DDL now? Is it comparable to MSSQL (which I found to be pretty good for tables in the many tens of GB)? Are workarounds like pt-online-schema-change and blue/green deployments still required? Are there any specific areas to watch out for? Any feedback appreciated. Cheers, Rhys

2 1

[Maria-discuss] can not kill process
by Radisson97＠gmx.de 17 Mar '23

17 Mar '23

hi i have a strange effect a simple insert got stuck (why ?) i tried to kill but no success, it is even marked as "killed" (see below) 1398867 kpc localhost kpc 1716 Killed INSERT INTO i can kill other processes- my system: 10.5.15-MariaDB engine aria 5.10.0-14-amd64 any ideas what may the problem ? (no usefull entries in any log, as you expected ) re,

3 4

[Maria-discuss] Optimized way to build MariaDB
by Otto Kekäläinen 12 Mar '23

12 Mar '23

Hi! I have spent a lot of time teaching various people how to build (and re-build) MariaDB. If you are interested to learn how I use entr + cmake --fresh + ninja + clang + ccache check out https://optimizedbyotto.com/post/quick-builds-and-rebuilds-of-mariadb-with-… - Otto

1 0

[Maria-discuss] MariaDB Connector C 3.1 ABI contract broken in latest release
by Otto Kekäläinen 04 Mar '23

04 Mar '23

Hello! After uploading MariaDB 10.3.38 in Debian I started seeing reports about libmariadb3 being broken for some consumers. Turns out that the function mysql_get_client_version() had a major behavior change in the latest MariaDB Connector C 3.1.20 which is embedded in the MariaDB 10.3 build. Details in PR https://github.com/mariadb-corporation/mariadb-connector-c/pull/219. Affected library consumers are listed in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031863#86 Unfortunately it might be too late to fix this, the milk is already spilled. Could you please in future keep as the engineering policy that *libraries are not allowed to have backwards-incompatible changes* unless the soname/version is also bumped. You might add a new symbol in an existing library version without causing breakage in library consumers, but you cannot remove any symbols/functions, or change their behavior significantly. In this case you completely changed what version the library returns. Please acknowledge these base rules and be more diligent in the future to avoid risks. Also, if you want to introduce libmariadb4, please do it on MariaDB 11.0 or newer only, and don't change any ABI in existing stable releases of libmariadb3. Thanks!

1 0

[Maria-discuss] EXPLAIN Analyser / Usage statistics
by Colin Charles 02 Mar '23

02 Mar '23

Hi, What might have happened to the EXPLAIN Analyzer https://mariadb.org/explain_analyzer/ It is now returning 404 not found Similarly, if you click https://mariadb.org/about/#usage-statistics (usage statistics at the bottom of every page) you’re also getting things like “oops that page can’t be found” e.g. https://mariadb.org/feedback_plugin/stats/server_count_by_month/ While I’m at it, it might also be that the download stats haven’t been updated since January? https://mariadb.org/statistics/ (I think Andrew fixed this in December, but maybe the Docker container died again) cheers, -c -- Colin Charles, http://bytebot.net/blog/ twitter: @bytebot | skype: colincharles "First they ignore you, then they laugh at you, then they fight you, then you win." -- Mohandas Gandhi

2 2

[Maria-discuss] Creating a trigger on information_schema.processlist
by Gordan Bobic 28 Feb '23

28 Feb '23

A long standing annoyance has been that stored procedures are opaque when it comes to profiling them with more granularity than just the entire procedure. SHOW FULL PROCESSLIST shows the individual queries while they are running, but polling SHOW FULL PROCESSLIST or SELECT * FROM information_schema.PROCESSLIST is not a practical solution because you can only poll it so many times per second without crippling the server. So I had the idea to hook triggers to information_schema.PROCESSLIST to capture the changes to the INFO column and shunt them into mysql.slow_log. But MariaDB informs them that root isn't privileged enough to crate a trigger in information_schema. Is there another way to augment slow logging granularity of queries that execute as part of the stored procedure?

2 1

[Maria-discuss] Mariadb 10.4 to 10.6 performance regression
by Reinis Rozitis 30 Jan '23

30 Jan '23

Hello, for testing the upgrade path I did an in-place slave server (with optimistic parallel replication) upgrade from 10.4.26 to 10.6.11, but the performance difference is so significant/abysmal I don't even know where to look at (see the attached QPS chart before and after upgrade - it's like 2-3x times lower!). All the previous settings/configuration are kept. I have various server parameters monitored but none seem to indicate why would everything become way slower (see for example the disk write operations also are 2-3 times lower). Maybe someone can give an idea where to look or what to check what could make such an impact? p.s. I know about https://jira.mariadb.org/browse/MDEV-29988 but I don't think that I'm hitting it since the cpu load hasn't changed (is even lower). wbr rr

1 1

[Maria-discuss] Branch 10.4 conflict
by Otto Kekäläinen 30 Jan '23

30 Jan '23

Hello! My local branch of 10.4 was at commit https://github.com/MariaDB/server/commit/9335e2a2b1d555d2ad83ae6f44acb8467c… and pulling from upstream causes now conflicts. Did somebody (Jan?) overwrite git history recently for mariadb-server or what might be going on? - Otto

3 2

[Maria-discuss] base64 text
by Ruben Safir 29 Jan '23

29 Jan '23

is there a way to encode text blogs to base64 so I can handle emails that are stored in the database so they don't look like binary trash like this? http://www.nylxs.com/messages.html?id=553339&archive=2023-01-01 ------ So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

2 1

[Maria-discuss] subscribe
by Ruben Safir 29 Jan '23

29 Jan '23

subscribe -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

1 0

[Maria-discuss] subscribe
by Ruben Safir 29 Jan '23

29 Jan '23

subscribe -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

1 0

[Maria-discuss] Query got stuck while we fetch for specific timestamp
by ragul rangarajan 27 Jan '23

27 Jan '23

Hi all, I have a TableA with data around 1.4Billion which was partitioned by timestamp. <https://stackoverflow.com/posts/75231474/timeline> TableA ( Column1, Column 2....Column N, timestamp) Partition by times I was able to see a weird behavior recently on this table where querying specific timestamps ended in an issue (Query getting stuck more than expected time - 10min and block IO utilization which inturn lead to service degradation). 127.0.0.1:3307>select * from TableA where cloumn1=v1 and cloumn2=v2 and cloumn3=v3 and cloumn4=v4 and cloumn5=v5 and (timestamp between 1673319600 and 1673319600);^CCtrl-C -- query killed. Continuing normally. ERROR 1317 (70009): Query execution was interrupted 127.0.0.1:3307> analyze select * from TableA where cloumn1=v1 and cloumn2=v2 and cloumn3=v3 and cloumn4=v4 and cloumn5=v5 and (timestamp between 1673319600 and 1673319600);^CCtrl-C -- query killed. Continuing normally. ERROR 1317 (70100): Query execution was interrupted 127.0.0.1:3307> explain select * from TableA where cloumn1=v1 and cloumn2=v2 and cloumn3=v3 and cloumn4=v4 and cloumn5=v5 and (timestamp between 1673319600 and 1673319600); +------+-------------+--------------+------+---------------+------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+------+-------------+--------------+------+---------------+------+---------+------+------+-------------+| 1 | SIMPLE | TableA | ALL | PRIMARY | NULL | NULL | NULL | 14 | Using where |+------+-------------+--------------+------+---------------+------+---------+------+------+-------------+1 row in set (0.042 sec) while the same was working under different timestamp 127.0.0.1:3307> explain select * from TableA where cloumn1=v1 and cloumn2=v2 and cloumn3=v3 and cloumn4=v4 and cloumn5=v5 and (timestamp between 1673319600+3600 and 1673319600+7200+7200+7200+7200+7200+7200);+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+-------------+| 1 | SIMPLE | TableA | ref | PRIMARY | PRIMARY | 7 | const,const,const,const,const | 12 | Using where |+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+-------------+1 row in set (0.002 sec) 127.0.0.1:3307> analyze select * from TableA where cloumn1=v1 and cloumn2=v2 and cloumn3=v3 and cloumn4=v4 and cloumn5=v5 and (timestamp between 1673319600+3600 and 1673319600+7200+7200+7200+7200+7200+7200);+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+--------+----------+------------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+--------+----------+------------+-------------+| 1 | SIMPLE | TableA | ref | PRIMARY | PRIMARY | 7 | const,const,const,const,const | 12 | 600.00 | 100.00 | 91.67 | Using where |+------+-------------+--------------+------+---------------+---------+---------+-------------------------------+------+--------+----------+------------+-------------+1 row in set (0.003 sec) Suspecting something might get corrupted in the DB entry for a specific time but not sure how to verify it. 1. Is there any way to check the Data corruption issue in the specific table? a. Tried to check via mysqlcheck but it consumes more time and IO utilization. b. Do we have any other option to check the same in a better way? 1. If we have a problem with DB table entries do we have options to solve this a. Either restarting the active DB or other ways to repair the data to solve the issue like mysqlcheck repair. 1. Not sure why we see NULL in key, Key_len, and possible_key for the first explain statement for the same table. Regards, Ragul R

2 1

[Maria-discuss] [newbie] Include latest ID into INSERT?
by Gilles 23 Jan '23

23 Jan '23

Hello, I need to write a script in SQL to import topics + posts into a phpBB forum. The issue I have is: How can I reference the last topic ID when adding a post to it, since a post must reference a topic? ============ #Create new topic (ie. discussion thread) INSERT INTO `phpbb_topics` (forum_id,topic_title,topic_poster,topic_time) VALUES (); #Add first post to topic #INSERT INTO `phpbb_posts` (topic_id,forum_id,poster_id,post_time,post_username,post_text) VALUES (); INSERT INTO `phpbb_posts` (SELECT LAST_INSERT_ID(),forum_id,poster_id,post_time,post_username,post_text) VALUES (); ============ Can it be done in SQL, or should I write it in eg. Python? Thank you.

1 0

[Maria-discuss] Inspect SQL sentencies in aria_log files
by woodyst 22 Jan '23

22 Jan '23

Hello! Please, forgive me if I am asking in the wrong list so I do not know if it is the correct place for asking this question. I was using MyISAM engine in the past, and I used binlog files for inspecting sql queries executed in the system for debuging purposes. Now I use Aria and I would need to list queries as I did with mysqlbinlog, but for the aria_log files. I have been reading a lot of documentation but I only find that the aria_read_log utility with the display-only option does not print the sql queries as mysqldump did. Please, can anybody help me with this? Thanks in advance. -- Edu.

2 1

[Maria-discuss] Re-starting mariadb again
by Jogchum Reitsma 19 Jan '23

19 Jan '23

Hi, 12th of October I started the topic" Maria-db refuses to start". With the help offered in the conversation following I could access my data by manually starting an older version of Mariadb. I then dumped my tables with mysqldump, one by one, successfully. I also dumped the mysql table. Final advice was to purge all reminiscences of the older versions, and make a fresh start by installing the version of my OS (OpenSuse Tumbleweed). In that new environment, import the dumped data. A few questions remain: - should I, after the install process of the OS version (at the moment 10.10.2-1.1) take further actions to get a fully functioning system? - should I import the dumped "mysql" table? - if yes, should I do that before or after importing the user data? Or doesn't that matter? Again, thanks for your answers! kind regards, Jogchum

1 1

[Maria-discuss] MySQLTuner-perl 2.0.10 with MariaDB 10.3.35 + Plesk
by Dragos Pacher 18 Jan '23

18 Jan '23

Hello, I am looking for an expert opinion on MySQLTuner-perl recommendations and usefulness in my specific case of MariaDB 10.3.35 with Plesk Web Host Edition 18.0.49 Update #2 - with a somehow default installation: one demo site added + some popular components in default state idling - on Rocky Linux 8.7 kernel 4.18.0-425.10.1.el8_7.x86_64. What are your opinions on the recommendations below? My machine is a VirtualBox 7.0.4 one(on Windows 11 22H2 build 22621.1105) with slower Seagate Barracuda PRO disks (non RAID), 8 GB RAM, processor Ryzen 7 3700x(all cores assigned). This was meant just for testing but I am trying to extrapolate some of the results here for a production environment with generous hardware(over 48 GB RAM and newer generation CPUs, NVME/RAIDs). The only way (as of now) for Plesk to work is by coexisting on the same machine with the database server - please take this into consideration. Please see the results below after ~15-24h of Plesk idling on the machine. Also please let me know if you need more data from my side or if I should enable something and rerun a specific test, I can provide detailed statistics if needed. I am really looking forward on how to make this database a bit faster if possible(without enabling really dangerous options). Thank you. [root@pensive-aryabhata ~]# cat /etc/my.cnf # # This group is read both both by the client and the server # use it for options that affect everything # [client-server] # # include all files from the config directory # !includedir /etc/my.cnf.d [mysqld] sql_mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION bind-address = ::ffff:127.0.0.1 local-infile=0 join_buffer_size = 5M # I changed this from 3 to 5 24h before running the MySQLTuner [root@pensive-aryabhata ~]# sysctl -a | grep swapp vm.force_cgroup_v2_swappiness = 0 vm.swappiness = 5 [root@pensive-aryabhata ~]# perl mysqltuner.pl --host 127.0.0.1 >> MySQLTuner 2.0.10 * Jean-Marie Renouard <jmrenouard(a)gmail.com> * Major Hayden <major(a)mhtx.net> >> Bug reports, feature requests, and downloads at http://mysqltuner.pl/ >> Run with '--help' for additional options and output filtering [--] Skipped version check for MySQLTuner script [--] Performing tests on 127.0.0.1:3306 [OK] Currently running supported MySQL version 10.3.35-MariaDB [OK] Operating on 64-bit architecture -------- Log file Recommendations ------------------------------------------------------------------ [OK] Log file /var/log/mariadb/mariadb.log exists [--] Log file: /var/log/mariadb/mariadb.log (13K) [OK] Log file /var/log/mariadb/mariadb.log is not empty [OK] Log file /var/log/mariadb/mariadb.log is smaller than 32 Mb [OK] Log file /var/log/mariadb/mariadb.log is readable. [!!] /var/log/mariadb/mariadb.log contains 15 warning(s). [!!] /var/log/mariadb/mariadb.log contains 2 error(s). [--] 5 start(s) detected in /var/log/mariadb/mariadb.log [--] 1) 2023-01-17 23:36:33 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 2) 2023-01-17 23:36:20 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 3) 2023-01-17 23:23:33 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 4) 2023-01-17 18:18:58 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 5) 2023-01-17 18:18:24 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 4 shutdown(s) detected in /var/log/mariadb/mariadb.log [--] 1) 2023-01-17 23:36:29 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 2) 2023-01-17 23:36:16 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 3) 2023-01-17 23:20:59 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 4) 2023-01-17 18:18:50 0 [Note] /usr/libexec/mysqld: Shutdown complete -------- Storage Engine Statistics ----------------------------------------------------------------- [--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE [--] Data in InnoDB tables: 8.2M (Tables: 267) [OK] Total fragmented tables: 0 -------- Analysis Performance Metrics -------------------------------------------------------------- [--] innodb_stats_on_metadata: OFF [OK] No stat updates during querying INFORMATION_SCHEMA. -------- Views Metrics ----------------------------------------------------------------------------- -------- Triggers Metrics -------------------------------------------------------------------------- -------- Routines Metrics -------------------------------------------------------------------------- -------- Security Recommendations ------------------------------------------------------------------ [OK] There are no anonymous accounts for any database users [OK] All database users have passwords assigned [!!] There is no basic password file list! -------- CVE Security Recommendations -------------------------------------------------------------- [--] Skipped due to --cvefile option undefined -------- Performance Metrics ----------------------------------------------------------------------- [--] Up for: 13h 39m 22s (45K q [0.920 qps], 1K conn, TX: 38M, RX: 3M) [--] Reads / Writes: 86% / 14% [--] Binary logging is disabled [--] Physical Memory : 7.6G [--] Max MySQL memory : 3.9G [--] Other process memory: 0B [--] Total buffers: 417.0M global + 23.7M per thread (151 max threads) [--] Performance_schema Max memory usage: 0B [--] Galera GCache Max memory usage: 0B [OK] Maximum reached memory usage: 724.6M (9.33% of installed RAM) [OK] Maximum possible memory usage: 3.9G (51.40% of installed RAM) [OK] Overall possible memory usage with other process is compatible with memory available [OK] Slow queries: 0% (0/45K) [OK] Highest usage of available connections: 8% (13/151) [OK] Aborted connections: 0.30% (3/1012) [!!] Name resolution is active: a reverse name resolution is made for each new connection which can reduce performance [OK] Query cache is disabled by default due to mutex contention on multiprocessor machines. [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 306 sorts) [!!] Joins performed without indexes: 228 [!!] Temporary tables created on disk: 90% (8K on disk / 9K total) [OK] Thread cache hit rate: 86% (140 created / 1K connections) [OK] Table cache hit rate: 98% (42K hits / 43K requests) [!!] table_definition_cache (400) is less than number of tables (428) [OK] Open file limit used: 0% (84/32K) [OK] Table locks acquired immediately: 100% (48 immediate / 48 locks) -------- Performance schema ------------------------------------------------------------------------ [!!] Performance_schema should be activated. [--] Sys schema is not installed. -------- ThreadPool Metrics ------------------------------------------------------------------------ [--] ThreadPool stat is disabled. -------- MyISAM Metrics ---------------------------------------------------------------------------- [!!] Key buffer used: 18.3% (23.4M used / 128.0M cache) [OK] Key buffer size / total MyISAM indexes: 128.0M/123.0K [!!] Read Key buffer hit rate: 73.3% (75 cached / 20 reads) -------- InnoDB Metrics ---------------------------------------------------------------------------- [--] InnoDB is enabled. [--] InnoDB Thread Concurrency: 0 [OK] InnoDB File per table is activated [OK] InnoDB buffer pool / data size: 128.0M / 8.2M [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (75%): 48.0M * 2 / 128.0M should be equal to 25% [OK] InnoDB buffer pool instances: 1 [--] Number of InnoDB Buffer Pool Chunk: 1 for 1 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances [OK] InnoDB Read buffer efficiency: 99.08% (182560 hits / 184256 total) [!!] InnoDB Write Log efficiency: 11.48% (114 hits / 993 total) [OK] InnoDB log waits: 0.00% (0 waits / 1107 writes) -------- Aria Metrics ------------------------------------------------------------------------------ [--] Aria Storage Engine is enabled. [OK] Aria pagecache size / total Aria indexes: 128.0M/0B [!!] Aria pagecache hit rate: 91.0% (95K cached / 8K reads) -------- TokuDB Metrics ---------------------------------------------------------------------------- [--] TokuDB is disabled. -------- XtraDB Metrics ---------------------------------------------------------------------------- [--] XtraDB is disabled. -------- Galera Metrics ---------------------------------------------------------------------------- [--] Galera is disabled. -------- Replication Metrics ----------------------------------------------------------------------- [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: MIXED [--] XA support enabled: ON [--] Semi synchronous replication Master: OFF [--] Semi synchronous replication Slave: OFF [--] This is a standalone server -------- Recommendations --------------------------------------------------------------------------- General recommendations: Check warning line(s) in /var/log/mariadb/mariadb.log file Check error line(s) in /var/log/mariadb/mariadb.log file MySQL was started within the last 24 hours: recommendations may be inaccurate Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1 We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found. See https://dev.mysql.com/doc/internals/en/join-buffer-size.html (specially the conclusions at the bottom of the page). When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries which have no LIMIT clause Performance schema should be activated for better diagnostics Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU Variables to adjust: skip-name-resolve=1 join_buffer_size (> 5.0M, or always use indexes with JOINs) tmp_table_size (> 16M) max_heap_table_size (> 16M) table_definition_cache (400) > 428 or -1 (autosizing if supported) performance_schema=ON key_buffer_size (~ 24M) innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals 25% of buffer pool size. # end of MySQL Tuner-perl script #1 # more details [root@pensive-aryabhata ~]# perl mysqltuner.pl --buffers --dbstat --idxstat --sysstat --pfstat --tbstat # # skipped part as the output was too big # -------- Views Metrics ----------------------------------------------------------------------------- -------- Triggers Metrics -------------------------------------------------------------------------- -------- Routines Metrics -------------------------------------------------------------------------- -------- Security Recommendations ------------------------------------------------------------------ [OK] There are no anonymous accounts for any database users [OK] All database users have passwords assigned [!!] There is no basic password file list! -------- CVE Security Recommendations -------------------------------------------------------------- [--] Skipped due to --cvefile option undefined -------- Performance Metrics ----------------------------------------------------------------------- [--] Up for: 13m 5s (3K q [4.741 qps], 1K conn, TX: 1M, RX: 549K) [--] Reads / Writes: 97% / 3% [--] Binary logging is disabled [--] Physical Memory : 7.6G [--] Max MySQL memory : 3.9G [--] Other process memory: 1.1G [--] Total buffers: 417.0M global + 23.7M per thread (151 max threads) [--] Performance_schema Max memory usage: 0B [--] Galera GCache Max memory usage: 0B [--] Global Buffers [--] +-- Key Buffer: 128.0M [--] +-- Max Tmp Table: 16.0M [--] Query Cache Buffers [--] +-- Query Cache: OFF - DISABLED [--] +-- Query Cache Size: 1.0M [--] Per Thread Buffers [--] +-- Read Buffer: 128.0K [--] +-- Read RND Buffer: 256.0K [--] +-- Sort Buffer: 2.0M [--] +-- Thread stack: 292.0K [--] +-- Join Buffer: 5.0M [OK] Maximum reached memory usage: 535.3M (6.90% of installed RAM) [OK] Maximum possible memory usage: 3.9G (51.40% of installed RAM) [OK] Overall possible memory usage with other process is compatible with memory available [OK] Slow queries: 0% (0/3K) [OK] Highest usage of available connections: 3% (5/151) [OK] Aborted connections: 0.00% (0/1594) [!!] Name resolution is active: a reverse name resolution is made for each new connection which can reduce performance [OK] Query cache is disabled by default due to mutex contention on multiprocessor machines. [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 93 sorts) [!!] Joins performed without indexes: 4 [!!] Temporary tables created on disk: 73% (1K on disk / 1K total) [OK] Thread cache hit rate: 99% (5 created / 1K connections) [OK] Table cache hit rate: 89% (3K hits / 3K requests) [!!] table_definition_cache (400) is less than number of tables (428) [OK] Open file limit used: 0% (59/32K) [OK] Table locks acquired immediately: 100% (18 immediate / 18 locks) -------- Performance schema ------------------------------------------------------------------------ [!!] Performance_schema should be activated. [--] Sys schema is not installed. -------- ThreadPool Metrics ------------------------------------------------------------------------ [--] ThreadPool stat is disabled. -------- MyISAM Metrics ---------------------------------------------------------------------------- [!!] Key buffer used: 18.2% (23.3M used / 128.0M cache) [OK] Key buffer size / total MyISAM indexes: 128.0M/123.0K -------- InnoDB Metrics ---------------------------------------------------------------------------- [--] InnoDB is enabled. [--] InnoDB Buffers [--] +-- InnoDB Buffer Pool: 128.0M [--] +-- InnoDB Buffer Pool Instances: 1 [--] +-- InnoDB Buffer Pool Chunk Size: 128.0M [--] +-- InnoDB Log File Size: 48.0M [--] +-- InnoDB Log File In Group: 2 [--] +-- InnoDB Total Log File Size: 96.0M(75 % of buffer pool) [--] +-- InnoDB Log Buffer: 16.0M [--] +-- InnoDB Log Buffer Free: 6.2K [--] +-- InnoDB Log Buffer Used: 8.0K [--] InnoDB Thread Concurrency: 0 [OK] InnoDB File per table is activated [OK] InnoDB buffer pool / data size: 128.0M / 8.2M [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (75%): 48.0M * 2 / 128.0M should be equal to 25% [OK] InnoDB buffer pool instances: 1 [--] Number of InnoDB Buffer Pool Chunk: 1 for 1 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances [!!] InnoDB Read buffer efficiency: 83.63% (8790 hits / 10511 total) [!!] InnoDB Write Log efficiency: 154.55% (17 hits / 11 total) [OK] InnoDB log waits: 0.00% (0 waits / 28 writes) -------- Aria Metrics ------------------------------------------------------------------------------ [--] Aria Storage Engine is enabled. [OK] Aria pagecache size / total Aria indexes: 128.0M/0B [OK] Aria pagecache hit rate: 95.4% (29K cached / 1K reads) -------- TokuDB Metrics ---------------------------------------------------------------------------- [--] TokuDB is disabled. -------- XtraDB Metrics ---------------------------------------------------------------------------- [--] XtraDB is disabled. -------- Galera Metrics ---------------------------------------------------------------------------- [--] Galera is disabled. -------- Replication Metrics ----------------------------------------------------------------------- [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: MIXED [--] XA support enabled: ON [--] Semi synchronous replication Master: OFF [--] Semi synchronous replication Slave: OFF [--] This is a standalone server -------- Recommendations --------------------------------------------------------------------------- General recommendations: Consider stopping or dedicate server for additional process other than mysqld. Check warning line(s) in /var/log/mariadb/mariadb.log file Check error line(s) in /var/log/mariadb/mariadb.log file Limit charset for column to one charset if possible for psa database. Limit collations for column to one collation if possible for psa database. Limit collations for column to one collation if possible for phpmyadmin database. Check all table collations are identical for all tables in roundcubemail database. Limit collations for column to one collation if possible for roundcubemail database. Limit charset for column to one charset if possible for apsc database. Limit collations for column to one collation if possible for apsc database. Add at least a primary key on table psa.SchemaVersions Add at least a primary key on table psa.suspend_handler_history Add at least a primary key on table psa.WebsitesDiagnosticDomains MySQL was started within the last 24 hours: recommendations may be inaccurate Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1 We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found. See https://dev.mysql.com/doc/internals/en/join-buffer-size.html (specially the conclusions at the bottom of the page). When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries which have no LIMIT clause Performance schema should be activated for better diagnostics Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU Variables to adjust: DON'T APPLY SETTINGS BECAUSE THERE ARE TOO MANY PROCESSES RUNNING ON THIS SERVER. OOM KILL CAN OCCUR! skip-name-resolve=1 join_buffer_size (> 5.0M, or always use indexes with JOINs) tmp_table_size (> 16M) max_heap_table_size (> 16M) table_definition_cache (400) > 428 or -1 (autosizing if supported) performance_schema=ON key_buffer_size (~ 24M) innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals 25% of buffer pool size. # end of MySQLTuner command #2 ################################################### # here we can see the databases and count number of tables if this matters MariaDB [psa]> show databases; +--------------------+ | Database | +--------------------+ | apsc | | information_schema | | mysql | | performance_schema | | phpmyadmin | | psa | | roundcubemail | +--------------------+ 7 rows in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'apsc'; +----------+ | COUNT(*) | +----------+ | 23 | +----------+ 1 row in set (0.003 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'mysql'; +----------+ | COUNT(*) | +----------+ | 31 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'phpmyadmin'; +----------+ | COUNT(*) | +----------+ | 19 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'psa'; +----------+ | COUNT(*) | +----------+ | 208 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'roundcubemail'; +----------+ | COUNT(*) | +----------+ | 17 | +----------+ 1 row in set (0.001 sec) # Thank you, Dragos

3 6

[Maria-discuss] command history - ignore cmds including keywords - ?
by lejeczek 06 Jan '23

06 Jan '23

Hi guys. I'm sroogling and the Internet says there should be a couple of ways to tell mySQL, MariaDB too I'd hope, to ignore command lines which contain certain keywords. So I tried bash/shell variable and also put some things into '.my.cnf' but to no avail. Does anybody know how to achieve such a basic, I'd like to think, thing? many thanks, L.

1 0

[Maria-discuss] MariaDB 10.5.18: Performance-Problems with CONCAT
by Johannes Ody 05 Jan '23

05 Jan '23

I'm german. Please excuse my bad english. MariaDB version 10.5.18 is much slower than version 10.5.16 in string-manipulation (CONCAT). In version 10.5.16 my code in the bottom needs for 800 records in tmp_tartikel_live 1-2 seconds. In version 10.5.18 my code needs for 800 records in tmp_tartikel_live 20-30 seconds. In version 10.5.16 my code needs for 26000 records in tmp_tartikel_live 5-6 minutes. It is ok. In version 10.5.18 my code needs for 26000 records in tmp_tartikel_live more than 2 hours and was terminating by the system and was not ready. The execution-time glows much more than linear. The calling script is internal for maintenance-work at night. The purpose of this code is to build our search-tables in MySQL (concordance). The code must work for strings with special characters in the coding-systems html (è), ansi and utf-8. In german and france we have the special characters: ß ü ö äàá èé and so on. I have found other users with performance-problems in MariaDB version 10.5.18, and they also use CONCAT: https://dba.stackexchange.com/questions/284073/query-performance-drop-after… My code is the following: /*#JODY_20221010 Die Datenbank muss fuer diesen Code auf Kollektion=latin1_swedish_ci eingestellt sein. ALTER DATABASE d03a5b1c COLLATE latin1_swedish_ci; Das CHARACTER SET scheint nicht so wichtig zu sein. */ /* #JODY_20211228 (ACHTUNG): Diese Datei muss ASCII-Codiert sein (nicht UTF8) Funktion zum Loeschen von HTML-Tags. Kann nur vom browser aus installiert werden. * / DROP PROCEDURE IF EXISTS mysql_log;*/ DROP FUNCTION IF EXISTS mysql_striptags; DROP FUNCTION IF EXISTS mysql_replace_utf8_sonderzeichen; DROP FUNCTION IF EXISTS mysql_replace_single_char; DROP FUNCTION IF EXISTS mysql_replace_char; DROP FUNCTION IF EXISTS mysql_replace_umlaute; /*DROP FUNCTION IF EXISTS mysql_delete_fuellwoerter;*/ DELIMITER $$ /*CREATE PROCEDURE mysql_log(strText VARCHAR(255)) BEGIN INSERT INTO debugout(cText) VALUES (strText); END$$*/ CREATE FUNCTION mysql_striptags(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE posOpen INT; DECLARE posClose INT; SET str = strParam; loop1: LOOP SET posOpen = LOCATE("<", str); SET posClose = LOCATE(">", str, posOpen); /*#JODY_20211228 Wenn posOpen 0 ist, dann ist posClose auch immer 0.*/ IF posClose = 0 THEN LEAVE loop1; END IF; SET str = CONCAT(LEFT(str, posOpen - 1), " ", MID(str, posClose + 1)); END LOOP loop1; RETURN str; END$$ /*#JODY_20220201: Die Moeglichkeit zur URL-Erzeugung mit UTF8-Sonderzeichen besteht darin, dass ein Anwender Strings aus einem Eingabefeld in die Zwischenablage kopiert und dann im URL-Feld des Browsers einfuegt. Dann stehen in der URL z.B. fuer ä CONCAT(CHAR(195), CHAR(164)) Das Problem ist also die Eingabe von Sonderzeichen im URL-Feld des Browsers.*/ CREATE FUNCTION mysql_replace_utf8_sonderzeichen(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nPos INT; DECLARE nOrd INT; DECLARE strChar VARCHAR(2); SET str = strParam; SET nPos = 1; loop_for: LOOP SET nPos = LOCATE(CHAR(195), str, nPos); /* ACHTUNG (#JODY_20220201): Hier kommen auch Positionen auf ein einfaches a zurueck. Liegt an MySQL. Diese brauche ich hier zwar nicht. Aber der Code kann damit umgehen. Durch das LOCATE spare ich mir aber fuer die grosse Menge an Zeichen die Schleifendurchlaeufe.*/ IF nPos = 0 THEN LEAVE loop_for; END IF; SET strChar = MID(str, nPos, 1); SET nPos = nPos + 1; IF ORD(strChar) != 195 THEN ITERATE loop_for; END IF; SET strChar = MID(str, nPos, 1); SET nOrd = ORD(strChar); IF nOrd >= 132 AND nOrd <= 188 THEN SET str = REPLACE(str, CONCAT(CHAR(195), strChar), CHAR(nOrd + 64)); SET nPos = nPos + 1; END IF; END LOOP; RETURN str; END$$ /* ACHTUNG (#JODY_20220120): Da strNewStr auch nur Blanks enthalten darf/kann, kann hier nicht mit CHAR(2) gearbeitet werden. Variablen vom Typ CHAR(x) waeren bei einem Wert von z.B. " " leer (Laenge 0), da die Laenge sich aus dem ersten Zeichen von rechts, da kein Blank ist, ergibt. Es sind gewissermassen Fortran-Strings, die ja auch so arbeiten. VARCHAR sind dagegen C-Strings, die 0-terminiert sind. */ CREATE FUNCTION mysql_replace_single_char(strParam TEXT, nChr INT, strName CHAR(10), strNewStr VARCHAR(2)) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE strNewStrTmp VARCHAR(2); DECLARE strChar195 VARCHAR(2); DECLARE nPosEnd INT; SET strNewStrTmp = strNewStr; /*ACHTUNG (#JODY_20220120): Nach meinem aktuellen Kenntnisstand koennen Parameter nicht geaendert werden.*/ IF strNewStrTmp = "<" THEN SET strNewStrTmp = LEFT(strName, 1); END IF; SET str = REPLACE(strParam, CONCAT("&", strName, ";"), strNewStrTmp); IF nChr <= 255 THEN SET str = REPLACE(str, CHAR(nChr), strNewStrTmp); END IF; SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr, CHAR), ";"), strNewStrTmp); RETURN str; END$$ CREATE FUNCTION mysql_replace_char(strParam TEXT, nChrMin INT, nChrMax INT, strNamesExt VARCHAR(40), strNewStr CHAR(1)) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nChr INT; DECLARE strNames VARCHAR(40); DECLARE nPosStart INT; DECLARE nPosEnd INT; DECLARE bUseNames TINYINT; DECLARE strName VARCHAR(40); DECLARE strNewStrUpper CHAR(1); SET str = strParam; SET nChr = nChrMin; IF strNamesExt != "" THEN /*#JODY_20211229 In MySQL gibt es in Stored Procedures keine Array-Unterstuetzung. s. https://forums.mysql.com/read.php?98,406197,406216*/ SET strNames = CONCAT("grave;acute;circ;", strNamesExt); /*#JODY_20220408 Dieser Code ist fuer die Sonderzeichen à usw. gedacht. Nicht fuer &grave;*/ SET bUseNames = 1; SET strNewStrUpper = UPPER(strNewStr); SET nPosStart = 1; ELSE SET bUseNames = 0; END IF; loop_for: LOOP IF bUseNames != 0 THEN SET nPosEnd = LOCATE(";", strNames, nPosStart); SET strName = MID(strNames, nPosStart, nPosEnd - nPosStart + 1); /*#JODY_20211229 Das ; will ich mit haben.*/ SET str = REPLACE(str, CONCAT("&", strNewStr, strName), strNewStr); SET str = REPLACE(str, CONCAT("&", strNewStrUpper, strName), strNewStr); /*SET strNames = MID(strNames, nPos + 1);*/ SET nPosStart = nPosEnd + 1; /*#JODY_20211229 So ersprare ich mir dieses MID.*/ END IF; SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr, CHAR), ";"), strNewStr); SET str = REPLACE(str, CHAR(nChr), strNewStr); /*#JODY_20211229 Diese Funktion wird nur fuer Umlaute und nur fuer die kleinen augerufen. Diese liegen zwischen à und ü.*/ SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr - 32, CHAR), ";"), strNewStr); SET str = REPLACE(str, CHAR(nChr - 32), strNewStr); IF nChr >= nChrMax THEN LEAVE loop_for; END IF; SET nChr = nChr + 1; END LOOP; RETURN str; END$$ CREATE FUNCTION mysql_replace_umlaute(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nPos INT; DECLARE nChr INT; SET str = mysql_replace_utf8_sonderzeichen(strParam); /* ACHTUNG (#JODY_20220201): Die UTF8-Sonderzeichen muessen hier ganz oben behandelt werden. Sie werden in ASCII-Sonderzeichen konvertiert, die ja nachfolgend auch noch wieder konvertiert werden. So kann ich das aber allgemein gueltig fuer alle UTF8-Sonderzeichen abhandeln.*/ SET str = REPLACE(LOWER(str), "make-up", "makeup"); /*#JODY_20220315 Make-Up soll intern als ein Wort behandelt werden.*/ /*#JODY_20220120 Die vor- und nachlaufenden Blanks werden benoetigt, damit der Wortanfaeng/-ende richtig und leicht erkannt wird. */ SET str = CONCAT(" ", mysql_striptags(str), " "); SET str = mysql_replace_char(str, 224, 230, "tilde;uml;ring;elig;", "a"); SET str = mysql_replace_char(str, 232, 235, "uml;", "e"); SET str = mysql_replace_char(str, 236, 239, "uml;", "i"); SET str = mysql_replace_char(str, 242, 246, "tilde;uml;slash;", "o"); /*#JODY_20211228 So brauche ich den Ignore-Parameter in mysql_replace_char() nicht mehr. × und ÷ sollen hier nicht beachtet werden.*/ SET str = mysql_replace_single_char(str, 216, "Oslash", "o"); SET str = mysql_replace_single_char(str, 248, "oslash", "o"); SET str = mysql_replace_char(str, 249, 252, "uml;", "u"); SET str = mysql_replace_single_char(str, 223, "szlig", "ss"); SET str = REPLACE(str, "ae", "a"); SET str = REPLACE(str, "oe", "o"); SET str = REPLACE(str, "ue", "u"); SET str = mysql_replace_single_char(str, 191, "iquest", ""); /*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 199, "Ccedil", "<");/*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 231, "ccedil", "<");/*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 241, "ntilde", "<");/*#JODY_20211228 porteño*/ SET str = mysql_replace_single_char(str, 215, "times", "x"); SET str = mysql_replace_single_char(str, 160, "nbsp", " "); SET str = mysql_replace_single_char(str, 178, "sup2", "2"); /*#JODY_20211228 Hochgestellte 2 z.B. bei dem Hersteller dsquared²*/ SET str = mysql_replace_single_char(str, 179, "sup3", "3"); /*#JODY_20211228 Hochgestellte 3 z.B. "wella care³ balance clean anti"*/ SET str = mysql_replace_single_char(str, 185, "sup1", ""); /*#JODY_20211228 Hochgestellte 1; wird aber nicht benoetigt.*/ SET str = mysql_replace_single_char(str, 8216, "lsquo", "");/*#JODY_20211228 j‘adore*/ SET str = mysql_replace_single_char(str, 8217, "rsquo", "");/*#JODY_20211228 j’adore*/ SET str = REPLACE(str, "'", ""); /*#JODY_20211228 l'oreal*/ SET str = mysql_replace_single_char(str, 34/*"*/, "quot", "");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 169, "copy", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 173, "shy", ""); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 174, "reg", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 176, "deg", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 183, "middot", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8194, "ensp", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8218, "sbquo", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8226, "bull", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8230, "hellip", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8364, "euro", "euro"); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8482, "trade", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8722, "minus", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = REPLACE(str, "̈", ""); /*#JODY_20211228 Beispiel: "ascorbinsaure natürliches aroma"; Bei "ä" wird ein ä angezeigt.*/ SET str = mysql_replace_single_char(str, 60, "lt", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 62, "gt", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 171, "laquo", " ");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 187, "raquo", " ");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 189, "frac12", "");/*#JODY_20211228 "mit ca ½ liter"*/ /*#JODY_20211228 (ACHTUNG): Prinzipiell soll ´ durch ein Blank ersetzt werden. Nur bei loreal ist dies auf Grund des Eigennamens nicht erwuenscht.*/ SET str = mysql_replace_single_char(str, 96, "grave", " ");/*#JODY_20220408"*/ /*SET str = REPLACE(str, "l´oreal", "loreal"); #JODY_20220408*/ SET str = mysql_replace_single_char(str, 180, "acute", " ");/*#JODY_20211228 "save´n" oder "rock´n"*/ SET str = REPLACE(str, "&", " "); /*#JODY_20220119 Es exisztieren so viele Begriffe, die in [] eingeklammert sind. z.B. [iln48006]*/ SET str = REPLACE(str, "[", " "); SET str = REPLACE(str, "]", " "); SET str = REPLACE(str, "#", " "); SET str = REPLACE(str, "*", " "); SET str = REPLACE(str, "&", " "); SET str = REPLACE(str, "%", " "); SET str = REPLACE(str, "|", " ");/*#JODY_20220316*/ SET str = REPLACE(str, ".", " "); SET str = REPLACE(str, ":", " "); SET str = REPLACE(str, ",", " "); SET str = REPLACE(str, ";", " "); SET str = REPLACE(str, "?", " "); SET str = REPLACE(str, "!", " "); SET str = REPLACE(str, "/", " "); SET str = REPLACE(str, "\\", " "); /*#JODY_20220120 Macht mir z.B. beim Suchbegriff "wax\" die SQL-Anweisung kaputt.*/ SET str = REPLACE(str, "-", " "); SET str = REPLACE(str, "+", " "); SET str = REPLACE(str, "=", " "); SET str = REPLACE(str, "\r", " "); SET str = REPLACE(str, "\n", " "); SET str = REPLACE(str, "\t", " "); SET str = REPLACE(str, "(", " "); SET str = REPLACE(str, ")", " "); /* #JODY_20220121 Erstmal nicht. Die Anwender suchen meistens nach Woertern, wo ein loeschen dieser Buchstaben das Suchergebnis eigentlich verschlechtert, da weniger Zeichenuebereinstimmungen bei Teilwoertern gefunden werden. In Teilwoertern #JODY_20220120 Dies soll die Menge der Wortredundanzen verringern. z.B. zugewandte/zugewandt* / SET str = REPLACE(str, "em ", " "); SET str = REPLACE(str, "en ", " "); SET str = REPLACE(str, "es ", " "); SET str = REPLACE(str, "er ", " "); SET str = REPLACE(str, " sie ", " "); /*#JODY_20220120 Dies ist so noetig, da "sie" durch das naechste REPLACE zu "si" wird und eine Gleichbehandlung von "sie" und "si" wegen der vielen Suchen nach "si" nicht erwunescht ist. Ausserdem macht die Suche nach "sie" eigentlich auch keinen Sinn.* / SET str = REPLACE(str, "e ", " "); SET str = REPLACE(str, "ig ", " ");*/ /*#JODY_20211229 Bisher koennen durchaus mehrere Leerzeichen hintereinander sein. Diese jetzt zu einem zusammenstreichen.*/ loop1: LOOP SET nPos = LOCATE(" ", str); IF nPos = 0 THEN LEAVE loop1; END IF; SET str = REPLACE(str, " ", " "); END LOOP loop1; SET str = TRIM(str); /*#JODY_20220120 Die vor- und nachlaufenden Blanks wieder loeschen. Wurde nur eingefuegt, damit der Wortanfaeng/-ende richtig und leicht erkannt wird. */ RETURN str; END$$ DELIMITER ; CREATE TABLE tmp_tartikel_live ( kArtikel INT UNSIGNED, cArtNr VARCHAR(31), cSeo VARCHAR(255), cSeoU TEXT, cBeschreibung TEXT, cBeschreibungU TEXT, cName VARCHAR(255), cSuchbegriffe TEXT, cKurzBeschreibung TEXT, kurztyp TEXT, kurzserie TEXT, kurzname TEXT ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci; SELECT tmp_tartikel_live.kArtikel, tmp_tartikel_live.cArtNr, tmp_tartikel_live.cSeo, tmp_tartikel_live.cBeschreibung, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cName), " ") AS cNameU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cSuchbegriffe), " ") AS cSuchbegriffeU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cKurzBeschreibung), " ") AS cKurzBeschreibungU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cBeschreibung), " ") AS cBeschreibungU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cSeo), " ") AS cSeoU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurztyp), " ") AS kurztypU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurzserie), " ") AS kurzserieU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurzname), " ") AS kurznameU FROM tmp_tartikel_live This code prepare article-text for search in. Is this problem known or is it new. I hope, I do it right in this way. Johannes Ody

3 3

[Maria-discuss] problems getting MariaDB to start after install
by Tim McConnell 03 Jan '23

03 Jan '23

Hello list, I have installed and re-installed MariaDB trying to get prelude to work and I'm at a loss. When I try to run 'sudo systemctl start mariadb.service' I get these errors: Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xeu mariadb.service" for details. so I read the KB and found I'm supposed to use 'sudo systemctl status mariadb.socket' which gives this: ● mariadb.socket - MariaDB 10.6.11 database server (socket activation) Loaded: loaded (/lib/systemd/system/mariadb.socket; static) Active: active (listening) since Thu 2022-12-29 18:39:25 CST; 26s ago Until: Thu 2022-12-29 18:39:25 CST; 26s ago Triggers: ● mariadb.service Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Listen: @mariadb (Stream) /run/mysqld/mysqld.sock (Stream) [::]:3306 (Stream) Tasks: 0 (limit: 8190) Memory: 4.0K CPU: 2ms CGroup: /system.slice/mariadb.socket and then if I try to run 'sudo mysqlcheck -A' I'll get this error: mysqlcheck: Got error: 2013: Lost connection to server at 'handshake: reading initial communication packet', system error: 104 when trying to connect I then check the status of the socket and get this output: sudo systemctl status mariadb.socket × mariadb.socket - MariaDB 10.6.11 database server (socket activation) Loaded: loaded (/lib/systemd/system/mariadb.socket; static) Active: failed (Result: service-start-limit-hit) since Thu 2022-12-29 18:4> Duration: 1min 11.173s Triggers: ● mariadb.service Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Listen: @mariadb (Stream) /run/mysqld/mysqld.sock (Stream) [::]:3306 (Stream) CPU: 2ms I then ran 'sudo journalctl -n 20 -u mariadb.service' (as per the kb) and get: Dec 29 19:12:45 DebianTim mariadbd[3660996]: 2022-12-29 19:12:45 0 [Note] /usr/sbin/mariadbd (server 10.6.11-MariaDB-1) starting as process 3660996 ... Dec 29 19:12:45 DebianTim systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Dec 29 19:12:45 DebianTim systemd[1]: mariadb.service: Failed with result 'exit-code'. Dec 29 19:12:45 DebianTim systemd[1]: Failed to start MariaDB 10.6.11 database server. I then found this in my logs: 2022-12-29 19:12:45 0 [Warning] Plugin 'OQGRAPH' is of maturity level gamma while the server is stable 2022-12-29 19:12:45 0 [Note] CONNECT: Version 1.07.0002 March 22, 2021 2022-12-29 19:12:45 0 [Note] InnoDB: Compressed tables use zlib 1.2.13 2022-12-29 19:12:45 0 [Note] InnoDB: Number of pools: 1 2022-12-29 19:12:45 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2022-12-29 19:12:45 0 [Note] InnoDB: Using liburing 2022-12-29 19:12:45 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 2022-12-29 19:12:45 0 [Note] InnoDB: Completed initialization of buffer pool 2022-12-29 19:12:45 0 [Note] InnoDB: 128 rollback segments are active. 2022-12-29 19:12:45 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2022-12-29 19:12:45 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2022-12-29 19:12:45 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2022-12-29 19:12:45 0 [Note] InnoDB: 10.6.11 started; log sequence number 33836; transaction id 4 2022-12-29 19:12:45 0 [Note] Plugin 'FEEDBACK' is disabled. 2022-12-29 19:12:45 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2022-12-29 19:12:45 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded 2022-12-29 19:12:45 0 [Note] InnoDB: Buffer pool(s) load completed at 221229 19:12:45 2022-12-29 19:12:45 0 [Warning] You need to use --log-bin to make -- expire-logs-days or --binlog-expire-logs-seconds work. 2022-12-29 19:12:45 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist 2022-12-29 19:12:45 0 [Note] Using systemd activated socket host :: port 3306 2022-12-29 19:12:45 0 [Note] Using systemd activated unix socket /run/mysqld/mysqld.sock 2022-12-29 19:12:45 0 [Note] Using systemd activated unix socket @mariadb 2022-12-29 19:12:45 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist 2022-12-29 19:12:45 0 [ERROR] Aborting Warning: Memory not freed: 280 So, how do I get this to work? I even tried the apparmor suggestion from the KB and it didn't work, I even tried stopping apparmor service completely and still couldn't get it to work. I'm using Debian testing (Bookworm) with Kernel version 6.0.0-6-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.0.12-1 (2022-12-09) x86_64 GNU/Linux and MariaDB 10.6.11 from the Debian repository. Thanks in advance for any help. -- Tim McConnell <tmcconnell168(a)gmail.com>

4 10

[Maria-discuss] debian 11 + mariadb remote connections
by Aaron Axelsen 29 Dec '22

29 Dec '22

Howdy all! I'm having trouble figuring out why a new install on a debian 11 box of mariadb 10.10 is not allowing remote connections. I'm really at a loss for what's going on. I've adjusted the bind address and made sure it's setup for remote connections. From this box, I am able to access mysql using 127.0.0.1, 192.168.0.143 or the external IP without any issue. When trying to connect from any other box, it's acting like it's blocked by the firewall. * iptables is what I generally use but i've cleared out all rules, flushed all tables, and all policies are set to accept. * There is no firewall running from my cloud host - verified with their support team. * Other services are connecting just fine externally, which would really point towards an issues specific with mariadb. MariaDB is currently binding to app ipv4 and ipv6 interfaces. I've tried having only bind to ipv4, changing the default port to something other than 3306. I even went as far as spinning up a test server, same config and it's working as expected. This server is setup for email - so it's also running postfix, dovecot, rspam, redis and apache. Any suggestions or thoughts are greatly appreciated! -- Aaron Axelsen

3 3

[Maria-discuss] MySQL Workbench Issue
by Software Info 28 Dec '22

28 Dec '22

Hi All, I have a very strange problem here. I have mariadb 10.6.10 running on FreeBSD 13.1. I have SSL enabled globally. When I use MySQL workbench to log on from Windows 10 machines, I can authenticate but when I use the same credentials to log on from a Windows 2012 server, I cannot. I get the error user access denied. The credential is configured to log on from anywhere. Is there some kind of SSL/TLS incompatibility in Windows 2013 that anyone knows of? Any help would be appreciated. Best regards, SI

2 1

[Maria-discuss] Code of Conduct
by Ian Gilfillan 23 Dec '22

23 Dec '22

In the light of recent events, we're implementing a Code of Conduct. Details at https://mariadb.org/going-for-a-code-of-conduct/ - comments are welcome!

10 24