- discuss - lists.mariadb.org

[Maria-discuss] Inspect SQL sentencies in aria_log files
by woodyst 22 Jan '23

22 Jan '23

Hello! Please, forgive me if I am asking in the wrong list so I do not know if it is the correct place for asking this question. I was using MyISAM engine in the past, and I used binlog files for inspecting sql queries executed in the system for debuging purposes. Now I use Aria and I would need to list queries as I did with mysqlbinlog, but for the aria_log files. I have been reading a lot of documentation but I only find that the aria_read_log utility with the display-only option does not print the sql queries as mysqldump did. Please, can anybody help me with this? Thanks in advance. -- Edu.

2 1

[Maria-discuss] Re-starting mariadb again
by Jogchum Reitsma 19 Jan '23

19 Jan '23

Hi, 12th of October I started the topic" Maria-db refuses to start". With the help offered in the conversation following I could access my data by manually starting an older version of Mariadb. I then dumped my tables with mysqldump, one by one, successfully. I also dumped the mysql table. Final advice was to purge all reminiscences of the older versions, and make a fresh start by installing the version of my OS (OpenSuse Tumbleweed). In that new environment, import the dumped data. A few questions remain: - should I, after the install process of the OS version (at the moment 10.10.2-1.1) take further actions to get a fully functioning system? - should I import the dumped "mysql" table? - if yes, should I do that before or after importing the user data? Or doesn't that matter? Again, thanks for your answers! kind regards, Jogchum

1 1

[Maria-discuss] MySQLTuner-perl 2.0.10 with MariaDB 10.3.35 + Plesk
by Dragos Pacher 18 Jan '23

18 Jan '23

Hello, I am looking for an expert opinion on MySQLTuner-perl recommendations and usefulness in my specific case of MariaDB 10.3.35 with Plesk Web Host Edition 18.0.49 Update #2 - with a somehow default installation: one demo site added + some popular components in default state idling - on Rocky Linux 8.7 kernel 4.18.0-425.10.1.el8_7.x86_64. What are your opinions on the recommendations below? My machine is a VirtualBox 7.0.4 one(on Windows 11 22H2 build 22621.1105) with slower Seagate Barracuda PRO disks (non RAID), 8 GB RAM, processor Ryzen 7 3700x(all cores assigned). This was meant just for testing but I am trying to extrapolate some of the results here for a production environment with generous hardware(over 48 GB RAM and newer generation CPUs, NVME/RAIDs). The only way (as of now) for Plesk to work is by coexisting on the same machine with the database server - please take this into consideration. Please see the results below after ~15-24h of Plesk idling on the machine. Also please let me know if you need more data from my side or if I should enable something and rerun a specific test, I can provide detailed statistics if needed. I am really looking forward on how to make this database a bit faster if possible(without enabling really dangerous options). Thank you. [root@pensive-aryabhata ~]# cat /etc/my.cnf # # This group is read both both by the client and the server # use it for options that affect everything # [client-server] # # include all files from the config directory # !includedir /etc/my.cnf.d [mysqld] sql_mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION bind-address = ::ffff:127.0.0.1 local-infile=0 join_buffer_size = 5M # I changed this from 3 to 5 24h before running the MySQLTuner [root@pensive-aryabhata ~]# sysctl -a | grep swapp vm.force_cgroup_v2_swappiness = 0 vm.swappiness = 5 [root@pensive-aryabhata ~]# perl mysqltuner.pl --host 127.0.0.1 >> MySQLTuner 2.0.10 * Jean-Marie Renouard <jmrenouard(a)gmail.com> * Major Hayden <major(a)mhtx.net> >> Bug reports, feature requests, and downloads at http://mysqltuner.pl/ >> Run with '--help' for additional options and output filtering [--] Skipped version check for MySQLTuner script [--] Performing tests on 127.0.0.1:3306 [OK] Currently running supported MySQL version 10.3.35-MariaDB [OK] Operating on 64-bit architecture -------- Log file Recommendations ------------------------------------------------------------------ [OK] Log file /var/log/mariadb/mariadb.log exists [--] Log file: /var/log/mariadb/mariadb.log (13K) [OK] Log file /var/log/mariadb/mariadb.log is not empty [OK] Log file /var/log/mariadb/mariadb.log is smaller than 32 Mb [OK] Log file /var/log/mariadb/mariadb.log is readable. [!!] /var/log/mariadb/mariadb.log contains 15 warning(s). [!!] /var/log/mariadb/mariadb.log contains 2 error(s). [--] 5 start(s) detected in /var/log/mariadb/mariadb.log [--] 1) 2023-01-17 23:36:33 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 2) 2023-01-17 23:36:20 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 3) 2023-01-17 23:23:33 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 4) 2023-01-17 18:18:58 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 5) 2023-01-17 18:18:24 0 [Note] /usr/libexec/mysqld: ready for connections. [--] 4 shutdown(s) detected in /var/log/mariadb/mariadb.log [--] 1) 2023-01-17 23:36:29 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 2) 2023-01-17 23:36:16 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 3) 2023-01-17 23:20:59 0 [Note] /usr/libexec/mysqld: Shutdown complete [--] 4) 2023-01-17 18:18:50 0 [Note] /usr/libexec/mysqld: Shutdown complete -------- Storage Engine Statistics ----------------------------------------------------------------- [--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE [--] Data in InnoDB tables: 8.2M (Tables: 267) [OK] Total fragmented tables: 0 -------- Analysis Performance Metrics -------------------------------------------------------------- [--] innodb_stats_on_metadata: OFF [OK] No stat updates during querying INFORMATION_SCHEMA. -------- Views Metrics ----------------------------------------------------------------------------- -------- Triggers Metrics -------------------------------------------------------------------------- -------- Routines Metrics -------------------------------------------------------------------------- -------- Security Recommendations ------------------------------------------------------------------ [OK] There are no anonymous accounts for any database users [OK] All database users have passwords assigned [!!] There is no basic password file list! -------- CVE Security Recommendations -------------------------------------------------------------- [--] Skipped due to --cvefile option undefined -------- Performance Metrics ----------------------------------------------------------------------- [--] Up for: 13h 39m 22s (45K q [0.920 qps], 1K conn, TX: 38M, RX: 3M) [--] Reads / Writes: 86% / 14% [--] Binary logging is disabled [--] Physical Memory : 7.6G [--] Max MySQL memory : 3.9G [--] Other process memory: 0B [--] Total buffers: 417.0M global + 23.7M per thread (151 max threads) [--] Performance_schema Max memory usage: 0B [--] Galera GCache Max memory usage: 0B [OK] Maximum reached memory usage: 724.6M (9.33% of installed RAM) [OK] Maximum possible memory usage: 3.9G (51.40% of installed RAM) [OK] Overall possible memory usage with other process is compatible with memory available [OK] Slow queries: 0% (0/45K) [OK] Highest usage of available connections: 8% (13/151) [OK] Aborted connections: 0.30% (3/1012) [!!] Name resolution is active: a reverse name resolution is made for each new connection which can reduce performance [OK] Query cache is disabled by default due to mutex contention on multiprocessor machines. [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 306 sorts) [!!] Joins performed without indexes: 228 [!!] Temporary tables created on disk: 90% (8K on disk / 9K total) [OK] Thread cache hit rate: 86% (140 created / 1K connections) [OK] Table cache hit rate: 98% (42K hits / 43K requests) [!!] table_definition_cache (400) is less than number of tables (428) [OK] Open file limit used: 0% (84/32K) [OK] Table locks acquired immediately: 100% (48 immediate / 48 locks) -------- Performance schema ------------------------------------------------------------------------ [!!] Performance_schema should be activated. [--] Sys schema is not installed. -------- ThreadPool Metrics ------------------------------------------------------------------------ [--] ThreadPool stat is disabled. -------- MyISAM Metrics ---------------------------------------------------------------------------- [!!] Key buffer used: 18.3% (23.4M used / 128.0M cache) [OK] Key buffer size / total MyISAM indexes: 128.0M/123.0K [!!] Read Key buffer hit rate: 73.3% (75 cached / 20 reads) -------- InnoDB Metrics ---------------------------------------------------------------------------- [--] InnoDB is enabled. [--] InnoDB Thread Concurrency: 0 [OK] InnoDB File per table is activated [OK] InnoDB buffer pool / data size: 128.0M / 8.2M [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (75%): 48.0M * 2 / 128.0M should be equal to 25% [OK] InnoDB buffer pool instances: 1 [--] Number of InnoDB Buffer Pool Chunk: 1 for 1 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances [OK] InnoDB Read buffer efficiency: 99.08% (182560 hits / 184256 total) [!!] InnoDB Write Log efficiency: 11.48% (114 hits / 993 total) [OK] InnoDB log waits: 0.00% (0 waits / 1107 writes) -------- Aria Metrics ------------------------------------------------------------------------------ [--] Aria Storage Engine is enabled. [OK] Aria pagecache size / total Aria indexes: 128.0M/0B [!!] Aria pagecache hit rate: 91.0% (95K cached / 8K reads) -------- TokuDB Metrics ---------------------------------------------------------------------------- [--] TokuDB is disabled. -------- XtraDB Metrics ---------------------------------------------------------------------------- [--] XtraDB is disabled. -------- Galera Metrics ---------------------------------------------------------------------------- [--] Galera is disabled. -------- Replication Metrics ----------------------------------------------------------------------- [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: MIXED [--] XA support enabled: ON [--] Semi synchronous replication Master: OFF [--] Semi synchronous replication Slave: OFF [--] This is a standalone server -------- Recommendations --------------------------------------------------------------------------- General recommendations: Check warning line(s) in /var/log/mariadb/mariadb.log file Check error line(s) in /var/log/mariadb/mariadb.log file MySQL was started within the last 24 hours: recommendations may be inaccurate Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1 We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found. See https://dev.mysql.com/doc/internals/en/join-buffer-size.html (specially the conclusions at the bottom of the page). When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries which have no LIMIT clause Performance schema should be activated for better diagnostics Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU Variables to adjust: skip-name-resolve=1 join_buffer_size (> 5.0M, or always use indexes with JOINs) tmp_table_size (> 16M) max_heap_table_size (> 16M) table_definition_cache (400) > 428 or -1 (autosizing if supported) performance_schema=ON key_buffer_size (~ 24M) innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals 25% of buffer pool size. # end of MySQL Tuner-perl script #1 # more details [root@pensive-aryabhata ~]# perl mysqltuner.pl --buffers --dbstat --idxstat --sysstat --pfstat --tbstat # # skipped part as the output was too big # -------- Views Metrics ----------------------------------------------------------------------------- -------- Triggers Metrics -------------------------------------------------------------------------- -------- Routines Metrics -------------------------------------------------------------------------- -------- Security Recommendations ------------------------------------------------------------------ [OK] There are no anonymous accounts for any database users [OK] All database users have passwords assigned [!!] There is no basic password file list! -------- CVE Security Recommendations -------------------------------------------------------------- [--] Skipped due to --cvefile option undefined -------- Performance Metrics ----------------------------------------------------------------------- [--] Up for: 13m 5s (3K q [4.741 qps], 1K conn, TX: 1M, RX: 549K) [--] Reads / Writes: 97% / 3% [--] Binary logging is disabled [--] Physical Memory : 7.6G [--] Max MySQL memory : 3.9G [--] Other process memory: 1.1G [--] Total buffers: 417.0M global + 23.7M per thread (151 max threads) [--] Performance_schema Max memory usage: 0B [--] Galera GCache Max memory usage: 0B [--] Global Buffers [--] +-- Key Buffer: 128.0M [--] +-- Max Tmp Table: 16.0M [--] Query Cache Buffers [--] +-- Query Cache: OFF - DISABLED [--] +-- Query Cache Size: 1.0M [--] Per Thread Buffers [--] +-- Read Buffer: 128.0K [--] +-- Read RND Buffer: 256.0K [--] +-- Sort Buffer: 2.0M [--] +-- Thread stack: 292.0K [--] +-- Join Buffer: 5.0M [OK] Maximum reached memory usage: 535.3M (6.90% of installed RAM) [OK] Maximum possible memory usage: 3.9G (51.40% of installed RAM) [OK] Overall possible memory usage with other process is compatible with memory available [OK] Slow queries: 0% (0/3K) [OK] Highest usage of available connections: 3% (5/151) [OK] Aborted connections: 0.00% (0/1594) [!!] Name resolution is active: a reverse name resolution is made for each new connection which can reduce performance [OK] Query cache is disabled by default due to mutex contention on multiprocessor machines. [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 93 sorts) [!!] Joins performed without indexes: 4 [!!] Temporary tables created on disk: 73% (1K on disk / 1K total) [OK] Thread cache hit rate: 99% (5 created / 1K connections) [OK] Table cache hit rate: 89% (3K hits / 3K requests) [!!] table_definition_cache (400) is less than number of tables (428) [OK] Open file limit used: 0% (59/32K) [OK] Table locks acquired immediately: 100% (18 immediate / 18 locks) -------- Performance schema ------------------------------------------------------------------------ [!!] Performance_schema should be activated. [--] Sys schema is not installed. -------- ThreadPool Metrics ------------------------------------------------------------------------ [--] ThreadPool stat is disabled. -------- MyISAM Metrics ---------------------------------------------------------------------------- [!!] Key buffer used: 18.2% (23.3M used / 128.0M cache) [OK] Key buffer size / total MyISAM indexes: 128.0M/123.0K -------- InnoDB Metrics ---------------------------------------------------------------------------- [--] InnoDB is enabled. [--] InnoDB Buffers [--] +-- InnoDB Buffer Pool: 128.0M [--] +-- InnoDB Buffer Pool Instances: 1 [--] +-- InnoDB Buffer Pool Chunk Size: 128.0M [--] +-- InnoDB Log File Size: 48.0M [--] +-- InnoDB Log File In Group: 2 [--] +-- InnoDB Total Log File Size: 96.0M(75 % of buffer pool) [--] +-- InnoDB Log Buffer: 16.0M [--] +-- InnoDB Log Buffer Free: 6.2K [--] +-- InnoDB Log Buffer Used: 8.0K [--] InnoDB Thread Concurrency: 0 [OK] InnoDB File per table is activated [OK] InnoDB buffer pool / data size: 128.0M / 8.2M [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (75%): 48.0M * 2 / 128.0M should be equal to 25% [OK] InnoDB buffer pool instances: 1 [--] Number of InnoDB Buffer Pool Chunk: 1 for 1 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances [!!] InnoDB Read buffer efficiency: 83.63% (8790 hits / 10511 total) [!!] InnoDB Write Log efficiency: 154.55% (17 hits / 11 total) [OK] InnoDB log waits: 0.00% (0 waits / 28 writes) -------- Aria Metrics ------------------------------------------------------------------------------ [--] Aria Storage Engine is enabled. [OK] Aria pagecache size / total Aria indexes: 128.0M/0B [OK] Aria pagecache hit rate: 95.4% (29K cached / 1K reads) -------- TokuDB Metrics ---------------------------------------------------------------------------- [--] TokuDB is disabled. -------- XtraDB Metrics ---------------------------------------------------------------------------- [--] XtraDB is disabled. -------- Galera Metrics ---------------------------------------------------------------------------- [--] Galera is disabled. -------- Replication Metrics ----------------------------------------------------------------------- [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: MIXED [--] XA support enabled: ON [--] Semi synchronous replication Master: OFF [--] Semi synchronous replication Slave: OFF [--] This is a standalone server -------- Recommendations --------------------------------------------------------------------------- General recommendations: Consider stopping or dedicate server for additional process other than mysqld. Check warning line(s) in /var/log/mariadb/mariadb.log file Check error line(s) in /var/log/mariadb/mariadb.log file Limit charset for column to one charset if possible for psa database. Limit collations for column to one collation if possible for psa database. Limit collations for column to one collation if possible for phpmyadmin database. Check all table collations are identical for all tables in roundcubemail database. Limit collations for column to one collation if possible for roundcubemail database. Limit charset for column to one charset if possible for apsc database. Limit collations for column to one collation if possible for apsc database. Add at least a primary key on table psa.SchemaVersions Add at least a primary key on table psa.suspend_handler_history Add at least a primary key on table psa.WebsitesDiagnosticDomains MySQL was started within the last 24 hours: recommendations may be inaccurate Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1 We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found. See https://dev.mysql.com/doc/internals/en/join-buffer-size.html (specially the conclusions at the bottom of the page). When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries which have no LIMIT clause Performance schema should be activated for better diagnostics Consider installing Sys schema from https://github.com/mysql/mysql-sys for MySQL Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU Variables to adjust: DON'T APPLY SETTINGS BECAUSE THERE ARE TOO MANY PROCESSES RUNNING ON THIS SERVER. OOM KILL CAN OCCUR! skip-name-resolve=1 join_buffer_size (> 5.0M, or always use indexes with JOINs) tmp_table_size (> 16M) max_heap_table_size (> 16M) table_definition_cache (400) > 428 or -1 (autosizing if supported) performance_schema=ON key_buffer_size (~ 24M) innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals 25% of buffer pool size. # end of MySQLTuner command #2 ################################################### # here we can see the databases and count number of tables if this matters MariaDB [psa]> show databases; +--------------------+ | Database | +--------------------+ | apsc | | information_schema | | mysql | | performance_schema | | phpmyadmin | | psa | | roundcubemail | +--------------------+ 7 rows in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'apsc'; +----------+ | COUNT(*) | +----------+ | 23 | +----------+ 1 row in set (0.003 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'mysql'; +----------+ | COUNT(*) | +----------+ | 31 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'phpmyadmin'; +----------+ | COUNT(*) | +----------+ | 19 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'psa'; +----------+ | COUNT(*) | +----------+ | 208 | +----------+ 1 row in set (0.001 sec) MariaDB [psa]> SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'roundcubemail'; +----------+ | COUNT(*) | +----------+ | 17 | +----------+ 1 row in set (0.001 sec) # Thank you, Dragos

3 6

[Maria-discuss] command history - ignore cmds including keywords - ?
by lejeczek 06 Jan '23

06 Jan '23

Hi guys. I'm sroogling and the Internet says there should be a couple of ways to tell mySQL, MariaDB too I'd hope, to ignore command lines which contain certain keywords. So I tried bash/shell variable and also put some things into '.my.cnf' but to no avail. Does anybody know how to achieve such a basic, I'd like to think, thing? many thanks, L.

1 0

[Maria-discuss] MariaDB 10.5.18: Performance-Problems with CONCAT
by Johannes Ody 05 Jan '23

05 Jan '23

I'm german. Please excuse my bad english. MariaDB version 10.5.18 is much slower than version 10.5.16 in string-manipulation (CONCAT). In version 10.5.16 my code in the bottom needs for 800 records in tmp_tartikel_live 1-2 seconds. In version 10.5.18 my code needs for 800 records in tmp_tartikel_live 20-30 seconds. In version 10.5.16 my code needs for 26000 records in tmp_tartikel_live 5-6 minutes. It is ok. In version 10.5.18 my code needs for 26000 records in tmp_tartikel_live more than 2 hours and was terminating by the system and was not ready. The execution-time glows much more than linear. The calling script is internal for maintenance-work at night. The purpose of this code is to build our search-tables in MySQL (concordance). The code must work for strings with special characters in the coding-systems html (è), ansi and utf-8. In german and france we have the special characters: ß ü ö äàá èé and so on. I have found other users with performance-problems in MariaDB version 10.5.18, and they also use CONCAT: https://dba.stackexchange.com/questions/284073/query-performance-drop-after… My code is the following: /*#JODY_20221010 Die Datenbank muss fuer diesen Code auf Kollektion=latin1_swedish_ci eingestellt sein. ALTER DATABASE d03a5b1c COLLATE latin1_swedish_ci; Das CHARACTER SET scheint nicht so wichtig zu sein. */ /* #JODY_20211228 (ACHTUNG): Diese Datei muss ASCII-Codiert sein (nicht UTF8) Funktion zum Loeschen von HTML-Tags. Kann nur vom browser aus installiert werden. * / DROP PROCEDURE IF EXISTS mysql_log;*/ DROP FUNCTION IF EXISTS mysql_striptags; DROP FUNCTION IF EXISTS mysql_replace_utf8_sonderzeichen; DROP FUNCTION IF EXISTS mysql_replace_single_char; DROP FUNCTION IF EXISTS mysql_replace_char; DROP FUNCTION IF EXISTS mysql_replace_umlaute; /*DROP FUNCTION IF EXISTS mysql_delete_fuellwoerter;*/ DELIMITER $$ /*CREATE PROCEDURE mysql_log(strText VARCHAR(255)) BEGIN INSERT INTO debugout(cText) VALUES (strText); END$$*/ CREATE FUNCTION mysql_striptags(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE posOpen INT; DECLARE posClose INT; SET str = strParam; loop1: LOOP SET posOpen = LOCATE("<", str); SET posClose = LOCATE(">", str, posOpen); /*#JODY_20211228 Wenn posOpen 0 ist, dann ist posClose auch immer 0.*/ IF posClose = 0 THEN LEAVE loop1; END IF; SET str = CONCAT(LEFT(str, posOpen - 1), " ", MID(str, posClose + 1)); END LOOP loop1; RETURN str; END$$ /*#JODY_20220201: Die Moeglichkeit zur URL-Erzeugung mit UTF8-Sonderzeichen besteht darin, dass ein Anwender Strings aus einem Eingabefeld in die Zwischenablage kopiert und dann im URL-Feld des Browsers einfuegt. Dann stehen in der URL z.B. fuer ä CONCAT(CHAR(195), CHAR(164)) Das Problem ist also die Eingabe von Sonderzeichen im URL-Feld des Browsers.*/ CREATE FUNCTION mysql_replace_utf8_sonderzeichen(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nPos INT; DECLARE nOrd INT; DECLARE strChar VARCHAR(2); SET str = strParam; SET nPos = 1; loop_for: LOOP SET nPos = LOCATE(CHAR(195), str, nPos); /* ACHTUNG (#JODY_20220201): Hier kommen auch Positionen auf ein einfaches a zurueck. Liegt an MySQL. Diese brauche ich hier zwar nicht. Aber der Code kann damit umgehen. Durch das LOCATE spare ich mir aber fuer die grosse Menge an Zeichen die Schleifendurchlaeufe.*/ IF nPos = 0 THEN LEAVE loop_for; END IF; SET strChar = MID(str, nPos, 1); SET nPos = nPos + 1; IF ORD(strChar) != 195 THEN ITERATE loop_for; END IF; SET strChar = MID(str, nPos, 1); SET nOrd = ORD(strChar); IF nOrd >= 132 AND nOrd <= 188 THEN SET str = REPLACE(str, CONCAT(CHAR(195), strChar), CHAR(nOrd + 64)); SET nPos = nPos + 1; END IF; END LOOP; RETURN str; END$$ /* ACHTUNG (#JODY_20220120): Da strNewStr auch nur Blanks enthalten darf/kann, kann hier nicht mit CHAR(2) gearbeitet werden. Variablen vom Typ CHAR(x) waeren bei einem Wert von z.B. " " leer (Laenge 0), da die Laenge sich aus dem ersten Zeichen von rechts, da kein Blank ist, ergibt. Es sind gewissermassen Fortran-Strings, die ja auch so arbeiten. VARCHAR sind dagegen C-Strings, die 0-terminiert sind. */ CREATE FUNCTION mysql_replace_single_char(strParam TEXT, nChr INT, strName CHAR(10), strNewStr VARCHAR(2)) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE strNewStrTmp VARCHAR(2); DECLARE strChar195 VARCHAR(2); DECLARE nPosEnd INT; SET strNewStrTmp = strNewStr; /*ACHTUNG (#JODY_20220120): Nach meinem aktuellen Kenntnisstand koennen Parameter nicht geaendert werden.*/ IF strNewStrTmp = "<" THEN SET strNewStrTmp = LEFT(strName, 1); END IF; SET str = REPLACE(strParam, CONCAT("&", strName, ";"), strNewStrTmp); IF nChr <= 255 THEN SET str = REPLACE(str, CHAR(nChr), strNewStrTmp); END IF; SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr, CHAR), ";"), strNewStrTmp); RETURN str; END$$ CREATE FUNCTION mysql_replace_char(strParam TEXT, nChrMin INT, nChrMax INT, strNamesExt VARCHAR(40), strNewStr CHAR(1)) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nChr INT; DECLARE strNames VARCHAR(40); DECLARE nPosStart INT; DECLARE nPosEnd INT; DECLARE bUseNames TINYINT; DECLARE strName VARCHAR(40); DECLARE strNewStrUpper CHAR(1); SET str = strParam; SET nChr = nChrMin; IF strNamesExt != "" THEN /*#JODY_20211229 In MySQL gibt es in Stored Procedures keine Array-Unterstuetzung. s. https://forums.mysql.com/read.php?98,406197,406216*/ SET strNames = CONCAT("grave;acute;circ;", strNamesExt); /*#JODY_20220408 Dieser Code ist fuer die Sonderzeichen à usw. gedacht. Nicht fuer &grave;*/ SET bUseNames = 1; SET strNewStrUpper = UPPER(strNewStr); SET nPosStart = 1; ELSE SET bUseNames = 0; END IF; loop_for: LOOP IF bUseNames != 0 THEN SET nPosEnd = LOCATE(";", strNames, nPosStart); SET strName = MID(strNames, nPosStart, nPosEnd - nPosStart + 1); /*#JODY_20211229 Das ; will ich mit haben.*/ SET str = REPLACE(str, CONCAT("&", strNewStr, strName), strNewStr); SET str = REPLACE(str, CONCAT("&", strNewStrUpper, strName), strNewStr); /*SET strNames = MID(strNames, nPos + 1);*/ SET nPosStart = nPosEnd + 1; /*#JODY_20211229 So ersprare ich mir dieses MID.*/ END IF; SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr, CHAR), ";"), strNewStr); SET str = REPLACE(str, CHAR(nChr), strNewStr); /*#JODY_20211229 Diese Funktion wird nur fuer Umlaute und nur fuer die kleinen augerufen. Diese liegen zwischen à und ü.*/ SET str = REPLACE(str, CONCAT("&#", CONVERT(nChr - 32, CHAR), ";"), strNewStr); SET str = REPLACE(str, CHAR(nChr - 32), strNewStr); IF nChr >= nChrMax THEN LEAVE loop_for; END IF; SET nChr = nChr + 1; END LOOP; RETURN str; END$$ CREATE FUNCTION mysql_replace_umlaute(strParam TEXT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE str TEXT; DECLARE nPos INT; DECLARE nChr INT; SET str = mysql_replace_utf8_sonderzeichen(strParam); /* ACHTUNG (#JODY_20220201): Die UTF8-Sonderzeichen muessen hier ganz oben behandelt werden. Sie werden in ASCII-Sonderzeichen konvertiert, die ja nachfolgend auch noch wieder konvertiert werden. So kann ich das aber allgemein gueltig fuer alle UTF8-Sonderzeichen abhandeln.*/ SET str = REPLACE(LOWER(str), "make-up", "makeup"); /*#JODY_20220315 Make-Up soll intern als ein Wort behandelt werden.*/ /*#JODY_20220120 Die vor- und nachlaufenden Blanks werden benoetigt, damit der Wortanfaeng/-ende richtig und leicht erkannt wird. */ SET str = CONCAT(" ", mysql_striptags(str), " "); SET str = mysql_replace_char(str, 224, 230, "tilde;uml;ring;elig;", "a"); SET str = mysql_replace_char(str, 232, 235, "uml;", "e"); SET str = mysql_replace_char(str, 236, 239, "uml;", "i"); SET str = mysql_replace_char(str, 242, 246, "tilde;uml;slash;", "o"); /*#JODY_20211228 So brauche ich den Ignore-Parameter in mysql_replace_char() nicht mehr. × und ÷ sollen hier nicht beachtet werden.*/ SET str = mysql_replace_single_char(str, 216, "Oslash", "o"); SET str = mysql_replace_single_char(str, 248, "oslash", "o"); SET str = mysql_replace_char(str, 249, 252, "uml;", "u"); SET str = mysql_replace_single_char(str, 223, "szlig", "ss"); SET str = REPLACE(str, "ae", "a"); SET str = REPLACE(str, "oe", "o"); SET str = REPLACE(str, "ue", "u"); SET str = mysql_replace_single_char(str, 191, "iquest", ""); /*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 199, "Ccedil", "<");/*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 231, "ccedil", "<");/*#JODY_20211228 françois*/ SET str = mysql_replace_single_char(str, 241, "ntilde", "<");/*#JODY_20211228 porteño*/ SET str = mysql_replace_single_char(str, 215, "times", "x"); SET str = mysql_replace_single_char(str, 160, "nbsp", " "); SET str = mysql_replace_single_char(str, 178, "sup2", "2"); /*#JODY_20211228 Hochgestellte 2 z.B. bei dem Hersteller dsquared²*/ SET str = mysql_replace_single_char(str, 179, "sup3", "3"); /*#JODY_20211228 Hochgestellte 3 z.B. "wella care³ balance clean anti"*/ SET str = mysql_replace_single_char(str, 185, "sup1", ""); /*#JODY_20211228 Hochgestellte 1; wird aber nicht benoetigt.*/ SET str = mysql_replace_single_char(str, 8216, "lsquo", "");/*#JODY_20211228 j‘adore*/ SET str = mysql_replace_single_char(str, 8217, "rsquo", "");/*#JODY_20211228 j’adore*/ SET str = REPLACE(str, "'", ""); /*#JODY_20211228 l'oreal*/ SET str = mysql_replace_single_char(str, 34/*"*/, "quot", "");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 169, "copy", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 173, "shy", ""); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 174, "reg", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 176, "deg", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 183, "middot", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8194, "ensp", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8218, "sbquo", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8226, "bull", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8230, "hellip", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8364, "euro", "euro"); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8482, "trade", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 8722, "minus", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = REPLACE(str, "̈", ""); /*#JODY_20211228 Beispiel: "ascorbinsaure natürliches aroma"; Bei "ä" wird ein ä angezeigt.*/ SET str = mysql_replace_single_char(str, 60, "lt", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 62, "gt", " "); /*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 171, "laquo", " ");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 187, "raquo", " ");/*#JODY_20211228 Ist fuer die Suche nicht relevant.*/ SET str = mysql_replace_single_char(str, 189, "frac12", "");/*#JODY_20211228 "mit ca ½ liter"*/ /*#JODY_20211228 (ACHTUNG): Prinzipiell soll ´ durch ein Blank ersetzt werden. Nur bei loreal ist dies auf Grund des Eigennamens nicht erwuenscht.*/ SET str = mysql_replace_single_char(str, 96, "grave", " ");/*#JODY_20220408"*/ /*SET str = REPLACE(str, "l´oreal", "loreal"); #JODY_20220408*/ SET str = mysql_replace_single_char(str, 180, "acute", " ");/*#JODY_20211228 "save´n" oder "rock´n"*/ SET str = REPLACE(str, "&", " "); /*#JODY_20220119 Es exisztieren so viele Begriffe, die in [] eingeklammert sind. z.B. [iln48006]*/ SET str = REPLACE(str, "[", " "); SET str = REPLACE(str, "]", " "); SET str = REPLACE(str, "#", " "); SET str = REPLACE(str, "*", " "); SET str = REPLACE(str, "&", " "); SET str = REPLACE(str, "%", " "); SET str = REPLACE(str, "|", " ");/*#JODY_20220316*/ SET str = REPLACE(str, ".", " "); SET str = REPLACE(str, ":", " "); SET str = REPLACE(str, ",", " "); SET str = REPLACE(str, ";", " "); SET str = REPLACE(str, "?", " "); SET str = REPLACE(str, "!", " "); SET str = REPLACE(str, "/", " "); SET str = REPLACE(str, "\\", " "); /*#JODY_20220120 Macht mir z.B. beim Suchbegriff "wax\" die SQL-Anweisung kaputt.*/ SET str = REPLACE(str, "-", " "); SET str = REPLACE(str, "+", " "); SET str = REPLACE(str, "=", " "); SET str = REPLACE(str, "\r", " "); SET str = REPLACE(str, "\n", " "); SET str = REPLACE(str, "\t", " "); SET str = REPLACE(str, "(", " "); SET str = REPLACE(str, ")", " "); /* #JODY_20220121 Erstmal nicht. Die Anwender suchen meistens nach Woertern, wo ein loeschen dieser Buchstaben das Suchergebnis eigentlich verschlechtert, da weniger Zeichenuebereinstimmungen bei Teilwoertern gefunden werden. In Teilwoertern #JODY_20220120 Dies soll die Menge der Wortredundanzen verringern. z.B. zugewandte/zugewandt* / SET str = REPLACE(str, "em ", " "); SET str = REPLACE(str, "en ", " "); SET str = REPLACE(str, "es ", " "); SET str = REPLACE(str, "er ", " "); SET str = REPLACE(str, " sie ", " "); /*#JODY_20220120 Dies ist so noetig, da "sie" durch das naechste REPLACE zu "si" wird und eine Gleichbehandlung von "sie" und "si" wegen der vielen Suchen nach "si" nicht erwunescht ist. Ausserdem macht die Suche nach "sie" eigentlich auch keinen Sinn.* / SET str = REPLACE(str, "e ", " "); SET str = REPLACE(str, "ig ", " ");*/ /*#JODY_20211229 Bisher koennen durchaus mehrere Leerzeichen hintereinander sein. Diese jetzt zu einem zusammenstreichen.*/ loop1: LOOP SET nPos = LOCATE(" ", str); IF nPos = 0 THEN LEAVE loop1; END IF; SET str = REPLACE(str, " ", " "); END LOOP loop1; SET str = TRIM(str); /*#JODY_20220120 Die vor- und nachlaufenden Blanks wieder loeschen. Wurde nur eingefuegt, damit der Wortanfaeng/-ende richtig und leicht erkannt wird. */ RETURN str; END$$ DELIMITER ; CREATE TABLE tmp_tartikel_live ( kArtikel INT UNSIGNED, cArtNr VARCHAR(31), cSeo VARCHAR(255), cSeoU TEXT, cBeschreibung TEXT, cBeschreibungU TEXT, cName VARCHAR(255), cSuchbegriffe TEXT, cKurzBeschreibung TEXT, kurztyp TEXT, kurzserie TEXT, kurzname TEXT ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci; SELECT tmp_tartikel_live.kArtikel, tmp_tartikel_live.cArtNr, tmp_tartikel_live.cSeo, tmp_tartikel_live.cBeschreibung, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cName), " ") AS cNameU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cSuchbegriffe), " ") AS cSuchbegriffeU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cKurzBeschreibung), " ") AS cKurzBeschreibungU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cBeschreibung), " ") AS cBeschreibungU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.cSeo), " ") AS cSeoU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurztyp), " ") AS kurztypU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurzserie), " ") AS kurzserieU, CONCAT(" ", mysql_replace_umlaute(tmp_tartikel_live.kurzname), " ") AS kurznameU FROM tmp_tartikel_live This code prepare article-text for search in. Is this problem known or is it new. I hope, I do it right in this way. Johannes Ody

3 3

[Maria-discuss] problems getting MariaDB to start after install
by Tim McConnell 03 Jan '23

03 Jan '23

Hello list, I have installed and re-installed MariaDB trying to get prelude to work and I'm at a loss. When I try to run 'sudo systemctl start mariadb.service' I get these errors: Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xeu mariadb.service" for details. so I read the KB and found I'm supposed to use 'sudo systemctl status mariadb.socket' which gives this: ● mariadb.socket - MariaDB 10.6.11 database server (socket activation) Loaded: loaded (/lib/systemd/system/mariadb.socket; static) Active: active (listening) since Thu 2022-12-29 18:39:25 CST; 26s ago Until: Thu 2022-12-29 18:39:25 CST; 26s ago Triggers: ● mariadb.service Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Listen: @mariadb (Stream) /run/mysqld/mysqld.sock (Stream) [::]:3306 (Stream) Tasks: 0 (limit: 8190) Memory: 4.0K CPU: 2ms CGroup: /system.slice/mariadb.socket and then if I try to run 'sudo mysqlcheck -A' I'll get this error: mysqlcheck: Got error: 2013: Lost connection to server at 'handshake: reading initial communication packet', system error: 104 when trying to connect I then check the status of the socket and get this output: sudo systemctl status mariadb.socket × mariadb.socket - MariaDB 10.6.11 database server (socket activation) Loaded: loaded (/lib/systemd/system/mariadb.socket; static) Active: failed (Result: service-start-limit-hit) since Thu 2022-12-29 18:4> Duration: 1min 11.173s Triggers: ● mariadb.service Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Listen: @mariadb (Stream) /run/mysqld/mysqld.sock (Stream) [::]:3306 (Stream) CPU: 2ms I then ran 'sudo journalctl -n 20 -u mariadb.service' (as per the kb) and get: Dec 29 19:12:45 DebianTim mariadbd[3660996]: 2022-12-29 19:12:45 0 [Note] /usr/sbin/mariadbd (server 10.6.11-MariaDB-1) starting as process 3660996 ... Dec 29 19:12:45 DebianTim systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE Dec 29 19:12:45 DebianTim systemd[1]: mariadb.service: Failed with result 'exit-code'. Dec 29 19:12:45 DebianTim systemd[1]: Failed to start MariaDB 10.6.11 database server. I then found this in my logs: 2022-12-29 19:12:45 0 [Warning] Plugin 'OQGRAPH' is of maturity level gamma while the server is stable 2022-12-29 19:12:45 0 [Note] CONNECT: Version 1.07.0002 March 22, 2021 2022-12-29 19:12:45 0 [Note] InnoDB: Compressed tables use zlib 1.2.13 2022-12-29 19:12:45 0 [Note] InnoDB: Number of pools: 1 2022-12-29 19:12:45 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2022-12-29 19:12:45 0 [Note] InnoDB: Using liburing 2022-12-29 19:12:45 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 2022-12-29 19:12:45 0 [Note] InnoDB: Completed initialization of buffer pool 2022-12-29 19:12:45 0 [Note] InnoDB: 128 rollback segments are active. 2022-12-29 19:12:45 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2022-12-29 19:12:45 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2022-12-29 19:12:45 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2022-12-29 19:12:45 0 [Note] InnoDB: 10.6.11 started; log sequence number 33836; transaction id 4 2022-12-29 19:12:45 0 [Note] Plugin 'FEEDBACK' is disabled. 2022-12-29 19:12:45 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2022-12-29 19:12:45 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded 2022-12-29 19:12:45 0 [Note] InnoDB: Buffer pool(s) load completed at 221229 19:12:45 2022-12-29 19:12:45 0 [Warning] You need to use --log-bin to make -- expire-logs-days or --binlog-expire-logs-seconds work. 2022-12-29 19:12:45 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist 2022-12-29 19:12:45 0 [Note] Using systemd activated socket host :: port 3306 2022-12-29 19:12:45 0 [Note] Using systemd activated unix socket /run/mysqld/mysqld.sock 2022-12-29 19:12:45 0 [Note] Using systemd activated unix socket @mariadb 2022-12-29 19:12:45 0 [ERROR] Fatal error: Can't open and lock privilege tables: Table 'mysql.db' doesn't exist 2022-12-29 19:12:45 0 [ERROR] Aborting Warning: Memory not freed: 280 So, how do I get this to work? I even tried the apparmor suggestion from the KB and it didn't work, I even tried stopping apparmor service completely and still couldn't get it to work. I'm using Debian testing (Bookworm) with Kernel version 6.0.0-6-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.0.12-1 (2022-12-09) x86_64 GNU/Linux and MariaDB 10.6.11 from the Debian repository. Thanks in advance for any help. -- Tim McConnell <tmcconnell168(a)gmail.com>

4 10

[Maria-discuss] debian 11 + mariadb remote connections
by Aaron Axelsen 29 Dec '22

29 Dec '22

Howdy all! I'm having trouble figuring out why a new install on a debian 11 box of mariadb 10.10 is not allowing remote connections. I'm really at a loss for what's going on. I've adjusted the bind address and made sure it's setup for remote connections. From this box, I am able to access mysql using 127.0.0.1, 192.168.0.143 or the external IP without any issue. When trying to connect from any other box, it's acting like it's blocked by the firewall. * iptables is what I generally use but i've cleared out all rules, flushed all tables, and all policies are set to accept. * There is no firewall running from my cloud host - verified with their support team. * Other services are connecting just fine externally, which would really point towards an issues specific with mariadb. MariaDB is currently binding to app ipv4 and ipv6 interfaces. I've tried having only bind to ipv4, changing the default port to something other than 3306. I even went as far as spinning up a test server, same config and it's working as expected. This server is setup for email - so it's also running postfix, dovecot, rspam, redis and apache. Any suggestions or thoughts are greatly appreciated! -- Aaron Axelsen

3 3

[Maria-discuss] MySQL Workbench Issue
by Software Info 28 Dec '22

28 Dec '22

Hi All, I have a very strange problem here. I have mariadb 10.6.10 running on FreeBSD 13.1. I have SSL enabled globally. When I use MySQL workbench to log on from Windows 10 machines, I can authenticate but when I use the same credentials to log on from a Windows 2012 server, I cannot. I get the error user access denied. The credential is configured to log on from anywhere. Is there some kind of SSL/TLS incompatibility in Windows 2013 that anyone knows of? Any help would be appreciated. Best regards, SI

2 1

[Maria-discuss] Code of Conduct
by Ian Gilfillan 23 Dec '22

23 Dec '22

In the light of recent events, we're implementing a Code of Conduct. Details at https://mariadb.org/going-for-a-code-of-conduct/ - comments are welcome!

10 24

[Maria-discuss] DBMS vs Storage Engine Alone
by Jonathan M. Wilbur 20 Dec '22

20 Dec '22

Sorry to ask this question here, since it is not strictly MariaDB-related, but I need the knowledge of people that develop with databases: is there significant overhead to using a DBMS versus a storage engine directly? I have a project for which speed / latency is very important, and I was wondering if I could make significant performance gains by using a storage engine (such as WiredTiger, RocksDB, LevelDB, etc.) directly instead of using a DBMS (like MariaDB). More concretely, are there any studies out there on this to give me some objective numbers on this? Assume it is over a local loopback connection, so pretty much only the overhead of the networking itself would apply. Thank you in advance, - Jonathan M. Wilbur

4 3