- discuss - lists.mariadb.org

unexpected swap usage for mysqld (but mariadb) process
by cyusedfzfb 25 Sep '24

25 Sep '24

Hi all! New here, signed up just new, to discuss an interesting mariadb behaviour we are seeing, related to mariadb unexpectedly using swap space. Look at this example: RHEL 8.10, running mariadb-server-utils.x86_64 3:10.5.22-1.module+el8.8.0+20134+a92c7654: > [user@db ~]$ free -g > total used free shared buff/cache > available > Mem: 187 9 11 0 165 175 > Swap: 3 3 0 and: > top - 08:52:46 up 39 days, 15:50, 2 users, load average: 1.70, 2.05, > 1.99 > Tasks: 616 total, 2 running, 614 sleeping, 0 stopped, 0 zombie > %Cpu(s): 6.0 us, 0.8 sy, 0.0 ni, 88.2 id, 4.7 wa, 0.1 hi, 0.1 > si, 0.0 st > MiB Mem : 191529.7 total, 11761.7 free, 9764.9 used, 170003.1 > buff/cache > MiB Swap: 4096.0 total, 1006.1 free, 3089.9 used. 179429.5 avail > Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND *SWAP * > 1952811 mysql 20 0 14.4g 5.0g 15172 S 65.7 2.7 9663:14 > mysqld* 1.7g * > 2934963 root 20 0 5208 2056 1408 S 8.9 0.0 10:39.29 > gzip 0 > 2934962 root 20 0 33612 8268 6960 S 7.9 0.0 2:35.31 > mysqldump 0 while: > [user@db ~]$ cat /proc/sys/vm/swappiness > 1 We could (we should..) allocate more ram to mariadb, but the point is: there is plenty of RAM available, and there always has been, since boot. And yet, mariadb is using all swap. I guess this has no performance impact, probably the swapped pages are not actually used, but zabbix does not like 100% swap usage. And frankly: neither do I. Why? Can anyone here explain? Swappiness set to 1...is mariadb ignoring swappiness..? Is there something else we can configure..? I read some older posts on ram/swap, and here is some data, requested in that older thread: > [user@db ~]$ sudo ps -o pid,vsz,comm `pgrep mysqld` > PID VSZ COMMAND > 1952811 15117612 mysqld and > [user@db ~]$ for file in /proc/*/status ; do awk > '/VmSwap|Name/{printf $2 " " $3}END{ > > print ""}' $file; done | sort -k 2 -n -r | grep mysql > mysqld 1982672 kB We are seeing similar behaviour on another machines, running more recent mariadb version: 10.5.22-MariaDB MariaDB Server: > [user@db_another ~]$ free -g > total used free shared buff/cache > available > Mem: 62 40 0 1 21 20 > Swap: 4 3 1 Again: enough RAM available, and swap is used 3/4, mostly by MariaDB. Hope the provided information is enough. Let me know if there is anything else I can provide. Looking forward for any insight or help on the matter. :-) Thanks! MJ

5 6

Logging options to debug SSL problem
by Andrew Wood 22 Sep '24

22 Sep '24

Hi Is there a way to get the server to log verbosely everything to do with connection attempts in order to try and debug why a client keeps getting error 2026 SSL connection error: socket layer receive error? There is currently nothing being logged on the server other than: [Warning] Aborted connection 332 to db: 'unconnected' user: 'unauthenticated' host: '192.168.253.231' (This connection closed normally without authentication) SHOW GLOBAL VARIABLES LIKE 'tls_version'; gives TLSv1.1,TLSv1.2,TLSv1.3 and the client is based on a relatively recent version of libmysqlclient so Im struggling to understand what is going wrong without some more detailed logging. Thanks Andrew

2 2

InnoDB database corrupt after unplaned server restart
by Rainer Dorsch 22 Sep '24

22 Sep '24

Hello, I am using mariadb as backend for nextcloud. After an unplanned server restart by my provider I see in the log this error message repeating many times: Sep 21 23:14:06 contabo.bokomoko.de mariadbd[267649]: 2024-09-21 23:14:06 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/ kb/en/library/innodb-recovery-modes/ for information about forcing recovery. Sep 21 23:14:06 contabo.bokomoko.de mariadbd[267649]: 2024-09-21 23:14:06 0 [ERROR] InnoDB: Page [page id: space=0, page number=300] log sequence number 572290405174 is in the future! Current system log sequence number 572290220157. I have a backup of /var/lib/mysql from before the issue. The files which are currently in /var/lib/mysql seem to be stuck with a timestamp from immediately after the reboot. Can anybody advise on a good recovery strategy? Any hint or advice is welcome. Many thanks Rainer -- Rainer Dorsch http://bokomoko.de/

1 0

Database schema error while upgrading Koha 24.05 in Ubuntu 24.04 Noble LTS
by Md Monirul Islam Ansari 20 Sep '24

20 Sep '24

Dear folks, Greetings from Monirul this side, During an upgrade of Koha to version 24.05.03-1, the process failed due to a database access error. The upgrade script was unable to connect to the MySQL/MariaDB database, suggesting a potential issue with database permissions or configuration. Steps to Reproduce: Attempt to upgrade Koha-common package to version 24.05.03-1 Run sudo apt-get install mariadb-server Confirm the installation/upgrade process Observed Behavior: The upgrade process fails during the database schema upgrade step An error message indicates that access is denied for the Koha database user Error Message: "DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('database=koha_monirul;host=localhost;port=3306','koha_monirul',...) failed: Access denied for user 'koha_monirul'@'localhost' (using password: YES) at /usr/share/koha/lib/Koha/Database.pm line 91. at /usr/share/koha/lib/Koha/Database.pm line 139" Environment: Koha version being upgraded to: 24.05.03-1 Database: MariaDB (version 1:10.11.8-0ubuntu0.24.04.1) Operating System: Ubuntu (likely 24.04 based on package versions)

2 1

Joining two tables on a list of values
by Gingko 14 Sep '24

14 Sep '24

Hello, I would like to know if is there some (efficient!) way to join two tables on a list of values contained in (likely a single column) of the joined table, whatever form could have this list of values ? For example, I have a main table containing: main_table: +------------+------------------+------+-----+------------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------+------------------+------+-----+------------+----------------+ | index | int(11) | NO | PRI | 0 | | | to_join | int(11) | NO | | 0 | | | <other fields, irrelevant> | | || | +------------+------------------+------+-----+------------+----------------+ … and a secondary table containing: joined_table: +------------+------------------+------+-----+------------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------+------------------+------+-----+------------+----------------+ | index | int(11) | NO | PRI | | auto_increment | | int_list | <list_of_ints> | YES | | (empty) | | | <other fields, irrelevant> | | | | | +------------+------------------+------+-----+------------+----------------+ We could suppose, here, as a simple test case, that main_table has five rows where to_join contains respectively 1, 2, 3, 4, and 5, and the joined_table has a single row with the single column int_list combining 2 and 4 in any form (but of course, this list could actually have any length). I'd like to join these tables using a statement like: SELECT <anything> FROM main_table AS M LEFT JOIN joined_table AS J ON M.to_join <some_operator to> J.int_list; … in such a way that main_table would be joined to joined_table using any value contained in int_list, and the statement is expected to give two rows as a result. I tried to set int_list as a TINYTEXT containing "2,4" and use: SELECT <anything> FROM main_table AS M LEFT JOIN joined_table AS J ON M.to_join IN (J.int_list); … but this doesn't work, of course, my comma separated list is seen as a whole single value. I also tried, still using TINYTEXT, the list as "[2][4]" and use: SELECT <anything> FROM main_table AS M LEFT JOIN joined_table AS J ON J.int_listLIKE CONCAT('%[', M.to_join , ']%'); This works (more or less), but it is excruciatingly slow, especially knowing that main_table can actually contain millions of rows, and joined_table several thousands. Is there a better solution for doing that? As the final goal implies, somewhere later, associating to_join in main_table, via int_list, to the index auto increment primary key of the joined_table, using multiple rows there is not an option. Regards, Gingko

5 5

Fulltext index tokenization rules / DOC_ID
by wknauf＠hg-online.de 14 Sep '24

14 Sep '24

Hi all, first question: is there a document that describes the rules how MariaDB builds the fulltext index words? I set "innodb_ft_min_token_size=1" to include also one letter words in the index. But this does not seem to work for words like "D'Artagnan": the "d" letter is not found in "information_schema.INNODB_FT_INDEX_TABLE". It works for "D Artagnan". So I assume that single letters that are followed by a quote are ignored when building the index? You could also post the link to the source code file, but I don't know whether my C knowledge is sufficient ;-) Second question: the doc (https://mariadb.com/kb/en/information-schema-innodb_ft_index_table-table/) states that "information_schema.INNODB_FT_INDEX_TABLE.doc_id" matches to the id column of the original table: "DOC_ID: Document ID of the newly added row, either an appropriate ID column or an internal InnoDB value." But this did not happen for me - the doc_id values do not match my ID column. I tried the datatypes "int" and "bigint". Best regards Wolfgang

4 4

MariaDB ServerFest in Berlin on September 17
by Kristian Nielsen 26 Aug '24

26 Aug '24

The next MariaDB ServerFest will take place in Berlin, Germany. The event is on September 17, 2024, and is generously hosted by MariaDB Foundation sponsor Ionos: https://mariadb.org/fest-2024-berlin/ The event is open to all. I was at the MariaDB ServerFest in Helsinki last Autumn, and I really liked the close interaction with the many interesting participants: big and advanced users, developers, university students, service providers, etc. It is an excellent opportunity to meet many of the core developers, and hopefully a lot of other great people! I will be presenting at the event myself; my talk is titled: "MariaDB Replication – Fantastically Flexible and Fragile". I very much welcome questions during my talks, so feel free to prepare questions for me about anything replication. - Kristian.

2 1

MariaDB server crash status=11/SEGV on version 10.6.19
by zxc072211＠gmail.com 21 Aug '24

21 Aug '24

Hi, First, I apologize if my English isn't perfect. After upgrading my MariaDB server from version 10.6.18 to 10.6.19. I encountered an issue. When I execute a "SELECT ... FROM ( SELECT ... ) x WHERE x.colume1 .." statement, the server returns the error: "ERROR 2013 (HY000): Lost connection to server during query." I checked the `/var/log/messages` and found this message: "mariadb.service: Main process exited, code=killed, status=11/SEGV.". I used core dumping to investigate further and found that the stack frame count reaches #12171. The error at #0 is "... Item::build_clone ... at /usr/src/debug/MariaDB-/src_0/sql/item.h:1928". The previous stack frames keep repeating two frames: #1 Item_func_case_simple::do_build_clone ... at /usr/src/debug/MariaDB-/src_0/sql/item_cmpfunc.h:2448 #2 ... Item::build_clone ... at /usr/src/debug/MariaDB-/src_0/sql/item.h:1928 My OS is Rocky Linux 8.10. The MariaDB server versions where I encountered this issue are 10.6.19 and 10.11.9. The MariaDB server version without issues is 10.6.18. Has anyone else encountered this problem? Thanks in advance for any help.

2 2

External MyISAM partitions
by Jan Křístek 07 Aug '24

07 Aug '24

Hi, We have an old instance of MariaDB 10.3.9/10.3.39, with historical data. Every month I copy an old MyISAM partition (binary files) from the same version of the engine (but a different instance) to a big volume and replace an empty partition in the database with symlinks to the newly copied two files (MYD, MYI - when the engine is stopped). I have been doing it for a year or two, checking each time the data are accessible afterwards. This month it happened, that the data are not accessible anymore, and the database throwing an exception: *Got error 140: "Wrong create options" from storage engine MyISAM.* I tried doing it in a new instance, each time I replace a partition with a link to file elsewhere, the error appeared again. Does anyone have any ideas, why it's showing the error and how to avoid it? I cannot recall changing anything in the database configuration, upgrading any database, or doing any other changes. Thanks for any help you can provide, Jan

4 7

Query Performance Drops 10.3 -> 10.5 - Query Optimizer Refusing Index
by Michael Caplan 02 Aug '24

02 Aug '24

HI there, Looking for some advice. A common query "structure" that performed well in 10.3 is showing a significant performance decrease in 10.5.24. The root cause seems to be 10.5's refusal to use a key. The table in question is has a "key => value" like structure: CREATE TABLE `cases_meta` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `case_id` int(10) unsigned NOT NULL, `key` varchar(50) NOT NULL, `value` mediumtext DEFAULT NULL, PRIMARY KEY (`id`), KEY `key` (`key`), KEY `fk_caes_meta_1` (`case_id`), CONSTRAINT `fk_caes_meta_1` FOREIGN KEY (`case_id`) REFERENCES `cases` (`id`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci The query in question simply is looking for a `key` with a specific value: SELECT * FROM cases_meta AS cm WHERE cm.`key` = 'PAN_NUMBER' AND cm.`value` = '153566' In this example, the table has 2.8 million records. In 10.5, it skips the `key` key, and grinds through 2.8 million records: ANALYZE: { "query_block": { "select_id": 1, "r_loops": 1, "r_total_time_ms": 2743.92693, "table": { "table_name": "cm", "access_type": "ALL", "possible_keys": ["key"], "r_loops": 1, "rows": 1004130, "r_rows": 2820048, "r_table_time_ms": 2548.916341, "r_other_time_ms": 195.0068087, "filtered": 30.31918144, "r_filtered": 0, "attached_condition": "ddx_lab_801615.cm.`key` = 'PAN_NUMBER' and ddx_lab_801615.cm.`value` = '153566'" } } } In 10.3, it uses the `key` key, and reads through a radically smaller subset of tables to fetch the result: ANALYZE: { "query_block": { "select_id": 1, "r_loops": 1, "r_total_time_ms": 357.84, "table": { "table_name": "cm", "access_type": "ref", "possible_keys": ["key"], "key": "key", "key_length": "152", "used_key_parts": ["key"], "ref": ["const"], "r_loops": 1, "rows": 304484, "r_rows": 155123, "r_total_time_ms": 340.94, "filtered": 100, "r_filtered": 0, "index_condition": "ddx_lab_801615.cm.`key` = 'PAN_NUMBER'", "attached_condition": "ddx_lab_801615.cm.`value` = '153566'" } } } I'm not sure why 10.5 is taking this substandard approach. Adding a USE INDEX (`key`) seems to have no effect in 10.5 either, it just skips the index. Any ideas? Thanks, Mike Confidentiality Notice: The information contained in this electronic message and any attachments to this message are intended only for the individual(s) addressed in the message and may contain proprietary and confidential information. If you are not the intended recipient, you should not disseminate, distribute, or copy this e-mail. Please notify the sender and destroy this message. WARNING: Computer viruses can be transmitted via email. The recipient should scan this email before opening it. The company accepts no liability for any damage caused by any virus transmitted by this email. Confidentiality Notice: The information contained in this electronic message and any attachments to this message are intended only for the individual(s) addressed in the message and may contain proprietary and confidential information. If you are not the intended recipient, you should not disseminate, distribute, or copy this e-mail. Please notify the sender and destroy this message. WARNING: Computer viruses can be transmitted via email. The recipient should scan this email before opening it. The company accepts no liability for any damage caused by any virus transmitted by this email.

2 3