October 2018 - discuss - lists.mariadb.org

[Maria-discuss] why stdout? -> [Note] /usr/libexec/mysqld (mysqld 10.0.20-MariaDB) starting as process 115096
by Reindl Harald 05 Jul '20

05 Jul '20

Aug 6 12:24:24 testserver mysqld: 150806 12:24:24 [Note] /usr/libexec/mysqld (mysqld 10.0.20-MariaDB) starting as process 115096 ... why is this now stdout instead mysql logs? on systemd based distributions that spits now to the global syslog and finally on machines restarting 2 instances every hour for backup reasons it annoys until 10.0.19 or 10.0.20 that was not this way

5 13

[Maria-discuss] MariaDB-10.2.11 Long Time To Start
by Michael Caplan 15 Mar '19

15 Mar '19

Hi, I'm setting up a replication slave based off of MySQL 5.6. There are a large number of tables (400K). The MySQL 5.6 master takes around 10 minutes to start up, where the new MariaDB 10.2 slave takes close to 30 minutes. What is making the startup take soo long? In the below mysql log, you can see in the startup sequence between "Highest supported file format is Barracuda." and "InnoDB: 128 out of 128 rollback segments are active." the startup process takes the majority of its time: 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Uses event mutexes 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Compressed tables use zlib 1.2.8 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Using Linux native AIO 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Number of pools: 1 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Using SSE2 crc32 instructions 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Initializing buffer pool, total size = 54G, instances = 8, chunk size = 128M 2018-01-07 9:38:18 139980432365760 [Note] InnoDB: Completed initialization of buffer pool 2018-01-07 9:38:18 139918113195776 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2018-01-07 9:38:18 139980432365760 [Note] InnoDB: Highest supported file format is Barracuda. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: 128 out of 128 rollback segments are active. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Creating shared tablespace for temporary tables 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Waiting for purge to start 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: 5.7.20 started; log sequence number 1394303471760 2018-01-07 10:11:48 139919461500672 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2018-01-07 10:11:48 139980432365760 [Note] Server socket created on IP: '::'. 2018-01-07 10:11:49 139980432365760 [Note] Reading of all Master_info entries succeded 2018-01-07 10:11:49 139980432365760 [Note] Added new Master_info '' to hash table 2018-01-07 10:11:49 139980432365760 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.11-MariaDB-10.2.11+maria~xenial-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 mariadb.org binary distribution Any ideas why? I've included my my.conf file below. Thanks, Mike # Generated by Percona Configuration Wizard (http://tools.percona.com/) version REL5-20120208 # Configuration name usaddxprdsqlpm3 generated formcaplan(a)labnet.net at 2017-12-22 13:49:58 [mysql] # CLIENT # port = 3306 socket = /var/lib/mysql/mysql.sock [mysqladmin] # CLIENT # port = 3306 socket = /var/lib/mysql/mysql.sock [mysqld] # GENERAL # user = mysql default-storage-engine = InnoDB socket = /var/lib/mysql/mysql.sock pid-file = /var/lib/mysql/mysql.pid character-set-server = utf8 collation_server = utf8_general_ci performance_schema = 0 # MyISAM # key-buffer-size = 32M myisam-recover = FORCE,BACKUP # SAFETY # max-allowed-packet = 20M max-connect-errors = 1000000 skip-name-resolve sysdate-is-now = 1 innodb = FORCE # DATA STORAGE # datadir = /var/lib/mysql/ # BINARY LOGGING # log-bin = /var/lib/mysql/mysql-bin binlog_format = MIXED server-id = 3 expire-logs-days = 7 sync-binlog = 1 # REPLICATION # read-only = 1 skip-slave-start = 1 log-slave-updates = 1 relay-log = /var/lib/mysql/relay-bin slave-net-timeout = 60 sync-master-info = 1 sync-relay-log = 1 sync-relay-log-info = 1 # CACHES AND LIMITS # tmp-table-size = 32M max-heap-table-size = 32M query-cache-type = 0 query-cache-size = 0 max-connections = 500 thread-cache-size = 50 open-files-limit = 65535 table-definition-cache = 4096 table-open-cache = 10240 # INNODB # innodb-flush-method = O_DIRECT innodb-log-files-in-group = 2 innodb-log-file-size = 512M innodb-flush-log-at-trx-commit = 1 innodb-file-per-table = 1 innodb-buffer-pool-size = 54G # LOGGING # log-error = /var/lib/mysql/mysql-error.log log-queries-not-using-indexes = 0 slow-query-log = 0 slow-query-log-file = /var/lib/mysql/mysql-slow.log [mariadb] plugin-load-add=file_key_management file_key_management_filename=/var/lib/mysql/keys.enc file_key_management_filekey=FILE:/var/lib/mysql/mysql.key file_key_management_encryption_algorithm=AES_CTR encrypt_binlog=0

3 19

[Maria-discuss] Chewing Through Swap - Swappiness = 0
by Michael Caplan 09 Mar '19

09 Mar '19

Hi, I'm trying to figure out why my recently put into production MariaDB is so swap hungry. I'm running 10.2.14, with roughly 300GB data (1000K +/- tables). 95% tables are innodb. I have 64GB RAM, with INNODB buffer pool size set to 50GB (full my.cnf below). The OS is Ubuntu 16.04.4. This is a dedicated MariaDB server. I have swappiness set to 0. I started with available swap set to just shy of 1GB. When swap got to 85% used, I bumped swap to 3GB. A day later, swap again was 85% used, and I bumped it to 5GB. A day later, swap again was 85% used, so I bumped it to 15GB. 2 days later 50% of the available swap was used. I restarted the DB moments ago, freeing up all 15GB but 128MB of swap: total used free shared buff/cache available Mem: 62G 20G 4.7G 266M 37G 41G Swap: 14G 128M 14G My understanding is that the OS should never allocate swap when swappiness is 0, so that has me confused. I've had similar issues with MySQL 5.6 (what I upgraded from) dipping into swap, but that instance had swappiness set to 1, and never consumed more than 5GB of swap. Questions: 1) How much of this is an OS issue vs MariaDB issue? 2) What MariaDB config vars should I be looking at to fix this issue? 3) What linux config vars should I be looking at to fix this issue? Thanks, Mike [mysqld] # GENERAL # user = mysql default-storage-engine = InnoDB character-set-server = utf8 collation_server = utf8_general_ci performance_schema = 0 max_allowed_packet = 16777216 sql_mode = "NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" # MyISAM # key-buffer-size = 32M myisam-recover = FORCE,BACKUP # SAFETY # max-allowed-packet = 16777216 max-connect-errors = 1000000 skip-name-resolve sysdate-is-now = 1 innodb = FORCE local_infile = 0 secure_auth = 1 safe_user_create = 1 skip_symbolic_links = 1 wait_timeout = 28800 # DATA STORAGE # datadir = /var/lib/mysql/ # BINARY LOGGING # log-bin = /var/lib/mysql/mysql-bin binlog_format = MIXED server-id = 3 expire-logs-days = 7 sync-binlog = 1 # REPLICATION # read-only = 1 skip-slave-start = 1 log-slave-updates = 1 relay-log = /var/lib/mysql/relay-bin slave-net-timeout = 60 sync-master-info = 1 sync-relay-log = 1 sync-relay-log-info = 1 # CACHES AND LIMITS # tmp-table-size = 32M max-heap-table-size = 32M query-cache-type = 0 query-cache-size = 0 max-connections = 500 thread-cache-size = 50 open-files-limit = 65535 table-definition-cache = 4096 table-open-cache = 10240 # INNODB # innodb-flush-method = O_DIRECT innodb-log-files-in-group = 2 innodb-log-file-size = 512M innodb-flush-log-at-trx-commit = 2 innodb-file-per-table = 1 innodb-buffer-pool-size = 50G

3 8

[Maria-discuss] adding syslog crashes container
by Paul Chubb 30 Oct '18

30 Oct '18

hi I am trying to get Mariadb to log to syslog - this is for log consolidation etc. I am using Maria in a docker container from docker hub. Other things from the container are successfully logging to syslog on my logging server but Maria isn't. If I add the word 'syslog' to a [mysqld_safe] section the configuration is not read (as reported by mysqld --print-defaults). The container init script calls mysqld rather than mysqld_safe If I change the init script to call mysqld_safe instead of mysqld, syslog is set as reported by --print-defaults however the container continually crashes after restart. If I add the word 'syslog' to [mysqld] the configuration is read however after restart, mariadb continually crashes - at least the container does. I have checked for log-error statements, when checking global vars log_error is empty and it is not reported on the print-defaults output. If I setup to log to a file, this works fine. The container is running Ubuntu 18.04 Mariadb version is 10.3.10 rsyslog version is kinetic 8.32.0 Not sure what you want to see to help me, advise and I will do my best to give it to you. Cheers paul

3 3

[Maria-discuss] Secure replication with mariadb
by Danishka Navin 26 Oct '18

26 Oct '18

Hi Team, I am working on deploying moodle (using mariadb) in over 1000 centers and connect each remote center with central server. Each remote center has no identical data as users are totally different. (But same type of data across these 1000 remote center nodes). We can not gurrenty the connectivity between remote server and centralized server in all 1000 nodes (some of the remote servers may have connectivity issues but we need to recover all data once the connectivity establish). Is it possible to replicate each remote server in to single central server over TLS? or do we have better solution? Regards, Danishka

1 0

[Maria-discuss] My Solution to MariaDB Database Server Crashed in Experimental Online Store
by Turritopsis Dohrnii Teo En Ming 23 Oct '18

23 Oct '18

Good evening from Singapore, Here is my solution: # systemctl start mariadb.service # mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 5.5.60-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> quit Bye # systemctl enable mariadb.service ===BEGIN SIGNATURE=== Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 30 Oct 2017 [1] https://tdtemcerts.wordpress.com/ [2] http://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming ===END SIGNATURE===

4 8

[Maria-discuss] aws key management plugin
by Paul Chubb 19 Oct '18

19 Oct '18

Hi, on the repo there only seems to be copies of this for Ubuntu yacketty (16.10) and Ubuntu zesty (17.04). Both of these are end of life. Is there supposed to be support for Bionic (18.04)? Secondly, they are tagged with the mariadb version number. Are they supposed to work across versions because not every version of Mariadb is represented? Thirdly is there any reason I can't pull the code from Gidhub and compile it myself? Many thanks in advance Paul

3 4

[Maria-discuss] help
by Paul Chubb 18 Oct '18

18 Oct '18

1 0

[Maria-discuss] gtid_slave_pos row count
by Reinis Rozitis 15 Oct '18

15 Oct '18

Hi, before considering this as a bug wanted to ask in mailing list: I know that gtid_slave_pos table can have multiple entries, but (at least on 10.3.9 but maybe also earlier (just noticed now)) while running: slave_parallel_threads = 20 slave_parallel_mode = optimistic the table starts to grow continuously: MariaDB [mysql]> select count(*) from gtid_slave_pos; +----------+ | count(*) | +----------+ | 5577268 | +----------+ 1 row in set (1.553 sec) If I switch back to conservative mode it never goes above 2-3 rows even with the same 20 threads. Same happens also with gtid_pos_auto_engines - when in conservative each engine table has several records, once I switch to optimistic the gtid tables just grow. Is there something wrong with the purger? (something similar like in https://jira.mariadb.org/browse/MDEV-12147 ? ) rr

2 10

[Maria-discuss] MariaDB 10.3 and largepages on Linux
by Kenneth Penza 12 Oct '18

12 Oct '18

Good morning, I am currently running RHEL 7.5 and MariaDB 10.3.9. After setting "large_pages=on" I noticed that MariaDB is reporting an error on startup and shutdown as per below output. To use large pages I modified /etc/sysctl.conf, /etc/security/limits.conf and the systemd unit file. By monitoring /proc/meminfo I verified that largepage are being used. Configuration: mysqld would have been started with the following arguments: --port=3306 --socket=/var/lib/mysql/mysql.sock --thread_stack=256K --large_pages=on --key_buffer_size=16K --max_allowed_packet=512M --table_open_cache=32 --sort_buffer_size=512K --net_buffer_length=8K --read_buffer_size=256K --read_rnd_buffer_size=256K --slow_query_log=1 --slow_query_log_file=mysql-slow.log --long_query_time=0.1 --log-queries-not-using-indexes=on --thread_handling=pool-of-threads --thread_pool_idle_timeout=60 --thread_pool_max_threads=1000 --thread_pool_size=8 --innodb_buffer_pool_size=256M --innodb_strict_mode=1 --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --innodb_log_files_in_group=4 --innodb_log_file_size=50331648 --innodb_undo_logs=128 --innodb_undo_tablespaces=8 --slow_query_log=1 --slow_query_log_file=mysql-slow.log --long_query_time=30 --log-queries-not-using-indexes=off --log_bin=mariadblog --binlog_format=ROW --binlog_row_image=MINIMAL --max_binlog_size=10240000 --expire_logs_days=1 --server-id=1 --innodb_flush_log_at_trx_commit=1 --sync_binlog=1 --relay-log=mariadb-relay-bin --performance_schema=off --histogram_size=255 --histogram_type=SINGLE_PREC_HB --use_stat_tables=preferably --optimizer_use_condition_selectivity=4 Startup messages: Sep 25 10:40:53 srv1 mysqld: 2018-09-25 10:40:52 0 [Note] InnoDB: Opened 8 undo tablespaces Sep 25 10:40:53 srv1 mysqld: 2018-09-25 10:40:53 0 [Warning] InnoDB: Failed to set memory to DODUMP: Invalid argument ptr 0x2aaac5400000 size 2097152 Sep 25 10:40:53 srv1 mysqld: 2018-09-25 10:40:53 0 [Note] InnoDB: 128 out of 128 rollback segments in 8 undo tablespaces are active. Shutdown messages: Sep 25 10:41:18 srv1 mysqld: 2018-09-25 10:41:18 0 [Note] InnoDB: Buffer pool(s) dump completed at 180925 10:41:18 Sep 25 10:41:19 srv1 mysqld: 2018-09-25 10:41:19 0 [Warning] InnoDB: Failed to set memory to DODUMP: Invalid argument ptr 0x2aaac3400000 size 33554432 Sep 25 10:41:19 srv1 mysqld: 2018-09-25 10:41:19 0 [Note] InnoDB: Shutdown completed; log sequence number 1522725; transaction id 22 Do you have ideas on what is causing "InnoDB: Failed to set memory to DODUMP: Invalid argument ptr .. " warning? Kenneth

2 3