June 2018 - discuss - lists.mariadb.org

[Maria-discuss] why stdout? -> [Note] /usr/libexec/mysqld (mysqld 10.0.20-MariaDB) starting as process 115096
by Reindl Harald 05 Jul '20

05 Jul '20

Aug 6 12:24:24 testserver mysqld: 150806 12:24:24 [Note] /usr/libexec/mysqld (mysqld 10.0.20-MariaDB) starting as process 115096 ... why is this now stdout instead mysql logs? on systemd based distributions that spits now to the global syslog and finally on machines restarting 2 instances every hour for backup reasons it annoys until 10.0.19 or 10.0.20 that was not this way

5 13

[Maria-discuss] MariaDB-10.2.11 Long Time To Start
by Michael Caplan 15 Mar '19

15 Mar '19

Hi, I'm setting up a replication slave based off of MySQL 5.6. There are a large number of tables (400K). The MySQL 5.6 master takes around 10 minutes to start up, where the new MariaDB 10.2 slave takes close to 30 minutes. What is making the startup take soo long? In the below mysql log, you can see in the startup sequence between "Highest supported file format is Barracuda." and "InnoDB: 128 out of 128 rollback segments are active." the startup process takes the majority of its time: 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Uses event mutexes 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Compressed tables use zlib 1.2.8 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Using Linux native AIO 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Number of pools: 1 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Using SSE2 crc32 instructions 2018-01-07 9:38:15 139980432365760 [Note] InnoDB: Initializing buffer pool, total size = 54G, instances = 8, chunk size = 128M 2018-01-07 9:38:18 139980432365760 [Note] InnoDB: Completed initialization of buffer pool 2018-01-07 9:38:18 139918113195776 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2018-01-07 9:38:18 139980432365760 [Note] InnoDB: Highest supported file format is Barracuda. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: 128 out of 128 rollback segments are active. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Creating shared tablespace for temporary tables 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: Waiting for purge to start 2018-01-07 10:11:48 139980432365760 [Note] InnoDB: 5.7.20 started; log sequence number 1394303471760 2018-01-07 10:11:48 139919461500672 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool 2018-01-07 10:11:48 139980432365760 [Note] Server socket created on IP: '::'. 2018-01-07 10:11:49 139980432365760 [Note] Reading of all Master_info entries succeded 2018-01-07 10:11:49 139980432365760 [Note] Added new Master_info '' to hash table 2018-01-07 10:11:49 139980432365760 [Note] /usr/sbin/mysqld: ready for connections. Version: '10.2.11-MariaDB-10.2.11+maria~xenial-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 mariadb.org binary distribution Any ideas why? I've included my my.conf file below. Thanks, Mike # Generated by Percona Configuration Wizard (http://tools.percona.com/) version REL5-20120208 # Configuration name usaddxprdsqlpm3 generated formcaplan(a)labnet.net at 2017-12-22 13:49:58 [mysql] # CLIENT # port = 3306 socket = /var/lib/mysql/mysql.sock [mysqladmin] # CLIENT # port = 3306 socket = /var/lib/mysql/mysql.sock [mysqld] # GENERAL # user = mysql default-storage-engine = InnoDB socket = /var/lib/mysql/mysql.sock pid-file = /var/lib/mysql/mysql.pid character-set-server = utf8 collation_server = utf8_general_ci performance_schema = 0 # MyISAM # key-buffer-size = 32M myisam-recover = FORCE,BACKUP # SAFETY # max-allowed-packet = 20M max-connect-errors = 1000000 skip-name-resolve sysdate-is-now = 1 innodb = FORCE # DATA STORAGE # datadir = /var/lib/mysql/ # BINARY LOGGING # log-bin = /var/lib/mysql/mysql-bin binlog_format = MIXED server-id = 3 expire-logs-days = 7 sync-binlog = 1 # REPLICATION # read-only = 1 skip-slave-start = 1 log-slave-updates = 1 relay-log = /var/lib/mysql/relay-bin slave-net-timeout = 60 sync-master-info = 1 sync-relay-log = 1 sync-relay-log-info = 1 # CACHES AND LIMITS # tmp-table-size = 32M max-heap-table-size = 32M query-cache-type = 0 query-cache-size = 0 max-connections = 500 thread-cache-size = 50 open-files-limit = 65535 table-definition-cache = 4096 table-open-cache = 10240 # INNODB # innodb-flush-method = O_DIRECT innodb-log-files-in-group = 2 innodb-log-file-size = 512M innodb-flush-log-at-trx-commit = 1 innodb-file-per-table = 1 innodb-buffer-pool-size = 54G # LOGGING # log-error = /var/lib/mysql/mysql-error.log log-queries-not-using-indexes = 0 slow-query-log = 0 slow-query-log-file = /var/lib/mysql/mysql-slow.log [mariadb] plugin-load-add=file_key_management file_key_management_filename=/var/lib/mysql/keys.enc file_key_management_filekey=FILE:/var/lib/mysql/mysql.key file_key_management_encryption_algorithm=AES_CTR encrypt_binlog=0

3 19

[Maria-discuss] Chewing Through Swap - Swappiness = 0
by Michael Caplan 09 Mar '19

09 Mar '19

Hi, I'm trying to figure out why my recently put into production MariaDB is so swap hungry. I'm running 10.2.14, with roughly 300GB data (1000K +/- tables). 95% tables are innodb. I have 64GB RAM, with INNODB buffer pool size set to 50GB (full my.cnf below). The OS is Ubuntu 16.04.4. This is a dedicated MariaDB server. I have swappiness set to 0. I started with available swap set to just shy of 1GB. When swap got to 85% used, I bumped swap to 3GB. A day later, swap again was 85% used, and I bumped it to 5GB. A day later, swap again was 85% used, so I bumped it to 15GB. 2 days later 50% of the available swap was used. I restarted the DB moments ago, freeing up all 15GB but 128MB of swap: total used free shared buff/cache available Mem: 62G 20G 4.7G 266M 37G 41G Swap: 14G 128M 14G My understanding is that the OS should never allocate swap when swappiness is 0, so that has me confused. I've had similar issues with MySQL 5.6 (what I upgraded from) dipping into swap, but that instance had swappiness set to 1, and never consumed more than 5GB of swap. Questions: 1) How much of this is an OS issue vs MariaDB issue? 2) What MariaDB config vars should I be looking at to fix this issue? 3) What linux config vars should I be looking at to fix this issue? Thanks, Mike [mysqld] # GENERAL # user = mysql default-storage-engine = InnoDB character-set-server = utf8 collation_server = utf8_general_ci performance_schema = 0 max_allowed_packet = 16777216 sql_mode = "NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" # MyISAM # key-buffer-size = 32M myisam-recover = FORCE,BACKUP # SAFETY # max-allowed-packet = 16777216 max-connect-errors = 1000000 skip-name-resolve sysdate-is-now = 1 innodb = FORCE local_infile = 0 secure_auth = 1 safe_user_create = 1 skip_symbolic_links = 1 wait_timeout = 28800 # DATA STORAGE # datadir = /var/lib/mysql/ # BINARY LOGGING # log-bin = /var/lib/mysql/mysql-bin binlog_format = MIXED server-id = 3 expire-logs-days = 7 sync-binlog = 1 # REPLICATION # read-only = 1 skip-slave-start = 1 log-slave-updates = 1 relay-log = /var/lib/mysql/relay-bin slave-net-timeout = 60 sync-master-info = 1 sync-relay-log = 1 sync-relay-log-info = 1 # CACHES AND LIMITS # tmp-table-size = 32M max-heap-table-size = 32M query-cache-type = 0 query-cache-size = 0 max-connections = 500 thread-cache-size = 50 open-files-limit = 65535 table-definition-cache = 4096 table-open-cache = 10240 # INNODB # innodb-flush-method = O_DIRECT innodb-log-files-in-group = 2 innodb-log-file-size = 512M innodb-flush-log-at-trx-commit = 2 innodb-file-per-table = 1 innodb-buffer-pool-size = 50G

3 8

Re: [Maria-discuss] [MariaDB Announce] MariaDB 10.2.16 now available
by Reindl Harald 27 Jun '18

27 Jun '18

is there any QA on your side? mysql-test-run: *** ERROR: Could not find any of /usr/sql/mysql_tzinfo_to_sql /usr/bin/mysql_tzinfo_to_sql /usr/sql/mysql_tzinfo_to_sql where the hell is "mysql_tzinfo_to_sql" gone? that it's here in "mariadb-manpages" is a packaging detail but in 10.2.16 it seems to be missing completly [root@testserver:~]$ rpm -q --file /usr/bin/mysql_tzinfo_to_sql mariadb-manpages-10.2.15-1.0.fc27.20180613.rh.sandybridge.x86_64 Am 26.06.2018 um 17:45 schrieb MariaDB Announce List: > The MariaDB project is pleased to announce the availability > of MariaDB 10.2.16, the latest stable release in the > MariaDB 10.2 series. > > See the Release Notes and Changelogs for details. > > > - - Links - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > MariaDB 10.2.16 > - Release Notes: https://mariadb.com/kb/en/mdb-10216-rn/ > - Changelog: https://mariadb.com/kb/en/mdb-10216-cl/ > - Downloads: https://downloads.mariadb.org/mariadb/10.2.16 > > About MariaDB 10.2: > - https://mariadb.com/kb/en/what-is-mariadb-102/ > > APT and YUM Repository Configuration Generator: > - https://downloads.mariadb.org/mariadb/repositories/ > > > - - MariaDB Books - - - - - - - - - - - - - - - - - - - - - - - - - - > > There is an ever-growing library of MariaDB books available to help > you get the most out of MariaDB. See the MariaDB Books page for > details and links: > > - https://mariadb.com/kb/en/mariadb/books/ > > > - - User Feedback plugin - - - - - - - - - - - - - - - - - - - - - - - > > MariaDB includes a User Feedback plugin. This plugin is disabled by > default. If enabled, it submits basic, completely anonymous MariaDB > usage information. This information is used by the developers to > track trends in MariaDB usage to better guide development efforts. > > If you would like to help make MariaDB better, please add > "feedback=ON" to your my.cnf or my.ini file! > > See http://mariadb.com/kb/en/user-feedback-plugin for more > information. > > > - - Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > The project always strives for quality, but in reality, nothing is > perfect. Please take time to report any issues you encounter at: > > - http://jira.mariadb.org > > > - - Support MariaDB - - - - - - - - - - - - - - - - - - - - - - - - - > > If you would like to contribute to the MariaDB Foundation, please see > the "contributing" and "donations" pages. We also have merchandise > available in a cafepress store. All proceeds go to support the > MariaDB Foundation. > > - https://mariadb.com/kb/en/contributing > > - https://mariadb.org/donate/ > > > We hope you enjoy MariaDB!

2 3

[Maria-discuss] MariaDB crashes because of "long semaphore wait" after migrating from 10.1 to 10.3
by TD 22 Jun '18

22 Jun '18

Hi, I have a Windows server that has been running MariaDB 10.1 successfully for over a year. The server remains mostly idle for long times with some read access, but occasionally there are transactions that add data (about 500k rows per commit). There can be up to 10 such transactions (one per database) at the same time and during those times the server is under quite some load (the code processing the data resides on the same server as the database). We're using an 8 core AMD Ryzen, 32 GB RAM and a Samsung SSD. Like I said, this worked fine with MariaDB 10.1 all the time. We're currently in the process of setting up a new server (specs similar to the ones above) and because MariaDB 10.3 has been released recently, I was curious and installed it. The database now constantly shuts down because of "[FATAL] InnoDB: Semaphore wait has lasted > 600 seconds. We intentionally crash the server because it appears to be hung." whenever there is one of the load situations described above. If I install MariaDB 10.1 on the new server, it runs as fine as the old one. The crashes also happen when I start with a clean database (no data migrated from 10.1 databases). I'm not a huge MariaDB config pro, as it's only a tool to store the data, it just needs to run with reasonable performance and has always worked until now. As far as I know and can judge from comments in the file, the my.ini in use is (mostly?) based on my-innodb-heavy-4G.cnf, but coworkers might have made modifications. I've attached the contents below. I don't know how I can debug this issue further. Windows event viewer contains several log warnings with the text "InnoDB: A long semaphore wait:" but no further information why this wait happens. I couldn't find any other error logs. I guess it is some configuration error (I read about MariaDB switching from XtraDB to InnoDB, so I guess that the issue might be related). Migrating to 10.3 is not an absolute must, but it would be nice to be able to fix the problem. Any help would be greatly appreciated and if you need more information, I'll gladly try to provide it. Thanks, Tom. my.ini: [mysqld] max_connections = 100 max_allowed_packet = 16M binlog_cache_size = 1M max_heap_table_size = 64M read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M thread_cache_size = 10 query_cache_size = 64M query_cache_limit = 2M thread_stack = 240K tmp_table_size = 64M slow_query_log expire_logs_days = 14 log-bin = mysql-bin character-set-server=utf8 collation-server=utf8_general_ci #*** MyISAM Specific options key_buffer_size = 32M bulk_insert_buffer_size = 64M myisam_sort_buffer_size = 128M myisam_max_sort_file_size = 10G myisam_repair_threads = 1 myisam_recover # *** INNODB Specific options *** #innodb_fatal_semaphore_wait_threshold = 6000 #tried that, didn't help #innodb_adaptive_hash_index = 0 #tried that (suggested in some stack overflow post, didn't help) innodb_buffer_pool_size = 2G innodb_data_file_path = ibdata1:10M:autoextend innodb_write_io_threads = 8 innodb_read_io_threads = 8 innodb_thread_concurrency = 16 innodb_log_buffer_size = 8M innodb_log_file_size = 256M innodb_log_files_in_group = 3 innodb_max_dirty_pages_pct = 10 innodb_io_capacity = 2000 innodb_lock_wait_timeout = 120 [mysql] no-auto-rehash [myisamchk] key_buffer_size = 512M sort_buffer_size = 512M read_buffer = 8M write_buffer = 8M [mysqld_safe] open-files-limit = 8192

4 4

[Maria-discuss] MariaDB Developers Unconference next week
by Otto Kekäläinen 22 Jun '18

22 Jun '18

Hello everybody! If you plan to attend the MariaDB Developers Unconference and related activities next week,* make sure you have signed up <https://www.meetup.com/MariaDB-Developers/events/249024232/> to the meetup* . We will soon be posting important information to attendees and only people who have signed up will receive it. Info: - www.meetup.com/MariaDB-Developers/events/249024232/ <https://www.meetup.com/MariaDB-Developers/events/249024232/> - mariadb.org/2018-2-developers-unconference-in-finland/ - mariadb.org/travel-tips-to-tampere-finland/

1 0

[Maria-discuss] MariaDB files wiped out after reboot
by Allan Mendoza 20 Jun '18

20 Jun '18

Hi, Good Day! I have a weird experience on one of our mysql servers. After a reboot all the database files have been wiped out. May I know if someone had expereienced the same, My OS is Centos 6.9 and MariaDB version is 5.5.35 Attaching is the error log that we have salvaged after the reboot. Thank you lanz_s99zx :)

1 0

Re: [Maria-discuss] Chewing Through Swap - Swappiness = 0
by Michael Caplan 15 Jun '18

15 Jun '18

Just found this interesting article on the "The MySQL 'swap insanity' problem": https://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-numa-architect… Has anyone else tried the mentioned solutions? - Forcing interleaved allocation with numactl --interleave=all. - Flushing Linux’s buffer caches just before mysqld startup with sysctl -q -w vm.drop_caches=3. - Forcing the OS to allocate InnoDB’s buffer pool immediately upon startup, using MAP_POPULATE where supported (Linux 2.6.23+), and falling back to memset otherwise. One of the solutions seems dependent on the Twitter fork that was absorbed by the now removed XtraDB. Thanks, Mike On 2018-06-15 08:24 AM, Michael Caplan wrote: > Hi, > > I'm trying to figure out why my recently put into production MariaDB > is so swap hungry. > > I'm running 10.2.14, with roughly 300GB data (1000K +/- tables). 95% > tables are innodb. I have 64GB RAM, with INNODB buffer pool size set > to 50GB (full my.cnf below). The OS is Ubuntu 16.04.4. This is a > dedicated MariaDB server. > > I have swappiness set to 0. > > I started with available swap set to just shy of 1GB. When swap got > to 85% used, I bumped swap to 3GB. A day later, swap again was 85% > used, and I bumped it to 5GB. A day later, swap again was 85% used, > so I bumped it to 15GB. 2 days later 50% of the available swap was > used. I restarted the DB moments ago, freeing up all 15GB but 128MB > of swap: > > total used free shared buff/cache > available > Mem: 62G 20G 4.7G 266M 37G 41G > Swap: 14G 128M 14G > > My understanding is that the OS should never allocate swap when > swappiness is 0, so that has me confused. > > > I've had similar issues with MySQL 5.6 (what I upgraded from) dipping > into swap, but that instance had swappiness set to 1, and never > consumed more than 5GB of swap. > > > Questions: > > 1) How much of this is an OS issue vs MariaDB issue? > > 2) What MariaDB config vars should I be looking at to fix this issue? > > 3) What linux config vars should I be looking at to fix this issue? > > > Thanks, > > Mike > > > > [mysqld] > > # GENERAL # > user = mysql > default-storage-engine = InnoDB > character-set-server = utf8 > collation_server = utf8_general_ci > performance_schema = 0 > max_allowed_packet = 16777216 > sql_mode = > "NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" > > # MyISAM # > key-buffer-size = 32M > myisam-recover = FORCE,BACKUP > > # SAFETY # > max-allowed-packet = 16777216 > max-connect-errors = 1000000 > skip-name-resolve > sysdate-is-now = 1 > innodb = FORCE > local_infile = 0 > secure_auth = 1 > safe_user_create = 1 > skip_symbolic_links = 1 > wait_timeout = 28800 > > # DATA STORAGE # > datadir = /var/lib/mysql/ > > # BINARY LOGGING # > log-bin = /var/lib/mysql/mysql-bin > binlog_format = MIXED > server-id = 3 > expire-logs-days = 7 > sync-binlog = 1 > > # REPLICATION # > read-only = 1 > skip-slave-start = 1 > log-slave-updates = 1 > relay-log = /var/lib/mysql/relay-bin > slave-net-timeout = 60 > sync-master-info = 1 > sync-relay-log = 1 > sync-relay-log-info = 1 > > # CACHES AND LIMITS # > tmp-table-size = 32M > max-heap-table-size = 32M > query-cache-type = 0 > query-cache-size = 0 > max-connections = 500 > thread-cache-size = 50 > open-files-limit = 65535 > table-definition-cache = 4096 > table-open-cache = 10240 > > # INNODB # > innodb-flush-method = O_DIRECT > innodb-log-files-in-group = 2 > innodb-log-file-size = 512M > innodb-flush-log-at-trx-commit = 2 > innodb-file-per-table = 1 > innodb-buffer-pool-size = 50G > >

1 0

[Maria-discuss] Timezones Table
by Kurniadi Asrigo 12 Jun '18

12 Jun '18

Hi, I am following the instruction to update the timezone table from MySQL: https://mariadb.com/kb/en/library/time-zones/ And the file is downloaded from MySQL page. I got MariaDB from XAMPP, running on Window10. I downloaded the "timezone_2018e_posix_sql.zip<https://downloads.mysql.com/general/timezone_2018e_posix_sql.zip> - POSIX standard" https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html Once downloaded, I extracted the package into a directory. From the instruction, I executed the command: .\mysql_tzinfo_to_sql.exe C:\dev\xampp_download\timezone\timezone_posix.sql | .\mysql.exe -D mysql -u root -p Now, the timezone Table is all empty. I am wondering if I am missing a step or an option. Thanks, Ti.

2 1

[Maria-discuss] MariaDB Developers Unconference in a few weeks!
by Otto Kekäläinen 09 Jun '18

09 Jun '18

Hello everybody! The next developers unconference will be in a few weeks: https://mariadb.org/2018-2-developers-unconference-in-finland/ There are still open slots so we encourage you to sign up now. https://www.meetup.com/MariaDB-Developers/events/249024232/ We have plenty of break-out rooms this time and there is space, so we also encourage you to suggest sessions/discussions you would like to have to ian(a)mariadb.org. Around mid-summer is a great time to visit Finland. The nights that don't get dark are worth experiencing! Getting to the venue in Tampere., Finlnd is also very easy. https://mariadb.org/travel-tips-to-tampere-finland/#why-visit Hope to see you there! - Otto

1 0