- discuss - lists.mariadb.org

Re: [Maria-discuss] fsync necessary for synchronous page flush?
by Xiaofei Du 07 May '15

07 May '15

Hi Jan, The only place fsync is called after AIO that I could find is in function buf_LRU_remove_pages (buf0lru.cc) But after other AIO operations, I didn't see fsync is called. Do you know how this difference will affect the recovery process? Thanks. Xiaofei On Wed, May 6, 2015 at 8:24 PM, Jan Lindström <jan.lindstrom(a)mariadb.com> wrote: > Hi, > > There is sync for AIO pages also. There is no dirty page table but > naturally dirty pages are marked and they are on LRU list see > buf/buf0lru.cc. > > R: Jan > > On Thu, May 7, 2015 at 12:23 AM, Xiaofei Du <xiaofei.du008(a)gmail.com> > wrote: > >> Hi Jan, >> >> Thanks for clarification. I should have used synchronous write, instead >> of synchronous flush. My point is that I noticed for sync writes, fsync is >> called to force pages to be on persistent storage. while for AIO pages, >> fsync is not called to force pages to be on persistent storage. My question >> here is why fsync is required for sync IOs. Does InnoDB maintain a dirty >> page table? Is fsync called to guarantee the page to be on persistent >> storage so that the dirty page table can be updated? If this is the case, >> when is the dirty page table updated for asynchronous IOs? >> >> P.S. I couldn't find code for dirty page table. So I am not sure if >> InnoDB maintains a dirty page table for recovery. Could you please give me >> a pointer to related code and related resources? Thanks. >> >> Xiaofei >> >> On Wed, May 6, 2015 at 10:12 AM, Jan Lindström <jan.lindstrom(a)mariadb.com >> > wrote: >> >>> Hi, >>> >>> Terminology is little bit confusing here. Page flushing means that we >>> have done synchronous write to disk but that does not mean that write is >>> physically on device yet, therefore there is flush to force it to >>> persistent storage. >>> >>> R: Jan >>> >>> On Wed, May 6, 2015 at 12:57 AM, Xiaofei Du <xiaofei.du008(a)gmail.com> >>> wrote: >>> >>>> Hello, >>>> >>>> When a dirty page is flushed synchronously in buf_flush_write_block_low >>>> , >>>> fsync is called in the following snippet. I am wondering why >>>> we need this fsync for synchronous flush? The record should be in >>>> the log already, so recovery should be able to successfully redo it >>>> and apply to the disk during recovery. Maybe I am missing something >>>> here, >>>> please let me know if I am wrong. Thanks much! >>>> >>>> Xiaofei >>>> >>>> >>>> /* When doing single page flushing the IO is done synchronously and we >>>> flush the changes to disk only for the tablespace we are working on. */ if >>>> (sync) { ut_ad(flush_type == BUF_FLUSH_SINGLE_PAGE); fil_flush(buf_page_get_space(bpage)); >>>> /* true means we want to evict this page from the LRU list as well. */ >>>> buf_page_io_complete(bpage, true); } >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~maria-discuss >>>> Post to : maria-discuss(a)lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~maria-discuss >>>> More help : https://help.launchpad.net/ListHelp >>>> >>>> >>> >> >

1 0

[Maria-discuss] fsync necessary for synchronous page flush?
by Xiaofei Du 05 May '15

05 May '15

Hello, When a dirty page is flushed synchronously in buf_flush_write_block_low, fsync is called in the following snippet. I am wondering why we need this fsync for synchronous flush? The record should be in the log already, so recovery should be able to successfully redo it and apply to the disk during recovery. Maybe I am missing something here, please let me know if I am wrong. Thanks much! Xiaofei /* When doing single page flushing the IO is done synchronously and we flush the changes to disk only for the tablespace we are working on. */ if (sync) { ut_ad(flush_type == BUF_FLUSH_SINGLE_PAGE); fil_flush(buf_page_get_space(bpage)); /* true means we want to evict this page from the LRU list as well. */ buf_page_io_complete(bpage, true); }

1 0

[Maria-discuss] MariaDB 10 multi-source replciation with SSL enabled
by Vadim Pestovnikov 05 May '15

05 May '15

Hi all, I am having a replication issue with MariaDB 10 using multi-source replication with SSL enabled. I have MariaDB 10 db in China, MariaDB 10 db in US and MariaDB 10 db in Europe. Each of them are getting data from other database sources such as Percona Server, AWS RDS using Tungsten replication because MariaDB 10 is not compatible with AWS RDS and Percona with GTID enabled. US and China MariaDB 10 servers are masters for Europe MariaDB. When I restore US and China MariaDB 10 data in Europe MariaDB 10 slave and start the replication, slave and masers are in sync. Then new data is coming to a masters from Tungsten, but slave doesn't get it. I don't see any errors neither from mysql-error.log or slave status. When I create fake tables and insert records everything replicates perfectly. What can be possible wrong with my setup? -- Vadim

1 0

[Maria-discuss] New Question: Connecting .Net with MariaDB
by AskMonty KB 05 May '15

05 May '15

Hello, A new question has been asked in "MariaDB Connector/ODBC Release Notes" by swarejg. Please answer it at http://mariadb.com/kb/en/connecting-net-with-mariadb/ as the person asking the question may not be subscribed to the mailing list. -------------------------------- Hi, I've reading about MariaDB and the way we can connect to it via .Net Can we use MySQL .Net Connector to connect MariaDB? If not, I found that you provide Connector/ODBC for use in .Net. Which are the performance results? Regards Jorge Guevara -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/connecting-net-with-mariadb/

1 0

[Maria-discuss] New Question: Are there plans to support Spatial Indexes in InnoDB for the 10.1 Release?
by AskMonty KB 04 May '15

04 May '15

Hello, A new question has been asked in "Geographic Features" by johndailyjr. Please answer it at http://mariadb.com/kb/en/are-there-plans-to-support-spatial-indexes-in-inno… as the person asking the question may not be subscribed to the mailing list. -------------------------------- Are there plans to support Spatial Indexes in InnoDB/XtraDB for the 10.1 Release? I can't seem to find anything relevant in the changelogs. Thanks! John -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/are-there-plans-to-support-spatial-indexes-in-inno…

1 0

[Maria-discuss] Password Complexity Plugin
by Adam Scott 01 May '15

01 May '15

Hi, I noticed MySQL 5.6 has a validate_password plugin. Since it's open-source, can we add it to MariaDB 5.5. Or is that bad form, a hot potato, etc.? We need the features of this plugin in our environment, and the PAM integration is looking not a good "fit" (though it's working great). (I'm trying to get it to compile in right now, but it's having issues with the includes so far...working on it). Thanks, Adam C. Scott

4 7

[Maria-discuss] New Question: Cannot increase open-files-limit
by AskMonty KB 01 May '15

01 May '15

Hello, A new question has been asked in "Managing MariaDB" by raquel.j. Please answer it at http://mariadb.com/kb/en/cannot-increase-open-files-limit/ as the person asking the question may not be subscribed to the mailing list. -------------------------------- I have a LEMP stack on Ubuntu 14.04 with 10.0.17-MariaDB-1~trusty. Mysql is a huge bottleneck for my server. I have tried everything of which I am aware to raise the open files limit for mysql. I have added "open-files-limit = 2097152" to my.cnf. I have added the following to limits.conf: * soft nofile 2097152 * hard nofile 2097152 * soft nproc 2097152 * hard nproc 2097152 mysql soft nofile 2097152 mysql hard nofile 2097152 mysql soft nproc 2097152 mysql hard nproc 2097152 I have added "session required pam_limits.so" to /etc/pam.d/common-session* and /etc/pam.d/sudo and a couple of others. I have tried restarting mysql after reboot - no change. I also checked /etc/apparmor.d/usr.sbin.mysqld to see if it was involved but the file is empty. After all that I still get the following: # ulimit -Sa core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 256538 max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 2097152 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited When I run # mysqld --verbose I get this: 150501 0:19:35 [Warning] Could not increase number of max_open_files to more than 1024 Does anyone have any idea as to what is blocking this change??? -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/cannot-increase-open-files-limit/

1 0

[Maria-discuss] MariaDB 5.5.43 now available
by Daniel Bartholomew 01 May '15

01 May '15

The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.43. This is a Stable (GA) release. See the Release Notes and Changelog for details. - - Links - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - MariaDB 5.5.43 - Release Notes: https://mariadb.com/kb/en/mariadb-5543-release-notes - Changelog: https://mariadb.com/kb/en/mariadb-5543-changelog - Downloads: https://downloads.mariadb.org/mariadb/5.5.43 About MariaDB 5.5: - https://mariadb.com/kb/en/what-is-mariadb-55/ APT and YUM Repository Configuration Generator: - https://downloads.mariadb.org/mariadb/repositories/ - - MariaDB Webinars - - - - - - - - - - - - - - - - - - - - - - - - - There are several upcoming MariaDB-focused webinars and many previously held ones available to watch on an on-demand basis at: - https://mariadb.com/news-events/webinars - - MariaDB Books - - - - - - - - - - - - - - - - - - - - - - - - - - There is an ever-growing library of MariaDB books available to help you get the most out of MariaDB. See the MariaDB Books page for details and links: - https://mariadb.com/kb/en/mariadb/books/ - - User Feedback plugin - - - - - - - - - - - - - - - - - - - - - - - MariaDB includes a User Feedback plugin. This plugin is disabled by default. If enabled, it submits basic, completely anonymous MariaDB usage information. This information is used by the developers to track trends in MariaDB usage to better guide development efforts. If you would like to help make MariaDB better, please add "feedback=ON" to your my.cnf or my.ini file! See http://mariadb.com/kb/en/user-feedback-plugin for more information. - - Quality - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The project always strives for quality, but in reality, nothing is perfect. Please take time to report any issues you encounter at: - http://mariadb.org/jira - - Support MariaDB - - - - - - - - - - - - - - - - - - - - - - - - - - If you would like to contribute to the MariaDB Foundation, please see the "contributing" and "donations" pages. We also have merchandise available in a cafepress store. All proceeds go to support the MariaDB Foundation. - https://mariadb.com/kb/en/contributing - https://mariadb.org/en/donate/ We hope you enjoy MariaDB! -- MariaDB Website - http://mariadb.org Twitter - http://twitter.com/mariadb Google+ - http://google.com/+mariadb Facebook - http://fb.com/MariaDB.dbms Knowledge Base - http://mariadb.com/kb

1 0

[Maria-discuss] R: Re: issue with 2 node cluster
by absolutely_free＠libero.it 30 Apr '15

30 Apr '15

Hi Alexey, thank you for your reply. Yes I am using wsrep_sst_method="xtrabackup-v2" Should I simply delete ALL from /var/lib/mysql/* and restart mysql again? Thank you again. >----Messaggio originale---- >Da: alexey.yurchenko(a)galeracluster.com >Data: 30/04/2015 15.09 >A: <absolutely_free(a)libero.it> >Cc: <maria-discuss(a)lists.launchpad.net>, "Maria-discuss"<maria-discuss- bounces+alexey.yurchenko=codership.com(a)lists.launchpad.net> >Ogg: Re: [Maria-discuss] issue with 2 node cluster > >On 2015-04-30 15:13, absolutely_free(a)libero.it wrote: >> Hi, >> I have 2 mariadb nodes on Debian 7.8. >> After a crash on first node, I can't restart the second one. > >> how can I solve this issue? >> First node is ok, all databases are available. >> Thank you > >It depends. If you are using rsync or xtrabackup SST method, you can >just delete everything from the data directory. >

1 0

[Maria-discuss] issue with 2 node cluster
by absolutely_free＠libero.it 30 Apr '15

30 Apr '15

Hi, I have 2 mariadb nodes on Debian 7.8. After a crash on first node, I can't restart the second one. The message I see in /var/log/mysql/error.log is: 150430 12:55:48 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... 150430 12:55:48 InnoDB: Error: page 7 log sequence number 565808461991 InnoDB: is in the future! Current system log sequence number 565772838412. InnoDB: Your database may be corrupt or you may have copied the InnoDB InnoDB: tablespace but not the InnoDB log files. See InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: for more information. 150430 12:55:48 InnoDB: Error: page 2 log sequence number 565929024045 InnoDB: is in the future! Current system log sequence number 565772838412. InnoDB: Your database may be corrupt or you may have copied the InnoDB InnoDB: tablespace but not the InnoDB log files. See InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: for more information. .. .. InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 150430 12:55:48 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 5.5.42-MariaDB-1~wheezy-wsrep-log key_buffer_size=134217728 read_buffer_size=2097152 max_used_connections=0 max_threads=4098 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 25383890 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x0 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0x7f3cc9d0c53b] /usr/sbin/mysqld(handle_fatal_signal+0x422)[0x7f3cc993a4f2] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf0a0)[0x7f3cc900f0a0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f3cc786c165] /lib/x86_64-linux-gnu/libc.so.6(abort+0x180)[0x7f3cc786f3e0] /usr/sbin/mysqld(+0x73b5dc)[0x7f3cc9b795dc] /usr/sbin/mysqld(+0x709617)[0x7f3cc9b47617] /usr/sbin/mysqld(+0x70a41a)[0x7f3cc9b4841a] /usr/sbin/mysqld(+0x6f25d8)[0x7f3cc9b305d8] /usr/sbin/mysqld(+0x6c4733)[0x7f3cc9b02733] /usr/sbin/mysqld(+0x6b72d2)[0x7f3cc9af52d2] /usr/sbin/mysqld(+0x6b812c)[0x7f3cc9af612c] /usr/sbin/mysqld(+0x6bad41)[0x7f3cc9af8d41] /usr/sbin/mysqld(+0x6a3db5)[0x7f3cc9ae1db5] /usr/sbin/mysqld(+0x659d1f)[0x7f3cc9a97d1f] /usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x48)[0x7f3cc993ce78] /usr/sbin/mysqld(+0x3bb550)[0x7f3cc97f9550] /usr/sbin/mysqld(_Z11plugin_initPiPPci+0x652)[0x7f3cc97fa592] /usr/sbin/mysqld(+0x32b118)[0x7f3cc9769118] /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x2036)[0x7f3cc976b9b6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7f3cc7858ead] /usr/sbin/mysqld(+0x321a4d)[0x7f3cc975fa4d] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.' how can I solve this issue? First node is ok, all databases are available. Thank you

1 0