Hello, A new question has been asked in "XtraDB and InnoDB" by kushkash. Please answer it at http://mariadb.com/kb/en/mariadb-service-fails-to-start-after-innodbxtradb-c... as the person asking the question may not be subscribed to the mailing list. -------------------------------- Hi, I've been running a script, inserting a large amount of data to several MySQL tables. Apparently, InnoDB crashed at some point (when ibdata1 reached ~1.6 Gb). I had to restart the computer and when I tried to restart MariaDB service, I got the following errors: Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: 150611 8:58:02 InnoDB: Operating system error number 5 in a file operation. Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: InnoDB: Error number 5 means 'Input/output error'. Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: InnoDB: Some operating system error numbers are described at Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: InnoDB: File operation call: 'read'. Jun 11 08:58:02 honolulu.bi.technion.ac.il mysqld_safe[7970]: InnoDB: Cannot continue operation.' Jun 11 08:58:02 honolulu.bi.technion.ac.il systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE Jun 11 08:58:02 honolulu.bi.technion.ac.il systemd[1]: mariadb.service: control process exited, code=exited status=1 Jun 11 08:58:02 honolulu.bi.technion.ac.il systemd[1]: Failed to start MariaDB database server. Jun 11 08:58:02 honolulu.bi.technion.ac.il systemd[1]: Unit mariadb.service entered failed state. I checked /var/log/mariadb/mariadb.log to find out what might have caused the crash: 140911 10:40:38 InnoDB: Error: Write to file ./ibdata1 failed at offset 0 220200960. InnoDB: 1048576 bytes should have been written, only 0 were written. InnoDB: Operating system error number 28. InnoDB: Check that your OS and file system support files of this size. InnoDB: Check also that the disk is not full or a disk quota exceeded. InnoDB: Error number 28 means 'No space left on device'. InnoDB: Some operating system error numbers are described at InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html 140911 10:40:38 InnoDB: Assertion failure in thread 139942049851136 in file os0file.c line 4382 InnoDB: Failing assertion: ret InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately a150505 07:59:38 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 150505 07:59:38 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.SFCcCu' --pid-file='/var/lib/mysql/honolulu.bi.technion.ac.il-recover.pid' 150505 07:59:41 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1 150505 7:59:41 [Note] WSREP: wsrep_start_position var submitted: '00000000-0000-0000-0000-000000000000:-1' 150505 7:59:41 InnoDB: The InnoDB memory heap is disabled 150505 7:59:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins 150505 7:59:41 InnoDB: Compressed tables use zlib 1.2.8 150505 7:59:41 InnoDB: Using Linux native AIO 150505 7:59:41 InnoDB: Initializing buffer pool, size = 128.0M 150505 7:59:41 InnoDB: Completed initialization of buffer pool 150505 7:59:41 InnoDB: highest supported file format is Barracuda. 150505 7:59:41 InnoDB: Waiting for the background threads to start 150505 7:59:42 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-35.0 started; log sequence number 458740332 150505 7:59:42 [Note] Plugin 'FEEDBACK' is disabled. 150505 7:59:42 [Note] Server socket created on IP: '0.0.0.0'. 150505 7:59:42 [ERROR] mysqld: Table './mysql/user' is marked as crashed and should be repaired 150505 7:59:42 [Warning] Checking table: './mysql/user' 150505 7:59:42 [ERROR] mysql.user: 1 client is using or hasn't closed the table properly 150505 7:59:42 [ERROR] mysqld: Table './mysql/db' is marked as crashed and should be repaired 150505 7:59:42 [Warning] Checking table: './mysql/db' 150505 7:59:42 [ERROR] mysql.db: 1 client is using or hasn't closed the table properly 150505 7:59:42 [Note] Event Scheduler: Loaded 0 events 150505 7:59:42 [Note] WSREP: Read nil XID from storage engines, skipping position init 150505 7:59:42 [Note] WSREP: wsrep_load(): loading provider library 'none' 150505 7:59:42 [Note] [Debug] WSREP: dummy_init 150505 7:59:42 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.5.38-MariaDB-wsrep' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server, wsrep_25.10.r3997 150608 04:43:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 150608 04:43:00 mysqld_safe WSREP: Running position recovery with --log_error='/var/lib/mysql/wsrep_recovery.7Yb3bw' --pid-file='/var/lib/mysql/honolulu.bi.technion.ac.il-recover.pid' 150608 04:43:13 mysqld_safe WSREP: Failed to recover position: '150608 4:43:01 InnoDB: The InnoDB memory heap is disabled 150608 4:43:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins 150608 4:43:01 InnoDB: Compressed tables use zlib 1.2.8 150608 4:43:01 InnoDB: Using Linux native AIO 150608 4:43:01 InnoDB: Initializing buffer pool, size = 128.0M 150608 4:43:01 InnoDB: Completed initialization of buffer pool 150608 4:43:01 InnoDB: highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 4676403586 150608 4:43:01 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Error: tried to read 1048576 bytes at offset 0 1048576. InnoDB: Was only able to read 655360. 150608 4:43:13 InnoDB: Operating system error number 5 in a file operation. InnoDB: Error number 5 means 'Input/output error'. InnoDB: Some operating system error numbers are described at InnoDB: http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html InnoDB: File operation call: 'read'. InnoDB: Cannot continue operation.' I have to say that there shouldn't be any disk space problem: df -h Filesystem Size Used Avail Use% Mounted on /dev/sda5 819G 27G 750G 4% / devtmpfs 7.9G 0 7.9G 0% /dev tmpfs 7.9G 0 7.9G 0% /dev/shm tmpfs 7.9G 8.9M 7.9G 1% /run tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup tmpfs 7.9G 12K 7.9G 1% /tmp /dev/sda3 1.9T 123M 1.8T 1% /home /dev/sda2 477M 112M 336M 25% /boot The MariaDB is currently installed in default configuration. I really need help on this issue... 1. What might have cause the problem? Do I have to configure some InnoDB parameters? Maybe the ibdata file size is a problem? 2. How to recover my data and restart the MariaDB service? (I tried all kind of tips I found in other forums, but I couldn't backup the ibdata1 file or start InnoDB in recovery mode). -------------------------------- To view or answer this question please visit: http://mariadb.com/kb/en/mariadb-service-fails-to-start-after-innodbxtradb-c...