On Thu, Dec 8, 2022 at 10:59 PM Reindl Harald <h.reindl@thelounge.net> wrote:
the next reply-all clown.... boy i am subsribed so you can just respond to the list
You sure whine a lot about inconsequential things.
Am 08.12.22 um 21:13 schrieb Gordan Bobic:
On Thu, Dec 8, 2022 at 9:42 PM Reindl Harald <h.reindl@thelounge.net> wrote:
Am 08.12.22 um 18:59 schrieb Gordan Bobic:
On Thu, Dec 8, 2022 at 7:28 PM Reindl Harald <h.reindl@thelounge.net> wrote:
MariaDB does the same as the filesystem InnoDB in fact is more ore less a FS on top of a FS
So why do it at both levels?
because the FS layer can't detect MariaDB errors?
What is the net benefit of detecting said error? The way I see it, the options are: 1) MariaDB detects and error, crashes out 2) MariaDB doesn't detect an error, ingests garbage, crashes out
silent data corruption
At what layer? If at anything below, up to and including the file system level, ZFS will detect and correct it for you. MariaDB can at best only detect it for you. So catching it at a lower level wins.
The only way an error will creep in without the error checking FS spotting it is: 1) manually corrupting the block by writing garbage over it 2) MariaDB generated a duff block and wrote it out 3) Some other hardware failure corrupted the block in MariaDB memory, just before writing it to the file system
If any of the latter set happens, the data is toast anyway. If the former set happens, the data is toast anyway.
silent data corruption
At what layer? If at anything below, up to and including the file system level, ZFS will detect and correct it for you. MariaDB can at best only detect it for you. So catching it at a lower level wins.
Sure it's nice to get an error that confirms that your data is corrupted, but that won't bring it back silent data corruption
Shift it down to a layer where at least a subset of the problemspace can be fixed and we have a net gain in at least some cases.
different world
A far better world that you seem have a hard time accepting exists.
And what makes doing it at MariaDB level in any way better than doing it somewhere else?
which magic should do it somewehre else?
If a file system is in control of data mirroring and checksumming every 16KB block, then if the data is corrupted on disk, file system will detect it on a read and fetch a good copy from an uncorrupted mirror.
moron the files are fine from the viewpoint of the filessystem
At what layer? If at anything below, up to and including the file system level, ZFS will detect and correct it for you. MariaDB can at best only detect it for you. So catching it at a lower level wins. I am talking about total cumulative data integrity up to and including what MariaDB reads from disks. What layer in the stack, exactly, are you trying to fight "silent data corruption" you mentioned above?
Application never knows something went wrong, file system replaces the corrupted block on the other disk and everything carries on uninterrupted.
moron the files are fine from the viewpoint of the filessystem
AGAIN: THE FILESYSTEM CAN'T DO ANYTHING BECAUSE IT'S NOT AFFECTED
So your argument is that the page checksum is there to tell you that your data is corrupted because of either: 1) data was corrupted by the database itself, or 2) a superuser overwriting the block on the disk
and it's impressive how long it takes until you mororn understand such basics
And in that case what exactly will the ckecksum do for you? What do you gain from it? The error message that the block is corrupted just before the database crashes?
In either case, you are not getting the data back either way the the database will stop working.
yes, but it's a difference if you have silent data corruption detectet months later when clean backups are gone because retention times
So what is the point? I'd rather have the error fixable
moron your filesystem can't fix that type of error
that's why "Some of us run MariaDB on file systems that do their own block checksumming, and thus run innodb_checksum_algorithm=none" was an idiots response
a different response would have been "i don't think that i need innodb checksums and want to disable it" but you moron pretended your filesystem does the same
Let's look at the scenarios here, assuming a block gets corrupted: 1) InnoDB has checksums. 1.1) Database doesn't step on the duff block, error doesn't get detected. Until long after your backup retention expired. 1.2) Data corruption happens in memory before the checksum is calculated. Checksum is based on corrupted data and doesn't help you. 2) InnoDB has no checksums. Database steps on the duff block. 2.1) Database crashes during a query. You get the query and tables accessed during a stack trace. You run a check table and it finds a corrupted table. You restore it from backup. 2.2) Database doesn't crash because the damage merely corrupts a single value but the record structure remains sound. So it is that 2.2) point where the InnoDB checksum gives you anything. It is a vanishingly narrow case. Sufficiently narrow that if you never encountered a relatively common problem with upgrades due to redo log format changes (there have been several over the years), you are orders of magnitude less likely to have encountered this.