Reduce logging output from mariabackup
Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic? Some of the log lines are very welcome, for example I like the first three Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64) and the last two are really great, like a gift from god Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK! But there's about 500 between these every hour in the journal and most of them are quite boring like this Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done Or stuff I do not understand and suspect I probably don't need to undersand like Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().
Op 3 aug. 2023 om 19:30 heeft Tom Worster via discuss <discuss@lists.mariadb.org> het volgende geschreven:
Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic?
Some of the log lines are very welcome, for example I like the first three
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64)
and the last two are really great, like a gift from god
Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK!
But there's about 500 between these every hour in the journal and most of them are quite boring like this
Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done
Or stuff I do not understand and suspect I probably don't need to undersand
Until something happens and you want to debug it after the fact …
like
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
I simply grep out the noise, a backup can be rerun if you need to debug, but for nightly and snapshots, if it doesn't fail...the only portions of the logs i see, and need to see, are what i've determined after 100xN times of running mariadbbackup. YMMV, but its really easy not to filter out errors. On Thu, Aug 3, 2023 at 1:43 PM William Edwards via discuss < discuss@lists.mariadb.org> wrote:
Op 3 aug. 2023 om 19:30 heeft Tom Worster via discuss < discuss@lists.mariadb.org> het volgende geschreven:
Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic?
Some of the log lines are very welcome, for example I like the first three
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64)
and the last two are really great, like a gift from god
Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK!
But there's about 500 between these every hour in the journal and most of them are quite boring like this
Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done
Or stuff I do not understand and suspect I probably don't need to undersand
Until something happens and you want to debug it after the fact …
like
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
of course, but it seems like you also answered your own comment. I didn't do this blindly as I mentioned. Also the filter string is simply grep -vE "Copying|done|log scanned" Many/Most, but of course not all should be findable outside of those filters. On Thu, Aug 3, 2023 at 4:58 PM William Edwards <wedwards@cyberfusion.nl> wrote:
Op 3 aug. 2023 om 22:54 heeft Jeff Dyke <jeff.dyke@gmail.com> het volgende geschreven:
I simply grep out the noise, a backup can be rerun if you need to debug
That is not true if the error is not reproducible (although one might wonder if debugging is needed then) or in case of a heisenbug.
but for nightly and snapshots, if it doesn't fail...the only portions of the logs i see, and need to see, are what i've determined after 100xN times of running mariadbbackup. YMMV, but its really easy not to filter out errors.
On Thu, Aug 3, 2023 at 1:43 PM William Edwards via discuss < discuss@lists.mariadb.org> wrote:
Op 3 aug. 2023 om 19:30 heeft Tom Worster via discuss < discuss@lists.mariadb.org> het volgende geschreven:
Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic?
Some of the log lines are very welcome, for example I like the first three
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64)
and the last two are really great, like a gift from god
Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK!
But there's about 500 between these every hour in the journal and most of them are quite boring like this
Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done
Or stuff I do not understand and suspect I probably don't need to undersand
Until something happens and you want to debug it after the fact …
like
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
Thanks for the grep, Jeff. On 8/3/2023 5:04:51 PM, "Jeff Dyke" <jeff.dyke@gmail.com> wrote:
of course, but it seems like you also answered your own comment. I didn't do this blindly as I mentioned. Also the filter string is simply grep -vE "Copying|done|log scanned"
Many/Most, but of course not all should be findable outside of those filters.
On Thu, Aug 3, 2023 at 4:58 PM William Edwards <wedwards@cyberfusion.nl> wrote:
Op 3 aug. 2023 om 22:54 heeft Jeff Dyke <jeff.dyke@gmail.com> het volgende geschreven:
I simply grep out the noise, a backup can be rerun if you need to debug
That is not true if the error is not reproducible (although one might wonder if debugging is needed then) or in case of a heisenbug.
but for nightly and snapshots, if it doesn't fail...the only portions of the logs i see, and need to see, are what i've determined after 100xN times of running mariadbbackup. YMMV, but its really easy not to filter out errors.
On Thu, Aug 3, 2023 at 1:43 PM William Edwards via discuss <discuss@lists.mariadb.org> wrote:
Op 3 aug. 2023 om 19:30 heeft Tom Worster via discuss <discuss@lists.mariadb.org> het volgende geschreven:
Is there an option to reduce the volume of information mariabackup writes to the console as it does its magic?
Some of the log lines are very welcome, for example I like the first three
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Connecting to MariaDB server host: localhost, user: mariabackup, password: set, port: not set, socket: /run/mysqld/mysqld.sock Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 Using server version 10.6.14-MariaDB-1:10.6.14+maria~deb11-log Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: mariabackup based on MariaDB server 10.6.14-MariaDB debian-linux-gnu (x86_64)
and the last two are really great, like a gift from god
Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 Last binlog file ./bhs8-bin.001617, position 952620696 Aug 03 11:36:34 bhs6 v2_backup_maria.sh[1291196]: [00] 2023-08-03 11:36:34 completed OK!
But there's about 500 between these every hour in the journal and most of them are quite boring like this
Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 Copying ./sys/sys_config.frm to /pool2/mariabackup/sys/sys_config.frm Aug 03 11:36:31 bhs6 v2_backup_maria.sh[1291066]: [01] 2023-08-03 11:36:31 ...done
Or stuff I do not understand and suspect I probably don't need to undersand
Until something happens and you want to debug it after the fact …
like
Aug 03 11:35:03 bhs6 v2_backup_maria.sh[1291066]: [00] 2023-08-03 11:35:03 uses posix_fadvise().
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
_______________________________________________ discuss mailing list -- discuss@lists.mariadb.org To unsubscribe send an email to discuss-leave@lists.mariadb.org
participants (3)
-
Jeff Dyke -
Tom Worster -
William Edwards