Re: [Maria-developers] a2520676ba1: MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default
Hi, Kentoku! On Jun 05, Kentoku wrote:
revision-id: a2520676ba1 (mariadb-10.4.5-10-ga2520676ba1) parent(s): 31fe70290c5 author: Kentoku <kentokushiba@gmail.com> committer: Kentoku <kentokushiba@gmail.com> timestamp: 2019-05-23 14:36:18 +0900 message:
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default
please remember to add an empty line after the first line in the commit comment, see https://git-scm.com/docs/git-commit#_discussion
add checksum_null for setting null value of checksum
diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc index 4e7fccca3af..e47da94ab20 100644 --- a/sql/ha_partition.cc +++ b/sql/ha_partition.cc @@ -8213,7 +8214,11 @@ int ha_partition::info(uint flag) stats.delete_length+= file->stats.delete_length; if (file->stats.check_time > stats.check_time) stats.check_time= file->stats.check_time; - stats.checksum+= file->stats.checksum; + if (!file->stats.checksum_null) + { + stats.checksum+= file->stats.checksum; + stats.checksum_null= FALSE; + }
I'm not sure about it. When a checksum can be NULL? I mean, normally, if you do A+B+C and one of the values is NULL, then the whole sum is NULL. But here you simply skip NULLs, like in aggregate SUM() function. To understand what semantics works better in this case, I need to understand when a checksum can be NULL in spider.
} if (stats.records && stats.records < 2 && !(m_file[0]->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT))
All other server changes look good, thanks. Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Hi Sergei,
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default
please remember to add an empty line after the first line in the commit comment, see https://git-scm.com/docs/git-commit#_discussion
O.K.
add checksum_null for setting null value of checksum
diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc index 4e7fccca3af..e47da94ab20 100644 --- a/sql/ha_partition.cc +++ b/sql/ha_partition.cc @@ -8213,7 +8214,11 @@ int ha_partition::info(uint flag) stats.delete_length+= file->stats.delete_length; if (file->stats.check_time > stats.check_time) stats.check_time= file->stats.check_time; - stats.checksum+= file->stats.checksum; + if (!file->stats.checksum_null) + { + stats.checksum+= file->stats.checksum; + stats.checksum_null= FALSE; + }
I'm not sure about it. When a checksum can be NULL?
I mean, normally, if you do A+B+C and one of the values is NULL, then the whole sum is NULL. But here you simply skip NULLs, like in aggregate SUM() function.
To understand what semantics works better in this case, I need to understand when a checksum can be NULL in spider.
Spider gets checksum from remote servers and Spider just uses this value. Some cases a checksum table command returns NULL from remote servers to Spider. Spider should use this result as is. This is the reason that a checksum can be NULL in Spider.
} if (stats.records && stats.records < 2 && !(m_file[0]->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT))
All other server changes look good, thanks.
O.K. Thanks. Best Regards, Kentoku 2019年6月6日(木) 4:22 Sergei Golubchik <serg@mariadb.org>:
Hi, Kentoku!
On Jun 05, Kentoku wrote:
revision-id: a2520676ba1 (mariadb-10.4.5-10-ga2520676ba1) parent(s): 31fe70290c5 author: Kentoku <kentokushiba@gmail.com> committer: Kentoku <kentokushiba@gmail.com> timestamp: 2019-05-23 14:36:18 +0900 message:
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default
please remember to add an empty line after the first line in the commit comment, see https://git-scm.com/docs/git-commit#_discussion
add checksum_null for setting null value of checksum
diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc index 4e7fccca3af..e47da94ab20 100644 --- a/sql/ha_partition.cc +++ b/sql/ha_partition.cc @@ -8213,7 +8214,11 @@ int ha_partition::info(uint flag) stats.delete_length+= file->stats.delete_length; if (file->stats.check_time > stats.check_time) stats.check_time= file->stats.check_time; - stats.checksum+= file->stats.checksum; + if (!file->stats.checksum_null) + { + stats.checksum+= file->stats.checksum; + stats.checksum_null= FALSE; + }
I'm not sure about it. When a checksum can be NULL?
I mean, normally, if you do A+B+C and one of the values is NULL, then the whole sum is NULL. But here you simply skip NULLs, like in aggregate SUM() function.
To understand what semantics works better in this case, I need to understand when a checksum can be NULL in spider.
} if (stats.records && stats.records < 2 && !(m_file[0]->ha_table_flags() & HA_STATS_RECORDS_IS_EXACT))
All other server changes look good, thanks.
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Hi, Kentoku! On Jun 06, kentoku wrote:
add checksum_null for setting null value of checksum
diff --git a/sql/ha_partition.cc b/sql/ha_partition.cc index 4e7fccca3af..e47da94ab20 100644 --- a/sql/ha_partition.cc +++ b/sql/ha_partition.cc @@ -8213,7 +8214,11 @@ int ha_partition::info(uint flag) stats.delete_length+= file->stats.delete_length; if (file->stats.check_time > stats.check_time) stats.check_time= file->stats.check_time; - stats.checksum+= file->stats.checksum; + if (!file->stats.checksum_null) + { + stats.checksum+= file->stats.checksum; + stats.checksum_null= FALSE; + }
I'm not sure about it. When a checksum can be NULL?
I mean, normally, if you do A+B+C and one of the values is NULL, then the whole sum is NULL. But here you simply skip NULLs, like in aggregate SUM() function.
To understand what semantics works better in this case, I need to understand when a checksum can be NULL in spider.
Spider gets checksum from remote servers and Spider just uses this value. Some cases a checksum table command returns NULL from remote servers to Spider. Spider should use this result as is. This is the reason that a checksum can be NULL in Spider.
So, looking at mysql_checksum_table(), CHECKSUM TABLE can be NULL in three cases * table doesn't exist * one uses QUICK and the table doesn't support live checksum * ha_rnd_init() failed, meaning, I presume, table is corrupted. What would this mean for spider? If one of partitions is missing or corrupted, the whole table is, basically, corrupted. Correct? So the checksum should be NULL with an appropriate error message. And I'd also say that in the second case - if some partitions support live checksum and others don't - the table as a whole doesn't support live checksum, so the result should be NULL as well. That is, I think you should use the normall addition semantics, not the aggregation semantics: stats.checksum_null= FALSE; .... if (file->stats.checksum_null) stats.checksum_null= TRUE; else stats.checksum+= file->stats.checksum; Regards, Sergei Chief Architect MariaDB and security@mariadb.org
participants (2)
-
kentoku -
Sergei Golubchik