On Thu, Apr 4, 2019 at 2:43 PM Sergei Golubchik wrote:

Hi, Aleksey!

On Apr 04, Aleksey Midenkov wrote:

...

Hello, Sergei!

In unversioned -> versioned scenario in the code below it first gets to Set time 4, creates some records (on slave) and then seconds on slave increased (X+1) while on master seconds are not yet increased (X). Then we get to Set time 3 and reset time on slave to X.0 therefore we get back in time and all stored records with timestamp X.n will be in future. 'n' came from ++system_time.sec_part in Set time 4.

That's not how it is supposed to work. As long as the master sends events with seconds=X, the slave will generate microsecons, X.0, X.1, X.2, etc.

When the master sends an event with a new timestamp, Y, the slave goes back to Y.0, and continues with Y.1, etc.

...

Why did you decided to use such logic of getting seconds from master and microseconds from slave? Since microseconds sooner or later reset to 0 it's not better than just assigning some random number. What sending microseconds from master conditionally only is good for?

Because the master was sending microseconds conditionally, since 5.3. The slave had to cope with that somehow anyway.

If there is an installation from unversioned 5.3 to versioned 10.3 we can warn user about lost microseconds. This is minor issue since such setups are rare, I guess. But for what microseconds are sent in 5.3?

And I didn't want to force the master to include microseconds in every single event for every single user just in case someone would decide to do unversioned->versioned replication.

If it's critical, this can be configured. But is it really performance issue?

Also, I thought that processing of 1000000 Query_log_event's in a second is not realistic.

Now it fails just with several events. I guess because system_time.sec_part is not reset to 0 initially.

But now I see some issues with that. One can freeze the time on the master with 'SET TIMESTAMP' and send an arbitrary number of events with the same timestamp.

Or one can generate a query event that includes microseconds, to force the slave to count not from X.0, but from, say, X.999998.

So, a wraparound is possible and we need some fix for it.

Looks like a lots of complications for a minor issue. Other DBMS don't use microseconds for System Versioning at all. I guess user should cope either with sent microseconds unconditionally (for both SBR and RBR) or he should not use microseconds-precise System Versioning on slave.

Regards, Sergei Chief Architect MariaDB and security@mariadb.org

-- All the best, Aleksey Midenkov @midenok

On Thu, Apr 4, 2019 at 5:08 PM Sergei Golubchik wrote:

Hi, Aleksey!

On Apr 04, Aleksey Midenkov wrote:

...

On Thu, Apr 4, 2019 at 2:43 PM Sergei Golubchik wrote:

...

If there is an installation from unversioned 5.3 to versioned 10.3 we can warn user about lost microseconds. This is minor issue since such setups are rare, I guess. But for what microseconds are sent in 5.3?

When it's needed for replication. Say, in

INSERT t1 VALUES (NOW(6));

thd->query_start_sec_part() sets the flag.

...

...

And I didn't want to force the master to include microseconds in every single event for every single user just in case someone would decide to do unversioned->versioned replication.

If it's critical, this can be configured. But is it really performance issue?

A small performance and storage issue. At least 3 bytes per event.

But is it really an issue: do you know setups where replication communication is a bottleneck?

And it's a general principle - there will be definitely less than 1% of users, who will use this. Less than 0.1% too. Most probably, less than 0.01%. So, the remaining 99.99% should not pay the price for it.

Btw, it would be good to see the stats. We have some feedback plugin that does the job, don't we?

...

...

Also, I thought that processing of 1000000 Query_log_event's in a second is not realistic.

Now it fails just with several events. I guess because system_time.sec_part is not reset to 0 initially.

You said yourself that is is reset:

start_time_sec_part= system_time.sec_part= 0;

Initially it's reset in THD::THD() too:

system_time.start.val= system_time.sec= system_time.sec_part= 0;

It is synchronized with hardware clock on each set_start_time().

...

...

But now I see some issues with that. One can freeze the time on the master with 'SET TIMESTAMP' and send an arbitrary number of events with the same timestamp.

Or one can generate a query event that includes microseconds, to force the slave to count not from X.0, but from, say, X.999998.

So, a wraparound is possible and we need some fix for it.

Looks like a lots of complications for a minor issue. Other DBMS don't use microseconds for System Versioning at all. I guess user should cope either with sent microseconds unconditionally (for both SBR and RBR) or he should not use microseconds-precise System Versioning on slave.

one cannot possibly use microseconds-precise System Versioning on slave, if the server does not send microseconds. the slave merely tries to distingush between different statements, it doesn't try being precise.

But it can't recover correct statements order anyway. The statements came from many master threads to single slave thread in some arbitrary order. What is the point in ordering them at slave end?

Regards, Sergei Chief Architect MariaDB and security@mariadb.org

-- All the best, Aleksey Midenkov @midenok

Hi, Aleksey! On Apr 04, Aleksey Midenkov wrote:

On Thu, Apr 4, 2019 at 5:08 PM Sergei Golubchik wrote:

...

...

...

And I didn't want to force the master to include microseconds in every single event for every single user just in case someone would decide to do unversioned->versioned replication.

But is it really an issue: do you know setups where replication communication is a bottleneck?

It's few percent increase of the binlog size. Not much.

...

And it's a general principle - there will be definitely less than 1% of users, who will use this. Less than 0.1% too. Most probably, less than 0.01%. So, the remaining 99.99% should not pay the price for it.

Btw, it would be good to see the stats. We have some feedback plugin that does the job, don't we?

Yes.

...

...

...

Also, I thought that processing of 1000000 Query_log_event's in a second is not realistic.

Now it fails just with several events. I guess because system_time.sec_part is not reset to 0 initially.

You said yourself that is is reset:

start_time_sec_part= system_time.sec_part= 0;

Initially it's reset in THD::THD() too:

system_time.start.val= system_time.sec= system_time.sec_part= 0;

It is synchronized with hardware clock on each set_start_time().

It must be a bug. Hardware clock shouldn't overwrite the counter as it comes from the slave. Regards, Sergei Chief Architect MariaDB and security@mariadb.org