Hi, andrei.elkin! On Oct 18, andrei.elkin@pp.inet.fi wrote:
Sergei,
Hi, Andrei!
First, when I run it I got:
Warning: 112 bytes lost at 0x7f212f26a670, allocated by T@0 at sql/sql_binlog.cc:68, sql/sql_binlog.cc:181, sql/sql_parse.cc:5625, sql/sql_parse.cc:7465, sql/sql_parse.cc:1497, sql/sql_parse.cc:1124, sql/sql_connect.cc:1330, sql/sql_connect.cc:1243
so you have a memory leak with your thd->lex->comment.str
Turns out LEX cleaned up at the end of decoded event execution. Thanks for alerting!
Btw, do you compile with clang or gcc? Asking seeing my stack unresolved.
gcc. It should not matter, I think. You need to build with -g and have addr2line in the path.
Also, I'm thinking whether BINLOG (in this last form) could automatically clear its variables @a and @b. Without an explicit SET @a=NULL. It's a bit weird to have such a side effect. But it'll help to save memory - the event size it over 3GB, right? So you'll have it split in two variables. Then you create a concatenated copy (it's over 3GB more), then you decode it (already more than 9GB in memory), and then it's executed where copies of huge blob values are created again, may be more than once. Freeing @a and @b and the concatenated copy as soon as possible will save over 6GB in RAM usage.
Well, in the regular case of replaying all three phases of SET, BINLOG, UNSET time between the last two is not really big.
It's not a question of time, but of how much memory you need *at the same time*. Now you do 1. allocate >3GB for both variables 2. allocate >3GB for a combined concatenated event 3. allocate ~3GB for a base64-decoded event 4. allocate memory for copies of blobs (2*3GB?) --- now you have 15GB allocated 5. free blobs 6. free base64 decoded event 7. free concatenated event 8. free variables And I suggest to do it like 1. allocate >3GB for both variables 2. allocate >3GB for a combined concatenated event 3. free variables 4. allocate ~3GB for a base64-decoded event 5. free concatenated event 6. allocate memory for copies of blobs (2*3GB?) --- now you have 9GB allocated 7. free blobs 8. free base64 decoded event So the peak memory consumption is >6GB less. Regards, Sergei Chief Architect MariaDB and security@mariadb.org