So i broke out mtaylor's patch that we were bumming around with at the UC
to replace ptr_compare with a simple memcmp call.
At the UC I benched that this patch actually caused a measurable performance
regresssion.
So what's the difference?
(same benchmark and machine as in previous mail)
MAX_FIELDS=64 with std::bitset
read/write requests: 70000 (3374.50 per sec.)
read/write requests: 70000 (3102.42 per sec.)
read/write requests: 70000 (3113.47 per sec.)
read/write requests: 70000 (3401.89 per sec.)
read/write requests: 70000 (3164.61 per sec.)
AVERAGE= 3231.37
With ptr_compare replaced with memcmp:
read/write requests: 70000 (3090.33 per sec.)
read/write requests: 70000 (3066.97 per sec.)
read/write requests: 70000 (2954.93 per sec.)
read/write requests: 70000 (2875.57 per sec.)
read/write requests: 70000 (2953.99 per sec.)
AVERAGE= 2988.35
With ptr_compare replaced with __builtin_memcmp:
read/write requests: 70000 (3245.37 per sec.)
read/write requests: 70000 (3001.91 per sec.)
read/write requests: 70000 (3254.99 per sec.)
read/write requests: 70000 (3177.13 per sec.)
read/write requests: 70000 (3195.49 per sec.)
AVERAGE= 3174.97
So, I thought that with just using the __builtin_memcmp automatically, we
may be okay with merging this.
However... I do have the following concerns:
1) I'm pretty sure that __builtin_memcmp is gcc only, and that SunStudio
may have an issue (yay - wrappers!)
2) Is this perf difference going to be true on different platforms?
3) the various memcmp implementations do seem to be dependent on a few
things for performance: alignment, size of data.
For what we're seeing in sysbench, the parameters are things like this:
"SELECT c from sbtest where id between 9942 and 10041 order by c"
0xc30780,0xc2c340,240
0xc2e560,0xc2c340,240
0xc30780,0xc2e560,240
0xc37390,0xc32f50,240
0xc35170,0xc32f50,240
0xc37390,0xc35170,240
0xc3dcc8,0xc39888,240
0xc3baa8,0xc2e560,240
0xc35170,0xc2e560,240
0xc3baa8,0xc35170,240
0xc2c340,0xc35170,240
0xc35170,0xc3dcc8,240
0xc2c618,0xc35170,240
0xc35170,0xc3d9f0,240
0xc2c8f0,0xc35170,240
0x7fbed85f10f0,0x7fbed85eccb0,240
and from the CREATE TABLE:
`c` VARCHAR(120) NOT NULL COLLATE utf8_general_ci DEFAULT `` ,
So here it's large, and aligned.
For some other micro-benchmarks I've clocked things looking much
different.... so it's possibly query dependent as to what ends up being
better (i.e. how much data we're comparing).
(1st parameter is number of repetitions, 2nd is number of bytes to memcmp)
For comparing equal values:
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 8
168435455 repetitions
Testing memcmp ..... done, 7.432 seconds
Testing builtin memcmp ....... done, 16.313 seconds
Testing loop .... done, 6.144 seconds
Testing loop32 .... done, 2.764 seconds
Testing loop64 .... done, 2.128 seconds
Testing no-op .... done, 1.488 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 16
168435455 repetitions
Testing memcmp ..... done, 5.912 seconds
Testing builtin memcmp ....... done, 26.418 seconds
Testing loop .... done, 11.201 seconds
Testing loop32 .... done, 3.980 seconds
Testing loop64 .... done, 2.564 seconds
Testing no-op .... done, 1.492 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 32
168435455 repetitions
Testing memcmp ..... done, 6.828 seconds
Testing builtin memcmp ....... done, 46.891 seconds
Testing loop .... done, 21.473 seconds
Testing loop32 .... done, 6.536 seconds
Testing loop64 .... done, 3.804 seconds
Testing no-op .... done, 1.476 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 64
168435455 repetitions
Testing memcmp ..... done, 9.549 seconds
Testing builtin memcmp ....... done, 87.513 seconds
Testing loop .... done, 41.455 seconds
Testing loop32 .... done, 11.669 seconds
Testing loop64 .... done, 6.368 seconds
Testing no-op .... done, 1.468 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 128
168435455 repetitions
Testing memcmp ..... done, 15.081 seconds
Testing builtin memcmp ....... done, 169.143 seconds
Testing loop .... done, 86.397 seconds
Testing loop32 .... done, 21.877 seconds
Testing loop64 .... done, 11.445 seconds
Testing no-op .... done, 1.488 seconds
stewart@willster:~/src/test/memcmp$ gcc -O3 -fno-builtin memcmpbench.c
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 256
168435455 repetitions
Testing memcmp ..... done, 26.134 seconds
Testing loop64 .... done, 21.549 seconds
Testing no-op .... done, 1.500 seconds
Completely inequal values are all about the same:
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 8
168435455 repetitions
Testing memcmp ..... done, 3.204 seconds
Testing builtin memcmp ....... done, 13.945 seconds
Testing loop .... done, 1.896 seconds
Testing loop32 .... done, 2.092 seconds
Testing loop64 .... done, 2.136 seconds
Testing no-op .... done, 1.488 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 240
168435455 repetitions
Testing memcmp ..... done, 5.520 seconds
Testing builtin memcmp ....... done, 13.973 seconds
Testing loop .... done, 1.912 seconds
Testing loop32 .... done, 2.124 seconds
Testing loop64 .... done, 2.104 seconds
Testing no-op .... done, 1.468 seconds
For half the same:
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 8
168435455 repetitions
Testing memcmp ..... done, 7.340 seconds
Testing builtin memcmp ....... done, 19.193 seconds
Testing loop .... done, 4.212 seconds
Testing loop32 .... done, 2.956 seconds
Testing loop64 .... done, 2.112 seconds
Testing no-op .... done, 1.476 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 32
168435455 repetitions
Testing memcmp ..... done, 5.920 seconds
Testing builtin memcmp ....... done, 34.526 seconds
Testing loop .... done, 11.885 seconds
Testing loop32 .... done, 4.444 seconds
Testing loop64 .... done, 3.388 seconds
Testing no-op .... done, 1.468 seconds
stewart@willster:~/src/test/memcmp$ ./a.out 168435455 64
168435455 repetitions
Testing memcmp ..... done, 6.948 seconds
Testing builtin memcmp ....... done, 54.923 seconds
Testing loop .... done, 22.081 seconds
Testing loop32 .... done, 7.000 seconds
Testing loop64 .... done, 4.444 seconds
Testing no-op .... done, 1.488 seconds
Is concurrency an issue here and not raw single threaded performance?
Let's try with 64 threads (on the same 2 core box):
64 threads, __builtin_memcmp:
read/write requests: 70000 (3184.20 per sec.)
read/write requests: 70000 (3184.87 per sec.)
read/write requests: 70000 (3166.83 per sec.)
read/write requests: 70000 (3085.69 per sec.)
AVERAGE=3155.39
64 threads, memcmp:
read/write requests: 70000 (3218.07 per sec.)
read/write requests: 70000 (3219.04 per sec.)
read/write requests: 70000 (3222.48 per sec.)
read/write requests: 70000 (3116.00 per sec.)
AVERAGE=3193.89
64 threads, 32bit cmp loop:
read/write requests: 70000 (3173.76 per sec.)
read/write requests: 70000 (3156.23 per sec.)
read/write requests: 70000 (3206.70 per sec.)
read/write requests: 70000 (3247.61 per sec.)
AVERAGE=3196.07
64 threads, 64bit cmp loop:
read/write requests: 70000 (3210.02 per sec.)
read/write requests: 70000 (3218.87 per sec.)
read/write requests: 70000 (3225.98 per sec.)
read/write requests: 70000 (3131.60 per sec.)
AVERAGE= 3196.61
64 threads, baseline (using original ptr_compare):
read/write requests: 70000 (3542.95 per sec.)
read/write requests: 70000 (3556.15 per sec.)
read/write requests: 70000 (3560.71 per sec.)
read/write requests: 70000 (3476.80 per sec.)
AVERAGE=3534.15
i.e. the old ptr_compare whoops the arse of any of the memcmp calls with
higher concurrency.
So, after this I can conclude:
- memcmp is nothing if not consistent across various microbenchmarks
- builtin_memcmp seems to help a bit at lower concurrency, not at all at
higher and is seldom useful in micro benchmark.
- the 64bit loop isn't so good at higher concurrency
- the 64bit loop is favourable in microbenchmarks
- memcmp is close to the 64bit loop performance (at most only 2x slower)
Unrolling loops being the key?
Testing loop .... done, 11.813 seconds
Testing unrollloop .... done, 7.628 seconds
Testing unrollloop32 .... done, 2.532 seconds
Testing unrollloop64 .... done, 2.536 seconds
Testing loop32 .... done, 4.432 seconds
Testing loop64 .... done, 3.792 seconds
Testing no-op .... done, 1.276 seconds
(32 and 64 do 32,64 bit compares in an unrolled loop)
Questions:
- Why does my 64bit compare loop not equal performance of ptr_compare
unrolled loop? (in a microbenchmark, it beats it). Concurrency again?
So what should we do?
Certainly a call to memcmp is easier to understand and keeps the code
nice and simple, but we're talking up to a 10% speed hit here at higher
concurrency (at least on my hardware).
I wonder what the difference is on various other hardware setups.
I'd be very interested to see what happens on sparc.
I'd also like to see it micro-benchmarked - preferably something we
could repeat in future (even in ./configure ? on server startup ?)
I'm voting to keep the current ptr_compare code, and hope that somebody
provides further explanation on top of what I've looked at here.
Below is the patch, followed by the code i used for microbenchmarking
(note the commented out 64bit loop i used for the above loop tests):
=== modified file 'drizzled/filesort.cc'
--- drizzled/filesort.cc 2009-04-28 00:17:10 +0000
+++ drizzled/filesort.cc 2009-05-08 05:59:11 +0000
@@ -1135,7 +1135,7 @@ int merge_buffers(SORTPARAM *param, IO_C
}
else
{
- cmp= get_ptr_compare(sort_length);
+ cmp= (qsort2_cmp)ptr_compare;
first_cmp_arg= (void*) &sort_length;
}
priority_queue