- developers - lists.mariadb.org

Re: [Maria-developers] [Commits] Rev 3398: Make testcase stable by adding --sorted_result for SHOW STATUS commands. in file:///home/psergey/dev2/5.3-look48/
by Sergei Golubchik 28 Jan '12

28 Jan '12

Hi, Sergey! What do you mean? SHOW STATUS returns a sorted list of variables. On Jan 27, Sergey Petrunya wrote: > At file:///home/psergey/dev2/5.3-look48/ > > revno: 3398 > revision-id: psergey(a)askmonty.org-20120127135140-4ues2tk6juf4iynx > parent: psergey(a)askmonty.org-20120126102034-tp33uzyuqnyk30mn > committer: Sergey Petrunya <psergey(a)askmonty.org> > branch nick: 5.3-look48 > timestamp: Fri 2012-01-27 17:51:40 +0400 > message: > Make testcase stable by adding --sorted_result for SHOW STATUS commands. Regards, Sergei

2 2

Re: [Maria-developers] [Commits] Rev 3241: Xtradb recently started to access thd members directly , e.g thd->stmt_da (ha_innodb.cc) in file:///H:/bzr/5.5/
by Kristian Nielsen 27 Jan '12

27 Jan '12

Vladislav Vaintroub <wlad(a)montyprogram.com> writes: > Xtradb recently started to access thd members directly ,e.g thd->stmt_da (ha_innodb.cc) > It needs recompilation for embedded server, as layout of THD is different in embedded. Did you consider instead fixing XtraDB to not access thd members directly? Eg. using proper accessors, maybe adding new accessors if needed? Or is this not possible for some reason? It would be much preferable if XtraDB didn't mess with server internals like this ... - Kristian.

3 2

[Maria-developers] mysql_client_test
by Kristian Nielsen 25 Jan '12

25 Jan '12

Just to finish our discussions on IRC... I found that mysql_client_test.test runs fine without any --plugin-dir=xxx option at all for mysql_client_test. The reason is that the correct option is in the my.cnf that we generate. So I decided to just remove the --plugin-dir option from the command line in mysql_client_test.test; this seems to work both when tested from source dir and when testing installed server. - Kristian.

1 0

[Maria-developers] Need help re. which files to include in 5.5 .debs
by Kristian Nielsen 25 Jan '12

25 Jan '12

Hi Serg, As you suggested, I compared .deb file contents with what `make install` produces in 5.5. I discovered that the following files are missing from .debs. I need help understanding which of these should be added to the .debs. (I think anything added should be to package libmariadbclient-dev): /usr/lib/libmysqlservices.a /usr/include/mysql.h /usr/include/mysql_com.h /usr/include/mysqld_ername.h /usr/include/mysqld_error.h /usr/include/mysql_embed.h /usr/include/mysql/psi /usr/include/mysql/psi/mysql_file.h /usr/include/mysql/psi/mysql_thread.h /usr/include/mysql/psi/psi_abi_v1.h /usr/include/mysql/psi/psi_abi_v2.h /usr/include/mysql/psi/psi.h /usr/include/client_plugin.h /usr/include/decimal.h /usr/include/errmsg.h /usr/include/keycache.h /usr/include/m_ctype.h /usr/include/m_string.h /usr/include/my_alloc.h /usr/include/my_attribute.h /usr/include/my_compiler.h /usr/include/my_dbug.h /usr/include/my_getopt.h /usr/include/my_global.h /usr/include/my_list.h /usr/include/my_net.h /usr/include/my_pthread.h /usr/include/mysql_time.h /usr/include/mysql_version.h /usr/include/my_sys.h /usr/include/my_valgrind.h /usr/include/my_xml.h /usr/include/plugin_audit.h /usr/include/plugin_auth.h /usr/include/plugin_ftparser.h /usr/include/plugin.h /usr/include/sql_common.h /usr/include/sql_state.h /usr/include/sslopt-case.h /usr/include/sslopt-longopts.h /usr/include/sslopt-vars.h /usr/include/typelib.h It seems pretty obvious that "mysql.h" is needed. But should I add _all_ the .h files? And what about libmysqlservices.a? - Kristian.

2 5

Re: [Maria-developers] something better than PMP
by Kristian Nielsen 25 Jan '12

25 Jan '12

Arun Sharma <asharma(a)fb.com> writes: > On 1/21/12 7:24 AM, Mark Callaghan wrote: >> 1 millisecond per thread would be much better than what we experience >> today. >> > > Is there a reason why latency per-thread is more interesting than the > absolute number? You are right of course - the interesting number is the time we spent with the target mysqld process suspended (stalled) under ptrace(). The reason I think in terms of latency-per-thread is this: Different mysqld instances can have widely different number of threads - from a few handful to perhaps thousands. Since we loop over threads while the process is ptrace()-suspended, it seems a nice measure to estimate the cost independent of number of threads in the process. >>> Arun: What I've done so far is mostly to cache reads of the same word, and >>> delay name resolve to after the target process is resumed. I think it can >>> be >>> made even faster by reading target process memory through /proc/PID/mem >>> instead of ptrace and if we can reduce repeated reads of /proc/PID/maps >>> and >>> such. > > Sounds like a good optimization to have. There is already a cache > internal to libunwind that maps a given IP address to it's unwind > info. If the repeated reads come from that code path, tweaking the > cache size might be a simpler fix. Ok, that sounds cool! - Kristian.

1 0

Re: [Maria-developers] [Commits] Rev 3129: remove few hard-coded checks from mtr in http://bazaar.launchpad.net/~maria-captains/maria/5.1/
by Kristian Nielsen 25 Jan '12

25 Jan '12

<serg(a)askmonty.org> writes: > At http://bazaar.launchpad.net/~maria-captains/maria/5.1/ > > ------------------------------------------------------------ > revno: 3129 > revision-id: sergii(a)pisem.net-20120124140839-iziwgt30gaalsiw9 > parent: sergii(a)pisem.net-20120124125357-b3o68crdtg2w3fzj > committer: Sergei Golubchik <sergii(a)pisem.net> > branch nick: 5.1 > timestamp: Tue 2012-01-24 15:08:39 +0100 > message: > remove few hard-coded checks from mtr Why?!? It's much faster to have the checks in mtr_cases.pm. Otherwise we have to start and stop the mysqld server(s), which is often the major part of time spent in a test case. The test suite is slow enough already ... Also, some of these tests _must_ be in mtr_cases, because just starting the server and executing the test for "skip" causes a failure. Eg. I remember that "not_valgrind" works this way, so I think your change will cause some tests to break in valgrind. If we want to improve things so that we have all checks in one place, I would prefer to move everything into mysql-test-run.pl. In the long term, not just for skipping tests but for everything. There is so much cruft in there just do deal with the testsuite being written partly in perl, partly in SQL (parse warnings...), partly in the mysqltest language, partly in C (mysqltest), ... > - if ( $tinfo->{'not_valgrind'} ) > - { > - if ( $::opt_valgrind_mysqld ) > - { > - $tinfo->{'skip'}= 1; > - $tinfo->{'comment'}= "Not compatible with Valgrind testing"; > - return $tinfo; > - } > - } - Kristian

2 2

Re: [Maria-developers] request for a feature -- LIMIT_ROWS_EXAMINED
by MARK CALLAGHAN 23 Jan '12

23 Jan '12

Yes, I accept On Mon, Jan 23, 2012 at 4:12 AM, Timour Katchaounov <timour(a)montyprogram.com> wrote: > Mark, > > >> The semantics are good for us. The potential overhead from the >> implementation isn't a concern for our uses. What was the feedback >> from the MPAB code review? > > > I haven't submitted the code for review because there are few more > small items to complete, however, the current solution was discussed > with Monty and Igor, and we all came to an agreement that this is the > best solution. > > Speaking of overhead, if the LIMIT_ROWS_EXAMINED is not present, then > the overhead is an additional IF statement in > handler::increment_statistics(): > > inline void handler::increment_statistics(ulong SSV::*offset) const > { > status_var_increment(table->in_use->status_var.*offset); > + if (table->in_use->examined_rows_limit_cnt < HA_POS_ERROR) > + table->in_use->check_examined_rows_limit(); > } > > If the LIMIT_ROWS_EXAMINED clause is present, the overhead is this inline > method: > > /** > Check if the number of rows accessed by a statement exceeded > LIMIT_ROWS_EXAMINED. If so, stop execution. > */ > void check_examined_rows_limit() > { > if (++status_var.accessed_rows_and_keys > examined_rows_limit_cnt) > { > push_warning_printf(this, MYSQL_ERROR::WARN_LEVEL_WARN, > ER_QUERY_EXCEEDED_EXAMINED_ROWS_LIMIT, > ER(ER_QUERY_EXCEEDED_EXAMINED_ROWS_LIMIT), > status_var.accessed_rows_and_keys); > killed= ABORT_QUERY; > } > } > > I believe this overhead for each handler call is negligible. If the overhead > is measurable, then we could improve by making handler::increment_statistics > a function pointer and choosing the right implementation before execution. > This can easily be done if needed. > > Ok, so I will assume your reply means that you accept this approach, and > will > complete the feature ASAP. > > > Timour > > >> >> On Fri, Jan 20, 2012 at 7:37 AM, MARK CALLAGHAN<mdcallag(a)gmail.com> >> wrote: >>> >>> OK, will have feedback in a few days. >>> >>> On Fri, Jan 20, 2012 at 6:45 AM, Timour Katchaounov >>> <timour(a)montyprogram.com> wrote: >>>> >>>> Hello Mark, >>>> >>>> >>>>> I want a variant of the LIMIT clause that limits the number of rows >>>>> examined during SELECT processing. This should return OK when the >>>>> limit is reached -- something like NESTED_LOOP_QUERY_LIMIT. While >>>>> LIMIT can be LIMIT x and LIMIT x,y. This only supports one argument -- >>>>> LIMIT_ROWS_EXAMINED x. Are you interested in implementing this feature >>>>> for me? We will sponsor the work. The purpose of the feature is to >>>>> prevent queries from taking too long when there isn't a great index >>>>> and many rows might be filtered. >>>>> >>>> >>>> After examining few alternatives, I implemented a pretty complete >>>> prototype of LIMIT_ROWS_EXAMINED. Please have a look at the description >>>> of the task here: >>>> https://mariadb.atlassian.net/browse/MDEV-28 >>>> >>>> It is important to notice that the current implementation will stop >>>> execution when LIMIT_ROWS_EXAMINED is reached irrespective of the >>>> query execution strategies used (thus not only for JOINs, but also >>>> writing/reading to temp tables, sorting, and single-table access methods >>>> that scan too many rows). This approach also is future-proof with >>>> respect to adding new query processing algorithms because the counting >>>> is done at the handler level. >>>> >>>> The current approach is such, that it should be fairly easy to backport >>>> the feature to almost any MariaDB/MySQL. >>>> >>>> Please let me know if this is what you want. If so, then I will proceed >>>> with the remaining polishing that is in the TODO list and will get an >>>> internal review. >>>> >>>> You can also have a look at the small example in the comments below the >>>> description to get a feeling what is the behavior. >>>> >>>> I can also send you a patch of the current version in case you want to >>>> play with the feature before it is pushed. >>>> >>>> Timour >>> >>> >>> >>> >>> -- >>> Mark Callaghan >>> mdcallag(a)gmail.com >> >> >> >> > -- Mark Callaghan mdcallag(a)gmail.com

1 0

[Maria-developers] MDEV-30: "mtr extensions for storage engine testing" review
by Kristian Nielsen 23 Jan '12

23 Jan '12

Hi Serg! As promised, I read through your MDEV-30, "mtr extensions for storage engine testing". Generally I think it looks good and reasonable. Here are a few comments: - --record should have options so that it can generate .rdiff or .COMBI.result. So one could say eg. ./mtr --record=rdiff <some test> to produce <whatever>.rdiff. It's not entirely trivial what the exact syntax should be due to the complicated structure of result file names... - Windows (the usual problem child :-( ) has no native `patch` tool. Will we require it to be installed, or will we gracefully skip tests that need it if no patch is available? - About the "options" at the end about file names wrt. suites: I did not fully understand - but it seems important that whatever is done, existing lots of test cases from Oracle/MySQL can be used without having to rewrite their --source statements. Will this be the case? Hope this helps, - Kristian.

2 1

Re: [Maria-developers] request for a feature -- LIMIT_ROWS_EXAMINED
by MARK CALLAGHAN 20 Jan '12

20 Jan '12

OK, will have feedback in a few days. On Fri, Jan 20, 2012 at 6:45 AM, Timour Katchaounov <timour(a)montyprogram.com> wrote: > Hello Mark, > > >> I want a variant of the LIMIT clause that limits the number of rows >> examined during SELECT processing. This should return OK when the >> limit is reached -- something like NESTED_LOOP_QUERY_LIMIT. While >> LIMIT can be LIMIT x and LIMIT x,y. This only supports one argument -- >> LIMIT_ROWS_EXAMINED x. Are you interested in implementing this feature >> for me? We will sponsor the work. The purpose of the feature is to >> prevent queries from taking too long when there isn't a great index >> and many rows might be filtered. >> > > After examining few alternatives, I implemented a pretty complete > prototype of LIMIT_ROWS_EXAMINED. Please have a look at the description > of the task here: > https://mariadb.atlassian.net/browse/MDEV-28 > > It is important to notice that the current implementation will stop > execution when LIMIT_ROWS_EXAMINED is reached irrespective of the > query execution strategies used (thus not only for JOINs, but also > writing/reading to temp tables, sorting, and single-table access methods > that scan too many rows). This approach also is future-proof with > respect to adding new query processing algorithms because the counting > is done at the handler level. > > The current approach is such, that it should be fairly easy to backport > the feature to almost any MariaDB/MySQL. > > Please let me know if this is what you want. If so, then I will proceed > with the remaining polishing that is in the TODO list and will get an > internal review. > > You can also have a look at the small example in the comments below the > description to get a feeling what is the behavior. > > I can also send you a patch of the current version in case you want to > play with the feature before it is pushed. > > Timour -- Mark Callaghan mdcallag(a)gmail.com

1 1

Re: [Maria-developers] [Commits] Rev 3227: Fix embedded build on Windows. in file:///H:/bzr/5.5/
by Sergei Golubchik 20 Jan '12

20 Jan '12

Hi, Vladislav! On Jan 20, Vladislav Vaintroub wrote: > revno: 3227 > revision-id: wlad(a)montyprogram.com-20120120155435-y8tyd01st3fpbq1e > parent: wlad(a)montyprogram.com-20120120113906-f6wcj47dyo6jyt45 > committer: Vladislav Vaintroub <wlad(a)montyprogram.com> > branch nick: 5.5 > timestamp: Fri 2012-01-20 16:54:35 +0100 > message: > Fix embedded build on Windows. > === modified file 'mysys/CMakeLists.txt' > --- a/mysys/CMakeLists.txt 2012-01-20 00:57:58 +0000 > +++ b/mysys/CMakeLists.txt 2012-01-20 15:54:35 +0000 > @@ -46,7 +46,7 @@ > SET (MYSYS_SOURCES ${MYSYS_SOURCES} my_addr_resolve.c) > ENDIF() > > -IF(WITH_SAFEMALLOC OR NOT HAVE_CXX_NEW) > +IF(NOT HAVE_CXX_NEW) > ADD_DEFINITIONS( -DUSE_MYSYS_NEW) Why is that? I thought that WITH_SAFEMALLOC is always off on Windows? Note, that we should always use mysys/new.cc when safemalloc is enabled. Regards, Sergei

2 2