Hi Sergei, Thanks for the quick reply. 1. I have tried your suggestion (which is a cool simplification to my bit-messing… ;). Unfortunately the code is crashing now: mariadb’s crash report shows the following stack trace: stack_bottom = 0x9e5de314 thread_stack 0x30000 mysys/stacktrace.c:246(my_print_stacktrace)[0x89912c5] sql/signal_handler.cc:153(handle_fatal_signal)[0x841593a] [0xb76f5c58] [0xb76f5c7c] /lib/i386-linux-gnu/libc.so.6(gsignal+0x47)[0xb70e6607] /lib/i386-linux-gnu/libc.so.6(abort+0x143)[0xb70e7d83] /lib/i386-linux-gnu/libc.so.6(+0x27757)[0xb70df757] /lib/i386-linux-gnu/libc.so.6(+0x27807)[0xb70df807] sql/field.cc:6876(Field_varstring::store(char const*, unsigned int, charset_info_st const*))[0x840032c] /usr/local/mysql/lib/plugin/ha_fsview.so(_ZN9ha_fsview8rnd_nextEPh+0xec)[0xb70adb00] sql/handler.cc:2553(handler::ha_rnd_next(unsigned char*))[0x841b003] sql/records.cc:465(rr_sequential(READ_RECORD*))[0x852cf50] sql/sql_select.cc:18532(join_init_read_record(st_join_table*))[0x82adff6] sql/sql_select.cc:17639(sub_select(JOIN*, st_join_table*, bool))[0x82ac256] sql/sql_select.cc:17304(do_select)[0x82abb38] sql/sql_select.cc:3079(JOIN::exec_inner())[0x82895c3] sql/sql_select.cc:2369(JOIN::exec())[0x8286b3a] sql/sql_select.cc:3307(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x8289e70] sql/sql_select.cc:372(handle_select(THD*, LEX*, select_result*, unsigned long))[0x8280acc] sql/sql_parse.cc:5269(execute_sqlcom_select)[0x8258632] sql/sql_parse.cc:2552(mysql_execute_command(THD*))[0x8250ee9] sql/sql_parse.cc:6415(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x825aa1f] sql/sql_parse.cc:1307(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x824e3db] sql/sql_parse.cc:1004(do_command(THD*))[0x824d7fd] sql/sql_connect.cc:1379(do_handle_one_connection(THD*))[0x8358b06] sql/sql_connect.cc:1293(handle_one_connection)[0x835887c] /lib/i386-linux-gnu/libpthread.so.0(+0x6f16)[0xb75fff16] /lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb71a3a3e] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x9891f840): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on Apparently it seems that Field_varchar::store is somehow unhappy. I am using 10.0.14 sources, the given function is: int Field_varstring::store(const char *from,uint length,CHARSET_INFO *cs) { ASSERT_COLUMN_MARKED_FOR_WRITE_OR_COMPUTED; uint copy_length; const char *well_formed_error_pos; const char *cannot_convert_error_pos; const char *from_end_pos; copy_length= well_formed_copy_nchars(field_charset, (char*) ptr + length_bytes, field_length, cs, from, length, field_length / field_charset->mbmaxlen, &well_formed_error_pos, &cannot_convert_error_pos, &from_end_pos); if (length_bytes == 1) *ptr= (uchar) copy_length; else int2store(ptr, copy_length); if (check_string_copy_error(this, well_formed_error_pos, cannot_convert_error_pos, from + length, cs)) return 2; return report_if_important_data(from_end_pos, from + length, TRUE); } (More specificallly the problematic line seems to be the well_formed_copy_nchars() invocation in this function). I am a bit puzzled about how this stuff is working. I mean i haven’t told the table struct buf pointer value from rnd_next(). How can the field class compute, where he should put the data? Thanks again for the quick help Andras -- Andras Szabo Sent with Airmail On 30 Jan 2015 at 17:19:21, Sergei Golubchik (serg@mariadb.org) wrote: Hi, Andras! On Jan 30, Andras Szabo wrote:
Hi Everyone,
I am developing a custom mariadb storage backend which is intended to give filesystem related information via mariadb tables. For example one could tell: CREATE TEMPORARY TABLE IF NOT EXISTS `/var/log` ENGINE=fsview; which would create a table (using the assisted discovery method) with the following schema:
Cool!
1. I am having trouble implementing rnd_next() in the backend:
int ha_fsview::rnd_next(uchar *buf) { DBUG_ENTER(__PRETTY_FUNCTION__); if (!dirp) { DBUG_RETURN(HA_ERR_INTERNAL_ERROR); } if (struct dirent* entry = readdir(dirp)) { uchar *pos = buf; *pos = 0; ++pos; // file name (1bytes size + str) *pos = strlen(entry->d_name); ++pos; size_t bytes_to_copy = strlen(entry->d_name); memcpy(pos, entry->d_name, bytes_to_copy); pos+=bytes_to_copy; DBUG_PRINT("info", ("fsview filename: %s", entry->d_name)); // content (4 bytes size + content ptr) static const std::string fake_content = "hello world"; int s = fake_content.size(); memcpy(pos, &s, sizeof(s)); pos+=sizeof(s); const char* c = fake_content.c_str(); memcpy(pos, &c, sizeof(c)); pos+=sizeof(c); DBUG_RETURN(0); } else { DBUG_RETURN(HA_ERR_END_OF_FILE); } }
My problem is that filenames (first field) is ok, but i cannot really see the expected content “hello world” in the second field. I am assuming that i am doing something wrong, but i cannot really find the reason. Could somebody help me, please?
This looks correct to me. May be, perhaps. std::string->c_str() isn't returning the pointer to your static string (but to a some temporary copy that gets invalidated on return)? I don't know how std::string is supposed to work in this case. Anyway it might've been easier to use Field methods than assembling the row directly. Like table->field[0]->store(entry->d_name, strlen(entry->d_name), system_charset_info); table->field[1]->store(STRING_WITH_LEN("hello world"), system_charset_info);
2. If i am issuing:
SELECT * FROM `/var`;
twice, then the server is not initiating a table scan again on my backend (most likely results from the first run are stored in some cache). So if filesystem contents are changed between the two statements, then it won’t be reflected in the resultset.
Of course if i do a:
SELECT SQL_NO_CACHE * FROM `/var`;
then everything is fine, however i would not like to put the burden on the user to specify the option to not cache the results.
My question is whether it’s possible to specify in the storage backend that the results from the tables should not be cached.
Yes. See handler::register_query_cache_table(). Regards, Sergei