Re: [Maria-developers] [Commits] MDEV-60 Support for Spatial Reference systems for the GIS data.
Hi, Alexey! First, please, please, *do not* send patches from Thunderbird. It took me a lot of time to fix it, the patch was badly garbled. May be broken formatting everywhere in the patch was also caused by this. Then, I didn't see any tests in the patch, not even syntax tests, but also not create field, store, read tests. Please add them. Now, the review:
diff --git a/sql/field.cc b/sql/field.cc --- a/sql/field.cc +++ b/sql/field.cc @@ -7808,6 +7808,35 @@ uint Field_blob::is_equal(Create_field *new_field)
#ifdef HAVE_SPATIAL
+uint gis_field_options_image(uchar *buff, List
&create_fields) +{ + uint image_size= 0; + uint n_field= 0; + List_iterator it(create_fields); + Create_field *field; + while ((field= it++)) + { + if (field->sql_type != MYSQL_TYPE_GEOMETRY) + continue; + if (buff) + int2store(buff + image_size, ((uint16) n_field)); + image_size+= 2; + if (buff) + int2store(buff + image_size, ((uint16) field->length)); + image_size+= 2; + if (buff) + int2store(buff + image_size, ((uint16) field->decimals)); + image_size+= 2; + if (buff) + int4store(buff + image_size, ((uint32) field->srid)); + image_size+= 4; + n_field++; + } + + return image_size; +}
Hmm... So, you basically, put an array of properties in frm, under EXTRA2_GIS Two comments: 1. Why n_field, length and decimals? I thought you only add srid, other properties are already stored elsewhere, aren't they? 2. Really think about upgrades downgrades. The "extra" section in the frm doesn't handle them well, I know how these problems happen. Think, what happens when you add more properties here? What if you remove some? How new server will parse old frms, how old server will read new frms?
+ + void Field_geom::sql_type(String &res) const { CHARSET_INFO *cs= &my_charset_latin1; @@ -9181,7 +9210,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, Item *fld_on_update_value, LEX_STRING *fld_comment, char *fld_change, List<String> *fld_interval_list, CHARSET_INFO *fld_charset, uint fld_geom_type, - Virtual_column_info *fld_vcol_info, + char *fld_srid, Virtual_column_info *fld_vcol_info,
fix the formatting, please
engine_option_value *create_opt, bool check_exists) { uint sign_len, allowed_type_modifier= 0; @@ -9232,6 +9261,7 @@ bool Create_field::init(THD *thd, char *fld_name, enum_field_types fld_type, pack_length= key_length= 0; charset= fld_charset; geom_type= (Field::geometry_type) fld_geom_type; + srid= fld_srid ? (uint)atoi(fld_srid) : 0;
and here
interval_list.empty();
comment= *fld_comment; @@ -9635,7 +9665,7 @@ Field *make_field(TABLE_SHARE *share, uchar *ptr, uint32 field_length, uint pack_flag, enum_field_types field_type, CHARSET_INFO *field_charset, - Field::geometry_type geom_type, + Field::geometry_type geom_type, uint srid,
and here, and everywhere, I won't comment on that anymore
Field::utype unireg_check, TYPELIB *interval, const char *field_name) diff --git a/sql/sql_lex.h b/sql/sql_lex.h --- a/sql/sql_lex.h +++ b/sql/sql_lex.h @@ -2358,6 +2358,7 @@ struct LEX: public Query_tables_list char *backup_dir; /* For RESTORE/BACKUP */ char* to_log; /* For PURGE MASTER LOGS TO */ char* x509_subject,*x509_issuer,*ssl_cipher; + char *srid;
really? it can only be used for field declarations, I don't think there's a need to put it in LEX see below.
String *wild; /* Wildcard in SHOW {something} LIKE 'wild'*/ sql_exchange *exchange; select_result *result; diff --git a/sql/sql_yacc.yy b/sql/sql_yacc.yy --- a/sql/sql_yacc.yy +++ b/sql/sql_yacc.yy @@ -6359,6 +6360,16 @@ precision: } ;
+srid_option: + /* empty */ + { Lex->srid= (char *)0; } + | + SRID_SYM EQ NUM + { + Lex->srid=$3.str; + }
return the value of srid here, don't store it in lex. Like $$ = $3.str; or even $$ = atoi($3.str);
+ ; + field_options: /* empty */ {} | field_opt_list {} diff --git a/sql/table.cc b/sql/table.cc --- a/sql/table.cc +++ b/sql/table.cc @@ -1451,6 +1464,20 @@ int TABLE_SHARE::init_from_binary_frm_image(THD *thd, bool write, #ifdef HAVE_SPATIAL geom_type= (Field::geometry_type) strpos[14]; charset= &my_charset_bin; + if (gis_options && gis_options_len >= 10) + { + gis_length= uint2korr(gis_options+2); + gis_decimals= uint2korr(gis_options+4); + srid= uint4korr(gis_options+6); + gis_options_len-= 10; + gis_options+= 10;
If you have the function to generate this data in field.cc, I'd put a small function to read it in field.cc too. Just to have all reading/writing code in the same place.
+ } + else + { + srid= 0; + gis_length= 0; + gis_decimals= 0; + } #else goto err; #endif diff --git a/sql/unireg.h b/sql/unireg.h --- a/sql/unireg.h +++ b/sql/unireg.h @@ -190,6 +190,7 @@ enum extra2_frm_value_type { #define EXTRA2_ENGINE_IMPORTANT 128
EXTRA2_ENGINE_TABLEOPTS=128, + EXTRA2_GIS=129,
better make it 2, not 129 (below EXTRA2_ENGINE_IMPORTANT threshold), because you de facto ignore it anyway when HAVE_SPATIAL is not defined.
};
int rea_create_table(THD *thd, LEX_CUSTRING *frm, diff --git a/unittest/mysys/CMakeLists.txt b/unittest/mysys/CMakeLists.txt --- a/unittest/mysys/CMakeLists.txt +++ b/unittest/mysys/CMakeLists.txt @@ -17,7 +17,7 @@ MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc LINK_LIBRARIES mysys)
MY_ADD_TESTS(ma_dyncol - LINK_LIBRARIES mysqlclient) + LINK_LIBRARIES mysqlclient stdc++)
Why?
IF(WIN32) MY_ADD_TESTS(my_delete LINK_LIBRARIES mysys)
Regards, Sergei
Hi, Sergei. Would like a bit of your opinion before the revised patch. 21.10.2014 22:22, Sergei Golubchik пишет:
First, please, please, *do not* send patches from Thunderbird.
Sorry about that.
diff --git a/sql/field.cc b/sql/field.cc --- a/sql/field.cc +++ b/sql/field.cc @@ -7808,6 +7808,35 @@ uint Field_blob::is_equal(Create_field *new_field)
#ifdef HAVE_SPATIAL
+uint gis_field_options_image(uchar *buff, List
&create_fields) +{ + uint image_size= 0; + uint n_field= 0; + List_iterator it(create_fields); + Create_field *field; + while ((field= it++)) + { + if (field->sql_type != MYSQL_TYPE_GEOMETRY) + continue; + if (buff) + int2store(buff + image_size, ((uint16) n_field)); + image_size+= 2; + if (buff) + int2store(buff + image_size, ((uint16) field->length)); + image_size+= 2; + if (buff) + int2store(buff + image_size, ((uint16) field->decimals)); + image_size+= 2; + if (buff) + int4store(buff + image_size, ((uint32) field->srid)); + image_size+= 4; + n_field++; + } + + return image_size; +}
Hmm... So, you basically, put an array of properties in frm, under EXTRA2_GIS Two comments: 1. Why n_field, length and decimals? I thought you only add srid, other properties are already stored elsewhere, aren't they?
the 'length' - it's the coordinate length, not the field length parameter that is presently stored in the FRM. Decimals can't be stored for the GEOMETRY field in ordinary way. #define FIELDFLAG_BLOB 1024 // mangled with decimals! #define FIELDFLAG_GEOM 2048 // mangled with decimals! n_field - well that is not really needed. Planned to use it as a link to the field initially, but that's lost it's sence.
2. Really think about upgrades downgrades. The "extra" section in the frm doesn't handle them well, I know how these problems happen. Think, what happens when you add more properties here? What if you remove some? How new server will parse old frms, how old server will read new frms?
I thought the way to add new properties is to introduce more EXTRA2_GIS_something sections in the 'extra' part. Do you think i'd better implement similar sections to be inside the EXTRA2_GIS block? Didn't think about removing ones. That 'sections inside EXTRA2_GIS' let's the removal to some extent for a little price, so doing that...
diff --git a/sql/sql_lex.h b/sql/sql_lex.h --- a/sql/sql_lex.h +++ b/sql/sql_lex.h @@ -2358,6 +2358,7 @@ struct LEX: public Query_tables_list char *backup_dir; /* For RESTORE/BACKUP */ char* to_log; /* For PURGE MASTER LOGS TO */ char* x509_subject,*x509_issuer,*ssl_cipher; + char *srid;
really? it can only be used for field declarations, I don't think there's a need to put it in LEX see below.
+srid_option: + /* empty */ + { Lex->srid= (char *)0; } + | + SRID_SYM EQ NUM + { + Lex->srid=$3.str; + }
return the value of srid here, don't store it in lex. Like
But then the 'srid_option' is used in the 'type' definition like this: type: .... | spatial_type float_options srid_option { #ifdef HAVE_SPATIAL Lex->charset=&my_charset_bin; Lex->uint_geom_type= (uint)$1; $$=MYSQL_TYPE_GEOMETRY; #else my_error(ER_FEATURE_DISABLED, MYF(0), sym_group_geom.name, sym_group_geom.needed_define); MYSQL_YYABORT; #endif } if the SRID is not stored in the Lex how do you think it should be transferred to the add_field_to_list?
--- a/unittest/mysys/CMakeLists.txt +++ b/unittest/mysys/CMakeLists.txt @@ -17,7 +17,7 @@ MY_ADD_TESTS(bitmap base64 my_vsnprintf my_atomic my_rdtsc lf my_malloc LINK_LIBRARIES mysys)
MY_ADD_TESTS(ma_dyncol - LINK_LIBRARIES mysqlclient) + LINK_LIBRARIES mysqlclient stdc++)
Why?
The 10.1 linking fails on my computer withouth that stdc++ thing. I don't actually plan to push that anyhow with this task. Will try to understand later what's going on. Best regards. HF
Hi, Alexey! On Nov 03, Alexey Botchkov wrote:
Hmm... So, you basically, put an array of properties in frm, under EXTRA2_GIS Two comments: 1. Why n_field, length and decimals? I thought you only add srid, other properties are already stored elsewhere, aren't they?
the 'length' - it's the coordinate length, not the field length parameter that is presently stored in the FRM.
Decimals can't be stored for the GEOMETRY field in ordinary way.
#define FIELDFLAG_BLOB 1024 // mangled with decimals! #define FIELDFLAG_GEOM 2048 // mangled with decimals!
Ok. So, decimals weren't stored anywhere before your patch? What these decimals are? What the syntax look like? It would be good if your commit would include some test cases...
n_field - well that is not really needed. Planned to use it as a link to the field initially, but that's lost it's sence.
2. Really think about upgrades downgrades. The "extra" section in the frm doesn't handle them well, I know how these problems happen. Think, what happens when you add more properties here? What if you remove some? How new server will parse old frms, how old server will read new frms?
I thought the way to add new properties is to introduce more EXTRA2_GIS_something sections in the 'extra' part. Do you think i'd better implement similar sections to be inside the EXTRA2_GIS block?
No, I was just asking. So, when you add more properties, it'll be EXTRA2_GIS2, etc ? Okay. Perhaps in that case I'd suggest to rename EXTRA2_GIS to EXTRA2_GIS_SRID_AND_DECIMALS (or something that explains what's stored there). I thought that EXTRA2_GIS is a storage for all new and future gis attributes.
diff --git a/sql/sql_lex.h b/sql/sql_lex.h --- a/sql/sql_lex.h +++ b/sql/sql_lex.h @@ -2358,6 +2358,7 @@ struct LEX: public Query_tables_list char *backup_dir; /* For RESTORE/BACKUP */ char* to_log; /* For PURGE MASTER LOGS TO */ char* x509_subject,*x509_issuer,*ssl_cipher; + char *srid;
really? it can only be used for field declarations, I don't think there's a need to put it in LEX ... if the SRID is not stored in the Lex how do you think it should be transferred to the add_field_to_list?
Right. This is a problem. Still, I don't like that everything a parser might ever need is simply dumped into the LEX structure. So I've create a patch to remove almost all field attributes from LEX - the parser now creates Create_field and puts everything in there directly. It's MDEV-7062, currently in bb-10.1-serg branch waiting for Bar to review it (because it changes some charset related behavior). I hope he'll review it first thing next week (or even earlier if he'll have time from 10.0 bugs) and then you'll be able to use it. Regards, Sergei
participants (2)
-
Alexey Botchkov
-
Sergei Golubchik