Hi Varun, Please add a comment above the declaration of Field::sort_string() that some implementations actually may write up to 8 bytes regardless of what size was requested. Ok to push after that.
revision-id: e14dcfe0bc743aa6b4ed2b3c4f7e9314aa01d6eb (mariadb-10.1.43-158-ge14dcfe0bc7) parent(s): db1f73b26edaace41345f19bca507a7848aa39a8 author: Varun Gupta committer: Varun Gupta timestamp: 2020-05-29 21:13:38 +0530 message:
MDEV-22715: SIGSEGV in radixsort_for_str_ptr and in native_compare/my_qsort2 (optimized builds)
For DECIMAL[(M[,D])] datatype max_sort_length was not being honoured which was leading to buffer overflow while making the sort key. The fix to this problem would be to create sort keys for decimals with atmost max_sort_key bytes
Important: The minimum value of max_sort_length has been raised to 8 (previously was 4), so fixed size datatypes like DOUBLE and BIGINIT are not truncated for lower values of max_sort_length.
... BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog