Hi, While importing 11.4.6 into Debian I ran into issues with git and quilt failing to convert text files properly between Unix and Windows line endings. Thus asking are Windows CRLF necessary in the new mysql-test/main/mysql-interactive.result test? commit e5147c8140450f28505a227159d2feba37dd1c39 Author: Sergei Golubchik <serg@mariadb.org> Date: Tue Jan 30 16:39:28 2024 +0100 regression introduced by MDEV-14448 diff --git a/mysql-test/main/mysql-interactive.result b/mysql-test/main/mysql-interactive.result new file mode 100644 index 00000000000..a18c018b932 --- /dev/null +++ b/mysql-test/main/mysql-interactive.result @@ -0,0 +1,24 @@ +# +# regression introduced by MDEV-14448 +# +delimiter $^M +select 1;^M +$^M +Welcome to the MariaDB monitor. Commands end with ; or \g.^M +Your MariaDB connection id is X^M +Server version: Y^M +Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.^M +^M +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.^M +^M +MariaDB [(none)]> delimiter $^M +MariaDB [(none)]> select 1;^M + -> $^M ++---+^M +| 1 |^M ++---+^M +| 1 |^M ++---+^M +1 row in set^M +^M +MariaDB [(none)]> All the other files with Windows line endings seem to be have them intentionally: ± find . -type f -exec file {} \; | grep -i "CRLF" ./storage/connect/mysql-test/connect/std_data/emp.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/boyswin.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/std_data/expenses.txt: ASCII text, with CRLF line terminators ./storage/connect/mysql-test/connect/r/xml.result: HTML document, UTF-8 Unicode text, with CRLF, LF line terminators ./mysql-test/std_data/loadxml2.dat: XML 1.0 document, ASCII text, with very long lines, with CRLF, LF line terminators ./mysql-test/main/loadxml.result: ASCII text, with CRLF, LF line terminators ./mysql-test/main/func_regexp_pcre.result: UTF-8 Unicode text, with very long lines, with CRLF, CR, LF line terminators ./mysql-test/main/mysql-interactive.result: ASCII text, with CRLF, LF line terminators ./mysql-test/main/mysql_binary_mode.result: ASCII text, with CRLF, LF line terminators ./extra/wolfssl/wolfssl/certs/test/smime-test-canon.p7s: MIME entity, ASCII text, with CRLF, CR line terminators ./extra/wolfssl/wolfssl/wolfssl.rc: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators
Otto Kekäläinen via developers <developers@lists.mariadb.org> writes:
Thus asking are Windows CRLF necessary in the new mysql-test/main/mysql-interactive.result test?
The .result files are compared verbatim against the test output to check for test passing or failing. So you can easily check this by removing the CR bytes and run the main.mysql-interactive to see if it still pases.
While importing 11.4.6 into Debian I ran into issues with git and quilt failing to convert text files properly between Unix and Windows line endings.
What issues? Why do you need to convert text files, why not simply keep the files as they are, regardless of CRLF or any other special byte sequences? - Kristian.
Hi!
Thus asking are Windows CRLF necessary in the new mysql-test/main/mysql-interactive.result test?
The .result files are compared verbatim against the test output to check for test passing or failing. So you can easily check this by removing the CR bytes and run the main.mysql-interactive to see if it still pases.
Firstly, I cannot use 'patch' to remove Windows line endings. I can't even use it to modify that file due to git attributes or patch itself mangling on them (e.g. https://salsa.debian.org/otto/mariadb-server/-/jobs/7569516). I didn't manage to fully debug which step exactly cleans them away and from what file. Secondly, I won'd have a Windows machine to test on if the test still works. On my Linux machine the test will surely pass if I remove those endings. My question was here for upstream if you know if that test was intentionally introduced with Windows line endings or if it just happened?
While importing 11.4.6 into Debian I ran into issues with git and quilt failing to convert text files properly between Unix and Windows line endings.
What issues? Why do you need to convert text files, why not simply keep the files as they are, regardless of CRLF or any other special byte sequences?
I need to patch this file in the Debian packaging. I could alternatively also just disable this new test, so it is not a showstopper.
Otto Kekäläinen via developers <developers@lists.mariadb.org> writes:
My question was here for upstream if you know if that test was intentionally introduced with Windows line endings or if it just happened?
Yes, the file mysql-interactive.result needs to have carriage-returns in it for the test to pass. It is not related to Windows though (this is a Linux-only test), but due to how the test runs the mysql client program in a PTY or something like that. That's the TL;DR, I got curious so got a bit carried away with a longer explanation below ;-)
The .result files are compared verbatim against the test output to check for test passing or failing. So you can easily check this by removing the CR bytes and run the main.mysql-interactive to see if it still pases.
Secondly, I won'd have a Windows machine to test on if the test still works. On my Linux machine the test will surely pass if I remove those endings.
Ok, then I'll test for you :-) I just meant, take an editor, remove a carriage-return character, and see what happens. As I suspected, the test will _not_ pass on Linux: ----------------------------------------------------------------------- main.mysql-interactive [ fail ] Test ended at 2025-05-13 07:30:25 CURRENT_TEST: main.mysql-interactive --- /kvm/src/my/dev/mariadb3/mysql-test/main/mysql-interactive.result 2025-05-13 07:30:23.046065796 +0200 +++ /kvm/src/my/dev/mariadb3/mysql-test/main/mysql-interactive.reject 2025-05-13 07:30:25.666029754 +0200 @@ -3,7 +3,7 @@ # delimiter $ select 1; -$ +$ exit Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is X Result length mismatch ----------------------------------------------------------------------- So you see, the test framework does nothing special about carriage-return characters, it just compares the output verbatim to the .result file. Therefore, if the output contains carriage-returns on some lines (as in this test), then those carriage-returns must be in the .result file. It is not related to "Windows line endings", that test is not even run on windows, it has: source include/not_windows.inc; The test runs the mysql client program interactively (in a terminal), this causes those carriage-return to appear in the output due to PTY or whatever reason. So yes, those carriage-returns in the .result file are "intentional" in the sense that the test as written does not work without them.
Firstly, I cannot use 'patch' to remove Windows line endings. I can't even use it to modify that file due to git attributes or patch itself mangling on them (e.g. https://salsa.debian.org/otto/mariadb-server/-/jobs/7569516). I didn't manage to fully debug which step exactly cleans them away and from what file.
patching file mysql-test/main/mysql-interactive.result Hunk #1 FAILED at 10 (different line endings).
This looks like the patch is incorrect, ie. that the carriage-returns in the patch file are not matching the actual file. This random googled answer suggests it's something with quilt, and has some suggested solution that may or may not work for you depending on how you produced the patch: https://unix.stackexchange.com/questions/793759/apply-debian-quilt-patch-on-... Try opening the patch file and the file mysql-test/main/mysql-interactive.result in an editor that shows carriage-returns, and check that the line endings in the two file match.
I need to patch this file in the Debian packaging. I could alternatively also just disable this new test, so it is not a showstopper.
I tried manually patching the file with the `patch` program, that seems to work fine (see attached example). It can modify the test in the file without corrupting the line endings: patch -p0 < /tmp/1.patch
I need to patch this file in the Debian packaging. I could alternatively also just disable this new test, so it is not a showstopper.
It seems to be some issue with how Debian tooling (or the particular usage of it in the mariadb package when generating or applying the patch), presumably there's some way to handle that, it can't be the first time that debian/patches needs to modify files with carriage-return in them. But yes, if it is tricky to sort out, then agree, not critical, just disable the test with a comment that there was some unresolved issue with required patching of a file with partial carriage-returns in it. - Kristian.
participants (2)
-
Kristian Nielsen -
Otto Kekäläinen