[Maria-developers] MTR fails on undefined {cpus} array in My/SysInfo.pm on armhf and armel
Hi! I noticed that the autopkgtests on Debian on armhf and armel that run the mariadb-test-run fail on error: starting mysql-test-tun.pl... Logging: ./mysql-test-run.pl --force --testcase-timeout=120 --suite-timeout=540 --retry=3 -- ... Collecting tests... Installing system database... Can't use an undefined value as an ARRAY reference at lib/My/SysInfo.pm line 166. Logs for full context: https://ci.debian.net/data/autopkgtest/unstable/armel/m/mariadb/33870767/log... https://ci.debian.net/data/autopkgtest/unstable/armhf/m/mariadb/33870768/log... This line 166 in mysql-test/lib/My/SysInfo.pm has: # Return the number of cpus found sub num_cpus { my ($self)= @_; return int(@{$self->{cpus}}) or confess "INTERNAL ERROR: No cpus in list"; } Can somebody with strong Perl skills help me understand 1) Where this cpus array is inherited from? 2) Why might it have now regressed in MariaDB 10.11.3 and why is this erroring only on armel/armhf but not other architectures? 3) What should I do wrap this section in so that if cpus is not set, it would just default to '1'? - Otto
Can't use an undefined value as an ARRAY reference at lib/My/SysInfo.pm line 166.
# Return the number of cpus found sub num_cpus { my ($self)= @_; return int(@{$self->{cpus}}) or
confess "INTERNAL ERROR: No cpus in list"; }
Can somebody with strong Perl skills help me understand 1) Where this cpus array is inherited from?
It's not inherited, $self is an "anonymous hash" Observe: root:~$ perl -Mstrict -le 'my $self={a=>"b"};print scalar(@{$self->{cpus}}) ' Can't use an undefined value as an ARRAY reference at -e line 1. root:~$ perl -Mstrict -le 'my $self={a=>"b"};print scalar(@{$self->{cpus} || []}) ' 0 So adding these " || []" will avoid the error ([] means anonymous array aka list) root:~$ perl -Mstrict -le 'my $self={a=>"b",cpus=>["a","b","c"]};print scalar(@{$self->{cpus} || []}) ' 3
2) Why might it have now regressed in MariaDB 10.11.3 and why is this erroring only on armel/armhf but not other architectures?
I can't answer this question right now, sorry, need some deeper investigation which I can't right now but can try later.
3) What should I do wrap this section in so that if cpus is not set, it would just default to '1'?
Add this line before the troublesome line: return 1 unless $self->{cpus}; best regards, Vadim
Hi, Otto, On May 25, Otto Kekäläinen wrote:
Hi!
I noticed that the autopkgtests on Debian on armhf and armel that run the mariadb-test-run fail on error:
starting mysql-test-tun.pl... Logging: ./mysql-test-run.pl --force --testcase-timeout=120 --suite-timeout=540 --retry=3 -- ... Collecting tests... Installing system database... Can't use an undefined value as an ARRAY reference at lib/My/SysInfo.pm line 166.
Logs for full context: https://ci.debian.net/data/autopkgtest/unstable/armel/m/mariadb/33870767/log... https://ci.debian.net/data/autopkgtest/unstable/armhf/m/mariadb/33870768/log...
This line 166 in mysql-test/lib/My/SysInfo.pm has:
# Return the number of cpus found sub num_cpus { my ($self)= @_; return int(@{$self->{cpus}}) or confess "INTERNAL ERROR: No cpus in list"; }
Can somebody with strong Perl skills help me understand 1) Where this cpus array is inherited from?
cpus is initialized to be an empty list on the line 119: 118 my $self= bless { 119 cpus => (), 120 }, $class; Then it tries to fill it from /proc/cpuinfo (line 67) and `kstat` (line 95). If nothing worked it'll create one dummy cpu: 145 push(@{$self->{cpus}}, 146 { 147 bogomips => DEFAULT_BOGO_MIPS, 148 model_name => "unknown", 149 }); At no point it sets $self->{cpus} to undef, so I don't understand how that error of yours is even possible.
2) Why might it have now regressed in MariaDB 10.11.3 and why is this erroring only on armel/armhf but not other architectures?
It doesn't, this file wasn't touched for ages. Could it be an incredibly ancient perl there? Just a shot in the dark, I don't think it'll explain much. Some distro patches, perhaps?
3) What should I do wrap this section in so that if cpus is not set, it would just default to '1'?
you shouldn't, it already defaults to 1 "unknown" cpu with 2000 bogomips. if you'll simply hack num_cpus() to return 1, it'll fail later on something like $self->{cpus}[0]->{bogomips} Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org
On May 25, Otto Kekäläinen wrote: cpus is initialized to be an empty list on the line 119:
118 my $self= bless { 119 cpus => (), 120 }, $class;
That should be cpus => [], Vadim
Hi Sergei and Vadim! On Fri, 26 May 2023 at 03:21, Sergei Golubchik <serg@mariadb.org> wrote:
Hi, Otto,
On May 25, Otto Kekäläinen wrote:
Hi!
I noticed that the autopkgtests on Debian on armhf and armel that run the mariadb-test-run fail on error:
starting mysql-test-tun.pl... Logging: ./mysql-test-run.pl --force --testcase-timeout=120 --suite-timeout=540 --retry=3 -- ... Collecting tests... Installing system database... Can't use an undefined value as an ARRAY reference at lib/My/SysInfo.pm line 166.
Logs for full context: https://ci.debian.net/data/autopkgtest/unstable/armel/m/mariadb/33870767/log... https://ci.debian.net/data/autopkgtest/unstable/armhf/m/mariadb/33870768/log...
..
2) Why might it have now regressed in MariaDB 10.11.3 and why is this erroring only on armel/armhf but not other architectures?
It doesn't, this file wasn't touched for ages. Could it be an incredibly ancient perl there? Just a shot in the dark, I don't think it'll explain much.
Some distro patches, perhaps?
Indeed, seems this was already failing on MariaDB 10.11.2: 2023-05-20: https://ci.debian.net/data/autopkgtest/unstable/armhf/m/mariadb/33731218/log... 2023-05-20 https://ci.debian.net/data/autopkgtest/unstable/armel/m/mariadb/33730848/log... (I just had not gotten any messages of it regressing in May) kernel: Linux 6.1.0-9-arm64 #1 SMP Debian 6.1.27-1 (2023-05-08) perl 5.36.0-7 libdbi-perl armhf 1.643-4 libconfig-inifiles-perl all 3.000003-2 Last passing one: 2023-04-28 https://ci.debian.net/data/autopkgtest/unstable/armel/m/mariadb/33218554/log... kernel: Linux 5.10.0-21-arm64 #1 SMP Debian 5.10.162-1 (2023-01-21) perl 5.36.0-7 libdbi-perl armel 1.643-4 libconfig-inifiles-perl all 3.000003-2 Seems the Perl and Perl library versions are the same, but the Linux kernel was upgraded on the Debian autopkgtest workers. Kernel upgrade seems plausible root cause to Perl not getting the cpu data in the same way as before. Even if the root cause is in the kernel/Perl library, as immediate mitigation I will try to test some way to wrap the cpus array check in code that gracefully degrades if cpus is not found.
Hi, Otto, On May 26, Otto Kekäläinen wrote:
Kernel upgrade seems plausible root cause to Perl not getting the cpu data in the same way as before.
Even if the root cause is in the kernel/Perl library, as immediate mitigation I will try to test some way to wrap the cpus array check in code that gracefully degrades if cpus is not found.
You've missed that graceful degradation is already there, by design. If you'll simply try to wrap the failing function mtr will fail later anyway. I've got an idea, could it be that your /proc/cpuinfo is *empty* ? If not, could you share it, please? Regards, Sergei VP of MariaDB Server Engineering and security@mariadb.org
Hi! Turns out indeed that /proc/cpu was not visible due to bug in lxcfs, where the ci.debian.net was running. Temporary ignore added in https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/8f8e112103e470... and will remove it once lxcfs version has upgraded on ci.debian.net workers. LXC tracker: https://github.com/lxc/lxcfs/issues/553
participants (3)
-
Otto Kekäläinen -
Sergei Golubchik -
Vadim V Konovalov