Hi! On Thu, 11 Apr 2024 at 22:26, Otto Kekäläinen <otto@kekalainen.net> wrote:
As Daniel wrote, there's MDEV-33073 "always green buildbot", and it's a blocker, which means it *will* be done before the next release.
Take a look at the 10.5 branch - I've done >30 commits in the last couple of weeks specifically to fix sporadic test failures.
This will be merged up soon.
Glad to hear!
While I understand that the natural reply is "we will get to green soon" and it makes a lot of sense, I am afraid it might be a overly optimistic. We've had in the past recurring the situation that Daniel, Sergei and Monty all say the same week they want to fix all failing tests, but it only lasts for a short while and then we are back to failures on mainline CI.
This is what branch protection is for. It cannot wasn't able to do much as tests were constantly failing. Now it can
Cool, looking forward to seeing branch protection enforcing CI stays green.
I wanted to follow up on the topic of enforcing that CI stays green by using branch protection in GitHub. The problem I have been witnessing is that new contributors such as Trevor and ParadoxV5 end up wasting a lot of time researching the failing CI only to discover that none of the failures were caused by their submission, and that the main branch already had CI failures, and essentially learn to not respect the CI status for MariaDB submissions. As Daniel and Sergei responded, the CI failures are tracked via https://jira.mariadb.org/browse/MDEV-33073 "always green buildbot" as critical bugs, and fixed as soon as possible. What I tried to argument is that it is an eternal game of catchup, and my recommendation would be to turn on the branch protection *now*, take the initial pain of having a stricter policy, but after than reaping the benefits of having a consistently green CI on main branch, and on most of the new PRs by new contributors. When I now look at https://github.com/MariaDB/server/commits/11.6/ I see a red cross on all commits since July 8th (last green one was 44af9bf). Looking at the summary by Hashim about 10 consecutive CI runs on 25b5c63 at https://github.com/MariaDB/server/pull/3425 at least the job amd64-fedora-38-last-N-failed is always failing and it would not have found its way into the codebase if a protected branch policy would require that all commits must have a passing CI status before getting pushed/merged. Thanks for considering the proposal, - Otto PS. If all MariaDB core developers tried using the GitHub PR workflow for a couple of weeks you would most likely run into this and many other smaller papercuts yourself, and a positive cycle would likely follow from such "dogfooding", which could lead to an improved external contributor experience overall.