Over the years, we've seen multiple kernel-testing systems emerge, which make results available to maintainers and developers: 0day, Syzbot, KernelCI, LKFT, CKI, and so on.
Yet, their impact remains uneven, and limited: test requirements are rare, pre-merge CI is rarer, and testing of the rest is opportunity-based. Maintainers and developers are cautious of their results: issues unrelated to changes are often reported, e.g. due to framework, infrastructure, test code problems, or previous changes.
This is a chicken-and-egg problem: unreliable results can't be used for gating, but they are not improving much until they are. Let's discuss ways to improve this, and how far they can possibly go.
The KCIDB project, developed by Linux Foundation's KernelCI, is aggregating data coming from testing systems and building unified dashboards and a notification system for developers. We are going to share some ideas we're working on for improving the testing side of the equation.
