So as I've noted in the past, Coverity has been using their tool to scan Open Source software as what I presume is a rather clever marketing campaign. This is the third year they've been doing this and claim that there is a 16 percent reduction is flaws found. They are of course claiming that the software has become 16 percent better, not that their tool is 16 percent worse.
A story like this is probably misleading and can be really hard to understand. What the real story is that Open Source software has gotten better at fixing the sort of things that Coverity scans for. It's another debate of course if this is the result of Coverity scanning the source, or better upstream practices. I would be most interested in seeing how many bugs Coverity found were fixed, if that 16 percent lines up with their previously found bugs.
Understanding statistics is really hard to do and even harder to properly analyze. A story such as this is often misleading, and the organization writing it will make sure it's in their favor. This is best explained with an example. Imagine you have a group of people at risk for heart attacks. It's also quite likely that these people have poor diets that put them in this position. Now suppose you conduct a study where they eat oatmeal for breakfast. It turns out, the folks who ate the oatmeal had fewer heart attacks. So obvious oatmeal is a magic food that prevents heart attacks! ... right? So there are a few ways to look at this. The most likely is that these folks were eating terrible things for breakfast, so basically by eating pretty much anything other than what they were, it would reduce their risk for heart attacks.
I'm not saying that coverity ISN'T responsible for better Open Source code, they might be, but I'm also saying that we can't really know they are without more information.
I agree with the deconstruction of this kind of messaging as marketing. But, I am completely in favor of the kind of automatic analysis that Coverity does, and having fixed many of the bugs they've found in X, I'm quite sure it's improved our code quality; most of the things it found were subtle and unlikely to get noticed by even a thorough reader.
I think, though, that that's mostly a condemnation of the languages we're using to build software. It's just too easy to write something syntactically valid but semantically implausible.