I am an Infinite Monkey

A friend passed along this blog posting from Microsoft:

Microsoft’s Many Eyeballs and the Security Development Lifecycle

The article is full of half truths and assumptions, but my favorite bit is probably this:

A million monkeys banging on a million keyboards will eventually produce Twelfth Night. Mathematically, the many-eyeballs argument, and the million-monkeys argument are equivalent.

I shall applaud Microsoft for comparing Open Source developers to monkeys randomly banging on keyboards. They've compared us to lots of things in the past, I'm not sure if this is a step up, or just lateral, either way, I'm happy to claim my infinite monkey status. It would be grand if one of you creative types could come up with a clever logo for our new social status.

In all seriousness though, the article makes a number of claims, I'll try to cover the big ones here:
1) Code review makes software more secure
2) Many Eyeballs is not true
3) Nobody is auditing Open Source software
4) The Microsoft Security Development Lifecycle, or SDL, is swell

1) Code review makes software more secure
I don't think anyone can argue against this. The author then goes on to pick out a handful of quotes about how Open Source doesn't get bugs fixed. I'm not even sure how one can come to this conclusion, so let's look at the facts presented instead: None. So in conclusion ... wait, what? The thing that makes an article like this hard to accept without any actual meat, is that it's all out there. Open Source development happens in the public. You don't get to hide bugs under the rug, you can't pretend development is or isn't happening. If the author had interest in backing up this claim, he could have picked a major project and proved his point.

Rather than make some crap up here about how secure Open Source is, I'll defer the reader to this link:
http://www.redhat.com/security/data/metrics/

That's Red Hat's security metric data. If you don't believe it, you can figure it out for yourself by reviewing the open source development data. Debian has something similar here. If what the author claims to be true that Open Source doesn't get any code reviews, or bugs fixed, it would be in a seriously sorry state. Keep in mind that there are a number of very widely used applications out there, Firefox, Apache, the Linux Kernel, and JBoss to name a few. If these applications were in near the sorry state presented in the article, nobody would run them, but they're quite widely used.

I think reality wins this one.


2) Many Eyeballs is not true
3) Nobody is auditing Open Source software
These two ideas are similar in their nature. Are they true? Who knows. I'm certainly not an expert and I suspect without a rather expensive study, we'll never know for sure. Here is what I do know. Microsoft has a finite number of employees. The number of Open Source developers eclipses this by magnitudes. This of course doesn't mean that they're doing audits, or even fixing bugs, but the numbers can't be ignored. Even if only one or two percent of Open Source developers are reviewing code, it's still a huge number. Unfortunately doing that sort of work isn't glamorous, so the folks doing it get little credit or attention.

My example is the Fedora Bug Zappers. These are folks that look at bug reports from Fedora. They're not well known, nor do they get lots of credit, but the job they do is amazing. Let's do a thought experiment here to show how bug reports are like echo chambers: they get noticed. If you report a bug in Fedora, one of the Bug Zappers will find it and take a look. If it's a security flaw, they'll pass it along to the security minded folks. Right here we've gone from one person, filing one bug, has just alerted about ten people (probably more, but let's be conservative) to the bug. Us security types will take a look, and if it's real, pass it along to all the other interested Open Source distributions, and upstream. We're now easily over 100 people. All of those people will collaborate to some degree to develop a patch. Upstream will accept the patch, distributions will patch their packages, and the users will upgrade. Once all this is done, you're talking about many hundreds of people involved in this one bug report. This is where the idea of many eyeballs comes from. It's not about people just auditing code, it's about the community working together to improve the software.

The thing we need to be mindful of here, is that Eric Raymond may be wrong with Linus' Law. It's not so much the why as it is the what. It's easy to claim that many eyeballs isn't true, but one cannot discount the fact that Open Source development works, and it works well. Why it works so well is no doubt an issue that can be debated at length. Perhaps he real power is in our infinite supply of monkeys banging on keyboards

4) The Microsoft Security Development Lifecycle, or SDL, is swell
I agree with this 100%. It is swell, and it's a great idea. It would be wonderful if every Open Source project started doing this.

I would be interested in knowing how many flaw are stopped as a result of SDL. I've not seen any metrics or papers on the effectiveness of this program. I'm certain it's stopped some number of flaws. If someone knows of such data, please pass it along. I won't comment further, as my understanding of this isn't all that deep, but from what I gather it is a good idea, and I applaud Microsoft for implementing this.

In Conclusion
The original article I'm mostly disagreeing with here concludes with the usual old data that Microsoft releases fewer security advisories than Open Source does. This is of course a red herring meant to distract the reader. They've been caught multiple times only releasing one advisory for multiple flaws. With closed source, there isn't a good way to tell what's all getting fixed. In Open Source, we can't hide anything, it's all there. This keeps us honest. I could go on with this argument at length, but it's not really worth it. It's been picked apart every which way for years now, and I don't think anyone really cares anymore. At the end of the day, all that matters is keeping the end users safe. If some friendly competition helps do this, we all win.