Open Source Security
Welcome at » metrics

Fedora Weekly News continues to be a(n unexpectedly) great source for security content. I’ve recently been cleaning up the backlog of my email and have discovered nuggets of valuable information such as

94% of Fedora 8 installs have SELinux enabled

in Fedora Weekly News Issue 121 (Feb. 18, 2008). Now if you read the article, the number I selected to highlight is the raw number that James got off-list. 47%, 50%, and 74% were also tossed out there. Dan Walsh said that the statistics are misleading but being improved and Yaakov Nemoy says that smolt only measures 10% of Fedora machines. So, they are still working out the details. Even so, what they have measured so far is a quite a bit different from the statistics that we see about enterprise customers. I expect it is probably because Fedora users are satisfied with a completely open source stack and do not install as many 3rd party ISV applications which are not as integrated and do not have application specific SELinux policy. Still, this is an incredibly encouraging statistic. Once the Fedora community has been collecting the statistics a little longer, collects whether SELinux is enforcing or not, and starts publicizing these statistics widely, they may be able to help drive ISV adoption (or at least tolerance) of SELinux which will encourage commercial customers to follow the Fedora wave of early adopters on short order.

P.S. Yes, the title is tongue in cheek with a nod to the guys who participated in the discussion.

http://fedoraproject.org/wiki/FWN/Issue121

Oh boy, I thought I had quibbles with the news story on the Coverity announcement yesterday and today someone points out the worst piece of yellow journalism that I have seen in quite some time: Open Source Code Contains Security Holes. First the title is atrocious and this quote “the popular open source backup and recovery software running on half a million servers, were all found to have dozens or hundreds of security exposures and quality defects” may (have) be(en) accurate, but without context sounds worse than it really is. The truth, as George Wilson said, is that this is an article along the lines “And in other news, fire is hot and water is wet.” I personally consider this irresponsible journalism. They had to willfully ignore older stories based on information from Coverity and Carnegie Mellon such as Open Scrutiny of Open Source Code which contains the nugget “The average defect rate of the open source applications was 0.434 bugs per 1000 lines of code. This compares with an average defect rate of 20 to 30 bugs per 1000 lines of code for commercial software, according to Carnegie Mellon University’s CyLab Sustainable Computing Consortium.” This is simply yellow journalism whose primary intention is to drive traffic and raise the ire of open source fans! Harrumph! Outrageous!

Note to Charles Babcock: software has bugs, even security bugs. If you want to drive down the number of bugs in the software that you are using, use open source.

This type of crappy response comes up almost every time Coverity announces a significant improvement. See this similar news story from ZDNet back in October 2006: Most open source is better.

Coverity has announced “Rung 2″ and that 11 open source projects have achieved “Rung 2″. This means that they have resolved all Rung 1 defects found by the latest release of Coverity Prevent. There is news coverage at news.com: 11 open-source projects certified as secure which claims that the projects “have been certified as free of security defects”. The 11 projects with bragging rights are Amanda, NTP, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and TCL. The Coverity announcement itself says “resolved all of the defects identified at Rung 1″. Looking at the Rung 2 page, it appears to me that there are uninspected defects remaining at Rung 2 which may or may not represent actual defects (and/or actual security flaws), so I’m not sure that the news article’s claim is justified. I also would quibble with the use of the word “certified” which is at risk of becoming overused and rendered meaningless when applied in this context. Despite my quibbles with the news story, Coverity has done us all a major service by exercising their excellent source scanning tools on hundreds of open source projects and reporting the results in a controlled fashion. The 11 projects: Amanda, NTP, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and TCL, have done themselves proud by grinding through the reports and fixing defects found. Thanks to Homeland Security for sponsoring this effort, I appreciate this use of taxpayer money. Congratulations and a hearty Thanks! to Coverity and Amanda, NTP, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and TCL!

http://scan.coverity.com/
http://www.news.com/8301-10784_3-9843682-7.html?tag=nefd.top
http://scan.coverity.com/rung2.html

If you are interested in security and security metrics, I highly recommend reading Dan Geer’s chart deck on “Measuring Security”. It weighs in at a hefty 426 pages, but it made me laugh out loud in parts and go hmmm. Highlights include p. 108 on “Decision Making” says “*Rational decisions are not enough, *Need to also allow for your preferences”. I really like the model for “Tracking Performance” that he shows for selected security software on pages 154-156, but caution still needs to be applied and meta-information about the numbers is important for full understanding - did the product undergo extensive review one year? are the CVE’s equivalent to each other in severity? etc. Well worth a read and on my list for more more comprehensive study.

[1] Dan Geer, Measuring Security