When It Comes to Protection from Vulnerabilities, Process Trumps “Many Eyes”

  • Article

Viewpoint

By Pat Edmonds
Senior Product Manager, Microsoft Corporation

See other Viewpoint articles.

There are distinct model differences in the way that Windows Server is developed and Linux is developed. Microsoft has focused on implementing a development model that allows for the inclusion of important software attributes in the planning process. One very clear example is interoperability. Interoperability by design is a key element that is enabled through the Microsoft development model. By taking into account the interoperability needs of Microsoft’s broad customer base, which includes the need to exchange data with software and hardware from more than 100,000 other companies, during the design phase Microsoft can implement appropriate standards and leverage relationships with other vendors to ease the burden on customers who need to integrate Microsoft products with software from other vendors including open source.

These model differences also translate to real-world security differences to customers:

  • How bugs are identified, reported, and fixed

  • Decreased severity of vulnerabilities

  • Reduced total Days of Risk for Windows relative to Linux

Security is often rated as the most important consideration when making a purchase decision about a server operating system. The reality is that Linux and Windows Server have very different philosophies about information security, which translate into different experiences for their customers. After years of building and improving a process-centric approach to security development, it is clear that the Microsoft Security Development Lifecycle model is helping to make Windows a highly secure enterprise server platform.

Nobody needs to tell CIOs that the stakes for information security are high and getting higher all the time. According to Forrester, on average, 10 percent of IT budgets go to security,1 and for good reason. For example:

  • A recent study by Consumer Reports estimates that Internet viruses, spyware, and phishing schemes have cost U.S. consumers more than $7 billion over the last two years, and that the chance of being a cyber-victim is 1 in 4.2

  • As many as one in every 10 Web sites is infected with malware.3

  • In 2006, the total average cost that an organization incurs per lost customer record is $182, an increase of 30 percent over 2005. Since February 2005, the Privacy Rights Clearinghouse has identified more than 93 million records of U.S. residents that have been exposed due to security breaches.4

  • A recent study conducted by Enterprise Management Associates of six companies with publicly disclosed data security breaches noted that the average price of these companies was depressed 5 percent by the sixteenth business day following disclosure (approximately 3 weeks later). For the four companies whose stock prices could be tracked for up to a year, their average remained within a range of -2.4 percent to -8.5 percent up to 195 business days post-disclosure (approximately 9 months). The average did not recover to a pre-disclosure level until 229 business days later—nearly a year.5 (See figure below. Colored lines represent the companies analyzed in the study.)

Figure 1

Changes in Closing Stock Prices of Six Companies Reporting Data Security Breaches, February 2005–June 20065

When it comes to countering security threats, Windows and Linux largely follow different approaches. Both Microsoft and leading, supported enterprise Linux vendors might use a number of similar proven security techniques and tools, but the development philosophies between Windows and Linux are quite divergent.

For example, Linux distributions follow the open source development model, which allows users flexibility to customize code. Depending on a user’s intentions and skill, Linux can be customized to be extremely secure; in fact, at the extreme there are anecdotes of users who have customized SELinux to be so secure that they could not log back in. With the ability to customize, however, comes increased responsibility on the part of the user for securing the code. Major supported Linux distributions have provisions in their contracts to stop their support of the product, including security support such as patch management, once the code has been customized past a certain point.

In addition, a common perception in the Linux community has been that “many eyes make bugs shallow,” a reference to the belief that the community-based nature of Linux and its visible, open-sourced code encourages and enables a large pool of users to identify bugs and offer debugging recommendations. In reality, the “many eyes” mantra for Linux security has largely been disproved for two primary reasons. First, it assumes that all of the “eyes” are qualified to know what they are looking for. In reality, security expertise is not widely distributed across most users, but is actually a fairly rare and valued skill set. Second, the “many eyes” argument implies that all the “eyes” want to voluntarily peruse code for bugs. Actually, debugging and testing code is not necessarily one of the more exciting pastimes for many volunteer developers, who more often than not would rather devote their spare time to creating the next great application. As a result, it is not surprising that Ben Laurie, Director of Security at the Apache Foundation, stated, that “although it’s still often used as an argument, it seems quite clear to me that the ‘many eyes’ argument, when applied to security, is not true.”6

Lack of a rigorous process-centric approach to security management appears to be catching up to Linux development as Linux continues to become increasingly complex. Andrew Morton, Lead Kernel Maintainer for Linux, has observed, "I believe the 2.6 kernel is slowly getting buggier. It seems we're adding bugs at a higher rate than we're fixing them." He also noted that few developers are motivated to work on bug fixes.7 Jonathan Corbet, co-founder of LWN.net, recently observed that “Linux developers seem to be letting bug reports slip through the cracks. With 1,500 open kernel bugs in the tracking system, and 50 going unanswered on the mailing list, do developers need a better process or just new priorities?”8

In contrast to the Linux model, Microsoft promotes security in its products through the Security Development Lifecycle (SDL). The SDL arose from a famous memo by Bill Gates 9 in 1992 to set the course for Microsoft to become a security leader after years of (often justified) customer concerns about Microsoft commitment to security. The purpose of the SDL is two-fold: Reduce the number of vulnerabilities in the code, and reduce the severity of bugs that get through. Since it is virtually impossible to catch all vulnerabilities, the key is to ensure that a disciplined process is in place to identify and correct for bugs in the most timely and prioritized manner possible. (Learn more about the SDL in a recent webcast with John Pescatore of Gartner and Michael Howard and James Whittaker of Microsoft.)

The advantage of the SDL is that security is “baked into” Microsoft products at every stage of development through a stringent series of gates and security expert oversight. Security is also reinforced in product development through the extensive investment Microsoft continues to make in security training and through top-down executive prioritization of security to make sure it is part of corporate culture. As part of this commitment, Microsoft spent more than $200 million training more than 13,000 Windows employees on security-focused development techniques and new engineering processes, resulting in a line-by-line security review of Windows Server 2003.10 In a twist on the Linux “many eyes” model, Microsoft often makes pre-release versions of its software available and has an extensive “dog-fooding” internal program for employees to try betas and provide feedback. For example, more than 2.25 million pre-release copies of Windows Vista were distributed,11 to help ensure that it was stable and met real-world requirements. Unlike the Linux “many eyes” model, however, Microsoft products are tested by a number of dedicated professionals whose livelihoods depend on them finding and debugging code. In fact, more than 2,500 testers ran assessments and testing on Windows Server 2003.12

Security does not stop at development with the SDL. When a software issue arises, customers can send an anonymous report to Microsoft. System failure issues engage the Microsoft Online Crash Analysis service, which does more extensive investigation and reporting. The data is used to improve later products, thereby reinforcing lessons learned into future releases and driving a cycle of continually improving security.

One security area that Microsoft has traditionally lagged behind Linux has been in its ability to customize distributions to exclude unnecessary components, thereby reducing the vulnerability surface area of the product. With Windows Server 2008, Microsoft has taken a strong step in the same direction by allowing users to strip out unnecessary applications to reduce vulnerability surface area through Server Core. The Server Core installation is only a fraction of the size of a default Windows Server installation because it excludes a number of traditional Windows features, including the Windows graphical user interface (with the exception of a minimal set of graphics capability), Microsoft Internet Explorer, and Windows File Explorer.

Given the obvious differences in approach to security between Linux and Windows, what do the results bear out for customers? By almost any measure, the SDL has proven to reward Windows customers with less vulnerable operating systems. In a recent comparison of vulnerabilities in competing operating systems during the first six months of their release, Windows Vista had far fewer vulnerabilities than its Linux rivals. In addition, as further evidence of the continuous improvement cycle generated by the SDL, Windows Vista also had less than half the number of vulnerabilities as its Windows XP predecessor at six months after release.13 (See figure below.)

Figure 2

Similarly, a comparison of vulnerabilities between Windows Server 2003 and its enterprise Linux rivals from January through July 2007 showed Windows Server 2003 to have fewer total vulnerabilities, including fewer high-criticality vulnerabilities.14 (See figure below.)

Figure 3

When comparing security across different products, a common measure of vulnerability is Days of Risk (DoR). DoR measures the time from when a vulnerability has been publically disclosed until a vendor update is available to close the vulnerability.

In a 2006 comparison of Microsoft, Red Hat, Novell, Sun, and Apple with respect to average Days–of-Risk, Windows again proved to be more secure than its rivals, including its Linux rivals. The analysis aggregated multiple product versions for each vendor since many customers have deployments of several versions. The results indicate that, on average, customers who use Windows are at risk of exposure to vulnerabilities, including high severity vulnerabilities, for a significantly shorter period of time than those who use rival operating systems. Novell Enterprise Linux had an average DoR more than 2.5 times longer than Windows, and Red Hat Enterprise Linux had an average DoR nearly 4 times longer than Windows. In addition, Novell Enterprise Linux customers were exposed to high-severity vulnerabilities nearly twice as long as Windows users, and Red Hat Enterprise Linux customers were exposed to high-severity vulnerabilities more than three times longer than Windows users.15 (See figure below.)

Figure 4

Windows and Linux are developed and supported through different models, and these divergent approaches lead to different security and interoperability experiences for end users. In the choice between Windows and Linux, the disciplined SDL process-centric approach as well as interoperability by design, both enabled by the Microsoft development model, offer compelling benefits for customers. The Microsoft approach provides customers with real choices when it comes to their infrastructure and helps ensure that the choices customers make are secure.

Learn more about security comparisons between Microsoft and Linux at https://www.microsoft.com/windowsserver/compare/linux/security.mspx

Sources

  1. Forrester Research, as quoted in article by Joanne VanAuken, “Managed Security Service Providers,” Network Computing.com, August 3, 2006. https://www.networkcomputing.com/showArticle.jhtml;jsessionid=NGR54YQR132R0QSNDLPCKH0CJUNN2JVN?articleID=191203015&pgno=10

  2. Consumer Report's 2007 "State of the Net" study, as profiled in Brian Prince, “Survey: Cost of Cybercrime Reaches $7B,” eWeek, August 6, 2007. https://www.eweek.com/article2/0,1759,2167203,00.asp

  3. Forrester Research analyst Chenxi Wang, as quoted in Brian Prince, “Survey: Cost of Cybercrime Reaches $7B,” eWeek, August 6, 2007. https://www.eweek.com/article2/0,1759,2167203,00.asp

  4. Ponemon Institute study sponsored by Vontu, Inc., and PGP Corp, “2006 Annual Study: Cost of a Data Breach.” https://www.pgp.com/downloads/research_reports/index.html

  5. Enterprise Management Associates, “The Convergence of Security and Systems Management: Towards IT Efficacy,” April 2007. https://download.microsoft.com/download/2/7/9/27940DDA-2B42-460A-A921-6D9E5B1226EE/EMA_Microsoft-Security-SystemsMgmt_WP.PDF

  6. Ben Laurie, “Open Sources 2.0,” p. 60. https://safari.oreilly.com/0596008023/opensources2-CHP-4-SECT-1

  7. Andrew Morton, as quoted in Ingrid Marson, “Linux kernel 'getting buggier,' leader says”, CNet News.com May 6, 2006. https://www.news.com/Linux+kernel+getting+buggier,+leader+says/2100-7344_3-6069363.html

  8. Jonathan Corbet, “Kernel space: Are Linux developers ignoring bug reports?” LinuxWorld.com, September 12, 2007. https://www.linuxworld.com/news/2007/091207-kernel.html?page=3

  9. Bill Gates, “Bill Gates: Trustworthy Computing,” Wired.com, January 17, 2002. https://www.wired.com/techbiz/media/news/2002/01/49826

  10. Microsoft, “Windows Server 2003 by the Numbers: One of the Biggest Product Launches in Microsoft History,” April 23, 2003. https://www.microsoft.com/presspass/features/2003/apr03/04-23WinServerFacts.mspx

  11. Microsoft, “The Business Case for Windows Vista.” 2007.

  12. Microsoft, “Windows Server 2003 by the Numbers.” https://www.microsoft.com/presspass/features/2003/apr03/04-23WinServerFacts.mspx

  13. Jeffrey R. Jones. “Windows Vista 6-Month Vulnerability Report,” CSO.com, June 15, 2007. https://www.csoonline.com/pdf/6_Month_Vista_Vuln_Report.pdf

  14. Jeff Jones, “July 2007 - Operating System Vulnerability Scorecard,” Technet.com, August 16, 2007. https://blogs.technet.com/security/archive/2007/08/16/july-2007-operating-system-vulnerability-scorecard.aspx

  15. Jeff Jones, “Days-of-risk in 2006: Linux, Mac OS X, Solaris, and Windows”, CSO.com, June 13, 2007. https://blogs.csoonline.com/days_of_risk_in_2006