Windows vs Linux Workstation Comparison

NOTE: I am not asserting that my vulnerability analysis demonstrates that Windows is more secure. Rather, I frequently hear and read Linux advocates making unsupported assertions to the opposite that Linux is inherently more secure than Windows. The "unsupported" part of that bothers me, so I check for myself. What I keep finding is that Linux distributions have more vulnerabilities, more serious vulnerabilities and the data does not support the assertions of security superiority for Linux and Open Source software. Also, this is my own work and any mistakes and opinions are mine and not necessarily those of Microsoft.

This post is part of a multi-part Q3 2006 Vulnerability Report. Here are links to all of the sections, in case you want to read the others:

2006 January through September Vulnerability Trends
Windows vs Linux Workstation Comparison (you are here)
Windows vs Linux Server Comparison (TBD real soon now)
Executive Summary
Many people won't have the time/patience (attention span) to read this excellent post through to the end. These folks are sometimes called "Executives". For them, let me just show the most important chart, graphing the weighted vulnerability count for workstation roles in Q3:

There it is - Windows XP with the least, Red Hat Enterprise Linux 4 WS with the most, and Ubuntu 6.06 falling in the middle. There is much more to this story however. To understand the definition of "weighted vulnerability count" and "workstation role" and to get more details, read on!

Introduction
So, now I will shift gears from disclosed vulnerabilities (as discussed in 2006 January through September Vulnerability Trends) to fixed vulnerabilities, as in fixed by the vendor. I will be using the same database as before for much of the information, especially disclosure dates, but will drive the primary analysis from vendor published security advisories at: http://rhn.redhat.com/errata, http://www.microsoft.com/technet/security/current.aspx, and http://www.ubuntu.com/usn.

In the past, I’ve done comparisons of Red Hat Enterprise Linux product with Windows products, a common criticism of the analysis has been that Red Hat ships with a lot of extra applications that should not be counted in an “apples to apples” comparison. I agree and I disagree, as I’ve outlined on my blog in the past under Apples, Oranges and Vulnerability Metrics. I also recently posted a discussion of how I might build a more Windows comparable workstation using Red Hat in Red Hat and Windows - Defining an Apples-to-Apples Workstation Build. To summarize, I think it best to look at things from multiple angles, since each perspective may provide different information. For example:

· A role-based comparison may provide useful information for comparing individual roles, such as a common workstation or Web server

· A comparison of all packages in a vendor product my provide some insight into the impact of using the product in multiple roles that leverage different combinations of packages

With that in mind, the set of metrics for each product will be measured against two configurations: the full configuration and one limited to a specific role. Details are provided in the appropriate sections that follow.

Products for analysis will be Microsoft Windows, Red Hat Enterprise Linux 4 WS and Ubuntu 6.06 LTS. Ubuntu products have been added because:

· I think Ubuntu clearly represents the “hot up and comer” position in the Linux distribution space

· Ubuntu added “Long Term Support” (LTS) with release 6.06, making this an Enterprise distribution

Both Novell Suse and Mandriva have Enterprise support lifecycle offerings, as well, and though I’m not including analysis of those here, I may include them in a year end analysis, once per year. In the meantime, Red Hat and Ubuntu should represent the Enteprise Linux community as both the leader and the interesting newcomer.

Vulnerability Metrics
In previous analyses, I have used total vulnerability counts as metrics, and also looked at severity breakdowns and most recently, a metric called WVI as defined by NIST and similar to that introduced by Mark Cox in Red Hat RHEL4 Risk Report. One of the benefits of the WVI metric is that is normalizes vulnerabilities by both severity and by time, in terms of weighted vulnerabilities per day.

As a first step in calculating WVI, the numerator is calculated by giving full value to High severity vulnerabilities, while Medium severity issues are divided by 5 and Low severity issues are divided by 20. I call this the Weighted Vulnerability Count (Vw). The second step in calculating the WVI is to divide the Vw by the time period involved. Dividing by time allows one to compare lifetime vulnerabilities for products that have been in the market for varying amounts of time.

In my comparisions, I am going to call out both the Vw, or weighted count, and the WVI. The Vw may be thought of as a rough approximation of the number of equivalent High severity issues that occurred in a period. Here are the formulas:

Vw = (High) + (Medium / 5) + (Low / 20)

WVI = Vw / days

Workstation Products
In this section, I will analyze workstation products. For the three products studied, the configurations were defined as follows:

· For Windows XP SP2, I took the worst case assumption that all components were present in a standard workstation role. This means that metrics for “all packages” and “workstation role” will be equivalent and yes, Internet Explorer is included in the analysis.

· For Red Hat Enterprise Linux 4 WS (rhel4ws), there were two distinct configurations.

o Rhel4ws-all consists of all components that Red Hat chose to ship and support as part of the official rhel4ws product. This configuration represents the union of all workstation roles and applications that might be deployed in an enterprise to support office workers, developers, network specialists, marketing professionals, etc.

o Rhel4ws-ws consists of just the default installation group components, excluding for OpenOffice and gimp, which were explicitly disabled. Note that by default, none of the optional “server” packages are installed either. This configuration represents a more minimal, but useful, configuration that is comparable to Windows XP. Firefox is included, for example, but Thunderbird is not.

· Ubuntu 6.06 LTS. For Ubuntu, there were two distinct configurations – Ubuntu-all and Ubuntu-ws.

o For Ubuntu-all, any vulnerability patched for Ubuntu 6.06 LTS by Ubuntu in an Ubuntu Security Notice was counted as part of analysis.

o For Ubuntu-ws, the configuration consisted of the default packages installed, except for OpenOffice and gimp, which were excluded.

The Quarter: Q3 2006
Looking first at all packages or components for each product (workstation-all), Figure 8 charts the weighted vulnerability counts, Vw. Keep in mind that this measure may be useful in terms of:

· Seeing the weighted equivalent number of High severity vulnerabilities

· That could apply in union across multiple workstation role deployments such as office worker, developer, network engineer, etc.

The chart shows that Red Hat Enterprise Linux 4 WS (rhel4ws) had the highest number of vulnerabilities across all components that are part of the product, doubling Ubuntu 6.06 and tripling Windows XP.

Figure 8: Weighted Workstation-all Vulnerabilities for Q3

But what about a basic workstation role that did not include all of the many optional components that ship with Red Hat and Ubuntu? Figure 9 charts the answer to this question, measuring only the vulnerabilities in components that might be in a basic workstation as defined in the configuration section above.


Figure 9: Weighted Workstation-ws Vulnerabilities for Q3

Note that the Windows Vw value for the quarter stays the same, since it is the same configuration, but that the Red Hat workstation and the Ubuntu workstation have lower Vw values, though the order of low to high remains the same.

2006 Year to date – January through September
Any quarter can be anomalous of course, so next I will look at the numbers for the entire year up to this point. Figure 10 charts the WVI for the first 3 quarters of the year. Note that though Ubuntu only shipped on June 1, we can still chart a Q2 value since the WVI formula normalizes for the time available.


Figure 10: Quarterly Workstation-ws WVI

Another way to look at the vulnerabilities patched by the vendors year to date is to calculate the WVI for the entire period instead of quarterly. This measurement helps us see the period as a whole when individual quarters could show a lot of fluctuation.


Figure 11: Workstation-ws 2006 WVI for January through September

Figure 11 charts the January through September WVI for us and does show that, for the entire year, the Red Hat workstation WVI is not as drastically higher than the other products as seen in Q3 alone. It might be that looking at an even longer period would smooth out any anomalous periods even further, so as a last check, let’s look at the lifetime of each product.

Product Lifetimes
Microsoft Windows XP has been generally available the longest, since October 2001. Red Hat Enterprise Linux 4 has been available since February 2005 and, as mentioned before, Ubuntu 6.06 has only been available since June 1st of this year. This section won’t affect Ubuntu much since its lifetime isn’t much longer than the last quarter, but it could provide a more normalized view of vulnerability disclosure rates for Red Hat and Windows XP.

Figure 12 charts WVI for all components for the lifetime of each product. We can easily observe that in this view, the weighted daily vulnerability fix rate for two Linux distributions is much closer than when we looked at the most recent quarter and year to date metrics.


Figure 12: Lifetime WVI for Workstation-all

I can also see that over the lifetime of the products, the two Linux distributions have a WVI roughly three times higher than Windows XP. Now let’s look at just the basic workstation configurations, rather than all components. Based upon popular perception, one might expect the two Linux distributions’ WVI to drop down below Windows XP when the extraneous optional components are removed. However, contrary to that perception, Figure 13 shows that the WVI for both Linux distribution is still over twice that of Windows XP.


Figure 13: Lifetime WVI for Workstation-ws

I can already anticipate some of the thoughts that this workstation analysis will cause for Linux advocates. The workstation builds include X-Windows, servers don’t have to include that. Workstations have browsers, servers don’t have to include that. All true, and in the next section, we’ll again look at configurations that represent the union of several roles (none of which include X or the browser!), as well as dig into a comparison of just a web server role.

Until this, I hope this analyis has been useful. Regards ~ Jeff