Open Source Testing: The Next Fad
Last week I spent two very worthwhile days in San Francisco at the Open Source Business Conference. Once again Matt Asay and friends put on a stellar conference. The speakers Matt was able to assemble were impressive and the halls were filled with many of the best thinkers on OSS business issues (in the U.S.). As a platinum sponsor of the show, I was more than pleased with the outcome.
If I had to pick one thing that jumped out at me during the conference, it is the fact that the ideology of open source has persisted and yet has diverged from the reality of how OSS is being used as a business strategy.
Ideology is important. Reaching for a social and business system that is inherently more fair and productive is what we should be doing. Enabling consumers to maximize their choice and vendors to compete based on the merits of their technologies is what should be driving the industry forward. And to an extent, this is what has made OSS so compelling – the potential for delivering on these promises.
Yet these compelling ideas are tied to a software development model. If it were able to stop there – and not layer on business strategy, operational realities, or the implications of existing intellectual property regimes - OSS would probably do a better job of meeting those goals.
I lost count of the number of presenters at the conference who invoked the dreaded “vendor lock-in” as the reason to look at OSS products. After all, the source code is made available for anyone to see or modify. If it is GPL-based code or any other reciprocal license, the code is mandated by the license to be made available for all. Yet none of these companies dealt with the fact that the commercial requirements of customers are driving them to place non-open terms in their support contracts and/or binary licensing agreements. Also, many of the startups I spoke with at the show have proprietary software built to help customers be more successful with OSS technologies. Thus, their customers are going to be waiting for them to fix the bugs and release the next versions.
Maybe vendors do more than just sit around and think of new ways to lock their customers in.
The second thing that has really stuck with me from the conference is the fact that testing of open source solutions is becoming very sexy. I’m not talking about the hyperbole of Linus’ Law – many eyes make all bugs shallow. That has turned out to be more fantasy than fact. I’m referring to the role that people like Spikesource and SourceLabs are playing. Also, systems integrators like Optaros are going to gain more celebrity as they are able to define tested solution sets for their customers.
Integration is the key to successful enterprise-class solutions. Large IT shops want to decrease complexity, manage risk, and reduce cost. I know it sounds like I am advocating hegemony – I’m not though. The best SIs are the ones who can take disparate components and tie them together into compelling solutions for their customers. Microsoft has put its efforts into bringing the integration into the solution stack itself, but that is not by any means the only way to do it (we think it is the most efficient way – but I know that’s not surprising).
For years, the focus has been on the romantic image of the OSS dev, sitting in his home office (or at some uber-hip wireless hotspot) hacking out some really groovy new function or feature. I think that music is going to start playing for the guy who knows how to build really cool test harnesses.
This posting is provided "AS IS" with no warranties, and confers no rights.
Comments
- Anonymous
April 11, 2005
When I first looked at your blog post name, I thought you mean using Open Source means to test any product and not testing of Open Source Code. So I will make the distinction in this comment.
Open Source Testing - using 3rd party and general public for testing through test script generation and verification. This is highly evident in the realm of Web Services. Say a provider publishes a service to a directory service. The provider does his own testing and releases his test cases which users can use to verify that the web service does what it intends to do. Then the end-users could generate their own test scripts and upload it to the directory service for other end-users to see and use. Also the directory service can also come up with the test scripts. Thus, the "open source" of test scripts. Everyone can verify the provider. Not to go into too much detail (as this is patent pending material at our Uni), but this is a novel means of testing in my mind.
What you were talking about is the use of test methodologies on Open Source Software. I'm sorry to say, I see nothing novel about it other than the fact that, as you rightly point out, the linus's law is a bunch of baloney and thus, there really is a need for real thorough testing. Am I missing something? - Anonymous
April 11, 2005
Is Open Source testing on it's way to becoming a little bit trendy? I think it is not unreasonable to assume that it might eventually. Then again, in some ways Open Source has been on the upswing to 'trendy' for quite some time now.... - Anonymous
April 12, 2005
Microsoft Director of Shared Source Jason Matusow, comments on the recent Open Source Development Conference: I lost count of the number of presenters at the conference who invoked the dreaded “vendor lock-in” as the reason to look at OSS products.... - Anonymous
April 21, 2005
I agree with several of your points. However, with respect to "vendor lock-in", I think you're failing to consider an important difference between a completely proprietary solution and one with an OSS base, and proprietary addons. The reason why many customers fear lock-in is because, if the vendor goes out of business or the terms of the contract become too onerous, the customer may be in a real bind. With OSS, the customer's data, at least, is not locked away in any way. Consultants can be hired to replace any addon sugar that goes away with a defunct vendor. Sure, the cost is non-trivial, but its a heck of a lot less than having to do a lot of reverse engineering just to get at the data.
I also take exception with your flat dismissal of the "many eyes" theory. It seems to work for those linux kernel writers. It may not be an inherent trait of all OSS programs, but certainly projects that are important to enough people who are capable of understanding the code, are going to get some very rapid bug turnaround times. - Anonymous
May 22, 2005
fad? i have been to reply to this. heh. soon - Anonymous
September 01, 2005
The comment has been removed - Anonymous
September 01, 2005
The comment has been removed - Anonymous
September 01, 2005
Your blog is very interesint - Anonymous
September 08, 2005
Stephen O'Grady posted a great response to my earlier posting on integration issues for OSS projects.... - Anonymous
March 09, 2006
Jeniffer