The Myth of "It Should Be Simple"
I've spent a lot of time over the past few years reading forum posts and submissions to tell_fs@microsoft.com. One thing that consistently catches my attention is the assertion that whatever is being requested "should be simple" to do. I considered providing examples from the past month or so but there were just too many.
I think what people really mean is "I hope it's simple to do because then maybe Microsoft will do it." At least I hope this what they mean because if they really believe it's simple they are setting themselves up to be disappointed. The fact is I've never seen a suggestion anywhere that I'd judge as "simple" to do.
Why? The answer to that question is simple: we develop software. Every idea or feature we consider has to be looked at with respect to standard phases of software development: design, engineering, testing and support. (Various software methodologies may use different terms but the concepts are the same.) Below I've listed the questions we ask when considering any new idea or feature.
Design: What exactly is the feature? How should it work? I see a lot of single-sentence suggestions like "it should be simple for ATC to issue speed restrictions to aircraft." There are a whole host of issues lurking behind that request. When should speed restrictions be used? Always? Only during heavy periods? How should speeds be calculated? Based on aircraft type? Model? A look-up table? How will it work with 3rd-party aircraft that haven't been developed yet? What if users can't maintain the assigned speed? What if they ignore the instruction? Should ATC cancel their clearence? Try to adjust the flow for other aircraft? And finally, what value does this feature bring to the different customers that use the product?
Engineering: Somebody's got to code this feature. How long will it take? Does it impact other systems? That's a question that often delivers a quick death to feature requests because an idea that seems simple reachs deep into other systems, requiring changes there. This gets to the question of risk. If you have one system that works perfectly well would changing it to accomodate a new feature risk breaking something? What about content? Does the feature recquire new assets such as art or audio? What about UI? How will the user interface with the feature? Does it need a menu option? A new dialog? For every single feature we develop we have to boil the answers to all these questions down to the notion of cost (in person-days, cash outlays and risk) and weigh it against the value computed during the design phase.
Testing: Everything's got to be tested. We use a rigourous testing methodology at Microsoft. That means we don't just mess around with the software waiting for it to break. It means that every feature or feature area has to have a test plan and test cases. The test plan outlines the expected results that we believe will yield a quality experience. What is the user experience for this feature? What are the parameters and their allowable ranges? What happens if the user tries to use the feature outside its designed range? The test team then needs to create test cases that target these scenarios and yield a clear pass/fail result. Creating a test case takes time for design, implemetation and execution. Plus, the more you test the more bugs you find; the more bugs you find the more you need to touch the code to fix it; the more you do that, the greater the risk of breaking something else.
Support: Supporting a new feature encompasses both educating users about it and answering their questions when they get stuck. How much documentation does the feature need? (Plus the cost of any new document is multiplied by the number of langauges we need to translate it into.) Is it likely to generate support calls? (If so, go back to the design to see if the feature is well understood.) What about future versions? When will the feature likely be re-written or dropped do to major changes in the product?
I remember watching a Frontline episode before the 2004 US Presidential elections that compared the two candidates lives and careers. It featured interviews with may of the candidates' friends and colleagues. John Kerry's former chief counsel, Jack Blum, described him as someone who would "tell you that the problem is much more complicated than you think it is." Software development is like that. The approach didn't serve Kerry very well but we don't have the choice of expressing things in simple, black and white terms. So keep those suggestions coming but, if you don't mind, let us decide if they're really simple.
Comments
- Anonymous
February 11, 2006
I agreed with you that a lot of people are using the "should be simple" argument to set themselves up for possible disappointment. The complexity of any good flight simulator is often overlooked by many who are not on the business in designing product, hardware or software. I have designed and managed projects from conception to production for the last 20 year (like ATMs and voting machines) so I concure with you statements. I am a avid flight simmer hoping for the best. Keep up the good work. - Anonymous
February 11, 2006
The comment has been removed - Anonymous
February 11, 2006
Hi Mike,
Thanks for giving us a glimpse into the decision making process. Its interesting that you mentioned the test plan and its attributes. In my Game Software Development class, we were just discussing the QA department's role, and the importance of a thorough test plan.
My impression was that there was only one test plan, and its purpose was to test the finished product. I never realized that there could be multiple test plans for individual features, or feature sets. Thanks for that insight. I'd love to be able to pick your brain for my mid-term!
Thanks again,
Nick - Anonymous
February 11, 2006
Ha! Don't get me started on security. Every piece of software that leaves Microsoft has to pass a security review that includes threat modeling plus auditing by automated tools and security experts. All good, but it does add to the cost, thus shifting the cost/value equation. - Anonymous
February 11, 2006
The comment has been removed - Anonymous
February 11, 2006
While I agree with all the things already said, I do think that we software guys like to solve complex problems - it's what gets us going. Far too often, in my experience, simply rephrasing the problem shows that simpler solutions are possible.
User requests aside - we really SHOULD be looking for simpler ways to "do it". - Anonymous
February 11, 2006
After over 30 years in IT myself, I can relate to your story, Mike.
And 'just in case' the users mean something else - like: it should be simple to OPERATE - let me say this:
They DO NOT MEAN IT ! <grin>
Especially in the forums (where the vocal FS minority is to be found) I see so many up-front protests against what people call 'dumbing down', that I doubt they REALLY want things made simple.
Let's face it; there is NO WAY you can make any FMC 'simple'. Not in terms of programming, nor in terms of actually operating it!
Same for MANY parts of the program.
That then of course opens the ancient debate: for whom are you/we making the simulator first and foremost !?? ;-)
No, 'simple' is not a word in the vocabulary of FS !
Francois - Anonymous
February 12, 2006
Thanks for the further insight, Mike. Keep these tidbits coming, as it helps us understand the true complexity of this system. To anyone who has done much coding, it's obvious that FS has to be one of the most complex pieces of software around, given all the things that have to be juggled in real time (or nearly so), and the interaction that all these things have.
But even given that, it's not always obvious to an outsider how complex a seemingly simple change can really be, even disregarding costs and testing (which you can't disregard, of course), since the mechanism used isn't apparent.
A good example is the "off runway" suggestion a while back. It seemed from here that a few parameter changes might do the job, but your post on that subject clarified the problems a lot -- there was a lot more to it than was apparent to us.
It's great that you folks are communicating with the community, and giving us more insight into what it really takes. Certainly it's good PR, but it also helps us understand FS better, too. And for many of us, the better we understand the sim, the better use we can make of it.
Thanks,
Larry N. - Anonymous
February 14, 2006
Great post Mike!
I think you should also mention that the new features that are put in also affect the SDKs as well. Lets say for instance the scenery portion... looks like the new roads in the screenshots do a much better job on mountainsides... great! people say.
Of course the Aces team wants to provide an SDK on how to do this. Well unless the feature is hard coded... which really you all want to avoid so designers can add to the FSX platform... then here is another feature that you all have to take a look at and decide how the feature gets implemented... pros and cons.
I work with CRM and early versions had the calendar inaccessible which really did not please the partner designers. Luckily now a new feature calendar also includes the ability to customize the calendar.
I understand what you are talking about and thanks for all the hard work you and the ACES team are doing!
Thanks,
Crashaz - Anonymous
February 16, 2006
The comment has been removed - Anonymous
April 02, 2006
Mike,
Thanks for the insight. I can certainly see where this proces has to be diligently followed in the case of any changes that will result in the end user haveing access to the change, and therefore the ability to mess it up!
However, there's more than one type of request... and some of them have nothing to do with the users, per se. As a gauge writer, my focus is on the tools and abilities afforded to me in order for me to create what's in my mind's eye... and it's on my shoulders to do the usability testing and error handling.
For example:
The XML parser for gauges, if there is more than one parser, has a list of things it can handle. That list determines what's possible and what's not. If there are, say, 100 items in that list then any changes to it shouldn't mess with the existing items because then you run into nightmares with xml that worked last week, but fails next week.
However, what about keeping the original 100 intact and simply adding a dozen or so new things that don't impact the old routines for handling already established capabilities?
IE:
We have the ability to mix fixed text and numerical variables... we also have the ability to do string comps and search for sub-strings within strings that are defined by C++ .gau files. But there is no way to set a text variable within the XML script itself.
I understand that it's dangerous to change the established ways of handling things... but what about adding the ability to store and recall variables of type text?
There is no GUI to deal with, it wouldn't affect any scripts in existance today because none of them assign a text variable and it would be 100% backwards compatable because no functionality would be removed or altered... only added.
The parser could be upgraded in a patch, if it's too late to do anything for FSX. That way, it's on the heads of those of us out here who are writing these scripts to ensure that they function as intended... which means that we'd need to learn the rules about whatever is added, but that's a small price to pay for the kind of things we could create with the new capabilities.
You may have already read the XML wish list for FSX in the FSX forum on avsim... we've been chatting about this very subject within the past week or so.
It's not simple, but it's less difficult than adding in speed controls to all areas of the world! We, the writers, need abilities and tools... leave it to us to create the products, test for usability and error handling.
Those of us who do it well will thrive, and those who write lousy scripts will share the fate of those who write less than acceptable code.
Thanks,
Scott / Vorlin - Anonymous
May 26, 2009
PingBack from http://castironbakeware.info/story.php?title=the-little-wheel-goes-in-back-the-myth-of-it-should-be-simple