The Problem with Dates, Timezones and XmlConvert

A recent post by Brad Abrams on DateTime, Serialization and TimeZones highlights one of the problems with the the System.Datetime structure. The best way to illustrate his problem is with a code fragment and the output from the code

DateTime now = DateTime.Now; 
DateTime now2 = now.ToUniversalTime(); 
    
Console.WriteLine(now);
Console.WriteLine(now2);

Console.WriteLine(XmlConvert.ToString(now));
Console.WriteLine(XmlConvert.ToString(now2));

which results in the following output

4/14/2004 9:29:31 AM
4/14/2004 4:29:31 PM
2004-04-14T09:29:31.9531250-07:00
2004-04-14T16:29:31.9531250-07:00

which at first seems right until you consider that all the values should reflect the same instant in time but do not. The problem is that the DateTime structure does not store time zone information which causes all sorts of problems when trying to transmit date or time values across the network or persist them. I've had to deal with this because I'm responsible for the System.Xml.XmlConvert class and have been working with folks that are responsible for XML Serialization, the DataSet and the CLR team as to how best to work around this problem. As Brad Abrams mentions in his blog post 

You can mark a DateTime instance as Local, UTC or Unspecified, and Unspecified means that it should not be serialized with a time zone. Unfortunately it won't magically take effect because it would break compatibility. But there will be ways to opt-in to this behavior for DataSet, XML Serialization and XML Convert.

 so there won't be time zone information added to the type but at the very least it should be possible to get my code fragment above to emit equivalent times in Whidbey. So time zone information still won't be supported but you can always normalize to UTC and have this preserved correctly instead of what happens now.

This is what has been proposed and I'm in the process of working out exactly how this should work for XmlConvert. We will most likely add overloads to the ToString method that takes a DateTime that allows people to control whether they want the backwards compatible (but incorrect) v1.0 behavior or the correct behavior when outputing a UTC date.

Comments

• Anonymous
  April 14, 2004
  Dare thanks for clearing this up and expressing your concerns. I’m sorry to ask (from a customer perspective), will this fix/workaround appear in .NET 1.0 & 1.1?
• Anonymous
  April 14, 2004
  (posted to both Dare's blog and Brad's blog)
  
  To be clear - fixes in this area are needed WELL in advance of Whidbey.
  
  This need cannot be overstated, as there are pre-Whidbey applications that simply cannot rely on DateTime's various local-time-related ambiguities, due to issues with savings time fallback, etc. nor can those applications wait until Whidbey in order to be released.
  
  Such applications must work with UTC DateTimes, but even that doesn't entirely work because the Framework still considers those times to be local, re-exposing the application to the same set of ambiguities. Once XmlConvert is added into the equation the problem is again multiplied, causing any internal standardization on UTC DateTimes to be wrecked by XmlConvert's behaviour.
  
  We're not exactly in "missing feature" territory here: this is well off into "critical short-sightedness" territory and as such should be considered a bug.
  
  If a change to this behaviour causes breaking changes, it will only break applications that are themselves aleady broken - coded on top of an outstanding known issue. Why do I consider these apps (including mine) to already be broken? Because there is presently no way to use DateTime without exposing one's self to these ambiguities.
  
  I frankly can't believe that fixes to these issues aren't occurring earlier than the Whidbey timeframe. Since there are definitely numerous .NET applications that ship DateTimes across time zones and/or have to keep running across savings time fallback windows, I can only assume that none of these applications are considered mission-critical enough to justify fixing these bugs. :(
• Anonymous
  April 15, 2004
  Please add this to your list of critical XML serialization/ deserialization fixes:
  
  When deserializing to a DateTime xml deserialization uses :
  DateTime.ParseExact
  
  The problem is that date time data in xml and SOAP messages is flexible.
  My current example is:
  
  this format string
  "yyyy-MM-dd'T'HH:mm:ss.fffzzzz" reads the time produced by the java serializer (3 digit millisecond)
  
  But the .Net output time uses
  "yyyy-MM-dd'T'HH:mm:ss.fffffffzzzz"
  
  so I'm guessing that it expects that back.
  
  The xml deserializer CAN NOT USE ParseExact
  the contents of an XML date time filed are far too variable.
  
  I agreee with the previous post. These are critical, must have now fixes. I can't imagine reccomending this technology to anyone to support web services until these datetime issues are fixed.
  
  
  
• Anonymous
  April 20, 2004
  I am a colleague of Brad's and I'm development owner of DateTime. I wanted to respond to the feedback on this thread, particularly the excellent feedback from Jeremy Gray and Mark Singletary. I also responded to Jeremy's identical post on Brad's blog.
  
  Serialization in XML is the Number 1 complaint associated with the DateTime. It is definitely something we are committed to fixing in Whidbey.
  
  Regarding the request for a servicing release. Introducing new functionality into our V1.0 and V1.1 releases is always significantly more risky and expensive than a regular release. Feedback such as you are providing here is very useful in convincing people that the extra cost and risk is justified. In fact the more specific business impact and adoption impact from as many distinct sources as I can get is useful in trying to get all the right teams and individuals bought in that this work should be done. If you know others impacted by this, or you can provide more information about its impact on you, that would be helpful in making something happen. Please contact me (amoore@microsoft.com) if there is more feedback you can provide. My personal opinion is that I agree with you and we should do something for V1.1 at the least.
  
  Regarding compatibility: the only circumstances when I see customers significantly more inflamed than you are right now about this is not when we have bugs in our product, but when fixing bugs causes reasonably functioning code to break. And compatibility issues often go directly to impacting your users and customers as they upgrade their operating systems or runtimes, or even when they install service packs. Unfortunately it is easy to take a dependency on the weird way DateTime serializes in XML, so we have to be very careful about how we introduce fixes in Whidbey, and we have to be even more careful about anything we consider in servicing, which has an even higher compatibility bar.
  
  Regarding ParseExact: I don't quite understand the problem with ParseExact. While the shorter overload of DateTime.ParseExact limits you to one format, the overload that XmlConvert calls is the one that takes an array of format strings. It passes in 24 different formats corresponding to various types of XML dates, and it includes both of the formats you mentioned in this response, including the one from the Java serialize.
  
  Mark, I'm curious to see if the reason you think XmlConvert is broken here is because it is using ParseExact, or because of the problems serializing date-only or UTC times.
  
  
• Anonymous
  April 20, 2004
  Paul,
  So far there isn't a plan to try and fix these issues either in the XmlConvert or DataSet classes in Everett. The XmlSerializer is considering a fix for Everett but nothing is set in stone at this time.
• Anonymous
  April 21, 2004
  Thank you for responding, hope you guys reconsider because 1.1 is here to stay for at least a few more years.
• Anonymous
  December 12, 2006
  The different systems in Commerce Server take somewhat different approaches for how they store DateTime