I invented the Common Query Runtime (CQR) to do for XML query languages what the Common Language Runtime (CLR) does for traditional programming languages -- provide a rich, common infrastructure for compiling and executing XML query and view definition languages. Like the CLR, the CQR features compilers for each XML query language (such as XSLT and XQuery), a common intermediate format (QIL), and multiple backend execution engines (over XML and SQL).
The CQR is implemented in the System.Xml.Query namespace and is slated to appear in Whidbey (the next version of .NET), Yukon (the next version of SQL Server), and Longhorn (the next version of Windows).
Because the CQR isn't yet released, I'm not comfortable saying anything more about it here. However, a couple of books have been published revealing the details here as well as API and examples -- see the Further Reading section below.
- ADO.NET and System Xml v. 2.0 - The Beta Version. Alex Homer, Dave Sussman, and Mark Fussell. Addison Wesley, to appear.
- First Look at ADO.NET and System Xml v 2.0. Alex Homer, Dave Sussman, and Mark Fussell. Addison Wesley, 2003.
- Early Adopter XQuery. Dan Maharry, Mark Fussell, et al. Wrox Press, 2002.
I think that although it's a step in the right direction, the implementation in 2004 falls far short of the (admittedly ambitious) goals we set for the project back in 2002. I probably just need more distance from it, but for now I'm very disappointed with the way the CQR turned out, both the end result and the process that got to there.
My experience designing the CQR and then participating in its creation taught me some important lessons about getting your ideas realized by a larger team, especially when no one reports to you. I hope that most of the unpleasant experiences were specific to the people involved, and not typical of Microsoft as a whole.
I learned that it's critical to own the definition and implementation of the API, and that there is a difficult balance between keeping the team small in the beginning and yet keeping everyone in the loop. I also learned to assign a higher priority to some common-sense things I already knew, like:
- Demo early
- If it's great, everyone wants to see it. But more than this, getting the end result you want is made a lot easier if you have a demo that shows how it needs to work. Test-driven development can do wonders to justify the design choices you've made to the people who end up implementing them, and to help constrain them to implement it the way you wanted it. Early demos can also help prevent over-engineering by requiring parts of the system to work right from the beginning (which implies a simple implementation to start with) and remain working at all times afterwards (which helps prevent massive neverending rewrites/redesigns).
- Accept dependencies late
- If it's great, everyone will want to use it right away, and then they'll be disappointed because it's not done yet. Users hate to wait, but even more they hate to use something before it's ready. And they end up waiting either way.
- Carefully record customer input
- If it's great, it'll last a long time. Commit all those great customer scenarios to writing up front, with enough detail to support the requirements. Otherwise, the team (which may change drastically in personnel over time) will inevitably lose sight of the end goals and stray from the path.