XQuery 1.0: An XML Query Language
https://www.w3.org/TR/2005/CR-xquery-20051103/

XQuery 1.0 and XPath 2.0 Functions and Operators
https://www.w3.org/TR/2005/CR-xpath-functions-20051103/

XQuery 1.0 and XPath 2.0 Data Model (XDM)
https://www.w3.org/TR/2005/CR-xpath-datamodel-20051103/

XQuery 1.0 and XPath 2.0 Formal Semantics
https://www.w3.org/TR/2005/CR-xquery-semantics-20051103/

XSLT 2.0 and XQuery 1.0 Serialization
https://www.w3.org/TR/2005/CR-xslt-xquery-serialization-20051103/

<foo id="1">foo 1</foo>
<foo id="2">foo 2</foo>
<foo id="3">foo 3</foo>

Suppose we have some client side logic where the user is able to choose which of the 3 foo elements that want to return. The XQuery expression used in this case is going to depend on the particular item requested and will not be known until the user makes their decision (i.e. it can not simply be hard-coded as a string). Since the XML data-type methods require a string constant as their input, there is no way to directly specify dynamic XQuery expressions. There are some workarounds for this problem, depending on exactly what part of the query is dynamic.

The easiest option is where the query simply needs to be parametrized. As is the case with the example above, the dynamic part of the query simply consists of a constant value that is different for each query execution. The easiest way to parametrize a query is to use the sql:variable() XQuery function and store the parameter value in the associated T-SQL variable. For example:

declare @index int
set @index=2

select data.query( N'
/foo[position()=sql:variable("@index")]
 ') from test

In this case, the index for the required item is stored in the T-SQL variable @index and this is then referenced in the XQuery expression through the sql:variable() function. Notice that we can not simply use the value of the sql:variable() function as an ordinal predicate - rather, we need to explicitly compare it's value to that of the position() function. This is due to a limitation of the XQuery implementation inside SQL Server 2005 that requires ordinal predicates to be numeric literal values (i.e. they can not be something that changes at query execution time).

While the parametrization of queries is a powerful tool, it does not solve the general problem of being able to execute truly dynamic XQuery expressions. In order to do this, you need to use the T-SQL exec function. The T-SQL exec function allows any string value to be executed as a T-SQL expression. For example:

declare @query nvarchar(200);

set @query=N'
select data.query( N''
/foo[1]
 '') from test1'

exec( @query )

The part of the query marked in red is the dynamic XQuery expression. Notice that the entire T-SQL expression (i.e. including the select ... from part of the statement) needs to be included in the dynamic query string. Also, notice that the ' characters that are used to contain the XQuery part of the expression need to be escaped (as '') in order to make the query string valid.

While this is not an ideal solution to the problem of dynamic queries, most cases can be handled by the parametrization option and the exec function should only be used as a last resort.

-
Disclaimer:
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

create xml schema collection attribute_types as N'
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="https://www.w3.org/2001/XMLSchema">	
	<xs:element name="root">
		<xs:complexType>
			<xs:choice maxOccurs="unbounded">
				<xs:element name="pointer">
					<xs:complexType>
						<xs:choice>
							<xs:element name="sibling" type="xs:string" />
						</xs:choice>
						<xs:attribute name="singleton_attribute" type="xs:IDREF" />
						<xs:attribute name="list_attribute" type="xs:IDREFS" />
					</xs:complexType>
				</xs:element>
				<xs:element name="target">
					<xs:complexType>
						<xs:attribute name="ID" type="xs:ID" />
					</xs:complexType>
				</xs:element>
			</xs:choice>
		</xs:complexType>
	</xs:element>
</xs:schema>'

In this case, we have defined a schema for a piece of XML that uses ID and IDREF types to create links between different elements. The important thing to notice here is the difference between the types of the singleton_attribute and list_attribute attributes of the pointer element. Note that the singleton_attribute has a type of xs:IDREF (implying a single IDREF value), while the list_attribute has a type xs:IDREFS (implying a list of IDREF values). Another key piece of information is the way in which list types are handled in XQuery. XQuery does not have a concept of list types as such, and handles these by simply treating their values as a sequence of singleton simple types. This is best illustrated by examining the static types of some queries over an instance of this data. Taking the following XML instance:

declare @x xml(attribute_types)
set @x='
<root>
	<pointer singleton_attribute="A">
		<sibling>test</sibling>
	</pointer>
	<pointer list_attribute="A B">
		<sibling>test</sibling>
	</pointer>

	<target ID="A"/>
	<target ID="B"/>
</root>'

If we execute a query to select the value of a single instance of the singleton_attribute:

select @x.query('
data( (/root/pointer)[1]/@singleton_attribute )
')

Examining the static type generated by this query, we get the following - xs:IDREF ? i.e. a single instance (optional) of an IDREF value. If we now execute a query to select the value of a single instance of the list_attribute:

select @x.query('
data( (/root/pointer)[2]/@list_attribute )
')

Examining the static type generated by this query, we get the following - xs:IDREF * i.e. multiple (optional) IDREF values. So, the result of selecting a single attribute of an XSD list type is a sequence of values, not a singleton. This is very useful since we can now use standard XQuery semantics (e.g. the ordinal predicate or FLWOR iteration) to navigate through this list of values and find the one we want.

This all works fine since the data function is able to handle sequences of values. But, remember that a lot of the functions and operators in XQuery require a singleton value and these will fail if applied to the value(s) of the list_attribute.

Now, remembering that the parent axis causes a loss of type fidelity and understanding that the above semantics require that the type of the attribute be known - what happens when we combine the parent axis with these attributes. Taking the following query against the singleton_attribute:

select @x.query('
data( (/root/pointer)[1]/sibling/../@singleton_attribute )
')

If we examine the static type generated by this query, we now get the following: xdt:anyAtomicType *. Two things to notice here, firstly we have lots type fidelity (IDREF has become  anyAtomicType) and secondly we have gone from an (optional) singleton value to a sequence! In fact, if we run a similar query against the list_attribute:

select @x.query('
data( (/root/pointer)[2]/sibling/../@list_attribute )
')

We get the exact same static type generated! The reasoning behind this is that, since we have lost all type fidelity by using the parent axis and there is a possibility that the attribute selected was a list type, we assume for the purposes of static typing that the result is a sequence i.e. we have lost the fact the the singleton_attribute was a singleton value!

This in and of itself is a problem, but (in most cases) the operations defined to deal with IDREF values can handle sequences of values, so this problem is moot in this case. But, armed with our knowledge of how the parent axis affects both the type and cardinality of attribute types, take the following new XSD schema:

create xml schema collection attribute_types2 as N'
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="https://www.w3.org/2001/XMLSchema">	
	<xs:element name="root">
		<xs:complexType>
			<xs:choice maxOccurs="unbounded">
				<xs:element name="value">
					<xs:complexType>
						<xs:choice>
							<xs:element name="sibling" type="xs:string" />
						</xs:choice>
						<xs:attribute name="integer" type="xs:integer" />
					</xs:complexType>
				</xs:element>
			</xs:choice>
		</xs:complexType>
	</xs:element>
</xs:schema>'

In this case, we have an attribute that stores a single integer value. Taking the following instance of the above schema:

declare @x xml(attribute_types2)
set @x='
<root>
	<value integer="12">
		<sibling>test</sibling>
	</value>
</root>
'

We can use this attribute value in conjunction with the arithmetic operators to do some calculations:

select @x.query('
(/root/value)[1]/@integer + 10
')

This works fine and returns the expected result of 22. Now, if we add in a parent axis to the mix:

select @x.query('
(/root/value)[1]/sibling/../@integer + 10
')

We get the following error message returned:

Msg 2389, Level 16, State 1, Line 11
XQuery [query()]: '+' requires a singleton (or empty sequence), found operand of type 'xdt:anyAtomicType *'

So, even though we had no list types in our schema at all. Because of the loss of typing fidelity from the parent operator and the necessary assumption that any attribute might be a list type in this case, we loose cardinality fidelity too.

Again, the moral of the story is to use the parent operator with extreme caution :)

-
Disclaimer:
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

To illustrate this, we create a simple trigger on our database that fires when we create a new table:

create trigger CreateTableTrigger on database for create_table as 
select eventdata()

In this case, our trigger is simply outputting the result of the eventdata() function each time a table is created. If we now create a table called Test in this database, we get the following XML instance returned:

<EVENT_INSTANCE>
  <EventType>CREATE_TABLE</EventType>
  <PostTime>...</PostTime>
  <SPID>51</SPID>
  <ServerName>...</ServerName>
  <LoginName>...</LoginName>
  <UserName>dbo</UserName>
  <DatabaseName>master</DatabaseName>
  <SchemaName>dbo</SchemaName>
  <ObjectName>Test</ObjectName>
  <ObjectType>TABLE</ObjectType>
  <TSQLCommand>
    <SetOptions ANSI_NULLS="ON" ANSI_NULL_DEFAULT="ON" ANSI_PADDING="ON" QUOTED_IDENTIFIER="ON" ENCRYPTED="FALSE" />
    <CommandText>create table [Test](c1 int )</CommandText>
  </TSQLCommand>
</EVENT_INSTANCE>

I have removed some of the information from the XML for security reasons :) Basically, we see from this that the eventdata() function, called from within a trigger, returns an instance of the XML data-type. This means that we are free to go ahead and apply any of the XML data-type methods to this in order to customize the output from our trigger. If we drop the trigger we created earlier and create a new one with the following definition:

drop trigger CreateTableTrigger on database
go

create trigger CreateTableTrigger on database for create_table as 
select eventdata().value('( /EVENT_INSTANCE/ObjectName )[1]', 'nvarchar(max)')

Now, if we create a new table (say Test2), we get the following output:

Test2

Which is exactly what we expect if we run the value() method above over the XML returned from the event.

-
Disclaimer:
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

Firstly, we create a simple schema with some typed elements that we will later query:

create xml schema collection small_type as N'
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="https://www.w3.org/2001/XMLSchema">
	<xs:element name="small">
		<xs:complexType>
			<xs:sequence>
				<xs:element maxOccurs="unbounded" name="item">
					<xs:complexType mixed="true">
						<xs:sequence>
							<xs:element name="embed-string" type="xs:string" nillable="true"/>
							<xs:element name="embed-int" type="xs:int" />
						</xs:sequence>
						<xs:attribute name="index" type="xs:unsignedByte" use="required" />
					</xs:complexType>
				</xs:element>
			</xs:sequence>
		</xs:complexType>
	</xs:element>
</xs:schema>'

Then we create a table and insert some instance data:

create table small_typed( id int IDENTITY PRIMARY KEY, xml_data xml(small_type), empty int)
go

insert into small_typed( xml_data ) values ('
<small>
	<item index="1">
		Item 1
		<embed-string>String Item 1</embed-string>
		<embed-int>1</embed-int>
	</item>
</small>
')

The next step is to set up an XQuery static type SQL profiler trace so we can examine the effects of various path expressions on the static type of the query. Once that is done, we execute the following path expression:

select xml_data.exist('
/small/item/embed-string
') from small_typed

From the SQL profiler trace, we see that the static type of this expression is:

element(embed-string,xs:string) *

Which is exactly what we expected, based on our XSD schema. Lets try another query, this time targeting the index attribute of the item element:

select xml_data.exist('
/small/item/@index
') from small_typed

In this case, the static type returned is the following:

attribute(index,xs:unsignedByte) *

Again, precisely what we expected based on our XSD schema. Now, throwing the parent axis into the mix, we execute the following query:

select xml_data.exist('
/small/item/embed-string/../@index
') from small_typed

If we take a look at the static type that results from this query, we now get the following:

attribute(index,xdt:anyAtomicType) *

Notice that the static type of the expression is now the generic xdt:anyAtomicType type, instead of xs:unsignedByte (which is what we would expect based on our XSD schema!). This loss of typing has consequences when the result of these expressions are used for further processing e.g. if they are passed through to functions.

Consider the following query which uses the XQuery number() function to extract and parse the value of the index attribute:

select xml_data.query('
number( (/small/item)[1]/@index )
') from small_typed

This query executes fine. We now alter the path expression to use the parent axis as follows:

select xml_data.query('
number( (/small/item/embed-string/..)[1]/@index )
') from small_typed

We now get the following error message:

Msg 2364, Level 16, State 1, Line 2
XQuery [small_typed.xml_data.query()]: Cannot implicitly convert from 'xdt:anyAtomicType ?' to 'xs:double'

Essentially, the use of the parent axis (and it's associated loss of typing) can mean that the resultant static type precludes the use of the expression with many of the XQuery functions and other expressions that are type dependent. As a further example of this, consider the following query where we use the + operator on the value of the index attribute:

select xml_data.query('
(/small/item)[1]/@index + 1
') from small_typed

This query executes and returns the expected results. Once again, if we alter the path expression to include the parent operator as follows:

select xml_data.query('
(/small/item/embed-string/..)[1]/@index + 1
') from small_typed

The query now returns the following error message:

Msg 9308, Level 16, State 1, Line 2
XQuery [small_typed.xml_data.query()]: The argument of '+' must be of a single numeric primitive type or 'https://www.w3.org/2004/07/xpath-datatypes#untypedAtomic'. Found argument of type 'xdt:anyAtomicType ?'.

The moral of the story - use the parent axis with caution :)

-
Disclaimer:
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

To illustrate this, I will use an example from our customer message boards. This particular case involved extracting the individual values from a list-typed simple value, using the nodes() method to create a separate relational row for each entry in the list. To start off with, I will create a simple schema which defines a the value of the foo element to be a string based list type:

create xml schema collection listExample as '
<xs:schema xmlns:xs="https://www.w3.org/2001/XMLSchema">
<xs:element name="foo">
	<xs:simpleType>
		<xs:list itemType="xs:string" />
	</xs:simpleType>
</xs:element>
</xs:schema>'

We now create an instance of the xml data-type using this schema and some values:

declare @xml_data xml(listExample)
set @xml_data='<foo>Entry1 Entry2 Entry3</foo>'

The XQuery data() function will extract each individual entry from the list type. Thus, if we run the following query, we get the 3 'entry' items returned:

select @xml_data.query( 'data(/foo)' )

At this point, we would simply execute the nodes method over this XQuery expression to extract each of the individual values into a separate relational row. For example, using the following query:

select ref.value('.', 'nvarchar(100)')
from @xml_data.nodes('data(/foo)') T(ref)

Unfortunately, if you try to execute this query, you get the following error message returned:

Msg 2374, Level 16, State 1, Line 8
XQuery [nodes()]: A node or set of nodes is required for 'nodes()'

Since the data() function causes the creation of new items which are not stored in the base XML data-type instance, we are unable to execute the nodes() method over the result of this expression! In order to run a nodes() method over this, we have to construct a new XML data-type instance in a sub-query and run nodes over the result of that. This can be done as follows:

select T.ref.value('.', 'nvarchar(100)')
from 
(
      select [Xml]=@xml_data.query('
			for $i in data(/foo) return 
			element temp { $i }
		')
) A
cross apply A.Xml.nodes('/temp') T(ref)

The sub-query (shown in green) extracts the items out of the list-typed element node foo, returning each individual item as the text node of a newly constructed element named temp. Since the result of the query expression is a new XML data-type instance, we can now run nodes() over this sub-query to get the desired results.

-
Disclaimer:
This posting is provided “AS IS” with no waranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

Results Instance

<results>
	<item name="First Item" section-id="Section1" result="PASS" />
	<item name="Second Item" section-id="Section1" result="PASS" />
	<item name="Third Item" section-id="Section2" result="PASS" />
	<item name="Fourth Item" section-id="Section3" result="FAIL" />
	<item name="Fifth Item" section-id="Section3" result="PASS" />
	<item name="Sixth Item" section-id="Section3" result="FAIL" />
</results>

Sections Instance

<sections>
	<section name="First Section" id="Section1" />
	<section name="Second Section" id="Section2" />
	<section name="Third Section" id="Section3" />
</sections>

The report we would like to produce over this data shows the results of each tests, along with a detailed description of the section of the specification that the test targets. For example, the final data we want may look like the following:

<result name="First Item" section="First Section">PASS</result>
<result name="Second Item" section="First Section">PASS</result>
<result name="Third Item" section="Second Section">PASS</result>
<result name="Fourth Item" section="Third Section">FAIL</result>
<result name="Fifth Item" section="Third Section">PASS</result>
<result name="Sixth Item" section="Third Section">FAIL</result>

In order to do this, we essentially need to perform a join across the two instances. Using standard XQuery and the document function, we could write a FLWOR expression to do this, along the lines of the following:

let $sections := document("sections.xml")/sections
for $result in document("results.xml")/results/item
return
<result 
	name="{$result/@name}" 
	section="{$sections/section[id=$result/@id]/@name}">
{ $result/text() }
</result>

Unfortunately, XQuery in SQL Server 2005 will not execute this query. Firstly, we do not support the let syntax. Secondly, there is no document() (or equivalent) function. So, we need to re-write this query to use T-SQL to get around the limitations of our XQuery implementation. In order to perform the join operation, we need to extract the relevant fields out of the XML instances and turn them into relational data, so we can use them in the standard T-SQL context - the recommended method for doing this is a combination of the the value() method for converting the XML into a relational type, and the nodes() method for extracting the multiple values from the single XML instance.

We use nodes() to extract the multiple item and section elements from the pair of XML instances. Once these are extracted, we can use the value() method on each individual entry to convert it to a relational value, using these to perform the join. The query to do this is as follows:

select R.i.value('@name', 'varchar(30)')   [Name], 
       S.s.value('@name', 'varchar(30)')   [Section],
       R.i.value('@result', 'varchar(30)') [Result]
from Results  cross apply xml_data.nodes('/results/item') R(i),
     Sections cross apply xml_data.nodes('/sections/section') S(s)
where R.i.value('@section-id', 'varchar(30)') = S.s.value('@id', 'varchar(30)')

The join operation in this case is being done through the where clause of the select statement. Running this query will produce the following relational result set:

First Item	First Section	PASS
Second Item	First Section	PASS
Third Item	Second Section	PASS
Fourth Item	Third Section	FAIL
Fifth Item	Third Section	PASS
Sixth Item	Third Section	FAIL

If a relational results is what you wanted, then you are done :) The initial goal was to output XML though, so we can add a for xml statement to this query to take these relational results and re-compose them back into XML. The query including the for xml statement is as follows:

select R.i.value('@name', 'varchar(30)')   [@name], 
       S.s.value('@name', 'varchar(30)')   [@section],
       R.i.value('@result', 'varchar(30)') [text()]
from Results  cross apply xml_data.nodes('/results/item') R(i),
     Sections cross apply xml_data.nodes('/sections/section') S(s)
where R.i.value('@section-id', 'varchar(30)') = S.s.value('@id', 'varchar(30)')
for xml path('result'), type

This query then produces the expected results from above. The for xml statement itself is pretty self-explanatory. The string (result) given as the path parameter is the name of the container element, while the column names of the various values indicate how they will be stored. Notice that the column names for @name and @section correspond to the name and section attributes respectively, while the text() column becomes the text value of the element.

Thanks again to Jinghao for help with the for xml part of the query.

-
Disclaimer:
This posting is provided “AS IS” with no waranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.

The first issue I will deal with is the concept of a default namespace. In XML 1.0, if you used an xmlns attribute without an associated name, all elements under that scope were considered to be in the namespace referenced by the associated URI. Notice I said all elements - attributes are not affected by default namespaces in XML. This functionality is mirrored in XQuery, either through the use of the declare default element namespace prolog entry, or by directly constructing the xmlns attribute on your constructed XML. Again, in XQuery the default element namespace does not affect non-element data i.e. attributes, function names and variables (all of which may have associated namespace prefixes) must be explicitly namespaced if required (NOTE: SQL Server 2005 XQuery does not currently allow variables to be placed into a namespace, even though this is allowed in the XQuery specification). Function names in XQuery have their own default namespace entry, accessible through the declare default function namespace prolog option.

In XQuery a previously declared namespace prefix can be used to add a namespace to elements, attributes, function calls and variables. Unlike the default options, the same namespace prefix can be used to apply to all 4 different identifier types.

Lastly, when doing XML construction, the order that a namespace is declared in does not affect the scope, since attribute order in XML is considered non normative. Take the following XQuery expression as an example:

<foo:element foo:index="1" xmlns:foo="https://example.com/foo" />

In this simple case, we have constructed an element in the foo namespace, containing an attribute which is also in the foo namespace. Notice that we were able to use the foo namespace prefix even before it was technically defined (by the xmlns attribute). In this case, the result of the query was pretty obvious. Now, consider the following XQuery:

declare namespace foo="https://example.com/foo1";
<foo:element xmlns:foo="https://example.com/foo2" />

In this case, we initially declare that the namespace prefix foo points to the URI https://example.com/foo1. But, when we construct the new element, the namespace prefix foo is redefined to point to the URI https://example.com/foo2, which is the URI used for the constructed XML element. Again, these results are pretty obvious, but the trick comes when we combine XML construction with XQuery expressions to generate dynamic content. Consider the following query:

declare namespace foo="https://example.com/foo1";
<foo:element index="{/foo:index}" xmlns:foo="https://example.com/foo2" />

Initially, the namespace prefix foo is declared to point to the URI https://example.com/foo1. The element constructor in the query contains an attribute constructor for the attribute index which uses an XQuery expression to dynamically calculate the value for this attribute. Inside this dynamic attribute content expression, the namespace foo is referenced. Even though the redefinition of the foo namespace prefix technically occurs after the XQuery expression used to generate the contents of the index attribute, since the scope of the redefinition applies to the entire element constructor, within the dynamic attribute expression, the namespace prefix foo is defined to point to the URI https://example.com/foo2!

The easiest thing is to avoid such inconsistencies by placing any namespace prefix definitions as the first attributes in an element constructor. Thus, the order of assignment of URI's to namespace prefixes will be more immediately apparent.

-
Disclaimer:
This posting is provided “AS IS” with no waranties, and confers no rights.
Use of included script samples are subject to the terms specified at https://www.microsoft.com/info/cpyright.htm.