How to: Stream Large Amounts of Data from a Web Service

Article
09/13/2006

To improve the performance of an ASP.NET Web service that returns more than 10 MB of data in a SOAP response, the data can be streamed. When the data is not streamed, the entire amount of data that will be sent in the response must be first buffered into the Web service computer's memory.

Streaming the data returned in a SOAP response has the following limitations:

Performance is not improved unless the data is very large - approximately 10 MB or greater.
When message-level security is applied to the SOAP message, streaming provides no benefit, because the entire SOAP response is buffered into memory before the SOAP response is returned to the client to perform the cryptographic operations.
When transport-level security, such as Secure Socket Layers (SSL) is used, streaming can be implemented.
Only SOAP responses from a Web service can be streamed.
WSE does not support the streaming of SOAP requests sent by a client.

To stream large amounts of data from a Web service

Open the Web service project in Microsoft® Visual Studio 2005.
Enable the project to use WSE and the WSE SOAP protocol factory.
1. In Solution Explorer, right-click the project name, and then click WSE Settings 3.0….
2. Select the General tab.
3. Select Enable this project for Web Services Enhancements and Enable Microsoft Web Services Enhancements SOAP Protocol Factory.
Specify that the Web service can accept SOAP messages encoded using MTOM.
1. In Solution Explorer, right-click the project name, and then click WSE Settings 3.0….
2. Select the Messaging tab.
3. Choose optional or always for the MTOM Mode.
  The optional MTOM Mode specifies that WSE processes incoming SOAP messages whether or not they are MTOM encoded and that all SOAP responses and SOAP faults are MTOM encoded.
  The always MTOM Mode specifies that all incoming and outgoing SOAP messages must be MTOM encoded. When a SOAP request is received that is not encoded using MTOM, a SOAP fault is returned to the sender.
4. Click OK to dismiss the dialog box.
  This adds an <mtom> Element to the Web service's Web.config file.

Create a class that implements the System.Xml.IXmlSerializable interface and represents the data that is returned by the Web service method.

Apply the System.Xml.XmlSchemaProvider attribute to the class to specify the name of a static method that returns an XML schema and a XmlQualifiedName that controls the serialization of the type public class.
The following code example applies the System.Xml.XmlSchemaProvider attribute to a class named GetFileResponseWrapper, which implements the System.Xml.IXmlSerializable interface.
```
<XmlSchemaProvider("GetMySchema")> _
Public Class GetFileResponseWrapper
    Implements IXmlSerializable, IDisposable
```
```
[XmlSchemaProvider("GetMySchema")]
public class GetFileResponseWrapper : IXmlSerializable, IDisposable
{
```

Add a static method that implements the method specified in the System.Xml.XmlSchemaProvider attribute.
The following code example returns an XML qualified name for Base64 binary.

Public Shared Function GetMySchema(ByVal xss As XmlSchemaSet) As XmlQualifiedName
    Return New XmlQualifiedName("base64Binary", "http://www.w3.org/2001/XMLSchema")
End Function 'GetMySchema

public static XmlQualifiedName GetMySchema(XmlSchemaSet xss)
{
    return new XmlQualifiedName("base64Binary", "http://www.w3.org/2001/XMLSchema");
}

Implement the System.Xml.IXmlSerializable.WriteXml method, which serializes the data into the SOAP response that is returned to the client.
The following code example serializes a GetFileResponseWrapper instance.

Public Sub WriteXml(ByVal w As XmlWriter) Implements IXmlSerializable.WriteXml
    Dim fs As New FileStream(_fileName, FileMode.Open)
    Dim buf(1023) As Byte
    Dim numRead As Integer = fs.Read(buf, 0, 1024)
    While (numRead >= 0)
        w.WriteBase64(buf, 0, numRead)
        numRead = fs.Read(buf, 0, 1024)
    End While
End Sub 'WriteXml

public void WriteXml(XmlWriter w)
{
    using (FileStream fs = new FileStream(_fileName, FileMode.Open))
    {
        byte[] buf = new byte[1024];
        int numRead = 0;
        while ((numRead = fs.Read(buf, 0, 1024)) > 0)
        {
            w.WriteBase64(buf, 0, numRead);
        }
    }
}

Implement the System.Xml.IXmlSerializable.ReadXml method, which deserializes the data that is received by the client and saves it into a file.
The following code example deserializes a GetFileResponseWrapper instance.

    Public Sub ReadXml(ByVal r As XmlReader) Implements IXmlSerializable.ReadXml
        ' Read the open tag of the encapsulating element
        r.ReadStartElement()

        '
        ' Read the binary data that represents the file contents
        ' into a temp file.
        '
        _fileName = tfc.AddExtension("fileContents", False)
        ReadContentsIntoFile(r, _fileName)

        ' Read the close tag of the encapsulating element
        r.ReadEndElement()
    End Sub 'ReadXml



...


        Sub ReadContentsIntoFile(ByVal r As XmlReader, ByVal fileName As String)
        Dim fs As New FileStream(fileName, FileMode.CreateNew)
        If r.CanReadBinaryContent Then
            Dim buf(1023) As Byte
            Dim numRead As Integer = r.ReadContentAsBase64(buf, 0, 1024)
            While (numRead > 0)
                fs.Write(buf, 0, numRead)
                numRead = r.ReadContentAsBase64(buf, 0, 1024)
            End While
        Else
            Throw New NotSupportedException()
        End If
    End Sub 'ReadContentsIntoFile

    public void ReadXml(XmlReader r)
    {
        // Read the open tag of the encapsulating element
        r.ReadStartElement();

        //
        // Read the binary data that represents the file contents
        // into a temp file.
        //
        _fileName = tfc.AddExtension("fileContents", false);
        ReadContentsIntoFile(r, _fileName);

        // Read the close tag of the encapsulating element
        r.ReadEndElement();
    }



...


        void ReadContentsIntoFile(XmlReader r, string fileName)
    {
        using (FileStream fs = new FileStream(fileName, FileMode.CreateNew))
        {
            if (r.CanReadBinaryContent)
            {
                byte[] buf = new byte[1024];
                int numRead = 0;
                while ((numRead = r.ReadContentAsBase64(buf, 0, 1024)) > 0)
                {
                    fs.Write(buf, 0, numRead);
                }
            }
            else
            {
                throw new NotSupportedException();
            }
        }
    }

Apply a WebMethod attribute with the BufferResponse parameter set to false on the Web service method that you want to stream data from.

The following code example applies the WebMethod attribute with the BufferResponse parameter set to false for the GetFile method.
```
<WebMethod(BufferResponse:=False)> _
Public Function GetFile(ByVal fileName As String) As GetFileResponseWrapper
```
```
[WebMethod(BufferResponse = false)]
public GetFileResponseWrapper GetFile(string fileName)
```
Change the return type for the Web service method that you want to stream to a user-defined class that implements or contains a field of a type that implements the System.Xml.IXmlSerializable interface.

The following code example is a Web service method that returns the previously defined GetFileResponseWrapper, which implements the System.Xml.IXmlSerializable interface.
```
<WebMethod(BufferResponse:=False)> _
Public Function GetFile(ByVal fileName As String) As GetFileResponseWrapper
```
```
[WebMethod(BufferResponse = false)]
public GetFileResponseWrapper GetFile(string fileName)
```
Open the Web service client project in Visual Studio 2005.
Specify that the client sends SOAP messages encoded using MTOM.
1. In Solution Explorer, right-click the project name, and then click WSE Settings 3.0….
2. Select the Messaging tab.
3. Choose On for the Client Mode.
  The On Client Mode specifies that proxy classes generated in the future set use the MTOM encoding by setting the RequireMtom property of the proxy class to true.
  
  Note
  
  When the TCP protocol is used, the RequireMtom property has no affect, as SOAP messages are always sent MTOM encoded.
4. Click OK to dismiss the dialog.
  This adds an <mtom> Element to the client's Web.config file.

Example

The following code example contains a Web service method named GetFile that streams the data that is returned in the SOAP response.

Imports System.CodeDom.Compiler
Imports System.Reflection
Imports System.IO
Imports System.Web
Imports System.Web.Services
Imports System.Web.Services.Protocols

Imports System.Xml
Imports System.Xml.Schema
Imports System.Xml.Serialization

Imports Microsoft.Web.Services3
Imports Microsoft.Web.Services3.Design


<Policy("ServerPolicy")> _
<WebService(Namespace:="http://stockservice.contoso.com/wse/samples/2005/10")> _
<WebServiceBinding(ConformsTo:=WsiProfiles.BasicProfile1_1)> _
Public Class BinaryDataMTOMService
    Inherits System.Web.Services.WebService

    ' This Web service method returns a class that contains a class that represents the data
    ' that is streamed on the Web service side. To stream the data, the class implements the
    ' IXmlSerializable interface.
    ' The client still caches the whole file as a byte array.
    <WebMethod(BufferResponse:=False)> _
    Public Function GetFile(ByVal fileName As String) As GetFileResponseWrapper
        Dim filePath As String = AppDomain.CurrentDomain.BaseDirectory & "App_Data\" & fileName
        Return New GetFileResponseWrapper(filePath)
    End Function 'GetFile

    ' This attribute tells the schema machinery to use the GetMysSchema
    ' method to get the schema for this class.
    <XmlSchemaProvider("GetMySchema")> _
    Public Class GetFileResponseWrapper
        Implements IXmlSerializable, IDisposable
        Private _fileName As String
        Private tfc As TempFileCollection = Nothing


        Public ReadOnly Property FileName() As String
            Get
                Return _fileName
            End Get
        End Property

        Public Sub New()
            MyClass.New(Nothing)
        End Sub 'New


        Public Sub New(ByVal fileName As String)
            _fileName = fileName
            ' Manages the temp file that contains the file contents
            ' Dispose this wrapper to clean up temp files.
            Dim tfc As New TempFileCollection()
        End Sub 'New

        Protected Overloads Overrides Sub Finalize()
            Dispose(False)
            MyBase.Finalize()
        End Sub 'Finalize


        Public Overloads Sub Dispose() Implements IDisposable.Dispose
            Dispose(True)
        End Sub 'Dispose


        Overloads Sub Dispose(ByVal isDisposing As Boolean)
            If isDisposing AndAlso Not (tfc Is Nothing) Then
                tfc.Delete()
            End If
            tfc = Nothing
        End Sub 'Dispose

        '/ <summary>
        '/ The schema for the file contents node is actually just
        '/ base 64 binary data so return the qname of the schema
        '/ type directly.
        '/ </summary>
        Public Shared Function GetMySchema(ByVal xss As XmlSchemaSet) As XmlQualifiedName
            Return New XmlQualifiedName("base64Binary", "http://www.w3.org/2001/XMLSchema")
        End Function 'GetMySchema

        '/ <summary>
        '/ Always return null.
        '/ </summary>
        Public Function GetSchema() As XmlSchema Implements IXmlSerializable.GetSchema
            Return Nothing
        End Function 'GetSchema

        '/ <summary>
        '/ Deserializes state out of an XmlReader
        '/ </summary>
        Public Sub ReadXml(ByVal r As XmlReader) Implements IXmlSerializable.ReadXml
            ' Read the open tag of the encapsulating element
            r.ReadStartElement()

            '
            ' Read the binary data that represents the file contents
            ' into a temp file.
            '
            _fileName = tfc.AddExtension("fileContents", False)
            ReadContentsIntoFile(r, _fileName)

            ' Read the close tag of the encapsulating element
            r.ReadEndElement()
        End Sub 'ReadXml

        '/ <summary>
        '/ Serializes state into an XmlWriter
        '/ </summary>
        Public Sub WriteXml(ByVal w As XmlWriter) Implements IXmlSerializable.WriteXml
            Dim fs As New FileStream(_fileName, FileMode.Open)
            Dim buf(1023) As Byte
            Dim numRead As Integer = fs.Read(buf, 0, 1024)
            While (numRead >= 0)
                w.WriteBase64(buf, 0, numRead)
                numRead = fs.Read(buf, 0, 1024)
            End While
        End Sub 'WriteXml

        Sub ReadContentsIntoFile(ByVal r As XmlReader, ByVal fileName As String)
            Dim fs As New FileStream(fileName, FileMode.CreateNew)
            If r.CanReadBinaryContent Then
                Dim buf(1023) As Byte
                Dim numRead As Integer = r.ReadContentAsBase64(buf, 0, 1024)
                While (numRead > 0)
                    fs.Write(buf, 0, numRead)
                    numRead = r.ReadContentAsBase64(buf, 0, 1024)
                End While
            Else
                Throw New NotSupportedException()
            End If
        End Sub 'ReadContentsIntoFile
    End Class 'GetFileResponseWrapper
End Class

using System;
using System.Collections;
using System.ComponentModel;
using System.CodeDom.Compiler;
using System.Data;
using System.Reflection;
using System.Diagnostics;

using System.IO;
using System.Web;
using System.Web.Services;
using System.Web.Services.Protocols;

using System.Xml;
using System.Xml.Schema;
using System.Xml.Serialization;

using Microsoft.Web.Services3;
using Microsoft.Web.Services3.Design;

[WebService(Namespace = "http://stockservice.contoso.com/wse/samples/2005/10")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[Policy("ServerPolicy")]
public class BinaryDataMTOMService : System.Web.Services.WebService
{
    public BinaryDataMTOMService()
    {
    }
    // This Web service method returns a class that contains a class that represents the data
    // that is streamed on the Web service side. To stream the data, the class implements the
    // IXmlSerializable interface.
    // The client still caches the whole file as a byte array.
    [WebMethod(BufferResponse = false)]
    public GetFileResponseWrapper GetFile(string fileName)
    {
        String filePath = AppDomain.CurrentDomain.BaseDirectory + @"App_Data\" + fileName;
        return new GetFileResponseWrapper(filePath);
    }

    // This attribute tells the schema machinery to use the GetMysSchema
    // method to get the schema for this class.
    [XmlSchemaProvider("GetMySchema")]
    public class GetFileResponseWrapper : IXmlSerializable, IDisposable
    {
        private string _fileName;
        private TempFileCollection tfc = null;

        public string FileName { get { return _fileName; } }

        public GetFileResponseWrapper()
            : this(null)
        {
        }

        public GetFileResponseWrapper(string fileName)
        {
            _fileName = fileName;
            // Manages the temp file that contains the file contents
            // Dispose this wrapper to clean up temp files.
            TempFileCollection tfc = new TempFileCollection();
        }

        #region Dispose logic
        ~GetFileResponseWrapper()
        {
            Dispose(false);
        }

        public void Dispose()
        {
            Dispose(true);
        }

        void Dispose(bool isDisposing)
        {
            if (isDisposing && tfc != null)
                tfc.Delete();
            tfc = null;
        }
        #endregion

        /// <summary>
        /// The schema for the file contents node is actually just
        /// base 64 binary data so return the qname of the schema
        /// type directly.
        /// </summary>
        public static XmlQualifiedName GetMySchema(XmlSchemaSet xss)
        {
            return new XmlQualifiedName("base64Binary", "http://www.w3.org/2001/XMLSchema");
        }
        
        /// <summary>
        /// Always return null.
        /// </summary>
        public XmlSchema GetSchema() { return null; }

        /// <summary>
        /// Deserializes state out of an XmlReader
        /// </summary>
        public void ReadXml(XmlReader r)
        {
            // Read the open tag of the encapsulating element
            r.ReadStartElement();

            //
            // Read the binary data that represents the file contents
            // into a temp file.
            //
            _fileName = tfc.AddExtension("fileContents", false);
            ReadContentsIntoFile(r, _fileName);

            // Read the close tag of the encapsulating element
            r.ReadEndElement();
        }
        
        /// <summary>
        /// Serializes state into an XmlWriter
        /// </summary>
        public void WriteXml(XmlWriter w)
        {
            using (FileStream fs = new FileStream(_fileName, FileMode.Open))
            {
                byte[] buf = new byte[1024];
                int numRead = 0;
                while ((numRead = fs.Read(buf, 0, 1024)) > 0)
                {
                    w.WriteBase64(buf, 0, numRead);
                }
            }
        }
        void ReadContentsIntoFile(XmlReader r, string fileName)
        {
            using (FileStream fs = new FileStream(fileName, FileMode.CreateNew))
            {
                if (r.CanReadBinaryContent)
                {
                    byte[] buf = new byte[1024];
                    int numRead = 0;
                    while ((numRead = r.ReadContentAsBase64(buf, 0, 1024)) > 0)
                    {
                        fs.Write(buf, 0, numRead);
                    }
                }
                else
                {
                    throw new NotSupportedException();
                }
            }
        }

    }
}

Share via

How to: Stream Large Amounts of Data from a Web Service

To stream large amounts of data from a Web service

Example

See Also

Tasks

Additional resources