==================

When using BizTalk WCF-SQL adapter to retrieve data from
SQL, there is a known issue that a blank namespace may be generated when a FOR
XML clause stored procedure is used. For example, the empty xmlns="" on FTP_KC_SERIAL_NO_HEAD in the
following XML instance.

<?xml version="1.0" encoding="utf-8"
?>

- <Root xmlns="http://TestEDI.SP_GetDataFromFTPKCSerialNo">

- <FTP_KC_SERIAL_NO_HEAD xmlns="">

  <INV_NO>30111112173</INV_NO>

  <ImportDate>8/16/2011 3:05:43
PM</ImportDate>

  <Status>processing</Status>

  <SendDate>8/16/2011 3:35:21
PM</SendDate>

  <FROM_DUNS>12490306</FROM_DUNS>

Problem Analysis

================

There are various of approaches to resolve the issue, a
simple workaround may be to set “FORM = UNQUALIFIED” on the message schema to
ignore it.

However sometimes this kind of blank namespaces will cause
errors within map. In this case, you may consider writing a simple pipeline
component to completely remove it from the XML message body.

Problem Resolution

=================

Sample code snippet of pipeline component to remove blank
xml namespaces.

        #region IComponent Members

        /// <summary>

        /// This method Executes
the pipeline component

        /// </summary>

        /// <param name="pContext">pipelinecontext Interface</param>

        /// <param name="pInMsg">the incoming XLang Message on which we have to operate</param>

        /// <returns></returns>

        public IBaseMessage Execute(IPipelineContext
pContext, IBaseMessage pInMsg)

        {

           
//Validate parameters

           
if (pContext == null)
throw new ArgumentNullException("pContext");

           
if (pInMsg == null)
throw new ArgumentNullException("pInMsg");

           
this.pipelineContext = pContext;

           
this.baseMessage = pInMsg;

           
Stream outStream = null;

           
string partName;

           
for (int i = 0;
i < baseMessage.PartCount; i++)

           
{

               
partName = null;

               
IBaseMessagePart part = baseMessage.GetPartByIndex(i, out
partName);

               
RemoveBlankNamespace(part.GetOriginalDataStream(), out
outStream);

               
outStream.Seek(0, SeekOrigin.Begin);

               
part.Data = outStream;

           
}

           
return baseMessage;

        }

       
#endregion

        #region Private-Methods

        /// <summary>

        /// This method remove
blank xml namespace attribute: xmlns="" from input message body
stream.

        /// </summary>

        /// <param name="xmlData">the actual stream of the data needs to be manipulated</param>

        /// <param name="modifiedXmlData">the modifiedStream is passed as a out parameter</param>

        /// <returns></returns>

        private void
RemoveBlankNamespace(Stream xmlData, out Stream
modifiedXmlData)

        {

           
modifiedXmlData = new MemoryStream();

           
StreamReader reader = new StreamReader(xmlData);

           
String messageBody = reader.ReadToEnd();

           
messageBody = messageBody.Replace("
xmlns=\"\"","");

           
byte[] messageBytes = System.Text.Encoding.UTF8.GetBytes(messageBody);

   
        modifiedXmlData.Write(messageBytes,
0, messageBytes.Length);

        }

       
#endregion

More Information

=================

A relevant forum thread:

Namespace appear in the child node when using FOR XML output
in BizTalk server 2009 and the map fails

http://social.msdn.microsoft.com/Forums/is/biztalkr2adapters/thread/f996af8b-1056-4a62-8628-d3845fa36e96

==================

When configuring the new feature FTPS adapter (receive or send) to connect FTP server in BizTalk Server 2010, there may raise the following error in the event log on BizTalk Server:

“The client and server cannot communicate, because they do not possess a common algorithm”

Problem Analysis

================

We collect dump files forBTSNTSvc.exe when reproducing this issue, and we have the following information.

The status code 0x80090331 means SEC_E_ALGORITHM_MISMATCH.

In addition, after we look into the source code, from the parameters which was passed to sspicli!AcquireCredentialsHandleW by the BizTalk xceedftpbiztalk module, we could see BizTalk FTPS only support TLS v1. So the problem is caused by that TLS v1 is disabled in the BizTalk Server box.

0:027> p

Time Travel Position:
FE3840000296.

eax=80090331 ebx=00000000
ecx=01a55038 edx=00000001 esi=5d02a0e8 edi=01a54c88

eip=74b3151e esp=254cebd4
ebp=254cebd4 iopl=0         nv up ei pl
zr na pe nc

cs=0023  ss=002b 
ds=002b  es=002b  fs=0053 
gs=002b            
efl=00000246

sspicli!AcquireCredentialsHandleW+0x27:

74b3151e
5d             
pop     ebp

0:027> !gle

LastErrorValue: (Win32) 0x1
(1) - Incorrect function.

LastStatusValue: (NTSTATUS) 0x80090331 - The client and server cannot
communicate, because they do not possess a common algorithm.

0:027> gu

Time Travel Position:
FE384000006A.

eax=80090331 ebx=00000000
ecx=768d0175 edx=1bf364e0 esi=00000000 edi=00000000

eip=74b253d8 esp=254cea84
ebp=254ceb10 iopl=0         nv up ei pl
nz na pe nc

cs=0023  ss=002b 
ds=002b  es=002b  fs=0053 
gs=002b            
efl=00000206

sspicli!SspipAcquireCredentialsHandle+0xf9:

74b253d8
8945e0         
mov     dword ptr [ebp-20h],eax ss:002b:254ceaf0=254ceb10

0:027> k

ChildEBP RetAddr 

254ceb10 74b2dccf sspicli!SspipAcquireCredentialsHandle+0xf9
[d:\w7rtm\minio\security\base\lsa\security\sspiwrap.cxx @ 196]

254ceb68 74b3030b
sspicli!LsaAcquireCredentialsHandleW+0x8e [d:\w7rtm\minio\security\base\lsa\security\sspicli\lsastubs.cxx
@ 581]

254ceba4 74b3151e
sspicli!AcquireCredentialsHandleCommon+0xce
[d:\w7rtm\minio\security\base\lsa\security\sspicli\newstubs.cxx @ 592]

254cebd4 5d01fc86
sspicli!AcquireCredentialsHandleW+0x27 [d:\w7rtm\minio\security\base\lsa\security\sspicli\newstubs.cxx
@ 463]

254cec54 5d02037d
xceedftpbiztalk!CXwlSslSocketWrapper::GetClientCredentials+0xc9
[d:\bt\9359137\private\source\runtime\msg\adapters\ftp\xceedftp\winsock\xwlsslsocketwrapper.cpp
@ 183]

254cec80 5d0128dd
xceedftpbiztalk!CXwlSslSocketWrapper::SetupTLS+0x51
[d:\bt\9359137\private\source\runtime\msg\adapters\ftp\xceedftp\winsock\xwlsslsocketwrapper.cpp
@ 303]

254ced18 5d0100c5
xceedftpbiztalk!CXceedFtp::ConnectPerformer+0x341 [d:\bt\9359137\private\source\runtime\msg\adapters\ftp\xceedftp\activex\xceedftpperformers.cpp
@ 473]

254ced60 1be164d6
xceedftpbiztalk!CXceedFtp::Connect+0xf3
[d:\bt\9359137\private\source\runtime\msg\adapters\ftp\xceedftp\activex\xceedftpmethods.cpp
@ 332]

254cedcc 1be16309
DomainBoundILStubClass.IL_STUB_CLRtoCOM()+0x86

254cee10 1be13fe5
Microsoft_BizTalk_CoreAdapter!Microsoft.BizTalk.Adapter.FtpAdapter.FtpUtil.Connect(XceedFtpLib.XceedFtp,
System.String, System.String)+0x29
[d:\bt\9359137\private\source\Runtime\Msg\Adapters\Ftp\Runtime\FtpUtil.cs @
201]

254cef18 1be12ec6
Microsoft_BizTalk_CoreAdapter!Microsoft.BizTalk.Adapter.FtpAdapter.FtpReceiverEndpoint.EndpointTask()+0x235
[d:\bt\9359137\private\source\Runtime\Msg\Adapters\Ftp\Runtime\FtpReceiverEndpoint.cs
@ 325]

254cef60 1be129ba
Microsoft_BizTalk_CoreAdapter!Microsoft.BizTalk.Adapter.FtpAdapter.FtpReceiverEndpoint.ControlledEndpointTask()+0x56
[d:\bt\9359137\private\source\Runtime\Msg\Adapters\Ftp\Runtime\FtpReceiverEndpoint.cs
@ 251]

254cef8c 1be12838 Microsoft_BizTalk_BaseAdapter!Microsoft.BizTalk.Adapter.Common.SimpleTask.Start()+0x2a
[d:\bt\9359137\private\source\Runtime\Msg\Adapters\BaseAdapter\SimpleTask.cs @
45]

254cefb8 1be1256d
Microsoft_BizTalk_Scheduler_Runtime!Microsoft.BizTalk.Scheduler.TaskController.StartTask()+0x28
[d:\bt\9359137\private\source\Runtime\Scheduler\Runtime\TaskController.cs @
638]

254cefc4 1be1241e
Microsoft_BizTalk_Scheduler_Runtime!Microsoft.BizTalk.Scheduler.TaskController+StartPendingState.OnThreadAvailable(Microsoft.BizTalk.Scheduler.TaskController)+0x1d

Problem Resolution

=================

Please check the following registry key in your BizTalk Server box to see if there is any TLS v1 registry key been disabled, that means the value is 0. If so, please change the TLS 1.0 Client registry key value to 1, that enable TLS 1.0 for client.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS1.0\Client

Microsoft APGC CSS is now promoting a new customer program named Anxin Service（安心服务）.

Currently, this program is only open for GCR customers, so the following introduction will be written in Chinese Simplified. If any of you come from SEA or ANZ area, that also want to get more information about this Anxin Service, please feel free to drop a mail to us, and we will send you detailed information.

And we will schedule to translate all the introduction into English in the future.

Thanks for visiting our Blog!!!

=============================

========

“Object reference not set to an instance of an object” was intermittently reported when executing a customized receiving pipeline when de-batch an incoming message.

Analysis:

========

1.       The error was caused by the following first chance Access Violation.

(2fac.347c): Access violation - code c0000005 (first/second chance not available)

eax=1e0a7a40 ebx=1e09d858 ecx=00000000 edx=00078272 esi=1e09c998 edi=100d455c

eip=08e16ee5 esp=05daf880 ebp=05daf8c8 iopl=0         nv up ei ng nz na pe cy

cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010287

08e16ee5 3909            cmp     dword ptr [ecx],ecx  ds:002b:00000000=????????

Microsoft_BizTalk_Pipeline_Components!Microsoft.BizTalk.Component.XmlDasmReader.get_MessageContext()

Microsoft_BizTalk_Pipeline_Components!Microsoft.BizTalk.Component.XmlDasmComp.GetNext2(Microsoft.BizTalk.Component.Interop.IPipelineContext)

Microsoft_BizTalk_Pipeline_Components!Microsoft.BizTalk.Component.XmlDasmComp.GetNext(Microsoft.BizTalk.Component.Interop.IPipelineContext)

Microsoft_BizTalk_Messaging!Microsoft.BizTalk.Internal.ComponentWrapper.GetNext(Microsoft.BizTalk.Component.Interop.IPipelineContext)

Microsoft_BizTalk_Pipeline!Microsoft.BizTalk.PipelineOM.DisassemblerComponent.GetNext(Microsoft.BizTalk.Component.Interop.IPipelineContext)

Microsoft_BizTalk_Pipeline!Microsoft.BizTalk.PipelineOM.DisassemblingParserStage.GetNext(Microsoft.BizTalk.PipelineOM.IBTMPipelineContext)

Microsoft_BizTalk_Pipeline!Microsoft.BizTalk.PipelineOM.ReceivePipeline.GetNext()

2.       The AV occurred because the field m_npStk of the object XmlDasmReader is NULL.

0:038> !u @eip

Normal JIT generated code

Microsoft.BizTalk.Component.XmlDasmReader.get_MessageContext()

Begin 08e16ee0, size 12

08e16ee0 8bc1            mov     eax,ecx

08e16ee2 8b4820          mov     ecx,dword ptr [eax+20h]

>>> 08e16ee5 3909            cmp     dword ptr [ecx],ecx            // Access Violation here since ecx is zero, ecx is coming from eax+20

08e16ee7 e844dbffff      call    Microsoft_BizTalk_Pipeline_Components!Microsoft.BizTalk.Component.NodeProcessorStack.get_Current()

08e16eec 8bd0            mov     edx,eax

08e16eee 8b4208          mov     eax,dword ptr [edx+8]

08e16ef1 c3              ret

0:038> !do @eax

Name: Microsoft.BizTalk.Component.XmlDasmReader

….

7933291c  40001e2       1c ...ections.ArrayList  0 instance 00000000 m_envelopes

080c923c  40001e3       20 ...odeProcessorStack  0 instance 00000000 m_npStk         // eax+20 is the field m_npStk(NodeProcessorStack) of the object Microsoft.BizTalk.Component.XmlDasmReader

793308ec  40001e4       24        System.String  0 instance 1e0aafd0 m_docType

7933291c  40001e5       28 ...ections.ArrayList  0 instance 00000000 propertyPromotionList

....

3.       The field m_npStk is set to NULL because the  XmlDasmReader.Close() method is called by the object method ReadOnlySeekableStream.Finalize() when it is Disposed by GC.

microsoft_biztalk_pipeline_components!Microsoft.BizTalk.Component.XmlDasmReader.Close()

microsoft_biztalk_streaming!Microsoft.BizTalk.Streaming.XmlTranslatorStream.Close()

microsoft_biztalk_streaming!Microsoft.BizTalk.Streaming.ReadOnlySeekableStream.Cleanup()

microsoft_biztalk_streaming!Microsoft.BizTalk.Streaming.ReadOnlySeekableStream.Finalize()

mscorwks!FastCallFinalize

mscorwks!MethodTable::CallFinalizer

mscorwks!SVR::CallFinalizer

mscorwks!SVR::DoOneFinalization

mscorwks!SVR::FinalizeAllObjects

mscorwks!SVR::GCHeap::FinalizerThreadWorker

4.       The object ReadOnlySeekableStream was disposed above was created by the  customized pipeline component MessageValidator used in the pipeline.

microsoft_biztalk_streaming!Microsoft.BizTalk.Streaming.ReadOnlySeekableStream..ctor(System.IO.Stream)

xxxx_xxxx_xxxx_xxxx!xxxx.xxxx.xxxx.xxxx.xxxxx (System.IO.Stream)

xxxx_xxxx_xxxx_xxxx!xxxx.xxxx.xxxx.xxxx.MessageValidator.Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext, Microsoft.BizTalk.Message.Interop.IBaseMessage)

microsoft_biztalk_messaging!Microsoft.BizTalk.Internal.ComponentWrapper.Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext, Microsoft.BizTalk.Message.Interop.IBaseMessage)

microsoft_biztalk_pipeline!Microsoft.BizTalk.PipelineOM.SimpleComponent.Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext, Microsoft.BizTalk.Message.Interop.IBaseMessage)

microsoft_biztalk_pipeline!Microsoft.BizTalk.PipelineOM.SimpleStage.Execute(Microsoft.BizTalk.PipelineOM.IBTMPipelineContext, Microsoft.BizTalk.Message.Interop.IBaseMessage, Int32)

114af168 1c7f38c6 microsoft_biztalk_pipeline!Microsoft.BizTalk.PipelineOM.Pipeline.Execute(Int32, Int32, Microsoft.BizTalk.Message.Interop.IBaseMessage)

….

5.       The problem can be reproduced using a simple pipeline component  as the below (at Validate stage) + the default XML receiving component(at Disassemble stage) when  de-batch an incoming XML message.

….

        public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

        {

            IBaseMessagePart bodyPart = inmsg.BodyPart;

            if (bodyPart!=null)

            {

                                Stream originalStrm                        = bodyPart.GetOriginalDataStream();

                                Stream strm = null;

                                if (originalStrm != null)

                                {

                                                        strm             = new ReadOnlySeekableStream(originalStrm);

                                                        bodyPart.Data    = strm;

//                                                    pc.ResourceTracker.AddResource( strm );  //  comments this line, the problem can be reproduced easily, the problem will be fixed by remaining the line.

                                    }

            }

            return inmsg;

        }

….

Solution:

==========

This is the coding  flaw in the developed pipeline component(MessageValidator in this case), please refer the description from the below document. (it is very good article for Streaming Pipeline component development).

http://download.microsoft.com/download/0/7/3/073B05EF-E629-4425-9851-295B98CDE699/DevelopingAStreamingPipelineComponent.docx

……

Before returning the message back to the pipeline in line 18, the StreamWriter object that was created is added to the context’s ResourceTracker (line 15). This ensures the .NET Garbage Collector does not dispose of this object and the stream it is using. If it did, the underlying stream would have been closed and disposed of also ll (the behaviour of the StreamWriter Dispose() method) and would not be available for other component’s or indeed BizTalk Server itself. Equally it also means that BizTalk Server will ensure disposing of this object once the processing of the message is complete.

….

To add the ResourceTracker.AddResource call in the pipeline component and let the BizTalk engine managing the lifetime of the new created Stream object for you.

http://support.microsoft.com/kb/2497794

Problem

========

When running PSSDiagForBizTalk in a BizTalk cluster which is not SQL cluster(no SQL server resource in the cluster), the following error is reported:

2011/02/25 15:14:35.48 PSSDIAG Error enumerating virtual servers.  Is your SQL Server instance clustered?

2011/02/25 15:14:35.48 PSSDIAG Error retrieving servers from the configuration file.  Aborting.  Please see Microsoft Knowledge Base article 906532 for more information

Cause

========

PSSDiagForBizTalk tool tries to retrieve the SQL server cluster resource from the BizTalk cluster and it fails with the above error.

Solution

========

Open pssdiag.ini, change the section [.] to [BizTalk_Machine_NetBios_Name]. For example, one node of the BizTalk cluster netbios name is birdgun, the section could be changed to [birdgun] when run the tool on this node. This makes the tool skips the steps to enumerate the cluster resources and continue to run.

NOTE: This article is migrated from Blog AsiaTech

Date: 2010-6-16 3:49 PM

Orignal URL: http://blogs.msdn.com/b/asiatech/archive/2010/06/16/remove-invalid-character-in-incoming-xml-message-using-custom-pipeline-component.aspx

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

BizTalk server is usually used as message bus for different platforms and applications. In most scenarios, BizTalk stores and processes messages in XML format. Internally, it calls .NET XML library to do the job. Thus it follows W3C XML standard to validate message and check encoding.

However, in some scenarios,   messages from different platforms or applications didn’t strictly follow W3C XML standard and may include invalid characters. These characters may lead to BizTalk message processing failure in various stages, for example, XML Receive pipelines, message mapping. The below is one sample error message you may find in BizTalk event log:

“Unable to read the stream produced by the pipeline.  Details: '', hexadecimal value 0x0F, is an invalid character.

*Note: According to W3C, XML processors should accept any character in following range

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

For more detail, please refer to:

http://www.w3.org/TR/REC-xml/#charsets

Generally speaking, the formal way of solving this problem is to fix the source system and prevent it from sending out invalid characters in XML message. Unfortunately, this way is not always available because the source system may not under our control. So it may be hard to persuade the owner to spend their money solving the problem. Writing a custom pipeline to remove the invalid characters before BizTalk processing the message would be one workaround in this scenario.

The below is the sample implementation for Execute interface of pipeline component. Here are the several stages in this sample:

·         Read original message stream into text

·         Use Regular expression to remove invalid characters

·         Add Unicode Byte Order Mark

·         Assign the new stream to message

Note: Here just a sample of using Unicode encoding. User can using UTF-8 instead or even implement design time support to switch between different supported encoding format.

        public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

        {

            string oldXMLText,newXMLText;

            Stream newStr;

            IBaseMessagePart bodyPart = inmsg.BodyPart;

            if (bodyPart != null)

            {

                using (StreamReader sr = new StreamReader(bodyPart.GetOriginalDataStream()))

                {

                    //Remove Invalid Character

                    oldXMLText = sr.ReadToEnd();

                    string re = @"[^\u0009\u000A\u000D\u0020-\uD7FF\uE000-\uFFFD\u10000-\u10FFFF]";

                    newXMLText = System.Text.RegularExpressions.Regex.Replace(oldXMLText, re, "");

                    //Add Byte Order Mark

                    byte[] b = new byte[newXMLText.Length * 2 + 2];

                    b[0] = 0xFF;

                    b[1] = 0xFE;

                    byte[] textb = System.Text.UnicodeEncoding.Unicode.GetBytes(newXMLText);

                    for (int i = 0; i < textb.Length; i++)

                    {

                        b[i + 2] = textb[i];

                     }

                    newStr = new MemoryStream(b);

                    newStr.Flush();

                    bodyPart.Data = newStr;

                    pc.ResourceTracker.AddResource(newStr);

                }

            }

           return inmsg;

        }

Best Regards,

Bryan Yang

NOTE: This article is migrated from Blog AsiaTech

Date: 2009-11-25 5:39 PM

Orignal URL: http://blogs.msdn.com/b/asiatech/archive/2009/11/25/how-does-biztalk-detect-if-a-host-instance-is-dead-in-bts2k6-2k6r2-and-2k9.aspx

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

This job is done by the cooperation between the front end BizTalk host instance process and the backend BizTalk SQL job MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb.  

1.       When a BizTalk host instance is started, it will call the store procedure bts_ProcessHeartbeat_<HostName> once as the below.

exec [dbo].[bts_ProcessHeartbeat_<HostName>] @uidProcessID=NULL,@dwCommand=1,@nHeartbeatInterval=60

Here @dwCommand=1 means Process Startup.

2.       When a BizTalk host instance is stopped, it will call the same store procedure bts_ProcessHeartbeat_<HostName> once again as the below.

exec [dbo].[bts_ProcessHeartbeat_<HostName>] @uidProcessID=NULL,@dwCommand=2,@nHeartbeatInterval=60

Here @dwCommand=2 means Process Shutdown.

3.       Look at the code of the store procedure bts_ProcessHeartbeat_<HostName>, could see the int_ProcessCleanup_<HostName> will be called if dwCommand=1(Process Startup) or dwCommand=2(Process Shutdown). The store procedure int_ProcessCleanup_<HostName> is used internally to clean up the records related with the host instance process and release all messages and service instances which are locked by this process.

4.       It is easy to understand why int_ProcessCleanup_<HostName> is called when a host instance is shutdown, the messages and service instances locked by the host instance can be freed before the shutdown so they can be picked up by the same restarted host instance again or they can be picked up by other host instances which are still running in a multiple boxes BizTalk environment.

5.       The questions is why int_ProcessCleanup_<HostName> is also called when a host instance is startup if the locked messages and services instances were already freed by the same SP when shutdown? Yes, it is redundant if the same SP was already called when the instance shutdown normally, but the problem is that the process could be terminated or crashed unexpectedly without calling the SP, You can see that abnormal shutdown easily by killing a host process, the SP bts_ProcessHeartbeat_<HostName> with the dwCommand=2 is not called if the host process is killed by the command “kill /f”. As we can’t guarantee the internal cleanup SP is already called before the host instance is started, to call the internal cleanup SP again at the startup is the good choice although it is redundant some times.

Another question is raised at this point, if a host instance process is totally hang or dead there, or it is just crashed or terminated unexpectedly without restarting, who can help to detect the situation and release these locked messages and service instances? How? Let’s continue.

6.       In each running BizTalk host instance, you can see one thread as the below.

0:016> kc

ntdll!NtWaitForSingleObject

kernel32!WaitForSingleObjectEx

kernel32!WaitForSingleObject

BTSMessageAgent!CAdminCacheRefresh::OnCall

BTSMessageAgent!CThreadPoolWrapper::ThreadWorker

ntdll!RtlpWorkerCallout

ntdll!RtlpExecuteWorkerRequest

ntdll!RtlpApcCallout

ntdll!RtlpWorkerThread

kernel32!BaseThreadStart

7. The thread calls the store procedure bts_ProcessHeartbeat_<HostName>  with @dwCommand=0 every 60 seconds by default. The interval can be modified through the column ConfigurationCacheRefreshInterval of the table adm_Group in BizTalkMGMTDb or through the settings of the BizTalk group in BizTalk admin UI.

…

exec [dbo].[bts_ProcessHeartbeat_Newhost] @uidProcessID=NULL,@dwCommand=0,@nHeartbeatInterval=60

…

Here @dwCommand=0 means Process Live.

8. Look at the code of the store procedure bts_ProcessHeartbeat_<HostName>, the internal SP int_ProcessCleanup_<HostName> will not be called if dwCommand=0(Process Live). The bts_ProcessHeartbeat_<HostName> will only update a record or insert a new record if there is no existing one in the table ProcessHeartbeats in BizTalkMsgBoxDb for the host instance.

9. Let’s look at the record in the table ProcessHeartbeats and see what it is look like.

uidProcessID is the unique GUID for each host instance. You can get the uidProcessID for a host instance by query the UniqueId column of the table adm_HostInstance in BizTalkMGMTDb. You also can get the uidProcessID for a host instance by checking the service property “Path to Executable” of the corresponding BizTalk service in Windows Service Control Manager on a BizTalk box. For example, the following is the “Path to Executable” setting for my "BizTalkServerApplication" in my BizTalk testing box. You can see the ID is after the command line option “-btsapp”.

"C:\Program Files (x86)\Microsoft BizTalk Server 2006\BTSNTSvc.exe" -group "BizTalk Group" -name "BizTalkServerApplication" -btsapp "{4D50992E-2AE1-4BBD-9D61-B712B052B99C}"

You can note that the dtNextHeartbeatTime is about equal to (dtLastHeartbeatTime+10*HeartbeatInterval). Since the default HeartbeatInterval is about 60 seconds by default(we mentioned how to modify that interval at the above), the dtNextHeartbeatTime is about 10 minutes after the dtLastHeartbeatTime by default. Of course if the HeartbeatInterval is modified to 30 seconds, then the dtNextHeartbeatTime would be 5 minutes after the dtLastHeartbeatTime.

10. Normally the record for a specified host instance in the table ProcessHeartbeat is updated every [HeartbeatInterval] seconds through the SP bts_ProcessHeartbeat_<HostName> by the running host instance process.

11. Take a look at the SQL job MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb, it is configured to execute the store procedure bts_CleanupDeadProcesses every minute.

12. Look at the code of the SP bts_CleanupDeadProcesses, it is simply go through the table ProcessHeartbeats and check if there is a host instance which dtNextHeartbeatTime is older than the current time. If there is, which means we already lost 10 heartbeats at least from this host instance, this host instance is then considered as dead and the store procedure int_ProcessCleanup_<HostName> for that instance is called by the SQL job to clean up the records related with the host instance process and release all messages and service instances which are locked by the process.

If this SQL job is disabled or get a problem when it is running, BizTalk will lost the capability to detect “dead” of a host instance.

Regards,

XiaoDong Zhu

NOTE: This article is migrated from Blog AsiaTech

Date: 2009-11-25 10:24 AM

Orignal URL: http://blogs.msdn.com/b/asiatech/archive/2009/11/25/unable-to-suspend-or-terminate-active-service-instance-in-biztalk.aspx

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

Symptom

When try to suspend or terminate a active service instance, the instance can not be suspended or terminated for a long time, it keeps active with Pending Job of the instance is set to Suspend or Terminate

Analysis

A service instance is in Active status means that it is still actively running in a host instance and hasn’t reached the next persistence point. Suspend and Terminate operations are designed as operation which will only be executed at the next persistable point. The Suspend or Terminate operation will be put into the pending operation table when the target service instance is in Active status. Please note BizTalk only hold one pending operation for a single instance. The instance will keep Active status with the queued pending operation until the next persistence point is reached. Let’s use a simple orchestration example to demo the behavior.

The code is in Expression_1 is as the below which is used to simulate long time processing in a host instance or hang in a host instance.

System.Threading.Thread.Sleep(5*60000);

When drop a testing message to activate the above simple orchestration, it will keep Active status for about 5 minutes and then an output message will be sent out. If try to suspend the active instance when the instance is sleeping in the host instance,  the instance will keep in Active status for a quite while with Pending Job is set to Suspend until the next persistence point - send shape is reached. One interesting thing is that the orchestration instance will be started from the last persistence point to continue the execution if the running host instance is restarted, our simple orchestration will start from the beginning to re-execute the whole orchestration code again if the host instance is restarted during System.Threading.Thread.Sleep(),we will see the instance will keep Active status with the pending operation Suspend for another 5 minutes. Now we got a problem, If the code in the expression shape is changed as the below to simulate a real hang situation.

while(1==1){System.Threading.Thread.Sleep(5*60000);}

We will find we can’t suspend the Active instance. The orchestration instance will keep Active with the pending operation Suspend forever even if the running host instance is restarted in BTS 2K6 and 2K6R2.

The ways to handle a long time active or hang instance as the above

1.  The format way should be to find where is the instance active or hang. The HAT debugging or a hang dump file for the running host instance can be used to find out where the instance processing is blocked. If the processing block or the hang can be fixed, then the instance can quickly move to the next persistable point, the pending operation or the other operations can get a chance to execute.

2. If you don’t want to spend the time to figure out where the blocking is and just want to simply clean out the instance from the MessageBox, use the tool Terminator to terminate these instance hardly. You can download the Terminator from the below link. The following is the captured screen for Terminate Instance (Hard) for the reference.

http://blogs.msdn.com/biztalkcpr/pages/biztalk-terminator-download-install-info.aspx

3. As the above, if just want to suspend or terminate the instance, besides the Terminator, you also can call the internal store procedure int_AdminSuspendInstance_<host> or int_AdminTerminateInstance_<host> directly to suspend or terminate the instance in MessageBox database. The following are the two SQL script sample to use the two store procedures.

Hard Suspend:

declare @ApplicationName nvarchar(128)

declare @uidInstanceID uniqueidentifier

declare @uidServiceID uniqueidentifier

declare @fKnownInstance int

declare @nvcErrorString nvarchar(512)

declare @dtTimeStamp datetime

declare @spname nvarchar(512)

Begin Tran

set @ApplicationName='testhost'

set @uidInstanceID='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

set @uidServiceID='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

set @dtTimeStamp=GETUTCDATE()

select @nvcErrorString = nvcError FROM dbo.LocalizedErrorStrings WHERE nID = 4

set @spname= 'int_AdminSuspendInstance_' + @ApplicationName

exec @spname @uidInstanceID, @uidServiceID, N'0xC0C01B50', -1, @nvcErrorString, 1, null, @dtTimeStamp, null, null, @fKnownInstance OUTPUT

DELETE FROM InstancesPendingOperations WITH (ROWLOCK) WHERE uidInstanceID = @uidInstanceID OPTION (KEEPFIXED PLAN)

Commit Tran

Hard Terminate:

declare @ApplicationName nvarchar(128)

declare @uidInstanceID uniqueidentifier

declare @uidServiceID uniqueidentifier

declare @fKnownInstance int

declare @spname nvarchar(512)

Begin Tran

set @ApplicationName='testhost'

set @uidInstanceID=' xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx '

set @uidServiceID=' xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx '

set @spname= 'int_AdminTerminateInstance_' + @ApplicationName

exec @spname @uidInstanceID, @uidServiceID, @fKnownInstance OUTPUT

DELETE FROM InstancesPendingOperations WITH (ROWLOCK) WHERE uidInstanceID = @uidInstanceID OPTION (KEEPFIXED PLAN)

Commit Tran

*note*    

(1) The two store procedures are used by BizTalk internally to suspend or terminate an instance, the implementation and the SP interface could be changed in future. The above sample scripts are based on BTS2K6 and BTS2K6R2

(2) You must modify the script to provide the BizTalk Host name, the uidInstanceID and uidServiceID of the target service instance to the variables @ApplicationName, @uidInstanceID and @uidServiceID before execute the script.

(3) Stop the running host instance before execute the script in order to avoid any inconsistence error between the status of the running instance in the host process memory and the status of the instance persisted in the database.

*note* In multiple boxes BizTalk environment, the host instance which an active instance is running in can be found by looking “Processing Server” and “Host” columns of the service instance in BizTalk admin console. The columns for “Processing Server” and “Host” are not listed in the BizTalk group hub UI by default and they can be added manually.

(4) As you can see in Terminator when use Terminate Instance (Hard), the above script should also be used as a last resort. You should be very clear what you want to do and what the impact could be in business before decide to use the script.

4. In BTS2K6 R2 + KB969987 (http://support.microsoft.com/kb/969987) and BTS2K9, it is easier to terminate or suspend such active instance. If you see the instance keep Active with the Pending Operation set after terminate or suspend an active instance in BTS2K6 R2 + KB968897 and BTS2K9, simply restart the host instance which the target service instance is running with, you will see the Pending operation(Terminate or Suspend) got executed immediately after the host instance is restarted. This is because that when a host instance is restarted in BTS2K6 R2 + KB969987 or in BTS2K9, it will check if there is a Pending Operation for every service instance which was owned by this host instance previously, the Pending Operation for an instance will get executed immediately if it is found.

Regards,

XiaoDong Zhu

NOTE: This article is migrated from Blog AsiaTech

Date: 2009-11-24 8:43 PM

Orignal URL: http://blogs.msdn.com/b/asiatech/archive/2009/11/24/0x800706d9-when-contact-a-sso-master-secret-cluster.aspx

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

Symptom:

==========

You may see the following warning event in a BizTalk machine.

Event Type: Warning

Event Source: ENTSSO

Event ID: 10536

Description:

SSO AUDIT

Function: GetConfigInfo

Tracking ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Client Computer: xxx.xxx.xxx (BTSNTSvc.exe:1234) Client User: Domain\BizAdmin Application Name: { xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx } Error Code: 0x800706D9, There are no more endpoints available from the endpoint mapper.

If try to backup the master secret on the SSO master secret cluster, you can get the below error:

C:\Program Files\Common Files\Enterprise Single Sign-On>ssoconfig -backupsecret test.bak
Password : *******
Confirm Password : *******
Password reminder : 1234567
ERROR: 0xC0002A0F : Could not contact the SSO server 'sso-cluster'. Check that SSO is configured and that the SSO service is running on that server.
(RPC: 0x800706D9: There are no more endpoints available from the endpoint mapper.)

Cause:

===========

In the SSO master secret cluster environment, the RPC service “SSOSecretServer” will not be registered successfully in the RPC endpoint mapper if the local ENTSSO service is started on a node before the clustered ENTSSO resource is brought online on this node.

For example, someone starts the local ENTSSO service from the service control manager or some tool executes a command “net start ENTSSO” on a cluster node.

In our case, we find there is a monitoring tool which periodically starts the local ENTSSO service on both cluster nodes if it finds they are not started, then the error will be reported when the cluster ENTSSO failover to another node which the local ENTSSO service is already started. Using “rpcdump” utility (the command “rpcdump -s <sso-cluster> -i”) or “portqry” utility (the command “portqry -n <sso-cluster> -e 135”) could list the registered RPC services in a RPC endpoint mapper. In this case, we only see the SSO related RPC services SSOMappingServer,SSOAdminServer,SSOLookupServer,SSOCSServer were registered but SSOSecretServer.

Solution:

=========

Stop the local ENTSSO service on another node and failover the cluster SSO to the node.

Configure the monitoring tool and stop to start the local ENTSSO service on both cluster nodes.

Regards,

XiaoDong Zhu