ENVIRONMENT
==============
Product  :  BizTalk Server 2004 SP2/SWIFT Accelerator  2.1 .
 O/S     :  Windows Server2003 

Cause
============
The users were provided multiple roles and the users use same certificate for different roles.
You will experience the behaviour  you are seeing when you use the same certificate for a user in multiples roles

Resolution:
=========
The issue was resolved by using different certificates for the same user in multiple roles

ENVIRONMENT
===========
BTS 2006 R2
Microsoft BizTalk Accelerator for SWIFT 3.0

CAUSE
======================
The behaviour described  is expected, because the same schema is deployed twice.
When the same schema is deployed twice then BizTalk run time will not be able to identify which schema to be used.

Resolution:
=============
The resolution here is to create a common assembly for the schema being used in more than one application and then adding a reference to that assembly in all the projects where we want to use that schema.

So follow the below steps to resolve it:
1. Create a dedicated assembly that will contain the shared schema(The schema being used in two or more applications).
2. GACed this dedicated assembly.
3. Modify any project containing the shared schema.
 a. Remove the schema from the project
 b. Add a reference to the assembly created in step #1
4. Deploy all the projects affected

"The adapter "MQSeries" raised an error message. Details "The adapter has encountered an 'Access Denied' error while attempting to contact the COM+ object on the MQSeries server. Ensure the BizTalk account is added to the Role on the MQSAgent COM+ application."

CAUSE
============
The pre production xml was deployed at production but the MQ Name and MQ server name was not changed in the xml to point to MQ name and the MQ server to be used by the Production box.

Resolution:
=========
The issue was resolved by changing the MQ Name and MQ server Name on Production xml.

We have the Biztalk 2006 R2 server communicating with the MQ server using MQ Series Adapter 2.0.

on the BizTalk machine we see the following error with Event ID:5740:

The adapter "MQSeries" raised an error message. Details "Error encountered on opening Queue Manager

name = QM_angvmbiztmq Reason code = 2354.".
 

On the MQ server we see the below error with Event ID:4171:

MS DTC Transaction Manager log write failed with error 0x8.

Also on the MQ server we see the error with event ID:264:

“Error encountered as BizTalk attempts to retrieve a message from MQSeries, description = Error

encountered on opening Queue Manager name =   QM_angvmbiztmq Reason code = 2354. hresult =

0XC0C11007. This error will also be reported on the BizTalk server”

Resolution:
=========
We stopped the SCOM software which Customer used as a system monitoring tool and after that we

didn’t see the occurrence of the problem.

Cause:
=========
This feature is by design to implement security restrictions to the connection.Ie we don’t want the password details to be shared with any resource other than the host specified while configuring the receive location.

Symptoms

When Installing MSMQ on Windows Server 2008 computer, the installation may fail with the following error:

"Attempt to install Message Queuing Server failed with error code 0x80070643. Fatal error during installation.

The follwoing features were not installed:

Message Queuing Services.

Message Queing Server.

Cause

This issue may occur if user permissions are inadequate for the installation. This can happen if UAC is turned on.

Resolution

To resolve this issue follow the steps listed below:

1.       Go to Control panel and click User Accounts

2.       Select User Accounts and click Turn User Account Control on/ Off

3.       Un-check the check box beside User Account Control

4.       Close the User Account Window, reboot the computer  and try to install Microsoft Message Queuing (MSMQ) again

Note: Be sure to turn User Account Controll back on when the issue is resolved

You can also give full permission to authenticated user for C:\Windows\System 32 and C:\Windows\debug folder and then try to install again.

Event Type:        Error

Event Source:    XLANG/s

Event Category:               None

Event ID:             10001

Date:                    23/01/2009

Time:                    9:15:21

User:                     N/A

Computer:          SCS002BZPR0002

Description:

Failed while delivering a message to a service instance. Message details follow.

Message ID: f2d316ab-db78-4c7d-aad0-dd96190d18b7

Service Instance ID: 2c641a56-80ff-462f-b5e7-df4eb4fb62bb

Service Type ID: 6498c049-0690-7758-a6b0-511f8e36f8fa

Subscription ID: 00000000-0000-0000-0000-000000000000

Body part name:

Service type: SAS.Int.GADU_SNS.NotificacionBajaServicioSalud.NotifBajaServicioSalud, SAS.Int.GADU_SNS.NotificacionBajaServicioSalud, Version=1.0.0.0, Culture=neutral, PublicKeyToken=dc6e1140c14ef65d

Exception type: BTXMessageDeliveryException

The following is a stack trace that identifies the location where the exception occured

at Microsoft.BizTalk.XLANGs.BTXEngine.BTXService.DeliverMessage(Guid subscriptionId, IBTMessage msg, Boolean& receiveCompleted)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXSession._tryReceiveOneMessage(Boolean& loggedError, Guid& instanceId, IBTMessage currMsg)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXSession._receiveOneMessage(Guid& instanceId, Guid& serviceId, IBTMessage currentMsg)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXSession.ReceiveMessages(IBTMessage[] messages, Int32 firstIdx, Int32 count)

   at Microsoft.BizTalk.XLANGs.BTXEngine.AppDomains.AppDomainRoot.Microsoft.XLANGs.BizTalk.ProcessInterface.IAppDomainStub.ReceiveMessages(Object objMsg)

   at Microsoft.XLANGs.BizTalk.CrossProcess.AppDomainStubProxy.Microsoft.XLANGs.BizTalk.ProcessInterface.IAppDomainStub.ReceiveMessages(Object msgs)

   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)

   at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(RuntimeMethodHandle md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)

   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext)

   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

   at System.Runtime.Remoting.Messaging.ServerObjectTerminatorSink.SyncProcessMessage(IMessage reqMsg)

   at System.Runtime.Remoting.Messaging.ServerContextTerminatorSink.SyncProcessMessage(IMessage reqMsg)

   at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessageCallback(Object[] args)

   at System.Threading.Thread.CompleteCrossContextCallback(InternalCrossContextDelegate ftnToCall, Object[] args)

   at System.Threading.Thread.InternalCrossContextCallback(Context ctx, IntPtr ctxID, Int32 appDomainID, InternalCrossContextDelegate ftnToCall, Object[] args)

   at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(IMessage reqMsg)

   at System.Runtime.Remoting.Channels.ChannelServices.SyncDispatchMessage(IMessage msg)

   at System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[] reqStmBuff, SmuggledMethodCallMessage smuggledMcm, SmuggledMethodReturnMessage& smuggledMrm)

   at System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCallback(Object[] args)

   at System.Threading.Thread.CompleteCrossContextCallback(InternalCrossContextDelegate ftnToCall, Object[] args)

Additional error information:

        String was not recognized as a valid DateTime.

Exception type: FormatException

Source: mscorlib

Target Site: System.DateTime Parse(System.String, System.Globalization.DateTimeFormatInfo, System.Globalization.DateTimeStyles)

The following is a stack trace that identifies the location where the exception occured

   at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles)

   at System.DateTime.Parse(String s, IFormatProvider provider)

   at System.Convert.ToDateTime(String value, IFormatProvider provider)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXService.HandleSpecialMessage(IBTMessage msg)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXService.DeliverMessageImpl2(Guid subscriptionId, IBTMessage msg, Boolean& receiveCompleted)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXService.DeliverMessageImpl(Guid subscriptionId, IBTMessage msg, Boolean& receiveCompleted)

   at Microsoft.BizTalk.XLANGs.BTXEngine.BTXService.DeliverMessage(Guid subscriptionId, IBTMessage msg, Boolean& receiveCompleted)

n:

Cause

======

If we have two machines for processing BizTalk messages, this issue occurs if the regional setting on the two machines are different.

So if an orchestration dehydrates on one box the timer message would be created on that box and if it rehydrates on another box then the same timer message would be consumed on the other box.

So if these two boxes have the different regional setting  this results in the Format exception.

We can check the regional setting by looking at the "Regional and Language options" in control panel.

Lets say on one box we have the “Standard and format” option for the Regional options tab in regional and language option page set to English and on the other box we have the same option set to Spain, this issue occurs.

Also make sure that the language selected on the advanced tab of the regional and language option page matches on both the servers.

  You should have same “Regional and language options” setting on regional options and advanced tab on all boxes in build and production to prevent any scenario that can cause this 

On two production processing servers SCS002BZPR0001 and SCS002BZPR0002 the customer had the below mismatched setting

SCS002BZPR0001

SCS002BZPR0002

SCS002BZPR0001

SCS002BZPR0002

===================

If the DTA database is huge in size Dtasp_clean HMData stored procedure may take days to execute in BizTalk 2004

CAUSE

=========

Dtasp_cleanHMData uses delete From table command in BizTalk 2004 and  it used truncate table command  in Biztalk 2006.So if you have a huge DTA database(of the order of 70-80 GB) it may take several days to execute the stored procedure to clean the DTA database in Biztalk 2004 environment.

CREATE PROCEDURE [dbo].[dtasp_CleanHMData]
@nCleanStatus int = 1
AS
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN
IF ((@nCleanStatus & 0x1) > 0)
BEGIN
 DELETE FROM dta_Adapter WITH (SERIALIZABLE)
 DELETE FROM dta_CallChain WITH (SERIALIZABLE)
 DELETE FROM dta_DebugTrace WITH (SERIALIZABLE)
 DELETE FROM dta_DecryptionSubject WITH (SERIALIZABLE)
 DELETE FROM dta_Host WITH (SERIALIZABLE)
 DELETE FROM dta_MessageBox WITH (SERIALIZABLE)
 DELETE FROM dta_MessageFields WITH (SERIALIZABLE)
 DELETE FROM dta_MessageFieldValues WITH (SERIALIZABLE)
 DELETE FROM dta_MessageInOutEvents WITH (SERIALIZABLE)
 DELETE FROM dta_MessageInstances WITH (SERIALIZABLE)
 DELETE FROM dta_PartyName WITH (SERIALIZABLE)
 DELETE FROM dta_PortName WITH (SERIALIZABLE)
 DELETE FROM dta_SchemaName WITH (SERIALIZABLE)
 DELETE FROM dta_ServiceInstanceExceptions WITH (SERIALIZABLE)
 DELETE FROM dta_ServiceInstances WITH (SERIALIZABLE)
 DELETE FROM dta_SigningSubject WITH (SERIALIZABLE)
 DELETE FROM dta_Url WITH (SERIALIZABLE)
END
IF ((@nCleanStatus & 0x2) > 0)
BEGIN
 DELETE FROM TDDS_FailedTrackingData WITH (SERIALIZABLE)
END
IF ((@nCleanStatus & 0x4) > 0)
BEGIN
 DELETE FROM dta_Services  WITH (SERIALIZABLE) WHERE nServiceId > 13
END
COMMIT TRAN
SET TRANSACTION ISOLATION LEVEL READ COMMITTED

GO

USE [BizTalkDTADb]

GO

/****** Object:  StoredProcedure [dbo].[dtasp_CleanHMData]    Script Date: 05/12/2009 14:57:07 ******/

SET ANSI_NULLS ON

GO

SET QUOTED_IDENTIFIER ON

GO

ALTER PROCEDURE [dbo].[dtasp_CleanHMData]

@nCleanStatus int = 1

AS

exec dtasp_DropViews

IF ((@nCleanStatus & 0x1) > 0)

BEGIN

      TRUNCATE TABLE dta_Adapter

      TRUNCATE TABLE dta_DecryptionSubject

      TRUNCATE TABLE dta_Host

      TRUNCATE TABLE dta_MessageBox

      TRUNCATE TABLE dta_MessageFields

      TRUNCATE TABLE dta_PartyName

      TRUNCATE TABLE dta_PortName

      TRUNCATE TABLE dta_SchemaName

      TRUNCATE TABLE dta_ServiceInstanceExceptions

      TRUNCATE TABLE dta_SigningSubject

      TRUNCATE TABLE dta_CallChain

      TRUNCATE TABLE dta_DebugTrace

      TRUNCATE TABLE dta_MessageFieldValues

      TRUNCATE TABLE dta_MessageInOutEvents

      TRUNCATE TABLE dta_ServiceInstances

      TRUNCATE TABLE Tracking_Fragments1

      TRUNCATE TABLE Tracking_Parts1

      TRUNCATE TABLE Tracking_Spool1

      TRUNCATE TABLE Tracking_Fragments2

      TRUNCATE TABLE Tracking_Parts2

      TRUNCATE TABLE Tracking_Spool2

END

IF ((@nCleanStatus & 0x2) > 0)

BEGIN

      TRUNCATE TABLE TDDS_FailedTrackingData

END

IF ((@nCleanStatus & 0x4) > 0)

BEGIN

      DELETE FROM dta_Services  WITH (SERIALIZABLE) WHERE nServiceId > 13

END

if ((@nCleanStatus & 0x8) > 0)

BEGIN

      TRUNCATE TABLE Tracking_Fragments1

      TRUNCATE TABLE Tracking_Parts1

      TRUNCATE TABLE Tracking_Spool1

      TRUNCATE TABLE Tracking_Fragments2

      TRUNCATE TABLE Tracking_Parts2

      TRUNCATE TABLE Tracking_Spool2

END

exec dtasp_CreateMessageFactsFindMsgViews

exec dtasp_CreateRealNamesView

exec dtasp_CreateServiceFactsView

RESOLUTION

===========

Don’t use dtasp_CleanHMData in Biztalk 2004 environment. You can use the script  Bts_tracking_shrinkexistingdatabase.sql from the KB 894253 to clean the DTA database as this script uses truncate  Table command.

===================

The log space used for message box database periodically increases upto the order of 14 GB and the performance of BizTalk becomes slow.

This happens intermittently.

CAUSE

=========

The issue occurs if the Backup BizTalk Server job is scheduled to perform log backups for long span of interval.

By default Backup BizTalk job is scheduled to perform log backups for every 15 minutes.

But in this case the customer had the Backup BizTalk Server job scheduled to perform log backups for every three hours. In such cases the Backup BizTalk job will not purge the logs from the message box until every 3^rd hour and it continues to consume space in memory until next run which happens after three hours.

So in such scenarios, under heavy load, the log space used for message box database increases up to the order of 14 GB when we are about to reach the three hours mark and it will be as  low as 2 Gb once we reach the 3 hours mark when the Backup job will purge the logs from the message box database.

RESOLUTION

===========

Please make sure that the Backup BizTalk Server job is scheduled to perform log backups in a shot interval of time. By default this job performs log back up for every 15 minutes.

You can check the scheduling of his job under Schedule column  in the message box viewer report under BizTalk jobs List.

===================

BizTalk Job 'TrackedMessages_Copy_BizTalkMsgBoxDb' fails  and we see in the job history of this job

deadlock as below:

Executed as user:PKUBL\clusteradmin.Transaction(Process ID:224) was deadlocked on lock resources with another process and has been chosen as the deadlock victim.Rerun the transaction[SQLSTate 40001][error 1205]

Warining:The join order has been enforced because a local join hint is used[SQLSTATE 010000][Error 8625].The step failed.

In the MBV we see the below:

 CRITICAL WARNINGS

In the non critical warning section in MBV, we see the below:

MsgBody Tracking for some ReceivePorts Yes (MsgBody can accumulate in MsgBox db if 'TrackedMessages_Copy' job is not running to send them to DTA) !

CAUSE

=========

We had the message body tracking enabled for some of the receive locations and those receive locations were having huge load in peak times.

So this was causing the job 'TrackedMessages_Copy_BizTalkMsgBoxDb' to fail with deadlock error when these receive locations with message body tracking enabled were having huge load.

We had two receive locations receiving messages under the receive port ReceivePort_KeepAlive.  The ReceivePort_KeepAlive had tracking enabled for message bodies before the port processing and after port processing.   This was  a two way port and this means that for each receive, there are 4 tracked message bodies for each inbound message.    

This was a high volume environment, if they were to receive 250,000 messages/day on these receive locations, they would be tracking 1 million message bodies.

RESOLUTION

===========

The issue was resolved by disabling the message body tracking for the receive port.

Enabling Message body tracking for many ports might cause deadlock for the job TrackedMessages_Copy_BizTalkMsgBoxDb.This is by design in BizTalk.