The purpose of this document is to demonstrate how SQL Server replication can be directed to use custom mapping while copying data to NON-SQL database.

This document requires the audience to know SQL server management. Replication and types of replication. It also requires you to SQL programming knowledge of DB2 database. That will be our example NON –SQL database.

Sample database:

In our example we will replicate two columns from Categories table of NorthWind database. Northwind database can be downloaded from https://www.microsoft.com/downloads/details.aspx?FamilyID=06616212-0356-46A0-8DA2-EEBC53A68034&displaylang=en

Original table definition for Categories table is like this:

Table create

USE [Northwind]

GO

/****** Object: Table [dbo].[Categories] Script Date: 08/03/2009 11:05:36 ******/

SET ANSI_NULLS ON

GO

SET QUOTED_IDENTIFIER ON

GO

CREATE TABLE [dbo].[Categories](

[CategoryID] [int] IDENTITY(1,1) NOT FOR REPLICATION NOT NULL,

[CategoryName] [nvarchar](15) NOT NULL,

[Description] [ntext] NULL,

[Picture] [image] NULL,

CONSTRAINT [PK_Categories] PRIMARY KEY CLUSTERED

(

[CategoryID] ASC

)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]

) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

Create a Snapshot publication for Article Categories. Select only two columns from Categories as the columns to be replicated - those are CategoryID and Description. Create a NON-SQL subscriber using

Create a Snapshot publication and NON-SQL subscriber:

Following steps will guide you through creating simple snapshot publication. Note that the security method selected is not recommended one. Your organisation may need more strict security and please alter your publication accordingly.

Creating subscription:

DB2OLEDB provider connection string looks like this:

Provider=DB2OLEDB;User ID=XXXXXX;Password=******;Initial Catalog=YYYYYY;Network Transport Library=TCP;Host CCSID=37;PC Code Page=1252;Network Address=YYYYYY;Network Port=446;Package Collection=XXXXXX;Default Schema=XXXXXX;Process Binary as Character=True;Units of Work=RUW;DBMS Platform=DB2/AS400;Defer Prepare=False;Rowset Cache Size=0;Authentication=Server;Persist Security Info=True;Connection Pooling=True;Derive Parameters=False;

Determining supported mappings for your remote DBMS:

SQL server has a mappings table that keeps track of what all source data types can be mapped to what all destination data types on different DBMS platforms. For e.g. Following query retrieves all the mappings for DB2 as a destination dbms :

select * from msdb.dbo.sysdatatypemappings

where destination_dbms ='DB2'

And following query does find all the mappings for DB2 for source SQL data type = next:

select * from msdb.dbo.sysdatatypemappings

where source_type= 'ntext' AND destination_dbms ='DB2'

If desired custom mapping does not exist then use sp_adddatatypemapping to define one, something like in e.g. below:

--if above query does not return anything then add maping yourself using something like below

exec sp_adddatatypemapping

@source_dbms= 'MSSQLSERVER',

@destination_dbms='DB2', @source_type = 'ntext',

@destination_type='VARCHAR', @destination_length=32703

Setting the Article properties to use custom create TABLE script :

Once you verify or create required mapping, we need to tell replication as to how to use this mapping. SQL server allows this through stored procedure sp_changearticle.

More reference on sp_changearticle can be found at: https://msdn.microsoft.com/en-us/library/ms175980(SQL.90).aspx

--set the creation_script property of Article to point to custom Create Script

sp_changearticle @publication = SUBREPL,

@article = Categories,

@property = 'creation_script',

@value = "d:\TEMP\RFC4789\SUBREPL.sql",

@force_invalidate_snapshot = 1,

@force_reinit_subscription = 1

-- then change the Schema option.

DECLARE @option AS int;

SET @option = (SELECT CAST(0x00 AS int));

EXEC sp_changearticle @publication = SUBREPL,

@article = Categories,

@property = N'schema_option',

@value = @option,

@force_invalidate_snapshot = 1,

@force_reinit_subscription = 1

We have changed the Article properties by doing above two steps, now its time to discuss our e.g. Script SUBREPL.sql that creates the TABLE. Since we want this Article to be replicated on DB2, it is required to use DB2’s sql syntax.

e.g. our SUBREPL.sql contains following commands in it to create table and index on it. Note that this is DB2 syntax for OS/400. bypass_translation is required to be mentioned in the script so that publication will not translate the syntax of any queries that follows.

bypass_translation

DROP TABLE CATEGORIES

GO

CREATE TABLE CATEGORIES (

CATEGORYID INT NOT NULL,

DESCRIPTION VARCHAR (32703))

GO

CREATE UNIQUE INDEX PK_CATEGORIES ON CATEGORIES (CATEGORYID)

GO

Match cases on object names:

Because DB2 is case sensitive we will make sure that Article property "Destination Object Name" is set to correct one . It has to be in the same case as it appears in the SUBREPL.sql Script

See the following screenshot:

That’s it you replication is set to go from ntext to VARCHAR using custom CREATE script.

here is code snippet using DataAccessLibrary that ships with :

1. Microsoft Host Integration Server AND

2. Microsoft Biztalk Adapters for Host Systems AND

3. Microsoft's DB2OLEDB (SQL server feature pack for DB2 ) .

if you have any of these , following code will help getting  your DB2 server's version and class ( MVS or AS400 or NT etc.)

//referencesspan style="mso-spacerun:yes">  required to be added in your project - Microsoft.HostIntegration.DataAccessLibrary.dll

using System;

using Microsoft.HostIntegration.DataAccessLibrary;

using System.Diagnostics;

namespace ConsoleApplication1

{

    class Program

    {

        static void Main(string[] args)

        {

            string serverClass = String.Empty;

            string serverVersion = String.Empty;

            try

            {

                DataAccessControl control = new DataAccessControl();

               //DB2.UDL is the connection to DB2 server( Created with DAT- Data Access Tool)

                IConnectionString cnstr = DB2OleDbConnectionString.ReadUDL(@"DB2.UDL");

                DataAccessControl.TestConnection(cnstr, out serverClass, out serverVersion);

                Console.WriteLine("serverClass" + serverClass);

                Console.WriteLine("serverVersion" + serverVersion);

            }

            catch(Microsoft.HostIntegration.DataAccessLibrary.ConnectionException e)

            {

                Console.WriteLine("failed to connect " + e.ToString());

            }

        }

    }

}

Just thought of writing a quick post on this new feature that ships with HIS 2009.

To give some introduction about this provider - HIS 2006 ships with first version of ADO.net provider for mainframe and AS400 files systems. At the time product required to have live connection to Host  to read the data from VSAM files and AS400 files. Because of popular demand of supporting the SQL on predownloded binary files, HIS 2009 now supports this provider in OFFLINE mode.

1) ADVOFFLINE is a OFFLINE file ( VSAM or AS400 file that  is FTP'd in BINARY format to a windows workstation ).

2) Lib_ADV.dll -  Metadata assembly that defines record structure(s) that a ADVOFFLINE can have.

3) ADVCOPY - is a copybook for the original ONLINE VSAM file.

4) program.cs is a sample that does a SELECT ON a ADVOFFLINE.

program.cs code :

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using Microsoft.HostIntegration.MsHostFileClient;

using System.IO;

using System.Reflection;

using System.Data;

namespace OFFLINEUnions

{

    class Program

    {

        static void Main(string[] args)

        {

            string cnstring = @"Local Folder=C:\HisTests\DataTests\OFFLINE;Metadata=C:\HISTests\DataTests\Binaries\HostFileProvider\Lib_ADV.DLL";

            HostFileConnection cn = new HostFileConnection(cnstring);

            try

            {

                Assembly HostFileMetadata   = Assembly.LoadFile(@"c:\histests\DataTests\Binaries\HostFileProvider\Lib_ADV.DLL"); 

                cn.Metadata = HostFileMetadata;

                cn.Open();

                HostFileCommand cm2 = new HostFileCommand(cn);

                cm2.CommandText = "SELECT * from ADVOFFLINE as Lib_ADV.OUT_FILE_REC1";

                HostFileRecordSet hfrs = cm2.ExecuteRecordSet();

                StreamWriter ADVstream = new StreamWriter(@"c:\Histests\datatests\Scripts\SELECT_ADV.txt" );

                while (hfrs.Read())

                {

                    ADVstream.WriteLine(hfrs.GetString(0));

                    int lev88_colNumber = hfrs.GetOrdinal("OUT1_LEV88");

                    ADVstream.WriteLine(hfrs.GetInt16(lev88_colNumber));

                    if (hfrs.GetInt16(lev88_colNumber).Equals(1))

                    {

                        System.Console.WriteLine(hfrs.GetString(hfrs.GetOrdinal("UNION1")));

                        ADVstream.WriteLine(hfrs.GetString(hfrs.GetOrdinal("UNION1")));

                    }

                    if (hfrs.GetInt16(lev88_colNumber).Equals(2))

                    {

                        ADVstream.WriteLine(hfrs.GetDecimal(hfrs.GetOrdinal("UNION1")).ToString());

                    }

                    if (hfrs.GetInt16(lev88_colNumber).Equals(3))

                    {

                        HostFileRecordSet _unionshfrs = (HostFileRecordSet)hfrs.GetRecordSet(hfrs.GetOrdinal("UNION1"));

                        while (_unionshfrs.Read())

                        {

                            ADVstream.WriteLine(_unionshfrs.GetInt16(_unionshfrs.GetOrdinal("OUT1_SMALL1")).ToString());

                            ADVstream.WriteLine(_unionshfrs.GetString(_unionshfrs.GetOrdinal("OUT1_SMALL2")));

                            ADVstream.WriteLine(_unionshfrs.GetInt16(_unionshfrs.GetOrdinal("OUT1_SMALL3")).ToString());

                        }

                        _unionshfrs.Close();

                    }

                    HostFileRecordSet _unionsnumbershfrs = (HostFileRecordSet)hfrs.GetRecordSet(hfrs.GetOrdinal("OUT1_NUMBERS"));

                    while (_unionsnumbershfrs.Read())

                    {

                        string logvalue = _unionsnumbershfrs.GetInt32(_unionsnumbershfrs.GetOrdinal("OUT1_LEADING")).ToString();

                        ADVstream.WriteLine(logvalue);

                        logvalue = _unionsnumbershfrs.GetInt32(_unionsnumbershfrs.GetOrdinal("OUT1_TRAILING")).ToString();

                        ADVstream.WriteLine(logvalue);

                        logvalue = _unionsnumbershfrs.GetInt32(_unionsnumbershfrs.GetOrdinal("OUT1_SEPALEAD")).ToString();

                        ADVstream.WriteLine(logvalue);

                        logvalue = _unionsnumbershfrs.GetInt32(_unionsnumbershfrs.GetOrdinal("OUT1_SEPATRAI")).ToString();

                        ADVstream.WriteLine(logvalue);

                    }

                }

                hfrs.Close();

                ADVstream.Close();

            }

            catch (Exception e)

            {

                Console.WriteLine(e.ToString());

            }

            finally

            {

                if (cn.State.Equals(ConnectionState.Open))

                {

                    cn.Close();

                }

            }

        }

    }

}

Lib_ADV.DLL

There is a small / Big (?) difference between ASMX and WCF with respect to hosting, and interestingly there is a compatibility mode for this same, THIS ARTICLE is great read on it.

 Now, without wasting much of time, here is the problem in hand. 

1. Need to create a WCF service which access Data Base

2. Then host it in IIS 6.0 (not IIS 7.0)

3. Make it Windows integrated Authentication

 4. Build an ASP.NET client and consume service.

 Simple enough? Interestingly there are some tweaks,

 Here are the steps. (I am using windows 2003 server and Visual studio 2008 with Sp1 and .NET framework 3.5)

1. With Visual studio, Create a WCF web service.

2. In IIS 6.0 create a different app pool and use that App pool for this service, Turn on the Windows integrated Auth and uncheck the Anonymous auth (For all the documents and pages or whatever)

3. Then edit the endpoint configuration. You need to use basicHTTPBinding, Here is the code snippet inside the service model tag

<system.serviceModel>

<bindings>

<basicHttpBinding>

<binding name="BasicHttpEndPointBinding">

<security mode="TransportCredentialOnly">

<transport clientCredentialType="Windows"/>

</security>

</binding>

</basicHttpBinding>

</bindings>

4. Once this is done, try debugging the service by requesting the .SVC file. You should get a credential prompt. Everything work fine till here.

5. Create a ASP.NET web app, turn on windows Auth and turn off Anonymous auth. Create a service ref and write a code to access the service.

6. Now here is the interesting part. In most of the cases, you will have a service account which has given the access to DB, and you would like web service to run under that user account. To do this you would go and run the app pool created for WCF service under service account (This is the AD account). Once you do the the application pool will crash saying the user identity should be in SPN. (Need to make changes on Domain controller and all)

7. The above is not the part of solution, so here is what you do, Enable the IIS metabase edit enabled.

8. Open the IIS metadata file wich is in System32/inetsrv/metabase.xml and find IISWebServer tag in there corrosponding to  you app pool name. Add this in there NTAuthenticationProviders="Negotiate,NTLM"

9. Close the IIS metabase xml file and restart the IIS. After doing this, Add this service account to IIS_WPG account. This is very important. Also you do not want <identity impersonate = "true"> for your service.

10. Now, on the client side (Your web app) if you are using the same app pool with Windows Auth, then it will work only on your local machine (Hop Problem), So for web application user the default app pool with network service as user.

You should be good now.

The other way to make use of this log information is if your shop face  the similar upgrade issues over and over you can refer to failed jobs for failure messages, return codes  etc.

these are the steps that would work for most of the shops:

1) go to the SDSF panel

2) locate the job you want to copy

3) in the command line print XD

4) this will pull  the  SDSF  open print dataset  , which looks like :

                         SDSF Open Print Data Set                          
 COMMAND INPUT ===>                                            SCROLL ===> CSR 
                                                                               
                                                                               
 Data set name  ===> 'HQ.SPROC.OUT'                                       
 Member to use  ===>                                                           
 Disposition    ===> NEW        (OLD, NEW, SHR, MOD)                           
                                                                               
 If the data set is to be created, specify the following.                      
 Volume serial will be used to locate existing data sets if specified.         
                                                                               
 Management class     ===>           (Blank for default management class)      
 Storage class        ===>           (Blank for default storage class)         
   Volume serial      ===>           (Blank for authorized default volume)     
   Device type        ===>           (Generic unit or device address)          
 Data class           ===> DBCLASS   (Blank for default data class)            
   Space units        ===> TRKS      (BLKS, TRKS, CYLS, BY, KB, or MB)         
   Primary quantity   ===> 75        (In above units)                          
   Secondary quantity ===> 5         (In above units)                          
   Directory blocks   ===>           (Zero for sequential data set)            
   Record format      ===> VBA                                                 
   Record length      ===> 240                                                 
   Block size         ===> 3120                                                 

5)  hit enter, it will take you to the  SDSF again. On the command line print "PRINT CLOSE" and hit enter.

6)  Now you should be able to see the job log captured in dataset - 'HQ.SPROC.OUT'

thanks !

Without talking much -> here are 3 ways to do it.

1. Accept attribute of fileUpload control - Doesnt work with most of the browsers (Bad with IE itself)

2. Regular Expression validator for FileUpload

<asp:RegularExpressionValidator id="UpLoadValidator" runat="server" ErrorMessage="Upload Images only." ValidationExpression="^(([a-zA-Z]:)|(\\{2}\w+)\$?)(\\(\w[\w].*))(.jpg|.png|.bmp|.jpeg|.gif)$" ControlToValidate="FileUpload1"> </asp:RegularExpressionValidator>

This is Not very handy when using AJAX controls on page- Both javascripts dont like each other :)

3. Write a code behind function in a very simple way - Most robust way to use it. - Very Generic can be used for any file types.

private bool IsFileValid()

string [] allowedImageTyps = {"image/gif", "image/pjpeg", "image/bmp","image/x-png"};

StringCollection imageTypes = new StringCollection();

imageTypes.AddRange(allowedImageTyps);

if (imageTypes.Contains(FileUpload1.PostedFile.ContentType))

return true;

else

{

return false;

}

 NOTE - use FileUpload1.PostedFile.ContentType and not the file name to check the Extension

Leave a comment if this helped you

https://www.microsoft.com/downloads/details.aspx?FamilyID=c6c3e9ef-ba29-4a43-8d69-a2bed18fe73c&DisplayLang=en

This provider  supports new datatypes added  in SQL 2008 .More details information on how the new datatypes correspond to OLEDB data types is available here :

https://msdn.microsoft.com/en-us/library/ms175970.aspx

Browser can be downloaded from https://www.google.com/chrome

1.  Silver light is not Supported  (No Silverlight to start with)

2.  Ajax toolkit project will not work as they work in IE. I have seen even IE8 breaks some functionality in AJAX enabled web pages.

3. You can debug/dump the JavaScript for a page as well as separate JavaScript console.

4. Basic problem from DEV perspective - Newlines are interpreted in much diff way in markup. Also Image (absolute)  alignment are bit different. 

Overall the browser is fast....feels much like  Firefox.

Things that I liked

1. No Window header (The Title of the Browser) Tabs start from (0x0) - maximizing the browse area

2. Caching is nice.

3. Creates the list of frequently visited site instead of Showing "You have just opened a new Tab"

4. Search and address bar are the same and gets very handy in 2 minutes.......no learning curve there.

using System;

using System.Collections.Generic;

using System.Text;

using Microsoft.HostIntegration.MsHostFileClient;

namespace ConsoleApplication1

class Program

static void Main(string[] args)

//connectionstring

string connectionstring = "Provider=SNAOLEDB;User ID=USERID;Password=PASSWORD;APPC Remote LU Alias=RUSDFM;APPC Local LU Alias=P31680;APPC Mode Name=PA62TKNU;Network Transport Library=TCP;Host CCSID=37;PC Code Page=1252;Network Address=HOST;Network Port=PORTNUMBER;Process Binary as Character=False;Persist Security Info=True;Cache Authentication=False;Cache Authentication=False;Encrypt Password=False;Impersonation Level=Impersonate;Location=HOST;Protection Level=Connect;Default Library=USERID;Repair Host Keys=False;Strict Validation=False;Mask Password=False;";

HostFileConnection cn = new HostFileConnection(connectionstring);

try

cn.Open();

HostFileCommand cmd = new HostFileCommand("EXEC 'SNDMSG MSG(HELLO) TOUSR(USERID)'", cn);

string RESULT = (string)cmd.ExecuteScalar();

// the result is dependednt upon the command getting executed.

// if command is wrong , it will give syntax error and will not execute at all.

if (RESULT.Equals(string.Empty))

{

// check for result to determine successfull execution of command.

}

catch (Exception e)

finally

{

cn.Close();

}