This could be real challenging if server is busy as it will create GBs of Data and we might now have such liberty of space.

What if we can run netmon in such a way that it will stop when event of interest occur and do this operation for burst of 5 minutes?

I was able to create PowerShell scrip which runs netmon for 5 minutes (can change time span from 1 sec to 1 hour and even more) then check if event occurred on particular machine (can mention localhost of event is going to occur on local machine itself) and if event occurred then stop execution of PowerSHell script and move files to separate location. And if event didn’t occur then keep running script and start nmcap again.

This Script could be used to run any exe which supports execution from command prompt or PowerShell script.

Certain Points to be considered:

Person running script should be admin on box from where we are checking event log entries otherwise we won’t be able to read logs.

Nmcap.exe should be accessible from command line without giving complete path to exe. If it is not happening then add location to exe in path variable of machine where we are running script.

Need to have both directories that we are using in script.

PowerShell Script:

#Variable which decides to whether run loop or not
 $a = 0
 Do
 {
 #Start time for network trace
 $StartDate = [datetime](Get-Date).ToString()
 #Start running chained network trace for 5 minutes
 nmcap.exe /Network * /Capture /StopWhen /TimeAfter 5 minutes /file F:\netmon\trace.chn:20M
 #get latest 10000 entries from Application event logs from machine "dilkush"
 $SysEvent = Get-Eventlog -Logname Application -ComputerName dilkush -Newest 10000
 #Filter event logs to get logs which start after netmon started, event ID 958
 $SysEvent = $SysEvent | Where-Object { $_.TimeWritten -gt $StartDate} | Where-Object { $_.EventID -Like "958"} | Format-Table TimeWritten, Source, EventID, Message -auto
 #Check if we have ant entries present with filter we mentioned above, if so then enter loop
 if($SysEvent.count -gt 0)
 {
 #Move files to separate directory
 Move-Item F:\netmon\* F:\netmon_zip\ -force
 #Change variable value so that loop doesn't run again
 $a = 1
 }
 else
 {
 #If no events occurred in last 5 minutes then delete files and run loop again.
 Remove-Item F:\netmon\*
 }
 }Until($a -ge 1)

I have shown screenshot of screen that it shows below. (two screenshots, start screen and failure screen)

Now what can I do with this error?

Pretty much nothing other than pulling my hair and scratching my head!!!

So I decided to enable logging for this MSI.

How to enable MSI logging?

https://support.microsoft.com/kb/223300

Once you enable logging, logs will get generated in %temp%

When I looked into this logs I saw below entries,

ExecFirewallExceptions:  Installing firewall exception2 Data Management Gateway Host Service (C:\Program Files\Microsoft Data Management Gateway\1.0\Shared\diahost.exe)

MSI (s) (6C!54) [16:41:36:552]: Closing MSIHANDLE (90) of type 790531 for thread 1620

MSI (s) (6C!54) [16:41:36:552]: Creating MSIHANDLE (91) of type 790531 for thread 1620

ExecFirewallExceptions:  Error 0x800706d9: failed trying to find existing app

MSI (s) (6C!54) [16:41:36:553]: Closing MSIHANDLE (91) of type 790531 for thread 1620

MSI (s) (6C!54) [16:41:36:553]: Creating MSIHANDLE (92) of type 790531 for thread 1620

ExecFirewallExceptions:  Error 0x800706d9: failed to add/update application exception for name 'Data Management Gateway Host Service', file 'C:\Program Files\Microsoft Data Management Gateway\1.0\Shared\diahost.exe'

MSI (s) (6C!54) [16:41:36:553]: Closing MSIHANDLE (92) of type 790531 for thread 1620

So I thought why it is not able to add firewall rule, so I checked firewall service and it was disabled!!

That explains why it is not able to add rule, so I went ahead and enabled firewall service. (you can disable firewall using firewall management utility  but there is no need of disabling firewall service as such.)

I again ran DMG setup after enabling firewall service and as expected it worked fine.

Of course error messages need to improve but till then happy fishing!!!!

Let me know if you have any questions or suggestions….

As per,

https://support.office.live.com/Article/Supported-Data-Sources-and-Data-Types-cb69a30a-2225-451f-a9d0-59d24419782e?ui=en-US&rs=en-US&ad=US

Supported data sources

So if it is not PowerQuery it will not work.

Also few connections in PowerQuery also has few limitations like SharePoint List,

For SPO list online we support Anonymous authentication for refresh however it almost impossible to get anonymous access to SPO resources like list under team site.

So through normal way of PowerBI scheduled data refresh we will not be able to refresh such workbooks.

So how to do that?

This thoughts lead me to do some digging and I end up writing a code which will load excel workbooks from SharePoint Online (PowerBI document library) and refresh them locally and then push those excel files back to SharePoint Online document library.

I have tested this code and it works well.

 Only glitch would be if you have connection which might not have credentials stored in them first time when we try to refresh it will ask for credentials and for subsequent runs it will not pop up anything.

First important thing would be assemblies we need to reference,

So you need to add 2^nd, 3^rd and 4^th ones manually.

Next thing will be imports in code:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using Microsoft.SharePoint.Client;

using System.Security;

using System.IO;

using Microsoft.Office.Interop.Excel;

And finally code itself,

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using Microsoft.SharePoint.Client;
 using System.Security;
 using System.IO;
 using Microsoft.Office.Interop.Excel;
 
 
 namespace SPO_ReadOnlyColumns
 {
 class Program
 {
 static void Main(string[] args)
 {
 //Create Connection to SPO for fetching Excel files
 ClientContext cont = new ClientContext("https://microsoft891.sharepoint.com/PowerBI");
 SecureString Password = new SecureString();
 string password = "*********";
 foreach (char c in password.ToCharArray())
 {
 Password.AppendChar(c);
 }
 cont.Credentials = new SharePointOnlineCredentials("********@microsoft891.onmicrosoft.com", Password);
 Web web = cont.Web;
 cont.Load(web);
 cont.ExecuteQuery();
 //At this point we are connected to SPO
 
 //Connect to Document Library which has Excel files which need to be refreshed , for me it was BI
 var PowerBI= web.Lists.GetByTitle("BI");
 var query = CamlQuery.CreateAllItemsQuery();
 query.FolderServerRelativeUrl = string.Format("/{0}","PowerBI"); //This is relative URL for document library, My BI document library was under PowerBI folder
 var items = PowerBI.GetItems(query);
 cont.Load(items,ic=>ic.Include(i=> i.FileSystemObjectType,i=> i.File)); //Load only files for this document library
 cont.ExecuteQuery();
 //At this point we have fetched list of files into items variable
 
 
 //Create object of excel application
 Application aexcel = new Application();
 aexcel.Visible = false;
 aexcel.DisplayAlerts = false;
 //At this point object of excel app is created
 
 
 //Loop through each file and either refresh all or refresh conditionally
 foreach (var Ex_f in items)
 {
 Microsoft.SharePoint.Client.File Ex_File =Ex_f.File;
 
 if (Ex_File.Name == "RefreshedFromCode.xlsx")
 {
 try
 {
 Console.WriteLine("Loading and refreshing file " + Ex_File.Name);
 FileInformation info = Microsoft.SharePoint.Client.File.OpenBinaryDirect(cont, Ex_File.ServerRelativeUrl);
 Stream sr = info.Stream;
 //Save local copy of file at C:\Temp
 FileStream fs = System.IO.File.Create(@"C:\temp\" + Ex_File.Name);
 sr.CopyTo(fs);
 fs.Close();
 //Local file created
 
 //Create workbook object and refresh
 Workbook wb = aexcel.Workbooks.Open(@"C:\temp\" + Ex_File.Name, 0, false, 5, "", "", false, Microsoft.Office.Interop.Excel.XlPlatform.xlWindows, "", true, false, 0, true, false, false);
 wb.RefreshAll();
 wb.Close(true);
 //refresh completed and close workbook
 Console.WriteLine("Refreshed file " + Ex_File.Name + " Successfully");
 Console.WriteLine();
 
 //Start uploading workbook file back to SPO
 Console.WriteLine("Starting upload of file "+ Ex_File.Name);
 FileStream fs_to_upload = System.IO.File.Open(@"C:\temp\" + Ex_File.Name,FileMode.Open);
 Microsoft.SharePoint.Client.File.SaveBinaryDirect(cont,Ex_File.ServerRelativeUrl,fs_to_upload,true);
 Console.WriteLine("Upload of file "+Ex_File.Name+" Succeeded; :)");
 //Upload to SPO completed
 }
 catch (Exception ex)
 {
 Console.WriteLine("Refreshing or Uploading of file " + Ex_File.Name + " failed. Error: " + ex.Message);
 }
 finally
 {
 Console.ReadLine();
 }
 }
 }
 //At this point we ahve refreshed file and uploaded it back to SPO
 
 }
 }
 }
 
 

I hope this helps few of you out there.
If any help needed or if something breaks in this code feel free to add comment here and I will try to have a look at it.

I have attached code which takes command line parameters for running from command line and scheduling, there are total 5 parameters
1) Site web app URL
2) Username
3) Password
4) Document library which has excel files to be refreshed
5) Relative path to document library from web app

Attached zip file is for 32 bit exe but if you have VS you can change that to 64 bit.

And That's all, i will sat bye at this point.

See you at some other time!!!

SPO_ReadOnlyColumns_X86 (5).zip

1) We can refresh power query connections created against SQL Server and Oracle only at this point.

2) PowerQyery connection against SQL or Oracle which has native query are not supported

What is native query?

When we create powerquery against SQL Server we will get below something like,

Here if you select SQL statement it will give an option to enter query, this query will be considered as native query.

So no refresh for native query and powerquery connection to sources other than SQL and Oracle. For example Odata feed. (Might be possible in future but not now!!!)

So once we have valid SQL PowerQuery connection in excel file next thing will be creating data source from PowerBI admin,

So go to PowerBI admin centre and select data sources -> New data Source -> Power Query

It will ask for connection string,

This connection string we can get from Excel file.

In excel file go to Data -> Connections

Here select your powerquery connection and select Properties.

Go to definition tab,

Select connection string from here.

It will not be very understandable but it works!!!

Sample connection string:

Provider=Microsoft.Mashup.OleDb.1;Data Source=$EmbeddedMashup(817587b2-a68c-4de3-89a4-d870e2ac80e5)$;Location=BinaryTest;Extended Properties="UEsDBBQAAgAIAGaW2kQlNJBXqwAAAPoAAAASABwAQ29uZmlnL1BhY2thZ2UueG1sIKIYACigFAAAAAAAAAAAAAAAAAAAAAAAAAAAAIWPzQqCQBSFX0Vm750fS0qu46JVkBEE0VbGSYd0DB3Td2vRI/UKBWW0a3fOx7c453G7YzLWlXfVbWcaGxMOjHjaqiY3tohJ707+giQSd5k6Z4X2XrLtorHLY1I6d4koHYYBhgCatqCCMU6P6WavSl1n5Cub/7JvbOcyqzSReHiPkQK4gCAMGfCZQDphTI2dMoc5BGIZAkP6g3HVV65vtdTWX2+ + /9xJkL/0TdG1BLAQItABQAAgAIAGaW2kQlNJBXqwAAAAAEwAAAAAAAAAAAAAAAAD3AAAAW0NvbnRlbnRfVHlwZXNdLnhtbFBLAQItABQAAgAIAGqW2kTpZ0KShwAAALwAAAATAAAAAAAAAAAAAAAAAOgBAABGb3JtdWxhcy9TZWN0aW9uMS5tUEsFBgAAAAADAAMAwgAAALwCAAAAAA=="

I have removed few of the characters from Extended Properties so it will not work if you try!!! :P

Next thing will be add this connection string in connection string text box of Power BI admin centre data source and click on next,

Next screen will look like,

Here initially you will not get status as “Configured”, you have to add correct Gateway and credentials to get it configured.

Then test power query connection.

If it shows success, you are ready to use this connection to refresh data in your Excel sheet.

That’s all from me…

If you have any questions feel to add in comment.

This is possible, not recommended and needs little careful settings regarding which ports should be open and which are not needed.

Supportability of this scenario might change in future as this is rapidly changing technology.

To start with steps need to be taken f we want this scenario to work,

Add TCP\IP endpoint for TCP port 1433 (SQL port) on Azure VM Dashboard

Public Port:1433

Private Port: 1433

Data Refresh from Excel workbook sheet:

DataSource.Error: OData: Request failed: Unable to connect to the remote server

Details:

                https://microsoft891.hybridproxy.powerbi.com/ODataService/v1.0/Azure

Test Connection from PowerBI admin on client box:

Login timeout expired A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. Named Pipes Provider: Could not open a connection to SQL Server [53].

Added TCP\IP endpoint for TCP port 8051 on Azure VM Dashboard

Public Port:8051

Private Port: 8051

Data Refresh from Excel workbook sheet:

Data Comes back

Test Connection from PowerBI admin on client box:

Successful.

One very important thing to take care when DMG is installed on Azure VM or any cloud VM , we need to add one entry in host file where we will map public IP of VM to netbios name of VM

If we can't ping netbios name of VM from client box then when we refresh connection from excel client it will give error similar to,

DataSource.Error: OData: Request failed: The remote name could not be resolved: 'sql2014dk'

Details:

    https://microsoft891.hybridproxy.powerbi.com/ODataService/v1.0/Azure

So edit host file

<Public IP of VM>            <netbios name of VM>

And after this follow other instructions mentioned above.

So not many steps but to summaries

n  Host file should have entry pointing public IP of Azure VM to netbios name of Azure VM

n  Need to add two endpoints for VM 8051 (port on which you have configured DMG) and port 1433 (SQL port)

n  Both this ports need to be part of Azure VM firewall rules exception

If even after this steps if you do not get needed data try adding TCP port 80 and 443 into firewall exception and see if that makes any different.

Let me know how this goes and if you have any questions for me.

Till then Hasta la vista

As we need to make sure we install right version of msodbcsql and also unixODBC

So lets start with what we have to do....

Steps to Install SQL ODBC driver on SuseLinux

We have followed following two articles (ans of course some common sense!!!):
https://technet.microsoft.com/en-us/library/hh568449%28v=sql.110%29.aspx
https://onefinepub.com/2013/03/ms-sql-odbc-ubuntu/


Pre requisite so that we have wget, make and gcc commands available

 sudo zypper addrepo https://download.opensuse.org/repositories/network:/utilities/openSUSE_12.1/ opensuse-network-utilities-x86_64
 sudo zypper install wget
 sudo zypper install make
 sudo zypper install gcc
 

1) Install msodbcsql archive for suse

wget https://download.microsoft.com/download/7/3/1/7318E2AD-E7C5-4301-AEFA-A15EDEDACFB4/msodbcsql-11.0.2260.0.tar.gz

2) unzip archive and put it in folder

tar -zxvf msodbcsql-11.0.2260.0.tar.gz

3)chnage directory to msodbcsql-11.0.2260.0

cd msodbcsql-11.0.2260.0

4) Install UnixODBC driver which is required for any ODBC connetcion to work

wget ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.0.tar.gz

5) Unzip unix odbc to install it

tar -zxvf unixODBC-2.3.0.tar.gz

6)Now we have to install unixODBC driver and make sure it is installed correctly

./build_dm.sh --download-url=file://unixODBC-2.3.0.tar.gz.

It will ask whether you want to install script or not, say YES
After this it will go ahead and install unixODBC on your machine.

5) At the end it will tell to change directory to some temp path and run "make install" command
just copy pase those commands and it will do the trick.
Command for my machine was(most probably it will same for everyone, run each commadn separately):  'cd /tmp/unixODBC.8503.14603.23788/unixODBC-2.3.0; make install'

6) After this make sure you get back to your home directory
For my environment it needed following commands

cd..
 cd..
 cd..
 cd home/<user name>/
 cd msodbcsql-11.0.2260.0/

6)Now we have to install SQLODBC (make sure we are in directory msodbcsql-11.0.2260.0)
    -> make sure we are in directory msodbcsql-11.0.2260.0

    -> First we have to Run install script
        

sudo bash install.sh install --accept-license --force

        
    -> make sure the SQL Server dependencies are installed:
        

sudo zypper install openssl libkrb5-3 libc6 e2fsprogs

    --> run sqlcmd -?
        if it gives error "sqlcmd: error while loading shared libraries: libcrypto.so.0.9.8: cannot open shared object file: No such file or directory"
        run below command
        

sudo zypper install libopenssl0_9_8

    
    --> run sqlcmd -?
        and it rill show below output
        

 Microsoft (R) SQL Server Command Line Tool
 Version 11.0.2260.0 Linux
 Copyright (c) 2012 Microsoft. All rights reserved.
 
 usage: sqlcmd [-U login id] [-P password]
 [-S server or Dsn if -D is provided]
 [-H hostname] [-E trusted connection]
 [-N Encrypt Connection][-C Trust Server Certificate]
 [-d use database name] [-l login timeout] [-t query timeout]
 [-h headers] [-s colseparator] [-w screen width]
 [-a packetsize] [-e echo input] [-I Enable Quoted Identifiers]
 [-c cmdend]
 [-q "cmdline query"] [-Q "cmdline query" and exit]
 [-m errorlevel] [-V severitylevel] [-W remove trailing spaces]
 [-u unicode output] [-r[0|1] msgs to stderr]
 [-i inputfile] [-o outputfile]
 [-k[1|2] remove[replace] control characters]
 [-y variable length type display width]
 [-Y fixed length type display width]
 [-p[1] print statistics[colon format]]
 [-R use client regional setting]
 [-b On error batch abort]
 [-D Dsn flag, indicate -S is Dsn]
 [-X[1] disable commands, startup script, environment variables [and exit]]
 [-x disable variable substitution]
 [-? show syntax summary]

    So now sqlcmd is ready for use.
        
    -> if it gives error regarding linking for crpto libraries (not needed if we have followed previous step correctly)
        Create some symlinks so everything works with the paths these binaries are expecting to find libraries: \        

 sudo ln -s /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.10;
 sudo ln -s /lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libssl.so.10;
 sudo ln -s /usr/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0 /usr/lib/x86_64-linux-gnu/libodbcinst.so.1;
 sudo ln -s /usr/lib/x86_64-linux-gnu/libodbc.so.2.0.0 /usr/lib/x86_64-linux-gnu/libodbc.so.1

I will finish this article here. 

If you have any questions feel free to add in comments section.      

Now if this user leaves company or somehow his account moves from one domain to another and he has not added anyone else as MDS users.

So  now everyone will get “Access Denied” when they try to access MDS website as they are not added as MDS user, MDS Admin no longer exist so basically MDS is gone for the toss and no one can do anything with it.

If you have backup of models then you can create new MDS DB and start deploying those models but what if you have too many huge models or if you don’t have backup for models.

In such scenarios we have another way which is not for light hearted people… :P

Let’s see how we can get our MDS back.

In MDS Db in SQL we have few tables and stored proc which will help us in doing this.

Table which is useful to us:

Mdm.tblUser

Stored proc which will help us:

Mdm.udpUserSave

Mdm.udpSecuritySetAdministrator

We will add new user to User table and then make that user Admin.

For adding that user in Users table we will need SID, email ID, friendly name and Domain Name of that user.

We can get this details from DC or we can create one new MDS DB under this user and get all this details from Users table of this new MDS DB.

Assume that I got below details from this new temporary DB and I want to add this user to my already existing MDS DB:

ID MUID Status_ID SID UserName DisplayName Description EmailAddress LastLoginDTM EnterDTM EnterUserID LastChgDTM LastChgUserID
 
 ----------- ------------------------------------ --------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------- --------------------------- --------------------------- ----------- --------------------------- -------------
 
 1 E4E5738E-BBBF-4D1E-917B-40811155E57E 1 S-1-5-21-2146773085-*********-*********-******** XYZ\ABC ABC LNAME NULL ABC@XYZ.COM 2014-04-21 21:32:31.900 2014-02-11 15:49:56.560 0 2014-02-11 15:50:01.297 1
 
 

Highlighted section will be useful for running stored proc which will add user.

Query to add user:

USE [MDS]
 
 GO
 
 
 
 DECLARE @RC int
 
 DECLARE @SystemUser_ID int
 
 DECLARE @User_ID int
 
 DECLARE @SID nvarchar(250)
 
 DECLARE @UserName nvarchar(100)
 
 DECLARE @Status_ID int
 
 DECLARE @DisplayName nvarchar(256)
 
 DECLARE @Description nvarchar(500)
 
 DECLARE @EmailAddress nvarchar(100)
 
 DECLARE @Return_ID int
 
 
 
 -- TODO: Set parameter values here.
 
 set @SystemUser_ID = 2
 
 set @User_ID = 2
 
 set @SID='S-1-5-21-2146773085-*********-*********-********'
 
 set @Username= 'XYZ\ABC'
 
 set @Status_ID = 1
 
 set @DisplayName = 'ABC LNAME'
 
 set @Description = ''
 
 set @EmailAddress = 'ABC@XYZ.COM'
 
 
 
 EXECUTE @RC = [mdm].[udpUserSave]
 
 @SystemUser_ID
 
 ,@User_ID
 
 ,@SID
 
 ,@UserName
 
 ,@Status_ID
 
 ,@DisplayName
 
 ,@Description
 
 ,@EmailAddress
 
 ,@Return_ID OUTPUT
 
 GO

After that when you query Users table you will see this new entry being added there:

4 A6DAA30F-CA91-4442-9166-A25674190A7F 1 S-1-5-21-2146773085-*********-*********-******** XYZ\ABC ABC LNAME ABC@XYZ.COM NULL 2014-05-27 19:27:27.307 2 2014-05-27 19:27:27.307 2
 
 

Now we have to make this user as new admin for which other query will help

USE [MDS]
 
 GO
 
 
 
 DECLARE @RC int
 
 DECLARE @UserName nvarchar(100)
 
 DECLARE @SID nvarchar(250)
 
 DECLARE @DisplayName nvarchar(256)
 
 DECLARE @Description nvarchar(500)
 
 DECLARE @EmailAddress nvarchar(100)
 
 DECLARE @PromoteNonAdmin bit
 
 
 
 set @UserName = 'XYZ\ABC'
 
 set @SID = 'S-1-5-21-2146773085-*********-*********-********'
 
 set @DisplayName = 'ABC LNAME'
 
 set @EmailAddress = 'ABC@XYZ.COM'
 
 set @PromoteNonAdmin = 1
 
 set @Description = ''
 
 
 
 EXECUTE @RC = [mdm].[udpSecuritySetAdministrator]
 
 @UserName
 
 ,@SID
 
 ,@DisplayName
 
 ,@Description
 
 ,@EmailAddress
 
 ,@PromoteNonAdmin
 
 GO
  

Once we make this user as admin it will delete other admin user present in Users table and after that we can login using this new user and then add new users as various roles so that we don’t run into same issue again.

Also make sure we change MDS DB to old DB again once we are done with adding user.

I hope this will help few people in getting out of such difficult situation as most of time when this happens we will be deep trouble as we have to re do all work done so far.

That’s all from me at this point, let me know if anyone has any questions.

When such scenario happens we have seen that SQL Browser memory consumption increases to almost 1.9 GB and after that it just stays there. (Compared to usual 10-20 MB memory consumption)

Now 1.9 GB is all that SQL Browser can use as it is 32 bit process.

So once it reaches that mark it will not have any flexibility to process new incoming requests.

Question comes why memory consumption increases to 1.9 GB at all at first point.

While troubleshooting we figures out that it is some sort of memory management problem where SQL Browser do not release memory and as a result in keeps on staggering till it reaches 1.9 GB.

Only good way to avoid this condition is to set affinity for SQL Browser service to less than 32 processors.

Usually affinity to 8 processor works fine as SQL Browser is not very heavy on CPU generally.

We have seen this problem with SQL 2012 (including machines with only SSAS 2012) and SQL 2008 r2 as well.

So this ends here…. Small and Sweet J

Let me know if you have any questions for me...

So we had SQL Azure server “abcxyz123.database.windows.net” which has user xyz@abc.com as administrator with password “abc123”

So when we try to login to this server from SSMS or if we click on manage on portal and give below credentials it gives error that login failed for user “xyz”

User Name: xyz@abc.com

Password: abc123

Error:

TITLE: Connect to Server

------------------------------

Cannot connect to abcxyz123.database.windows.net.

------------------------------

ADDITIONAL INFORMATION:

Login failed for user ‘xyz'.

This session has been assigned a tracing ID of 'd9c21eee-135c-4c08-86ab-1975bd4c607e'.  Provide this tracing ID to customer support when you need assistance. (Microsoft SQL Server, Error: 18456)

For help, click: https://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&EvtSrc=MSSQLServer&EvtID=18456&LinkId=20476

------------------------------

BUTTONS:

I looked into all possible logs to find out why it is not taking password even though it is correct but did not find anything interesting.

Later I realized that user name that it is using is “xyz” and not xyz@abc.com

SO it is breaking username as soon as it finds “@”

We worked on this at our and end fixed this issue but with small catch.

Now you have to use xyz@abc.com@abcxyz123 (basically username@servername) to connect instead of xyz@abc.com

Now it breaks at last “@” sign and not first “@” sign.

So even now if we use just xyz@abc.com it will break at last “@” and will consider “xyz” as user name so we have to use xyz@abc.com@abcxyz123 as user name.

I Hope this helps to few such people who has “@” sign in their user name and struggling to understand why login is not happening.

Feel free to drop you questions in comments.

Single Data Flow task which has

è Source , which fetches data from Table “xyz”

è Lookup which looks up in table “abc” and finds rows which are already there.

è For new rows look up will push them to no-match output and that will be mapped to OLEDB destination.

è For rows which are matching we will do update on specific column.

For OLEDB destination we are using Fast Load option.

Now when we run this package we get below error.

[OLE DB Command [39]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E14.

An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80040E14  Description: "This operation conflicts with another pending operation on this transaction. The operation failed.".

Package:

So what is going wrong?

We saw that we have table lock property checked, this could be issue as if Insert will have lock on table update will not be able to modify any rows.

We disabled “Table Lock” property but still we faced same error.

Next I took profiler to see what is happening in backend,

Profiler entries:

SQL:BatchStarting  insert bulk [OrdersNew]  62       2013-12-03 19:49:44.967         

SQL:StmtStarting   insert bulk [OrdersNew]  62       2013-12-03 19:49:44.970         

RPC:Starting exec sp_execute 2,187.4100,10395    59       2013-12-03 19:49:44.970         

SP:StmtStarting      update [dbo].[OrdersNew] set Freight = @P1 where OrderId= @P2    59            2013-12-03 19:49:44.970         

Exception      This operation conflicts with another pending operation on this transaction. The operation failed.     59       2013-12-03 19:49:44.970         

So we see that there are two queries started one for bulk insert and other for doing update. And as soon as update starts it gives exception.

I couldn’t find why it is not able to do update even though insert does not have table lock, however I was manage resolve this error.

How?

Use normal “Table or View” for insert and not “Fast Load” option.

So this fast load option is causing this issue.

Hmmm…. May be someone can find root cause for this but if you couldn’t we have solution…

We have seen that this has not caused any performance impact as such on package so should be viable solution.

So that’s all for today!!!!

Let me know if you have any questions about this post.