My customer has created a PowerPivot Model by using Excel 2013. As you may know, PowerPivot is now shipped with Excel 2013 and it is another addin shipped with Excel.

Please note that imagesbelow are illustrative images.

So Customer created very simple model

And also created Pivot Table in a workbook. Everything is working fine excel side.

As next step, we deployed our workbook to SharePoint PowerPivot Gallery and try to render workbook through sharepoint. When you first open workbook. You can see that Excel Services manages to render workbook properly. There is no problem so far. Because workbook didn't need to read data from source or doesn't need to refresh data.

So when you click one of drill down options or filter options, excel workbook needs to refresh data and also needs to load workbook into your powerpivot for SharePoint Instance. At this point we get following error message:

“Unable to refresh data for a data connection in the workbook. Try again or contact your system administrator. The following connections failed to refresh: ThisWorkbookDataModel.”

We couldn't find anything useful in ULS Logs.  After further investigations, we have found out that  This issue occurs because Excel 2013 uses a different method to maintain data models in workbooks than does SQL Server 2008 R2 or SQL Server 2012 PowerPivot for Excel 2010. Excel 2013 uses an internal connection for communication between the workbook and the embedded PowerPivot data. This internal connection does not reference a Microsoft OLE DB provider for Analysis Services (MSOLAP). Whereas PowerPivot for Excel 2010 uses an MSOLAP connection to load the embedded PowerPivot data model from a custom XML part in the workbook.
Excel Services and PowerPivot for SharePoint 2010 load PowerPivot data models from a custom XML part in the workbook. Therefore, this technology is incompatible with internal connections that do not reference an MSOLAP provider (such as the connections that are used by Excel 2013).
Note PowerPivot functionality is not supported by Excel 2013 workbooks that use data models in SharePoint Server 2010 environments.

Seems like only solution here to upgrade your sharePoint farm to 2013 farm or use excel 2010 to create your data models.

To work around this issue, upload the Excel 2013 workbook to a SharePoint Server 2013 farm.
Excel Services and PowerPivot for SharePoint 2013 load Excel 2013 workbooks that use advanced data models and PowerPivot workbooks that are created in PowerPivot for Excel 2010.

If you want to read more about it, here is KB Article:

http://support.microsoft.com/kb/2755126

Thanks

Kagan Arca

How can I learn Business Intelligence? I have been asked this question by my colleagues & customers several times. Actually It is great to see that people in market now realized that Business Intelligence doesn't mean SQL Server or Database. In other words, You shouldn't expect person who has deep SQL Skills to be able to deliver Business Intelligence. Although it took some time market to realise, finally we are there! We can understand from question “How can I learn Business Intelligence?” from people who are in SQL Market for ages.

I generally reply this question with another question: Which Business Intelligence layer are you interested in? Yes, I divided BI into layers:

-Data Warehousing Layer

-ETL Layer (Extract Transform Load)

-Data Processing Layer

-Data Management Information Later

-Data Visualisation & Content Management Layer.

I would recommend my customers and colleagues to start with Data Warehousing concepts. Because it is not easy to switch from stressful OLTP mind set like, transaction/ sec, High batch requests, concurrency, locking/blocking etc… to relaxed Data Warehousing OLAP mind set like, bulk loads, staging, transforming data etc…

Here is my few recommendation about Data Warehousing:

It is very useful to understand Enterprise DWH

http://msdn.microsoft.com/en-us/library/hh146875.aspx

There is also another useful link which provides wide range of resources about Data Warehousing:

http://technet.microsoft.com/en-us/sqlserver/dd421879.aspx

I also recommend you to read book called The Definitive Guide to Dimensional Modeling.

For our Premier customers, we have two modules in BI Academy. These are :

Module 1
BI Fundamentals
Overview of the fundamental concepts in a BI solution by introducing the different multi-dimensional models.

Module 2
Building and loading data warehouses with SSIS & T-SQL
Review how to improve the performance of a BI solution and review how to troubleshoot the most common scenarios, including DW Design, SSIS Development and how to load data into a Data Warehouse.

Covering these topics with some hands on experience should give good starting point to Data Warehousing Layer.

I will be covering rest of layers on following posts.

I was shown what the customer wanted to achieve, they needed to calculate some metrics including sampling errors using the method of “Conglomerados Últimos” (This can be translated to “Conglomerates Last” Method), this method was created by a Spanish Researcher and publish in a Spain Publication to calculate sampling errors a posteriori when the sample is stratified. This method is described in Spanish (Sorry I don’t an English version of this document) here http://dialnet.unirioja.es/descarga/articulo/250808.pdf

Just trying to understand the method and formulas was hard, but the customer had already an elegant solution using an MDX Script and the SCOPE statement. So I focused on the performance issue instead of trying to understand the formulas.

After a couple of hours I found out that the formula was using cell by cell calculation instead of block calculation, then found the operator in fault (the “^” operator) and after a small change (multiply the measure by itself) the query improved from around 30 seconds to 1 second.

You can find some details about the operators that support block computation in “Identifying and Resolving MDX Query Performance Bottlenecks in SQL Server 2005 Analysis Services” found in http://www.microsoft.com/en-us/download/details.aspx?id=661, you can also find some guidelines to improve query performance in http://msdn.microsoft.com/en-us/library/bb934106.aspx

What I want to focus in this blog is how you can detect that the issue is Cell by Cell Calculations.

We have two main tools to troubleshoot this kind of problems, SQL Server Profiler and Performance Monitor, the recommended order is to identify that the problem is Formula Engine using the profiler and then use the Performance monitor to confirm that the query is using Cell By Cell Calculation instead of block calculation (or computation).

Here is the recommended methodology, this is not new, but I haven’t found a good reference that goes through this steps.

Step 1. Identify if bottleneck is Formula engine

Use the profiler to capture the execution of your query, it is easiest if you do this in an isolated environment but if not possible you can still filter the events from your query using the ConnectionId or some other column.

You need to add the duration for each QuerySubcube, and then subtract that total from the duration shown in the QueryEnd Event. In this example you can see that most subcubes take 0 ms, some of them 16 ms and some other that is not shown in the image was 110 ms. You can also see that the total for the query was 3,953 ms, that means that, in the Storage Engine, Analysis Services spent 142 ms and the difference 3811 ms were spent in the Formula Engine.

In conclusion, we have a Formula Engine Bottleneck, let’s check is that bottleneck is caused by cell by calculation.

 The previous procedure may not scale well when you have hundreds of subcubes events or many potential queries using Cell by Cell in the same trace, in that situation you can use a procedure like this one:

1. Save the profiler trace to a SQL Server table
2. Execute the following SQL Query to obtain a list of the longest running queries
   SELECT ROWNUMBER, EVENTCLASS, EVENTSUBCLASS, TEXTDATA, DURATION, CONNECTIONID FROM <TABLE_NAME> WHERE EVENTCLASS=10
3. Execute the following SQL Query to get the beginning and ending points of the query in the trace
   SELECT TOP 2 ROWNUMBER, EVENTCLASS, EVENTSUBCLASS, TEXTDATA, DURATION, CONNECTIONID FROM <TABLE_NAME> WHERE EVENTCLASS IN (9,10) AND CONNECTIONID=<TARGET_CONNECTIONID> AND ROWNUMBER <= <ROWNUMBER_OF_TARGET_QUERY> ORDER BY ROWNUMBER DESC
4. Execute the following SQL Query to determine the amount of time spent in storage engine
   SELECT SUM(DURATION) FROM <TABLE_NAME> WHERE EVENTCLASS=11 AND CONNECTIONID=<TARGET_CONNECTIONID> AND ROWNUMBER BETWEEN <QUERY_BEGIN_ROWNUMBER> AND <QUERY_END_ROWNUMBER>

Step 2. Identify if the issue is Cell by Cell Calculation

You can use Performance Monitor to identify if the issue is that the calculations is being done Cell By Cell. To be successful in this task is necessary that you run your query in an isolated environment because we have a single counter for all queries and to be sure that our query is the one causing the problems you need to execute it alone, without other concurrent queries. If your query is using Cell By Cell calculation you will see something like the following image in your query.

 In the image you can see that the counter being used is “MSOLAP&InstanceName:MDX \ Total Cells Calculated” and the initial value is 0 and the value after you execute the query is a little above 1 million.

This was after restarting the instance and we can see that a single execution of the query had to calculate more than 1 million rows. If this is the first time you do this, you don’t know if 1 million is too much or too low. I can tell you that is very high for this specific query. 

Step 3. Modify your query

Modify the query using the guidelines in the links references until you stop seeing the big increments in cell calculated for your query.

In this particular case the Script was using a couple of expressions like:
Measures.Measure1 ^ 2

We changed the formula to something like:
Measures.Measure1 * Measures.Measure1

After the formula was changed in this particular case we got the following results.

 You can see that the total for the query is 156 ms now, considering that the Storage Engine must have used the same time, 132 ms, it means that now the Formula Engine is taking only 24 ms.

 In the performance Monitor we can see that the total number of cell calculated was 41. The query was executed after another restart of the instance.

Final Comments

This is just a basic procedure to find out what is causing the problem on your query. Modifying the query to avoid cell by cell calculation will be easy sometimes, like in this real case, and will be hard in other scenarios.
I hope this is helpful to start troubleshooting your own cases. 

General Guidelines and Limitations (Windows Azure SQL Database)

http://msdn.microsoft.com/en-us/library/windowsazure/ee336245.aspx#dlaps

Let’s start to see how to use SSIS to migrate data from on-premise database to Azure.

1. Create a DSN via ODBC. The most step is similar with traditional ODBC configuration, just pay more attention on the server name format, SQL authentication, and default database.
2. Now I had a DSN named “AzureDB”, the default database is destination Azure database
3. Using SQL Server Data Tools to create an Integration Service Project.
4. Drag a “Data Flow Task” in Control Flow
5. In Connection Managers, create a new “OLE DB Connection Manager” and choose the on-premise database; Then create a new “ODBC Connection Manager” and choose the Azure database
6. In Data Flow, drag “OLE DB Source” and “ODBC Destination”, then configure the “OLE DB Connection Manager” to the OLE DB Source and choose the table you want to migrate, meanwhile configure “ODBC Connection Manager” to ODBC Destination and choose the table you created in Azure database.
7. Then run the SSIS package, we can see the data migrate to Azure database now.

Enjoy it!

As Microsoft Premier Field Engineering team, we enjoy contributing events such as SQLPASS, SQLBits and SQL Saturdays. I am pleased to announce that I will be delivering a session about Using “SQL Server Integration Services with Hadoop Cluster”. SQL Saturday #258 will be held in Istanbul, Turkey on 5th October. You can find session details in following link. I look forward to meeting you there:

http://www.sqlsaturday.com/258/schedule.aspx

Thanks

Kagan Arca

In Part 2(which will be published in 2-3 weeks), we will show one how to create a basic table.  We will create a basic report.

In Part 3, we will work with filters, slicers and highlights.

In Part 4, we will explore maps and bubble charts.

This week we are talking about the basics of using Power View in a SharePoint environment, PowerPivot or Excel 2013. There are a number of options one can
choose from when working with Power View. We work with a number of clients that are heavy SharePoint or Excel users. They live to export data from a database
or import data from a text file or Azure Data. Then, import it into Excel to massage, manipulate, converge and report on data. Ultimately, they want to
write reports with this data, and they want to do it fast!!! Did we mention fast?

Traditionally reporting can be very fancy but it is not always fast. And most users, analysts or information workers rarely want to wait. And that is why we are going to
start looking at Power View.

Let’s start with Power View 101.

What is Power View? Power View provides intuitive ad-hoc reporting for business users such as data analysts, business decision makers, and information workers.

A Power View report is always presentable –one can explore data and present it at any time, because it is working with real data. It provides reading and full-screen presentation modes, in which the ribbon and other design tools are hidden to focus on the visualizations. However, the report is still fully interactive, with filtering and highlighting capability.

One can also export a Power View report to PowerPoint. Each view in Power View becomes a separate PowerPoint slide.

How cool does Power View sound? Here is how cool it looks. This is a Power View Pie Chart Report.

 

What does Power View require to work? One can utilize Power View in an enterprise solution such as SharePoint.  Or if one has Office 2013, one can use it in Excel 2013. One can easily create and
interact with views of data from data models based on PowerPivot workbooks or tabular models deployed to SQL Server 2012 Analysis Services (SSAS) instances.
Power View is a browser-based Silverlight application launched from SharePoint Server 2010.

If you would like to get started with Power View with a SharePoint environment, this is a great article to use for setup. For this post, we are going to use Power View in Excel 2013.

http://technet.microsoft.com/en-us/library/jj219068.aspx
.

Let’s get started.

1. Open Excel 2013. (In order to verify what version of Office one are currently running, follow this link.)

2.  Click on the Insert Tab

3.  Click on Power View. It looks like this.

            (If one see Power View but it is greyed out, one will need to enable it.)

4. Ok, now we need to import data. After all interactive data exploration is what Power View is all about.

(Power View and Power Pivot are only available in the Office Professional Plus and Office 365 Professional Plus editions.)

5.  For this example, use the following data.  (I am using SQL Windows Azure Data from the Marketplace.)

The next steps will walk one through importing the data into Excel.

6. In Excel, on the PowerPivot tab > Manage Data Model.

Don’t see a PowerPivot tab? Start PowerPivot in
Excel 2013 add-in.

1. Click Get External
   Data > From Data Service > From Windows Azure Marketplace.
2. In the Table
   Import Wizard, under Type > Data, and under Price > Free
3. Search for
   “air carriers”.
4. Next to US
   Air Carrier Delays, click Subscribe.
5. Sign in with a
   Microsoft account. If one don’t have one, follow the steps to create one.

This window shows some sample data.

1. Scroll to the
   end of the sample data and click Select Query.
2. One can type a
   friendly name for the connection, and then click Next.
3. Click Preview
   and Filter.
4. In this window one can pick which columns one want to
   import, by unchecking the check boxes. We want all of them, so click OK,
   and then click Finish.

          The download process may take some time. The dataset has more   than 2.4 million rows.

1. Click OK.
2. Now you have imported data! Congrats!

Questions or comments?

http://msdn.microsoft.com/en-us/library/dn393915.aspx

Feel free to post your questions!

Thanks

Kagan

The development process typically presents a certain set of challenges, given that BI projects must be molded to meet often highly unique business requirements. So you've worked for several weeks building an optimal solution and it's finally time to deploy the solution only to encounter an error message something like the following:

The provider 'XXXOLEDB.1" is not registered.

The following system error occurred:  Class not registered

A connection could not be made to the data source with the DataSourceID of '4a023caf-72d5-4021-af2d-49568052e42c', Name of 'Custom 3'.

The error message is reasonably self-explanatory, and is raised because the provider used for the data connection doesn't exist on the system. But wait, the project was developed on the same machine that it was deployed to so obviously the OLE DB provider exists and is installed on the system since it could be visualized in the Visual Studio development environment.

Yes, the OLE DB provider does exist and it can be visualized from the Visual Studio IDE, so you can rest assured that you weren't dreaming (or having a nightmare) for several weeks. What really happened is that the Visual Studio IDE is a 32-bit application which will result in calls to the 32-bit versions of oledb32.dll and rundll32.dll which allows the Visual Studio shell to only see the 32-bit providers and not the 64-bit providers. In the vast majority of instances, the server processes are 64-bit applications which will result in calls to the 64-bit versions of oledb32.dll and rundll32.dll which will allow the server processes to see only the 64-bit OLE DB providers and not the 32-bit providers.

So now the question arises "How can I make certain that the correct 64-bit provider is installed on my server?" Fortunately, Faruk Celik created a blog posting that describes how to use a UDL file on a 64 bit machine.

You can test it out by creating a new text file on the desktop named "test.udl". Once that's created, execute the following command from an elevated command prompt to see the 32-bit providers that exist on the machine (note that you'll need to substitute your account for <user_alias>):

C:\Windows\syswow64\rundll32.exe "C:\Program Files (x86)\Common Files\System\Ole DB\oledb32.dll",OpenDSLFile  C:\users\<user_alias>\desktop\test.udl

You should see a Universal Data Link property sheet that lists all of the 32-bit providers that are currently registered on the system.

To open the same file and view the 64-bit providers that are installed, run the following command from an elevated command prompt:

C:\Windows\system32\rundll32.exe "C:\Program Files\Common Files\System\Ole DB\oledb32.dll", OpenDSLFile  C:\users\<user_alias>\desktop\test.udl

If you try to create one without installing SSIS, you will get following error message about SSISDBBackup.bak

The catalog backup file ‘C:\Program Files\Microsoft SQL Server\110\DTS\Binn\SSISDBBackup.bak’ could not be accessed. Make sure the database file exists, and the SQL Server service account is able to access it. (Microsoft.SqlServer.IntegrationServices.Common.ObjectModel)

In some sites, as a workaround, they suggest to restore SSISDBBackup.bak to be able to use SSISDB. It is true that you can restore bak file from machine which SSIS 2012 installed however you can not use SSISDB even you restore database successfully.

Some users say, they just want SSISDB to store packages but this is not possible as well. When it comes to deployment of this package to SSISDB, deployment will fail with ISServer.exe not found error message:

As conclusion, in order to functionality of SSISDB like store, deploy, run packages through SSISDB Catalog, you have to install SQL Server Integration Services 2012.

Let me know if you have any questions.

Kagan

The error message that was being returned was:

An error prevented the view from loading. (Microsoft Visual Studio)

===================================

An error occurred while attempting to start the report preview worker process. (Microsoft.ReportingServices.Designer)

Given that this scenario worked with previous editions of Reporting Services and BI Development Studio, the error was unexpected.  On further investigation, the same behavior and error could be reproduced by launching a Remote Desktop session in Seamless or Remote Applications Integrated Locally (RAIL) mode.

RAIL extends the RDP protocol presents a remote application running on a RAIL server as a local user application running on the client machine. This causes the remote application to appear as if it is running on the user's local computer rather than being presented in the desktop of the remote Server.

In this scenario, the behavior and the error message are both by design.

SSDT is a multi-targeted assembly that is built around the .NET 4.0 Framework. Business Intelligence Development Studio (BIDS) is built around the .NET 3.5 Framework and there are differences in the way that the .NET 4.0 Framework deals with child processes when the parent process dies. As a result, there was a deliberate decision to change the behavior of the report viewer in preview mode. This is largely because under the .NET 4.0 Code Access Security policy, executing under the current AppDomain is no longer supported, necessitating use of a sandbox. You may wish to read Brian Hartman's blog posting "Expression Evaluation in Local Mode" for a more detailed discussion of Visual Studio 2010 defaulting to the .NET 4.0 CAS policy. The net result of the change is that when a report is previewed, SSDT spawns a child process to allow previewing of the report. In the RDP using Seamless or RAIL mode, it would still be necessary for SSDT to spin up a new instance of the Reporting Services Preview Processing Host (PreviewProcessingService.exe) separate from the SSDT process. In the event that the user were to close SSDT or SSDT were to crash, it would be possible for orphaned instances of the Reporting Services Preview Processing Host application to remain in memory and continue execution after SSDT was terminated.

The change makes it cleaner to close SSDT and eliminates the possibility of orphaned child processes in the event that the SSDT environment is terminated for some reason. Had the change not been made, it would have been possible to allow orphaned child applications/processes to continue to execute, using CPU cycles and consuming server memory, until the user logged off the machine. If the user didn’t log off and rather elected to simply disconnect without logging off, any orphaned applications would have remained in memory for an indefinite period of time. In the scenario where a machine is being remotely accessed by multiple users, that could potentially mean thousands of orphaned applications would be in memory and actively executing at any given time.