I won’t be allowed to post to my MSDN blog any more, so I found a new home at http://cathydumas.com/. You can watch there for updates.  You can also contact me through my blog and on Linked In.

Cheers

Rosling made sure all his information was put up publically on gapminder.org. I resolved to use this data set to come up with some nifty visualizations, but never quite got around to this project – until now.

I had to do quite a bit of ETL to get the data into a form that was suitable for consumption by DAX. Also, I needed some pictures so my Power View reports would look nice, so I had to put in some flags from the CIA World Fact Book. The mapping of country to continent I got from the web, unfortunately I lost the link to the original data source but it is publically available information. Once I got all this data together, I put it into one single SQL Server data source for easy import into PowerPivot. I made a PowerPivot workbook, and then a Power View report. And now, we have exciting data in Microsoft tools as well:

Here are the samples:

• The SQL Server data source for the project, so you can muck about with the data yourself.
• The PowerPivot workbook for the project
• The Power View report for the project

Enjoy

If I were to write the MSDN on the Impersonations Settings property, I might write something like this:

Even that description is a bit convoluted. Here is a pictorial example, which may aid understanding.

Let’s assume we have the following:

• There is a user that has admin access to the AS instance and read access to the SQL Server data source. In this example, it is me – REDMOND\cathyk. Also, for illustration purposes, I am running the AS service account as myself.
• There is a second user that does not have access to either the AS instance or the SQL Server data source. In this example, it is REDMOND\sqlcl01.

Now we can create our model. I create a tabular model and set it to DirectQuery mode by selecting the Model.bim file, viewing the property grid, then setting the DirectQuery Mode property to On. Next, I import data from my SQL Server data source, specifying either REDMOND\cathyk or the service account for the impersonation credentials (in this case they are the same). I then open the Role Manager and create a Read role with REDMOND\sqlcl01 as a member. Now let’s run some queries for REDMOND\sqlcl01 in three scenarios.

Scenario 1: Using default impersonation settings

I right click the project, then select Properties. Here are my settings:

Query Mode=DirectQuery means that a VertiPaq cache is not constructed for the model, so end users always use DirectQuery to connect to the data source at query time. Impersonation Settings=Default says use the credentials from the import wizard to connect.

F5 to deploy the model. Now let’s see what happens when REDMOND\sqlcl01 tries to connect:

Success. REDMOND\sqlcl01 sees the data because the credentials on the data source, and not the current user’s credentials are used. To verify that this is indeed the case, we need two profiler traces.

First, let’s look at the Analysis Services Profiler trace. You will need to customize the trace events by clicking “Show all events”, then selecting Direct Query Begin, Direct Query End, VertiPaq SE Query Begin, VertiPaq SE Query End. Now check the results of the trace for the DAX query issued above.

You can see from the trace that DirectQuery was used to connect to the data source. Now we check the Profiler trace for the SQL Server instance, which shows the credentials:.

You see that the credentials specified on the data source, in this case REDMOND\cathyk, are passed along and used to execute the query. Thus, we have shown that using the Default impersonation setting allows users without read access to the underlying data sources to get data from SQL Server.

Scenario 2: Impersonating the current user in a DirectQuery only model

Now, let’s reconfigure our model and try a different setting. Let’s change the impersonation credentials to impersonate the current user, as pictured below.

F5 to redeploy the model. Now let’s see what happens when REDMOND\sqlcl01 tries to connect:

You can see that the query fails. The complete error message in this case was “A connection could not be made to the data source with the DataSourceID of 'bc0bbbec-9cc4-4c53-aca9-b3569b0edeb3', Name of 'SqlServer localhost BikeSales'. OLE DB or ODBC error: Login failed for user 'REDMOND\SQLCL01'.; 28000.” If you look in Profiler for the SQL Server data source, you see that there was no query executed by Analysis Services this time.

Observe the Profiler trace for AS:

What that shows is that AS tried but failed to connect directly to the data source as REDMOND\sqlcl01. Thus, we have showed that impersonating the current user can cause connection attempts to fail even though the data was imported successfully in the import wizard.

Scenario 3: Impersonating the current user in a hybrid model

Just to make things more complicated, now let’s consider what happens if we have configured our DirectQuery model to work in a hybrid mode, so that users can hit either the VertiPaq cache or the DirectQuery data source directly. Let’s change our deployment settings as pictured, setting the Query Mode to "In-Memory with DirectQuery”.

F5 to redeploy. This time you see that processing happens after deployment, which makes sense because we have a hybrid mode. Now let’s see what happens when REDMOND\sqlcl01 runs a query:

Query succeeds. Why? The Profiler trace on the SQL Server data source yields no clues, as there are no trace events showing.  The Analysis Services Profiler trace, however, is much more illuminating:

User REDMOND\sqlcl01 hit the VertiPaq cache to get the query results. This is as expected for the hybrid mode. So, even though the current user is not authorized to connect to the SQL Server data source, she can still see the data because it already existed in the VertiPaq cache and the results were fetched from there.

What’s the bottom line – which impersonation settings should be used?

Impersonating the current user has its pros and cons:

• Con: you have to configure access on both Analysis Services and the SQL Server data source
• Pro: the security model on SQL Server is much fancier than the one for tabular models, so you might find the pain of configuring access worth the gain of the fine grained security model
• Con: the behaviour of Analysis Services can be confusing in hybrid mode, as hitting the VertiPaq cache yields extremely different results from hitting the data source directly

Using default impersonation settings has its pros and cons:

• Pro: conceptually much simpler, same account is always used to access data, and it’s the same account you specify in the import wizard
• Con: all users with read access to the model can read everything. Row level security is not allowed for DirectQuery models in SQL Server 2012. Limiting access to a subset of data to some users is impossible.

The right answer for you depends on your use case. Have fun

The user settings file

First, the user settings file. When I refer to the user settings file, I am referring to the hidden file called “Model.bim_<username>.settings”. For RC0 models, this file is very small. It does a few things. This file is created whenever a project (new or existing) is opened for the first time by this user. The file stores personal settings such as the workspace database name, the workspace database retention policy, and the back up policy. Also, it stores a bit of metadata that is used to populate the error list on open as needed. This is all private, per-user data that you should not be mucking about with and also not sharing with other users. It is OK to delete this file if you want to force usage of the ABF (as described in my previous article), but you should not ever modify this file nor check it into source control.

The .layout file

Now, the Model.bim.layout file. We created this new file for RC0 to solve a specific problem. The problem is with persistence of view state information. We save out a lot of view state information for tabular models. Some of this view state information is something that you would want to share with other developers in a shared development environment. The position of measures in the measure grid and the position of tables and connectors in the diagram are two such examples. If you have ever rearranged a complicated diagram, you know how much work this takes and you would want to share your hard work with others!

We did not want to save this information in the .bim file. The reason – developers have a hard enough time trying to figure out what changed in the model between iterations without adding the view state information. Last thing you need is to force a change to the .bim file whenever you move something with your mouse in the table. It becomes very difficult very quickly to differentiate real changes (such as add/remove of measures, tables, columns, etc) vs simply cosmetic changes. What to do?

Thus, the .layout file is born. It stores all the view state information. This new .layout file is very smart. Triggering a view state change (such as moving a table in the diagram) shows the dirty flag on the .bim file, even though the .bim file itself is not dirty. This enables VS to prompt for save on close, so you never lose your view state changes (unless you close without saving). The .layout file is renamed automatically when the Model.bim file is renamed. Should the .layout file ever disappear, don’t worry. All the model metadata is safe. Only view state information is lost, and models can still be opened and edited without issue. Also, stale information in the .layout file (anything not matching model metadata) is simply ignored. We tested this pretty thoroughly and it works pretty well.

Behind the design of the .layout file

You might wonder where we had this view state information before RC0, as it wasn’t in the bim file. Previously, it was in the user settings file. That approach was problematic because the user settings file cannot be shared in a source control environment. The designer ensures that you only use the user settings file with your name on it. The .layout file is much better because it can be shared.

You might also wonder why, if this information in the .layout file is so important, the .layout file isn’t added by default to the .smproj. This is a cut we had to make due to time constraints. Getting the Solution Explorer to work with your custom project types is actually quite tricky and time consuming, so we took a shortcut there. Another shortcut that we made was that we just put all the view state information into the .layout file, which includes personal information such as sheet tab selection, diagram vs. grid view selection, filter selection, etc. So if you choose to share the .layout file, you are sharing all non-metadata things that each user can see. This is not ideal, but it beats the alternative of not being able to share anything at all.

Sharing the .layout file

If you are in a source control environment and you want to share the diagram and measure grid settings, here is what you do:

1. Create a new project or open an existing project.
2. Open the Solution Explorer. Right click the .smproj, select Add->Existing item.
3. Select the Model.bim.layout file (in the same folder as the .bim file you are currently using), OK.
4. Add the Model.bim.layout file to source control and check it in.

Obviously, sharing is optional. If you don’t want developers to be dirtying your project by switching back and forth between diagram and grid view, don’t share the file. However, I do highly recommend sharing out the layout settings as setting up the diagram is time consuming and there is no point in making every developer do it manually.

Wrap-up – the final file structure

With this, we can now summarize the final project structure for tabular models. You have a parent .smproj project file, with a single child .bim file that contains model metadata. To this project, you can optionally add the .bim.layout file to the project/source control that contains view state information.

We have a couple of auxiliary files in the project folder that help the designer know how to load data. These are the Model.bim_<username>.settings file and the ABF backup of the project.

When you build the project, you get three outputs – the .asdatabase file, the .deploymentoptions file, and the .deploymenttargets file. These build outputs are designed to be used by the Deployment Wizard, but can be used by anybody that wants to deploy the project using any method (AMO/PowerShell/executing XMLA in SSMS).

Eight files – that’s all you need to build and deploy your tabular project, with only two files (the .smproj and .bim) that are absolutely essential to manage. Pretty simple.

----------------------------------------------------------
-- PowerPivot measures command (do not modify manually) --
----------------------------------------------------------

This is actually a meaningful comment. Despite having PowerPivot in the name, it is used in the tabular designer in Visual Studio as well. This header is used to tell the measure grid where the measures are in the model.

If you modify the header manually, here is what will happen. Say you insert a carriage return between the <Text> and the first hyphen in the comment. When you reopen the designer, you will see that all of your measures are gone from the measure grid! The measures are still in the model, and in fact when you deploy to the server you can query the measures perfectly fine. The designer simply cannot see the measures because the text does not match exactly the text generated by the designer.

I found this quirk of design not by hand-editing my BIM file (which is bad), but when processing my .asdatabase file as shown in my msbuild customization example. PowerShell, not understanding that it could not treat the .asdatabase file as any other XML file, inserted line breaks. When I deployed the database, everything seemed fine. However, next time I created a new tabular project by importing my model, the measures were all gone from the database. In a panic (thinking my demo ruined!) I filed a high priority bug and got a bunch of people to gather round and help me with my conundrum. And as it turns out, I just had to delete one line break from the XMLA, and my model was restored.

Bottom line: that command above is not joking. Do not modify the PowerPivot measures command manually. Consider yourself warned.

What you need is the Existing Connections dialog. To start the Existing Connections dialog, go to Model->Existing Connections or click the Existing Connections button on the toolbar:

Either operation starts the Existing Connections dialog. Select the connection you want to reuse, then click Open:

This operation then takes you directly into the import wizard, as shown:

Now you can import your table as shown.

If you forget to take this step, and simply import through the import wizard multiple times, connections will simply proliferate in the Existing Connections dialog, as shown:

Unfortunately, since you cannot change the connection used by a table after it has been created, you are stuck with these multiple connections forever unless you delete all tables associated with the “bad” connection and re-import them using the “good” connection.

Moral of the story – the Existing Connections dialog box is your friend, I know it is awkward to import from there (I trip over this too!) but you will learn to do it with practice over time.

Now on to the summary of the security model for tabular models for SQL Server 2012. There are two types of permissions for tabular models: database permissions and allowed row sets. A database permission is simple – a user can administer, read, or process a database. Granting a user read access to the database allows the user to see all data (but not metadata such as the DAX calculations included in the model). If you would like to restrict the amount of data the user in the read role can see, you can change the number of rows returned for a particular table by applying a row filter. Row filters affect related tables in the one to many direction, so applying a filter to a lookup table always affects the related fact tables. Aggregates in the table always aggregate the filtered row set. If multiple row filters are applied to a table, the user sees the intersection of the rows returned by each individual filter. If a user is a member of multiple roles, the user sees the union of all results returned for a particular role.

For more basic overview information, see the MSDN documentation on roles and Paul te Braak’s security overview post.  The Create Roles tutorial shows you how to create basic roles and set row filters. There is an awesome new supplemental lesson for RC0, called Implement Dynamic Security Using Row Filters, that shows how to do dynamic security. This is exactly the demo I gave for SQL PASS on dynamic security, except on AdventureWorks. The lesson shows how to add a table with a list of securable objects, “secure” the table, and then modify the allowed rowset for users using the USERNAME() function. We do not have a meaningful example that shows how to use CUSTOMDATA() at this time.

Notes and tips:

• Perspectives, tables, and columns are not securable objects. Only databases and rows are securable objects.
• There is no way to allow a user to view the aggregate data for all rows in the table while restricting permission to view individual rows in the table. Visual totals always apply.
• As Paul te Braak notes in his blog, there is no concept of a strong deny. By setting security permissions, you are defining what the user in a particular role is allowed to see.
• A user with process permission cannot process the database in SSMS. This is because the user is not allowed to see the metadata for the database. The user with process permission can only process via automation. See my management post for details on automation options.
• To “secure” a table for a user, type =FALSE() as the DAX filter and then hide the table from view in the client tools. Note that hiding the table hides the table for all users in all rows. If you want the table itself to be visible in the field list so other users in other roles can manipulate the table, the user with the =FALSE() restriction will still be able to see the table metadata in the field list.
• Only Windows authenticated users can be added to roles.
• Security works best when you are using domain users, and not individual users on a local machine. Analysis Services just works best in a domain environment, and you may get strange results when trying to test out security in Excel or in the “cube browser” when working with users on a local machine.

For example, to change your font in the tabular designer? Change the environment font. Same method as described in my previous article, also have to restart VS for changes to apply. Now, the DAX formula editor is not the hostable editor, it’s a component built in to the tabular designer, so unlike the hostable editor it uses the environment font. That’s the only difference.

And keyboard shortcuts? Just like the Workflow Designer, which has all its keyboard shortcuts in its own context (in its case, WorkflowDesigner.*), the tabular designer has all of its keyboard shortcuts in its own context (AnalysisServices.*). Just like everybody else in VS, you can remap all the keyboard shortcuts using the Tools->Options dialog. The default keyboard shortcuts in the tabular designer are pretty much the same shortcuts as PowerPivot, except that F5 follows VS rules and “starts debugging” (which in our case, builds and deploys).

Finally, toggling between View Code and View Designer. Just like in Workflow, you’re not supposed to be hand editing the underling code (Analysis Services has XMLA, Workflow has XAML, it only took a few months to stop making typos). But just like in Workflow, some people like to look at XML. And just like in Workflow, you can use F7 to switch to code view. (Shift+F7 doesn’t work, much to my chagrin.)

But of course, there are differences. Unlike Workflow, when you change to a high contrast mode color scheme in the tabular designer, the changes take effect right away. So they did teach this old dog one new trick.

In the tabular designer things are a bit more complicated. Tables can have many partitions. Therefore, the set of rows included in the table must be defined in the Partition Manager for each individual partition, not in Edit Table Properties for the entire table. However, there is still a need to have a single definition of the set of columns in the table (the data source view). This can’t be handled at the partition level, as it is impossible to mash together all the partition definition queries for the table and come up with a coherent column set. What to do?

In the end, we decided to keep the Edit Table Properties dialog. For a table with a single partition, the Edit Table Properties dialog works the same as in PowerPivot. If you make a change to Edit Table Properties, the partition definition is automatically updated, the new data is fetched, and all is well. In a table with two or more partitions, the behaviour is different in three ways, as I describe below.

Editing the row set: Editing the row set in Edit Table Properties has no effect on a partitioned table. Even though you see the filter in the Edit Table Properties dialog, it is disregarded at processing time. The individual partition definition queries are used to fetch the data instead.

Adding a column: To add a column to a table, you must add the column using the Edit Table Properties dialog. However, for a partitioned table, just using Edit Table Properties is not sufficient. When you try to process this model after adding the column, you would get the error “The operation failed because the source does not contain the requested column. You can fix this problem by updating the column mappings.” To solve this problem, you must open the Partition Manager and update each individual partition definition to add the column you want. Once you have fixed up both Edit Table Properties and the partition definitions, you can then re-process your model and get the data.

Removing a column: You can remove a column using the Edit Table Properties dialog. This gets rid of the column in the column set, so you no longer see it in table and you can no longer query it. However, for a partitioned model, it is a good idea to modify the partition definition query and get rid of the column there as well to avoid unnecessary data coming down the wire during processing.

This information about having to use both the Edit Table Properties and Partition Manager dialog box to adjust the column set appears in the MSDN documentation, as a note in the Add Columns to a Table topic. Also there’s a little warning in the Edit Table Properties dialog for partitioned models, which tells you about both the column and the row behaviours.

If you miss the docs, you might think that Edit Table Properties is buggy. After all, the row set you see in Edit Table Properties is not necessarily the row set you get when you process. Because of the long winded explanation I gave above, the Edit Table Properties behaviour one of those things that we on the product team call “by design” (a resolution that must be the bane of many bug filers on Connect). Hopefully after this post you can now understand what is going on, and you can now add or remove rows or columns from partitioned models with impunity.

All that said, I would be interested in hearing from you if you consider the current design to be a bug, instead of a design feature. Alberto Ferrari filed a Connect item on Edit Table Properties that you can use. If you think the design should change, feel free to upvote that item and add some comments about what you would consider the correct behaviour to be. Thanks

Question 1: How many impersonation options are supported for tabular models?

1. 2, as shown in the tabular designer
2. 3
3. 4, as shown in SSMS

Question 2: My impersonation credentials are validated in the Import Wizard.

1. True
2. False

Question 3: If I see data in the Preview and Filter pane in the Import Wizard, then the user specified in the impersonation page has access to the data source and my data will be imported.

1. True
2. False

Question 4: The account that I select for the impersonation option on the data source is always used when the engine connects to the data source to fetch data.

1. True
2. False

Got your answers? Good. The correct (and possibly surprising) answer is b for all. Let’s review our answers.

On question 1, the number of impersonation options. Tabular models support the following three options for connecting to a data source: impersonating a specific Windows user, impersonating the service account, and default (which for tabular models means inherit the credentials specified at the database level).

We only expose the first two options in the Import Wizard/Existing Connections dialog because we did not implement any UI that allowed users to enter the Data Source Impersonation Info property at the database level. Without a way to enter the database credentials, we figured it made no sense to show the default option in the Import Wizard since the option is useless at design time. This leaves us with the two options in the import wizard that you see today.

We expose four options in SSMS (impersonating the current user + the three supported options) because we share the impersonation option selection UI with the multidimensional and data mining projects. Since data mining projects support impersonating the current user when connecting to the data source, we show it in the dialog. We did not have time in Denali to implement a custom UI in SSMS for tabular models with only three options, so you’re stuck with the extra useless option.

On question 2, the tabular designer does not validate the impersonation credentials. The designer just passes the credentials along to the engine blindly. You won’t know that the credentials are bad until you try and fail to import the data.

On question 3, import can fail even after successful preview and filter. This is because preview and filter always uses the current user’s credentials when connecting to the data source. Why is this? Well, the tabular designer has to start the data provider itself so it can fetch the data for the preview and filter operation. We decided not to use the credentials on the impersonation page because you would not ever be able to preview and filter if you are impersonating the service account – the engine cannot share with us those credentials. Using the current user’s credentials was simpler.

On question 4, credentials used to connect to the data source. When your model is an in-memory (VertiPaq) model, yes the credentials specified on the data source are always used by the engine when connecting to the data source. However, when your model is a DirectQuery model, things get a bit trickier. There is an Impersonation Settings deployment property that determines the credentials used to connect to the data source. I will go over DirectQuery impersonation in another post.