BCS PowerShell: Introduction and Throttle Management

Your external list worked great when you tested it, but it’s not working on the deployment server! Reading the SharePoint logs, you see a message like one of these:

Timeout expired. BDC Runtime Object Model has throttled the response. The response from the service took more than '180000' milliseconds. The limit can be changed via the "Set-SPBusinessDataCatalogThrottleConfig' cmdlet.

WCF Service Connector has throttled the response. The response from the WCF service contains more than '3000000' bytes. The maximum amount of data that can be read through WCF Service Connector is '3000000' bytes. The limit can be changed via the 'Set-SPBusinessDataCatalogThrottleConfig' cmdlet.

Web Service Connector has throttled the response. The response from the web service contains more than ‘3000000’ bytes. The maximum amount of data that can be read through Web Service Connector is ‘3000000’ bytes. The limit can be changed via the ‘Set-SPBusinessDataCatalogThrottleConfig’ cmdlet.

Opening of a new connection is throttled. Maximum enabled connections ‘200’ for proxy are already been opened.

Database response throttled. Maximum number of rows that can be read through the database system utility is 2000.

I’m Adam Outcalt, a Software Development Engineer for BCS, and in this post, we’ll look at what these messages mean and how to address them.

This type of error means that BCS has throttled an external system call for taking too long, or trying to transfer too much data. This feature is enabled by default to prevent Denial of Service attacks that could adversely affect SharePoint or external system health by performing huge transactions. If you have a business need, you may want to increase the limits or disable them entirely, but keep in mind that you increase your farm’s exposure to Denial of Service threats. (It is possible to throttle on the Office rich clients as well, but this is separately controlled by Local Policies and is outside the scope of this post.)

Only farm administrators can change throttling rules, and the rules cannot be changed through the web controls or SharePoint Designer; you must change them from one of the farm machines, using public APIs or PowerShell. This means that if you are a tenant administrator or site collection administrator, you cannot change throttling rules for your hosting farm; in this case, you should contact your host or administrator to discuss the limits with them.

If you are the farm administrator, good news! We’ve made a broad set of PowerShell cmdlets to help you with various administration tasks. Almost anything that you can do through the web interface, you can also automate through PowerShell, from importing models to managing permissions, and there are a few things that you can exclusively manage in PowerShell. For this post, we’ll be focusing on throttle management, but it helps to have a little context on the SharePoint 2010 Management Shell in general.

SharePoint 2010 Management Shell

The SharePoint 2010 Management Shell is just like any other PowerShell prompt, except that when you start it, it automatically loads the SharePoint cmdlets. If you want to use another shell, just type this command to get all of the BCS cmdlets:

Add-PSSnapin Microsoft.Sharepoint.PowerShell.dll

The Get-Command cmdlet is very useful for finding out about commands available to you. You can see all of the BCS cmdlets by typing:

Get-Command -Noun SP*BusinessData*

Throttle Management

The entry point for most BCS cmdlets is the Service Context, which represents a tenant using a BCS Service Application Proxy. Throttle Management works instead on the Proxy itself, irrespective of tenants. This means that if you are a host farm administrator, you cannot change limits per-tenant. (If you want to do so, you can set up a unique proxy for each tenant, and associate those proxies with their respective web applications.)

The first step, then, is to get the BCS proxy that you want to work with. You can get all of the proxies on your farm with Get-SPServiceApplicationProxy. You can filter this by name to get the one you want, or index into the results. One easy way to get the BCS proxy is:

$bdcProxy = Get-SPServiceApplicationProxy | where {$_.GetType().FullName -eq ('Microsoft.SharePoint.BusinessData.SharedService.' + 'BdcServiceApplicationProxy')}

Once you have a handle on the proxy, you can start working with the throttling rules. BCS throttle management works on a change-and-refresh strategy. If you do anything to change a throttling rule, you should re-fetch it from BCS to be sure that you have the latest copy. To view a rule, run a command like this one:

$dbRule = Get-SPBusinessDataCatalogThrottleConfig -Scope Database -ThrottleType Items -ServiceApplicationProxy $bdcProxy

If you then type $dbRule by itself, you’ll see something like this:

Scope: Database
ThrottleType: Items
Enforced: True
Default : 2000
Max: 1000000

Scope and ThrottleType describe the rule you’re working with. Enforced represents whether it’s turned on, and Default is the limit that is applied to external list. Custom web parts can override the Default limit and work on more data than external lists; they are instead limited by the Max limit.

Once you have a rule, you can change it using a command like any of these examples:

#Default and Maximum must be provided together. This increases the limit for external lists to 3000.
Set-SPBusinessDataCatalogThrottleConfig -Identity $dbRule -Maximum 1000000 -Default 3000

#This disables a throttling rule. Notice the “:” instead of a space.
Set-SPBusinessDataCatalogThrottleConfig -Identity $dbRule -Enforced:$false

#This enables a throttling rule.
Set-SPBusinessDataCatalogThrottleConfig -Identity $dbRule -Enforced:$true

Remember that if you run one of these commands and immediately type $dbRule, you won’t see the changes reflected; you should re-fetch the rule using Get-SPBusinessDataCatalogThrottleConfig again. Your changes are immediately committed by Set-SPBusinessDataCatalogThrottleConfig, but it may take a few minutes for them to be reflected in external lists and other runtime scenarios.

So, what kinds of rules can you set? There are four ThrottleTypes and five Scopes:

ThrottleType	Meaning
Items	The number of records returned
Size	The amount of data returned, in bytes
Connections	The number of connections opened to the database, web service, or .NET assembly
Timeout	The time until an open connection is terminated, in milliseconds

 

Scope	Meaning
Global	Applies to Database, Web Service, WCF, and .NET Assembly Connectors (not to Custom Connectors)
Database	Applies to Database Connectors
WebService	Applies to Web Service Connectors
Wcf	Applies to WCF Connectors
Custom	Applies to Custom Connectors

Here’s a table of the rules that exist:

	Global	Database	WebService	Wcf
Items
Size
Connections
Timeout

You’ll notice that there are no rules for the Custom Scope. This value is reserved for advanced scenarios outside the scope of this post.

Once you have your throttles set up how you like them, be sure to check out some of our other cmdlets with:

Get-Help<cmdlet> -Examples

They can save you a lot of time on repeated tasks!

- Adam Outcalt, Software Development Engineer