Note that RBS2016 will no longer use a credential store symmetric key that uses the TRIPLE_DES option.  It will instead use AES_128 which addresses the first issue in the paragraph above.  It's also worth noting that SQL Server 2016 will not even allow you  to create a TRIPLE_DES symmetric key when using compatibility mode 130.

One can determine the RBS credential store symmetric key properties by issuing the following T-SQL query on the RBS database:

SELECT * FROM sys.symmetric_keys WHERE name = 'mssqlrbs_encryption_skey';

If the output from the above statement shows that TRIPLE_DES is still used, then it is advised to rotate this key.

To rotate the RBS credential store symmetric key, one may use the attached script on the RBS database.  Do backup your database prior to key rotation.  At the script’s conclusion, it has some verification steps.

If security policies require different key properties (e.g., algorithm or key length) from the ones provided, then the script may be used as a template for modification at one’s own risk.  One would need to simply change the key properties in two places: 1) the creation of the temporary key 2) the creation of the permanent key.

The RBS credential store symmetric key rotation script is as follows:

IF EXISTS (SELECT * FROM sys.procedures WHERE name = 'sp_rotate_rbs_symmetric_credential_key')

BEGIN

       DROP PROC sp_rotate_rbs_symmetric_credential_key;

END

/* This stored procedure will replace the currently used RBS credential store

symmetric key with one of your choosing.  One may want to do this if there is

a security policy requiring periodic key rotation or if there are specific

algorithm requirements, for example.  In this stored procedure, a symmetric

key using AES_256 will replace the current one.  As a result of the symmetric

key replacement, secrets need to be re-encrypted with the new key.  This stored

procedure will also re-encrypt the secrets.  The database should be backed up

prior to key rotation. */

CREATE PROC sp_rotate_rbs_symmetric_credential_key

AS

BEGIN

BEGIN TRANSACTION;

BEGIN TRY

CLOSE ALL SYMMETRIC KEYS;

/* Prove that all secrets can be re-encrypted, by creating a

temporary key (#mssqlrbs_encryption_skey) and create a

temp table (#myTable) to hold the re-encrypted secrets. 

Check to see if all re-encryption worked before moving on.*/

CREATE TABLE #myTable(sql_user_sid VARBINARY(85) NOT NULL,

    blob_store_id SMALLINT NOT NULL,

    credential_name NVARCHAR(256) COLLATE Latin1_General_BIN2 NOT NULL,

    old_secret VARBINARY(MAX), -- holds secrets while existing symmetric key is deleted

    credential_secret VARBINARY(MAX)); -- holds secrets with the new permanent symmetric key

/* Create a new temporary symmetric key with which the credential store secrets

can be re-encrypted. These will be used once the existing symmetric key is deleted.*/

CREATE SYMMETRIC KEY #mssqlrbs_encryption_skey 

    WITH ALGORITHM = AES_256 ENCRYPTION BY

    CERTIFICATE [cert_mssqlrbs_encryption];

OPEN SYMMETRIC KEY #mssqlrbs_encryption_skey 

    DECRYPTION BY CERTIFICATE [cert_mssqlrbs_encryption];

INSERT INTO #myTable

    SELECT cred_store.sql_user_sid, cred_store.blob_store_id, cred_store.credential_name,

    encryptbykey(

        key_guid('#mssqlrbs_encryption_skey'),

        decryptbykeyautocert(cert_id('cert_mssqlrbs_encryption'),

            NULL, cred_store.credential_secret)

        ),

    NULL

    FROM [mssqlrbs_resources].[rbs_internal_blob_store_credentials] AS cred_store;

IF( EXISTS(SELECT * FROM #myTable WHERE old_secret IS NULL))

BEGIN

    PRINT 'Abort. Failed to read some values';

    SELECT * FROM #myTable;

    ROLLBACK;

END;

ELSE

BEGIN

/* Re-encryption worked, so go ahead and drop the existing RBS credential store

symmetric key and replace it with a new symmetric key.*/

DROP SYMMETRIC KEY [mssqlrbs_encryption_skey];

CREATE SYMMETRIC KEY [mssqlrbs_encryption_skey]

WITH ALGORITHM = AES_256

ENCRYPTION BY CERTIFICATE [cert_mssqlrbs_encryption];

OPEN SYMMETRIC KEY [mssqlrbs_encryption_skey]

DECRYPTION BY CERTIFICATE [cert_mssqlrbs_encryption];

/*Re-encrypt using the new permanent symmetric key. 

Verify if encryption provided a result*/

UPDATE #myTable

SET [credential_secret] =

    encryptbykey(key_guid('mssqlrbs_encryption_skey'), decryptbykey(old_secret))

IF( EXISTS(SELECT * FROM #myTable WHERE credential_secret IS NULL))

BEGIN

    PRINT 'Aborted. Failed to re-encrypt some values'

    SELECT * FROM #myTable

    ROLLBACK

END

ELSE

BEGIN

/* Replace the actual RBS credential store secrets with the newly

encrypted secrets stored in the temp table #myTable.*/             

SET NOCOUNT ON;

DECLARE @sql_user_sid varbinary(85);

DECLARE @blob_store_id smallint;

DECLARE @credential_name varchar(256);

DECLARE @credential_secret varbinary(256);

DECLARE curSecretValue CURSOR

    FOR SELECT sql_user_sid, blob_store_id, credential_name, credential_secret

FROM #myTable ORDER BY sql_user_sid, blob_store_id, credential_name;

OPEN curSecretValue;

FETCH NEXT FROM curSecretValue

    INTO @sql_user_sid, @blob_store_id, @credential_name, @credential_secret

WHILE @@FETCH_STATUS = 0

BEGIN

    UPDATE [mssqlrbs_resources].[rbs_internal_blob_store_credentials]

        SET [credential_secret] = @credential_secret

        FROM [mssqlrbs_resources].[rbs_internal_blob_store_credentials]

        WHERE sql_user_sid = @sql_user_sid AND blob_store_id = @blob_store_id AND

            credential_name = @credential_name

FETCH NEXT FROM curSecretValue

    INTO @sql_user_sid, @blob_store_id, @credential_name, @credential_secret

END

CLOSE curSecretValue

DEALLOCATE curSecretValue

DROP TABLE #myTable;

CLOSE ALL SYMMETRIC KEYS;

DROP SYMMETRIC KEY #mssqlrbs_encryption_skey;

/* Verify that you can decrypt all encrypted credential store entries using the certificate.*/

IF( EXISTS(SELECT * FROM [mssqlrbs_resources].[rbs_internal_blob_store_credentials]

WHERE decryptbykeyautocert(cert_id('cert_mssqlrbs_encryption'),

    NULL, credential_secret) IS NULL))

BEGIN

    print 'Aborted. Failed to verify key rotation'

    ROLLBACK;

END;

ELSE

    COMMIT;

END;

END;

END TRY

BEGIN CATCH

     PRINT 'Exception caught: ' + cast(ERROR_NUMBER() as nvarchar) + ' ' + ERROR_MESSAGE();

     ROLLBACK

END CATCH

END;

GO

/* Rotate the RBS credential store symmetric key, and notice that the secrets

remain the same despite the key rotation. */

SELECT *, decryptbykeyautocert(cert_id('cert_mssqlrbs_encryption'), NULL, credential_secret)

FROM [mssqlrbs_resources].[rbs_internal_blob_store_credentials];

EXEC sp_rotate_rbs_symmetric_credential_key;

SELECT *, decryptbykeyautocert(cert_id('cert_mssqlrbs_encryption'), NULL, credential_secret)

FROM [mssqlrbs_resources].[rbs_internal_blob_store_credentials];

/* See that the RBS credential store symmetric key properties reflect the new changes*/

SELECT * FROM sys.symmetric_keys WHERE name = 'mssqlrbs_encryption_skey';

Related Articles: https://blogs.msdn.com/b/sqlrbs/archive/2010/08/05/rbs-security-model.aspx

RBS 2016 Documentation: https://msdn.microsoft.com/en-us/library/gg638709.aspx

The problem

Release of a new RBS library will cause the providers developed with older version to not work due to version mismatch.

The symptoms

Let’s take an example of SharePoint to understand where and how the problem might appear. On a running SharePoint instance with the RBS client and RBS providers installed, after an upgrade of the RBS client (being used by SharePoint), an error is thrown as below (assuming the provider is a Filestream provider):

Microsoft.Data.SqlRemoteBlobs.RemoteBlobConfigurationException: No provider of type <Filestream> found. Check the server configuration or install the provider on the client.

One scenario where this can happen is when an existing setup on a machine has SharePoint front-end, RBS client (v10.5) and some provider (compiled using RBS client v10.5). Now, upon release of a new RBS client (v11), SharePoint is developed/compiled on a separate machine and simply deployed on this one. Thus it only copies the latest RBS client library along with SharePoint front-end but RBS client is not completely installed on this machine. So even though SharePoint can find the newer version of RBS client, this new RBS client will not be able to use the Provider. If RBS is explicitly installed on the machine where the application (e.g. SharePoint) is used, then the installer will install appropriate files in .NET’s Global Assembly Cache (GAC) to make things work seamlessly.

Why does it happen?

CLR uses strong names for libraries and classes (more details here). This strong name includes the version of the class/DLL, among other things. All classes in a specific DLL have the same version number as that of the DLL. Hence the Abstract Class (BlobStore), which providers need to implement, also has the version of RBS dll. And this is where the issues are.

For RBS v10.5, this abstract class is also v10.5 and any existing provider would have extended this. Now, if the app starts using a newer version of RBS without installing it on the app machine (or recompiling the provider with newer version of RBS, thus marking the provider as sub-class of abstract class BlobStore v11.0), the RBS client will not be able to use the older provider (as CLR is trying to load provider v10.5 for RS v11) even though nothing has changed except the version number. This is because the RBS client doesn't identify provider as a subclass of BlobStore v11.0.

How to fix it?

There are a couple of ways this issue can be fixed without installing RBS completely on the application machine. Users can either compile the provider with new RBS version or install a couple of Policy files to Global Assembly Cache (GAC) on the system.

In this post, I’ll only focus on using the policy files for GAC. Please read this and this before continuing. The link has more details about using policy files. I'd like to emphasize here that the most robust way is to install the new version of RBS on the application client machine. Following fix should only be used if this is not possible.

<?xml version="1.0" encoding="UTF-16"?>

<configuration>

  <runtime>

    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

      <dependentAssembly>

        <assemblyIdentity publicKeyToken="89845dcd8080cc91" name="Microsoft.Data.SqlRemoteBlobs" culture="neutral"></assemblyIdentity>

        <bindingRedirect oldVersion="10.5.0.0" newVersion="11.0.0.0"></bindingRedirect>

      </dependentAssembly>

    </assemblyBinding>

  </runtime>

Please save this in an xml file. The file need to have a specific name Policy.10.5.Microsoft.Data.SqlRemoteBlobs.config

Now, use Assembly Linker (Al.exe) to create the publisher policy assembly. The Assembly Linker is included with the .NET Framework SDK. To create the publisher policy assembly that redirects a binding from version 10.0 of Microsoft.Data.SqlRemoteBlobs.dll to version 11.0 using a publisher policy file named as above, run the following command:

al.exe /link:Policy.10.5.Microsoft.Data.SqlRemoteBlobs.config /out:Policy.10.5.Microsoft.Data.SqlRemoteBlobs.dll

This should create the publisher policy assembly. Copy this file to C:\Windows\Assembly. It should install the policy file to GAC. Now, the app should be able to find and use the appropriate provider.

Master Key Requirement

As part of the RBS installation a database master key is required to be set. This is used for RBS credential encryption and the module signing / schema separation of the RBS internal tables and external views / stored procedures. Detailed information on the specifics of this are below. The database administrator should back up the master key by using BACKUP MASTER KEY and store the backup in a secure, off-site location.

Provider Security

There are 4 ways of integrating blob store provider security with RBS and SQL Server. Use of each is dependent on the type and support included in each provider.

1.       The blob store supports Windows integrated security.

2.       Application specifies credentials to access blob store, while opening an RBS Connection. Credentials are managed by the application.

3.       A mapping of SQL Principals to Blob Store Principals is maintained by the administrator using RBS auxiliary tables. RBS manages this mapping and automatically flows credentials to the blob store after performing SQL security check.

4.       The blob store supports the SQL security model. This allows treating security for RBS blobs in a very similar way to traditional varbinary(max) blobs. Such a level of integration is possible with providers that use SQL Server, such as the RBS FILESTREAM provider.

Windows Integrated Security

Some blob stores such as the sample File Server provider support windows integrated security and ACL blobs based on windows user identity. For such blob stores, RBS doesn’t have to do anything specifically to provide security to blobs. The blob store can be secured independently from RBS, using the mechanisms provided by the blob store. RBS client library guarantees that the provider is called in the same user context that the application uses to call into RBS client library.

When an RBS call is made to access a blob, it can be accessed on the blob store if the windows user in whose context the call was made has permissions to access the blob in the blob store. Such a call can be made after the blob ID, blob reference or blob locator has been retrieved from the database, which are secured using database security and RBS roles. The granularity of the permissions (read/write blob, create pool etc.) is specific to the blob store and is configured by the administrator on the blob store independent of RBS.

Credentials Managed by Application

RBS APIs provide a very flexible way of passing config options at various places. The application can provide credentials to access the blob store by setting CommandOptions on a RemoteBlob or RemoteBlobCollectionManager object before performing an operation. These credentials are not cached and need to be passed whenever needed to connect to the blob store.

This is a very simple approach, but requires the application to store and manage credentials, which adds complexity at the application layer. A better approach is for the credentials to be managed by the administrator, which the following two options provide.

Mapping SQL Principals to Blob Store Principals

A mapping from SQL Principals to Blob Store Principals can be setup and the blob store credentials can be stored in the database. This allows the credentials to be managed by the administrator. The administrator has the responsibility of making sure that the access rights to the user tables are reflected in this mapping to maintain consistency.

The credential management is exposed through a series of stored procedures:

-- Adds a credential to the credential table.

create procedure [mssqlrbs].[rbs_sp_set_blob_store_credential] (

    @sql_user_sid varbinary(85),

    @blob_store_id smallint,

    @credential_name nvarchar(256),

    @credential_secret varbinary(max))

-- Retrieve all the decrypted credentials owned by the current user.

create procedure [mssqlrbs].[rbs_sp_get_blob_store_credentials] ()

-- Retrieves a list of all the credentials in the table. Does not decrypt any secrets.

create procedure [mssqlrbs].[rbs_sp_get_all_blob_store_credentials] (

    @sql_user_sid varbinary(85))

-- Removes a credential from the credential table.

create procedure [mssqlrbs].[rbs_sp_delete_blob_store_credential] (

    @sql_user_sid varbinary(85),

    @blob_store_id smallint,

    @credential_name nvarchar(256))

For each SQL user that has access to RBS blobs, there will be one or more rows in the credential store: one per blob store (represented by the blob_store_id) the user has access to. This row stores the credentials to use to connect to the blob store. There will be one row per user per provider. To create a default credential the sql_user_sid can be specified as 0x. This credential will be used by any user that does not have a specific user credential added to the credential store.

Blob Store Credential Name

The credential name is an unencrypted credential identifier used by the store library to distinguish a credential without requiring the decryption of the credential secret. This may be a username or other non-protected information.

Blob Store Credential Secret

The credential secret is a byte array protected using SQL Server symmetric encryption. The exact internal format of the binary fragment is decided by the provider, e.g. it may contain a password, profile name / secret combination or any other important information that should be protected. The provider will need to provide a tool or instructions to generate the Credential Secret fragment that the administrator can then insert into this table through a stored procedure.

SQL User SID Representation

A SQL user is represented by the SQL user SID (which can be retrieved using SUSER_SID function). SID is chosen because it is guaranteed to be unique for every user and cannot be repeated unlike user IDs and login names. This offers better protection in case a user is deleted and another is added with the same name.

Steps to Access a Blob

With this scheme, the sequence of steps for accessing the blob store is:

1.       Application calls RBS client library to initialize a new RemoteBlob or RemoteBlobCollectionManager object. It passes in a SqlConnection object.

Based on this SqlConnection’s (DataSource, Database, User), a RemoteBlobSession object is associated with this RemoteBlob or RemoteBlobCollectionManager object.

2.       RBS client library uses the provided SqlConnection and calls a stored procedure to get Provider Credentials – rbs_sp_get_blob_store_credentials.

3.       Stored procedure finds the user associated with the current connection using SUSER_SID, performs a lookup on the rbs_blob_store_credentials table and retrieves all the credentials corresponding to this user.

4.       RBS Client library stores the Provider Credentials in the RemoteBlobSession object.

5.       Application calls RBS client library to perform an operation that communicates with the blob store (e.g. blob store / fetch).

6.       RBS calls provider library, passing in any matching cached BlobStoreCredentials[] along with other configuration options.

7.       Using the given Blob Store Credential, the blob is accessed on the store. Provider library is responsible for managing connections to the blob store. It may maintain a pool of connections if needed.

Credential Encryption

Provider credentials need to be protected using encryption. Two things can be encrypted: column in which the credentials are stored and channel over which the credentials are transmitted.

Column Encryption

The Credential column is encrypted using symmetric encryption available on SQL Server, and requires a master key to be present on the database at the time RBS is enabled. A single symmetric key is created during RBS installation and used to encrypt / decrypt all the credential secrets in the table.

Channel Encryption

RBS opens a new encrypted SqlConnection in order to retrieve credentials based on the original connection string with the encrypted flag set, and optionally setting the TrustServerCertificate flag if the option has been specified during setup.

In-Memory Encryption

When credentials are cached on the client, they are kept in encrypted form so that a memory dump does not contain the credentials in clear text. This is done using the ProtectedMemory class which internally calls the DPAPI OS encryption APIs.

Blob Store supports SQL Security Model

A tightly integrated solution allows using SQL Principals to secure access to the blob store directly. This can be done if the provider uses SQL server, e.g. the SQL RBS FILESTREAM provider. When using a provider that is not located on the same SQL Server as the RBS metadata store, for example the RBS FILESTREAM provider in the remote configuration this may not be possible.

Module Signing and Schema

The RBS metadata is split between two schemas, mssqlrbs and mssqlrbs_resources.  Externally facing modules such as user and admin stored procedures, views and functions are located in the main mssqlrbs schema. Internal tables and stored procedures are located in the mssqlrbs_resources schema.

A master key in the database is required to create a signing certificate and certificate account, along with the symmetric key used to encrypt the credential data. The modules in the mssqlrbs schema are signed with the certificate, and the certificate account is granted various permissions to the tables in the mssqlrbs_resources schema. When a user executes a module in the mssqlrbs schema that has been signed with the RBS certificate, during the execution of the module further privileges are granted to access mssqlrbs_resources that have been explicitly granted to the certificate user. This allows the admin to control user access using the db_rbs_reader, db_rbs_writer and db_rbs_maintainer SQL roles without requiring in depth knowledge of RBS internals.

Inline Storage (max_size_inline_blob)

For very small blobs, the overhead required to store these in Filestream files may be significant. The time required to create a new Filestream file or open a handle to a Filestream file can be substantially more than that required to store small blobs in internal SQL Server data structures in the user’s database. If the incoming blob is at or below the max_size_inline_blob size threshold it will be directly stored in the blob_data_inline varbinary(max) column of the internal RBS data table. During a read request the blob ID is checked and the read request dealt with by returning a T-SQL in-band stream. The max_size_inline_blob configuration value can be changed at any time for new blob inserts, however blobs that have already been stored inline will not be moved or migrated. The default value of the max_size_inline_blob parameter is set to 61140 bytes which was determined as the break-even point where creating a new file or opening a handle to a file outside of the database is more expensive than the slower data access when stored internally.

In-Band Read (max_size_inband_read) and Write (max_size_inband_write)

There is a performance penalty for setting up an out-of-band handle to access a Filestream file. If the blob to be stored or retrieved is small it can be faster to stream using in-band T-SQL access rather than spinning up an out-of-band stream even if they are stored in a Filestream column. This applies for both read and write scenarios. The default value of the max_size_inband_read and max_size_inband_write parameters are set to 1258290 bytes. As this value is used dynamically it can be changed in the blob store configuration and subsequent read and writes will honor the new value.

Out-of-band Read and Write

If the blob size is greater than the configuration setting values a Filestream share path is generated and the data is streamed in or out of the Filestream data file using Win32 streaming APIs. The streaming performance is much greater than going through T-SQL or directly using varbinary(max) however the initial cost to set up the streaming handle can be greater. Performance testing suggests the break-even point for SQL Server 2008 Filestream operations is around 1.2 MB, hence the default upper limits for the in-band read configuration settings.

- mike

Connection Strings

RBS maintainer takes connection strings from a CLR config file (Microsoft.Data.SqlRemoteBlobs.Maintainer.exe.config) that is present in the same directory as the maintainer executable. You will need to add your connection strings to this file. You will need to add one connection string per database that you want to run RBS maintainer on. It is recommended that the connection strings be encrypted in case you are using SQL authentication (since the password is part of the connection string). If you are using Windows Authentication, you dont need to encrypt your connection strings.

The config file created by RBS installer contains the connection string in encrypted form. CLR config files have the limitation that all the connection strings need to be either encrypted or plaintext - you cannot have a combination of some connection strings encrypted and some plaintext. So, if you want to add more connection strings to the config file already created by RBS installer, you will need to either write a program to encrypt them or use a utility (aspnet_regiis.exe) to do it for you. See [1] for more details on how to do this. If you do not want to encrypt your connection strings, feel free to delete the encrypted connection string already present in the config file and add your connection strings in plaintext.

Command Line Parameters

Once you have all the connection strings in the config file, you can run maintainer executable with command line parameters telling it which database to run on and what set of tasks to do. Here is a brief description of the command-line parameters that you can get by running maintainer without any parameters:

Usage (Available Options):

ConnectionStringName    - This parameter is the name of the connection string.

                                It takes a single argument: <Connection String Name>.

                                The connection string needs to be present in the CLR xml config file.

Operation               - This parameter is the operation(s) to perform.

                                The parameter takes between 1 and 4 arguments: <Operation1 [Operation2 [Operation3 [Operation4]]]>.

                                Operations must be <ConsistencyCheck, GarbageCollection, Maintenance, ConsistencyCheckForStores, ForceFinalize>.

                                ForceFinalize cannot be specified in combination with any other operation.

GarbageCollectionPhases - This parameter is the phase(s) of garbage collection to run or finalize.

                                The parameter takes a single argument: <Phase(s) of garbage collection to run or finalize>.

                                This argument needs to be 1 or more letters from <r, d, o>.

                                r: Reference Scan, d: Delete Propagation, o: Orphan Cleanup.

ConsistencyCheckMode    - This parameter is the mode for consistency checks.

                                The parameter takes a single argument: <Consistency Check Mode>.

                                This argument needs to be <c, r, b>.

                                c: Check only, r: Check and attempt to repair any issues found,

                                b: Check, repair and re-build RBS internal information without losing customizations.

ConsistencyCheckExtent  - This parameter is the extent for consistency checks.

                                The parameter takes a single argument: <Consistency Check Extent>.

                                This argument needs to be <m, c>.

                                m: Metadata only, c: Complete check.

TimeLimit               - This parameter is the limit on the amount of time to run specified operation(s).

                                The parameter takes a single argument: <Time Limit in Minutes>.

                                This argument needs to be a positive integer.

ConsistencyCheckForStores - This parameter enables selecting blob stores on which to run store specific consistency check operations.

                                This parameter has a default value of All, and can take the following blob store arguments: <BlobStoreName1 [BlobStoreName2 [...]] Default:All>

                                Takes the names of blob stores as arguments.

The different operations available in maintainer are:

1.       GarbageCollection. See [2] for details on the different phases of garbage collection. Pool Slicing is a new feature added in RBS 2008 R2 that allows GC of pools to happen incrementally, i.e. one slice at a time instead of the whole pool at one go. The slicing is done based on the create timestamp of the blob. This allows making incremental progress on garbage collecting huge pools that contain hundreds of millions of blobs without requiring a long maintenance window. Slicing applies only to the Orphan Cleanup phase and is used only if the provider implements time filtered enumeration (EnumerationOptimizationLevel is higher than Basic). The RBS Filestream provider implements this. Config keys that can be used to tune Garbage Collection operation are: delete_scan_period, orphan_scan_period, garbage_collection_time_window. If you want to completely remove all deleted blobs immediately, set all the 3 config items above to 'time 00:00:00' and run maintainer for GC phases RS and DP (rd). This is useful if you want to unregister a blob store / uninstall a provider.

The command line switch GarbageCollectionPhases is required for this operation. Examples (these examples assume that you have a connection string in your config file named RBSMaintainerConnection, whch is the default string added by the RBS installer):

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection   -Operation GarbageCollection -GarbageCollectionPhases rdo

o   This runs the 3 phases of garbage collection.

2.       ConsistencyCheck. This does a consistency check on the RBS internal tables and can optionally try to repair any issues found. If you specify this operation, you also need to specify ConsistencyCheckMode. Optionally, you can also specify ConsistencyCheckExtent to choose whether to check metadata only or check for the validity of each BlobID (default is metadata). Config keys that can be used to tune it are: max_consistency_issues_found, max_consistency_issues_returned. Examples:

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation ConsistencyCheck -ConsistencyCheckMode c

o   This does consistency check only (no repair) on the RBS metadata.

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation ConsistencyCheck -ConsistencyCheckMode r -ConsistencyCheckExtent c

o   This does consistency check and attempts repair on RBS metadata as well as each BlobId.

3.       ConsistencyCheckForStores. This does consistency check on blob stores associated with the RBS database. For this option to work, the provider needs to implement consistency checks (its ConsistencyCheckLevel must not be None). This is a new feature added in RBS 2008 R2 and has been implemented by the RBS Filestream Provider. Config keys that can be used to tune it are: . Examples:

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation ConsistencyCheckForStores

o   This runs consistency check on all the registered stores if their providers support it.

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation ConsistencyCheckForStores -ConsistencyCheckForStores MyBlobStore1 MyBlobStore2

o   This runs consistency check on the two specified blob stores (if their providers support it).

4.       Maintenance. This does some maintenance on RBS internal tables, which mainly consists of reorganizing indexes. It is a good idea to run maintainer with this operation once in a while to ensure good performance. Examples:

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation Maintenance

o   This does maintenance of RBS indexes.

5.       ForceFinalize. RBS Maintainer is designed to make incremental progress on garbage collection operations. If a phase of GC is not completed before the allotted time is up, the progress is saved and it is picked up again the next time RBS Maintainer is run for that operation. This mechanism is explained in [2] below as well. Sometimes it may be desired to “forget” this saved information about in-progress garbage collections and start fresh. The ForceFinalize operation serves that purpose: it removes saved information about one or more GC phases. The next time maintainer is run with those GC phases, they start from the beginning. The command line switch GarbageCollectionPhases is required for this operation. Examples:

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation ForceFinalize -GarbageCollectionPhases rd

o   This does ForceFinalize for the RS and DP phases of garbage collection.

Options:

TimeLimit. Currently this is the only option available. If this is specified, RBS maintainer will try to stop after the specified duration even if all its tasks are not complete. If it is not specified, maintainer will continue to run until all the specified tasks (operations) are completed. This option allows scheduling RBS maintainer to run during regular maintenance windows (e.g. from midnight to 2 AM) and stop at a predictable time. If maintainer had to stop before the tasks were completed, they will be picked up the next time maintainer is run with those operations specified. [2] talks about how this option interacts with some config keys.

You can combine multiple operations from above in one command line. You can specify upto 3 operations at one time (the usage message pasted above says upto 4, but there is a bug currently which limits it to 3) Examples:

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection   -Operation GarbageCollection ConsistencyCheck  -GarbageCollectionPhases rdo -ConsistencyCheckMode r

o   This does the 3 phases of garbage collection and consistency check on RBS metadata only with attempt to repair.

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection   -Operation GarbageCollection ConsistencyCheck ConsistencyCheckForStores  -GarbageCollectionPhases rdo -ConsistencyCheckMode r

o   This does the 3 phases of garbage collection, consistency check on RBS metadata only with attempt to repair and consistency check on all registered stores whose providers support consistency check.

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection   -Operation GarbageCollection ConsistencyCheck Maintenance  -GarbageCollectionPhases rdo -ConsistencyCheckMode r

o   This does the 3 phases of garbage collection, consistency check on RBS metadata only with attempt to repair and maintenance of RBS indexes.

·         Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection   -Operation GarbageCollection ConsistencyCheck Maintenance  -GarbageCollectionPhases rdo -ConsistencyCheckMode r -TimeLimit 120

o   This does the 3 phases of garbage collection, consistency check on RBS metadata only with attempt to repair and maintenance of RBS indexes and puts a time limit of 2 hours for processing all these things. If they don’t complete in approximately 2 hours, maintainer saves its progress and stops.

Monitoring Progress

The view mssqlrbs.rbs_blob_details shows a list of all the blobs that RBS thinks are currently in use by the application. Running the Reference Scan (r) phase of GC moves blobs out of these and marks them internally for deletion during the Delete Propagation (d) phase after the backup/restore SLA time window (garbage_collection_time_window) has elapsed. The view mssqlrbs.rbs_consistency_issues shows the list of consistency issues found by the maintainer. For measuring performance, troubleshooting or just for curiosity, you can look at views mssqlrbs.rbs_history and mssqlrbs.rbs_counters to see the progress of maintainer tasks.

Scheduling RBS Maintainer

RBS does not come with its own way of scheduling maintainer. RBS maintainer is a standalone executable and you will need to schedule it yourself. One way to do that is to use Window Task Scheduler - this is what the RBS installer brings up if you select that option while installing RBS. [1] has some sample steps to schedule it using Windows Task Scheduler. Another way to schedule it is to have a job in SQL Server Agent. You are also free to use your own mechanisms to do the scheduling.

References

[1] https://sqlrbs.codeplex.com/Thread/View.aspx?ThreadId=204627 - this thread has sample steps to add encrypted connection strings to the maintainer config file and to schedule maintainer task using Windows Task Scheduler. Running maintainer once will only perform the tasks on one database. So if you have multiple databases, you will need to run maintainer multiple times (once for each database).

[2] https://blogs.msdn.com/sqlrbs/archive/2008/08/08/rbs-garbage-collection-settings-and-rationale.aspx has more information on the different phases of garbage collection and the TimeLimit option.

As always, feel free to ask questions if you have any – either through comments on this blog or by creating a new discussion thread on the discussions page on our codeplex site.

- Pradeep.

A blob store can be identified using Blob Store Name or Blob Store ID; there is a 1:1 correspondence between these and there is a different name/ID for each instance of blob store registered with the DB. Blob Store Name is different from Blob Store Type. Blob Store Type indicates a class of blob stores, e.g. “Filestream”, “File” or “Centera” – this is fixed by the provider developer and cannot be changed by the admin or application writer. Blob Store Name is associated with an instance of a blob store, usually by the admin who deploys RBS. The list of blob stores registered with a DB can be found using the view mssqlrbs.rbs_blob_stores.

Here is the logic (in priority order) that determines which blob store is used to store a new blob.

1.       If the application specifies a blob store name in CreateNewBlob() call, that blob store is used. This has the highest priority and all defaults are ignored. The blob store name needs to be specified inside the config parameter.

·         SqlRemoteBlobContext.CreateNewBlob(int collectionId, ConfigItemList config): set the BlobStoreName parameter in config.

2.       Otherwise, if there is a default blob store associated with the collection to which the new blob belongs, that blob store is used. The default blob store for a collection can be specified while creating a collection and can also be modified later. [Note: If the CreateNewBlob() call doesn’t specify a collectionId, the default collection with collectionId 0 is used. A default blob store can be associated with that collection as well.]

·         SqlRemoteBlobContext.CreateNewCollection(ConfigItemList config): set the DefaultBlobStoreName parameter in config.

·         SqlRemoteBlobCollectionManager.ModifyConfig(ConfigItemList config): set the DefaultBlobStoreName parameter in config.

·         To clear/remove default blob store for a collection, call ModifyConfig above with an empty string ("") as the DefaultBlobStoreName. Note that not specifying an item in the config structure means retaining the old value, so you need to explicitly pass in some value to change the default blob store. The value that indicates removal/clearing is the empty string.

·         You can see the current default blob store for a collection from the view mssqlrbs.rbs_collections: select blob_store_name from mssqlrbs.rbs_blob_stores as b join mssqlrbs.rbs_collections as c on c.default_blob_store_id = b.blob_store_id where c.collection_id = <Collection you are interested in>.

3.       Otherwise, if there is a default blob store for the DB, that blob store is used.

·         To set a default blob store for the DB, call Stored Procedure: mssqlrbs.rbs_sp_set_config_value(@config_key sysname, @config_value nvarchar(max)) with @config_key = N'default_blob_store_name' and @config_value = <name of the blob store you want to set as default>. Repeated calls to this SP will overwrite the default blob store and only the latest setting will be retained.

·         To clear/remove default blob store for the DB, call SP: mssqlrbs.rbs_sp_delete_config_value(N'default_blob_store_name').

·         You can see the current default blob store for the DB from the view mssqlrbs.rbs_config: select * from mssqlrbs.rbs_config where config_key = N'default_blob_store_name'.

4.       Otherwise, an error is returned that no blob store has been specified.

- Pradeep

The SQL Server 2008 R2 November Community Technology Preview Feature Pack is available for download now. This includes a refresh of the Remote Blob Storage package.

Significant enhancements have been made in this release for performance and integration with SharePoint 2010. These changes include implementing the IDisposable interface in the SqlRemoteBlobCollectionManager and SqlRemoteBlobContext classes. This allows RBS to manage the lifetime of resources used by these objects and reuse them through a pooling mechanism.

While this is not needed to ensure correctness, in order to take full advantage of the performance changes existing applications must be modified to use the IDisposable pattern:

SqlRemoteBlobContext context = new SqlRemoteBlobContext(conn);

// RBS operations

Should be replaced with

using (SqlRemoteBlobContext context = new SqlRemoteBlobContext(conn))

{

    // RBS operations

}

Note that this release contains breaking changes preventing access to data stored in FILESTREAM blob stores using previous CTP releases. Uninstalling previous versions and removing the FILESTREAM blob store data prior to installing this one is recommended for configurations using FILESTREAM stores.

SQL Server 2008 R2 November CTP Feature Pack link:

https://www.microsoft.com/downloads/details.aspx?familyid=020EE0D5-BCE4-4A45-9D64-B0C49C8831E5&displaylang=en

Install and configure Remote BLOB Storage on SharePoint Foundation 2010:

https://technet.microsoft.com/en-us/library/ee663474(office.14).aspx

In addition to the new provider, several features were added to the core RBS client libraries. These include asynchronous provider requests, deferred deletion of collections, maintainer optimizations and support for provider specific consistency checks.

Administrators and developers can look forward to significant performance enhancements when these are made available in future CTP releases.

BlobStoreException is meant to be used for conditions that RBS client library expects and knows how to handle (either now or in the future). For this reason, there is a defined list of cases where BlobStoreException should be thrown. This is the list of values of the BlobStoreExceptionCode enum. If a condition falls in this list, then a BlobStoreException must be thrown. If a condition does not fall into this list, then a different exception must be thrown.

So the next question is: which exception should a BlobStore implementation throw? .NET Framework guidelines say that as far as possible, we should throw system exceptions. This helps with easier handling of the error, possibly by higher level applications or by admins. In the case of RBS, the client library wraps any non-BlobStoreException Exceptions in another exception (RemoteBlobStoreException with ExceptionCode = BlobStoreUnhandledException) and sets the InnerException to the original exception thrown by the BlobStore. So applications will need to catch and handle this new exception. If LogLevel is configured to Error or higher, the log will also contain details of the exception as well.

In general, it would be very rare to have a condition not covered by the list in BlobStoreExceptionCode. One example where it is not covered is present in the sample provider. The sample FileStoreLibrary throws non-BlobStoreExceptions such as ArgumentNullException. In this case, if this exception gets thrown, it indicates a bug in RBS client code, and it should not be hidden by a BlobStoreException.

If other components/dlls called by a blob store provider can throw exceptions such as OutOfMemoryException, the BlobStore implementation should be careful about letting it bubble up directly. Often times the correct thing to do is to throw a BlobStoreException with BlobStoreExceptionCode set to OperationFailedAuthoritative. Remember, the list of conditions indicated by BlobStoreExceptionCode are the ones that RBS client library knows how to handle, and it always takes precedence.

- Pradeep.