• Remote Distributor required

• Distribution database AG needs a listener

• Replication jobs use the distributor listener

• A new job created to monitor distributor AG state

• Synchronous mode is recommended and preferred

• Transactional replication with immediate or queued updating subscriber is not supported

• P2P, Merge, Heterogeneous, Bi-directional TR not supported

• Configure by script only

• Must be new replication, existing distribution db not supported

• SQL Server 2017 RTM + CU6 Build# 14.0.3025.34

• Distributor in AG (SQLNODE1 & SQLNODE2)

• Publisher in AG (SQLNODE4 & SQLNODE5)  - Test database AW2014 (it is a copy of AdventureWorks sample database, the table in the demo person.address_test is a copy of existing table person.address)

• Subscriber (SQLTest\SQL2017Sub)

-- SECTION 1 ---- CONFIGURE THE DISTRIBUTOR SERVERS   -- Step1 - Configure the Distribution DB nodes (AG Replicas) to act as a distributor :Connect SQLNode1 sp_adddistributor @distributor = @@ServerName, @password = 'Pass@word1' Go :Connect SQLNode2 sp_adddistributor @distributor = @@ServerName, @password = 'Pass@word1' Go   -- Step2 - Configure the Distribution Database :Connect SQLNode1 USE master EXEC sp_adddistributiondb @database = 'DistributionDB', @security_mode = 1; GO Alter Database [DistributionDB] Set Recovery Full Go Backup Database [DistributionDB] to Disk = 'Nul' Go   -- Step 3 - Create AG for the Distribution DB. :Connect SQLNode1 USE [master] GO CREATE ENDPOINT [Hadr_endpoint] STATE=STARTED AS TCP (LISTENER_PORT = 5022, LISTENER_IP = ALL) FOR DATA_MIRRORING (ROLE = ALL, AUTHENTICATION = WINDOWS NEGOTIATE , ENCRYPTION = REQUIRED ALGORITHM AES) GO   :Connect SQLNode2 USE [master] GO CREATE ENDPOINT [Hadr_endpoint] STATE=STARTED AS TCP (LISTENER_PORT = 5022, LISTENER_IP = ALL) FOR DATA_MIRRORING (ROLE = ALL, AUTHENTICATION = WINDOWS NEGOTIATE , ENCRYPTION = REQUIRED ALGORITHM AES) GO   :Connect SQLNode1 -- Create the Availability Group CREATE AVAILABILITY GROUP [DistributionDB_AG] FOR DATABASE [DistributionDB] REPLICA ON 'SQLNode1' WITH (ENDPOINT_URL = N'TCP://SQLNode1.Features2014DC.local:5022', FAILOVER_MODE = AUTOMATIC, AVAILABILITY_MODE = SYNCHRONOUS_COMMIT, BACKUP_PRIORITY = 50, SECONDARY_ROLE(ALLOW_CONNECTIONS = ALL), SEEDING_MODE = AUTOMATIC), N'SQLNode2' WITH (ENDPOINT_URL = N'TCP://SQLNode2.Features2014DC.local:5022', FAILOVER_MODE = AUTOMATIC, AVAILABILITY_MODE = SYNCHRONOUS_COMMIT, BACKUP_PRIORITY = 50, SECONDARY_ROLE(ALLOW_CONNECTIONS = ALL), SEEDING_MODE = AUTOMATIC); GO   :Connect SQLNode2 ALTER AVAILABILITY GROUP [DistributionDB_AG] JOIN GO ALTER AVAILABILITY GROUP [DistributionDB_AG] GRANT CREATE ANY DATABASE Go   --STEP4 - Create the Listener for the Availability Group. This is very important. :Connect SQLNode1   USE [master] GO ALTER AVAILABILITY GROUP [DistributionDB_AG] ADD LISTENER N'DistributionDBList' ( WITH IP ((N'10.10.3.50', N'255.0.0.0')) , PORT=1433); GO   -- STEP 5 - Enable SQLNode2 also as a Distributor :CONNECT SQLNODE2 EXEC sp_adddistributiondb @database = 'DistributionDB', @security_mode = 1; GO     --STEP 6 - On all Distributor Nodes Configure the Publisher Details -- ******@Publisher is the current primary replca of the publisher, we will set redirect publisher to Publisher's listener in the end******------------- :CONNECT SQLNODE1 sp_helpdistpublisher EXEC sp_addDistPublisher @publisher = 'SQLNode4', @distribution_db = 'DistributionDB', @working_directory = '\\SQL2014DC\Dist_Work_Directory\' GO   :CONNECT SQLNODE2 --sp_helpdistpublisher EXEC sp_addDistPublisher @publisher = 'SQLNode4', @distribution_db = 'DistributionDB', @working_directory = '\\SQL2014DC\Dist_Work_Directory\' GO   -- ******@Publisher is the current secondary replca of the publisher, we will set redirect publisher to Publisher's listener in the end******------------- -- ******because after failover, the secondary replica of the publisher can also be a publisher to use the distributionDB****** :CONNECT SQLNODE1 EXEC sp_addDistPublisher @publisher = 'SQLNode5', @distribution_db = 'DistributionDB', @working_directory = '\\SQL2014DC\Dist_Work_Directory\' GO   :CONNECT SQLNODE2 EXEC sp_addDistPublisher @publisher = 'SQLNode5', @distribution_db = 'DistributionDB', @working_directory = '\\SQL2014DC\Dist_Work_Directory\' GO   --sp_helpdistpublisher   -- SECTION 2 ---- CONFIGURE THE PUBLISHER SERVER ON ALL PUBLISHER NODES :CONNECT SQLNODE4 --sp_helpdistributor EXEC sp_addDistributor @distributor = 'DistributionDBList', -- Listener for the Distribution DB. @password = 'Pass@word1' Go   :CONNECT SQLNODE5 EXEC sp_addDistributor @distributor = 'DistributionDBList', -- Listener for the Distribution DB. @password = 'Pass@word1' Go   --sp_helpdistributor   -- On the Secondary replicas of the Distribution DB, add the Subscriber as a linked server. :CONNECT SQLNODE2 EXEC master.dbo.sp_addlinkedserver @server = N'SQLTest\SQL2017Sub', @srvproduct=N'SQL Server' :CONNECT SQLNODE2 EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'SQLTest\SQL2017Sub',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL     /* For security reasons the linked server remote logins password is changed with ######## */ --EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'SQLNODE5',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL   -- On the Secondary replicas of the Publisher DB, add the Subscriber as a linked server. :CONNECT SQLNODE5 EXEC master.dbo.sp_addlinkedserver @server = N'SQLTest\SQL2017Sub', @srvproduct=N'SQL Server' :CONNECT SQLNODE5 EXEC master.dbo.sp_addlinkedsrvlogin @rmtsrvname=N'SQLTest\SQL2017Sub',@useself=N'True',@locallogin=NULL,@rmtuser=NULL,@rmtpassword=NULL       -- SECTION 3 ---- CONFIGURE DISTRIBUTION DATABASE FULL/LOG BACKUP PLAN -- ***Make sure Distribution database has Full and T-log backup scheduled since it is in FULL recovery mode ***--   -- SECTION 4 ---- CONFIGURE THE PUBLICATION FOR THE PRIMARY REPLICA (JUST TREAT IT AS STANDALONE)   -- On Publisher's primary replca, enable the dataabse AW2014 for transactional replication :CONNECT SQLNODE4 USE master; GO EXEC sys.sp_replicationdboption @dbname = 'AW2014', @optname = 'publish',  --this is for transactional replication @value = 'true';        -- On primary Publisher SQLNode4, create the publication using script that you previously saved as below -- created publication AW2014_Pub -- You can script out your existing publication, here is just an example script that only contains 4 articles -- execute the script to create the publication on SQLNode4 -- create publication AW2014_Pub on the primary replica SQLNode4   :CONNECT SQLNODE4   --use [AW2014] --exec sp_replicationdboption @dbname = N'AW2014', @optname = N'publish', @value = N'true' --GO   -- Adding the logreader job --@publisher_security_mode = 1: integrate authentication mode, no @publisher_login/password needed --@publisher_security_mode = 0: SQL authentication mode, requires sql login for @publisher_login/password use [AW2014] exec [AW2014].sys.sp_addlogreader_agent @job_login = null, @job_password = null, @publisher_security_mode = 1, @publisher_login = null, @publisher_password = null GO   -- Adding the transactional publication use [AW2014] exec sp_addpublication @publication = N'AW2014_Pub', @description = N'Transactional publication of database ''AW2014'' from Publisher ''SQLNODE4''.', @sync_method = N'concurrent', @retention = 0, @allow_push = N'true', @allow_pull = N'true', @allow_anonymous = N'false', @enabled_for_internet = N'false', @snapshot_in_defaultfolder = N'true', @compress_snapshot = N'false', @ftp_port = 21, @allow_subscription_copy = N'false', @add_to_active_directory = N'false', @repl_freq = N'continuous', @status = N'active', @independent_agent = N'true', @immediate_sync = N'false', @allow_sync_tran = N'false', @allow_queued_tran = N'false', @allow_dts = N'false', @replicate_ddl = 1, @allow_initialize_from_backup = N'false', @enabled_for_p2p = N'false', @enabled_for_het_sub = N'false' GO   -- no need snapshot agent job -- exec sp_addpublication_snapshot @publication = N'AW2014_Pub', @frequency_type = 1, @frequency_interval = 1, @frequency_relative_interval = 1, @frequency_recurrence_factor = 0, @frequency_subday = 8, @frequency_subday_interval = 1, @active_start_time_of_day = 0, @active_end_time_of_day = 235959, @active_start_date = 0, @active_end_date = 0, @job_login = null, @job_password = null, @publisher_security_mode = 1   use [AW2014] exec sp_addarticle @publication = N'AW2014_Pub', @article = N'Address', @source_owner = N'Person', @source_object = N'Address', @type = N'logbased', @description = null, @creation_script = null, @pre_creation_cmd = N'drop', @schema_option = 0x000000000803509F, @identityrangemanagementoption = N'manual', @destination_table = N'Address', @destination_owner = N'Person', @vertical_partition = N'false', @ins_cmd = N'CALL sp_MSins_PersonAddress', @del_cmd = N'CALL sp_MSdel_PersonAddress', @upd_cmd = N'SCALL sp_MSupd_PersonAddress' GO   use [AW2014] exec sp_addarticle @publication = N'AW2014_Pub', @article = N'Address_Test', @source_owner = N'Person', @source_object = N'Address_Test', @type = N'logbased', @description = null, @creation_script = null, @pre_creation_cmd = N'drop', @schema_option = 0x000000000803509F, @identityrangemanagementoption = N'manual', @destination_table = N'Address_Test', @destination_owner = N'Person', @vertical_partition = N'false', @ins_cmd = N'CALL sp_MSins_PersonAddress_Test', @del_cmd = N'CALL sp_MSdel_PersonAddress_Test', @upd_cmd = N'SCALL sp_MSupd_PersonAddress_Test' GO   use [AW2014] exec sp_addarticle @publication = N'AW2014_Pub', @article = N'AddressType', @source_owner = N'Person', @source_object = N'AddressType', @type = N'logbased', @description = null, @creation_script = null, @pre_creation_cmd = N'drop', @schema_option = 0x000000000803509F, @identityrangemanagementoption = N'manual', @destination_table = N'AddressType', @destination_owner = N'Person', @vertical_partition = N'false', @ins_cmd = N'CALL sp_MSins_PersonAddressType', @del_cmd = N'CALL sp_MSdel_PersonAddressType', @upd_cmd = N'SCALL sp_MSupd_PersonAddressType' GO   use [AW2014] exec sp_addarticle @publication = N'AW2014_Pub', @article = N'AWBuildVersion', @source_owner = N'dbo', @source_object = N'AWBuildVersion', @type = N'logbased', @description = null, @creation_script = null, @pre_creation_cmd = N'drop', @schema_option = 0x000000000803509F, @identityrangemanagementoption = N'manual', @destination_table = N'AWBuildVersion', @destination_owner = N'dbo', @vertical_partition = N'false', @ins_cmd = N'CALL sp_MSins_dboAWBuildVersion', @del_cmd = N'CALL sp_MSdel_dboAWBuildVersion', @upd_cmd = N'SCALL sp_MSupd_dboAWBuildVersion' GO   -- SECTION 4.2 ---- CONTINUE TO CONFIGURE THE PUBLICATION -- check the publication :CONNECT SQLNODE4 use [AW2014] EXEC sp_helppublication @publication = N'AW2014_Pub'   -- set the publication to allow_initialize_from_backup :CONNECT SQLNODE4 use [AW2014] EXEC sp_changepublication @Publication = N'AW2014_Pub', @property = N'allow_initialize_from_backup', @value = True   -- Section 4.3 ---- INIT SUBSCRIBER DATABASE USING BACK/RESTORE :CONNECT SQLNODE4 BACKUP DATABASE [AW2014] TO  DISK = N'C:\Backups\UserDB\AW2014-ForSubscriber.bak' WITH NOFORMAT, NOINIT, NAME = N'AW2014-Full Database Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10 GO   :CONNECT SQLNODE4 BACKUP LOG [AW2014] TO  DISK = N'C:\Backups\UserDB\AW2014-ForSubscriber.trn' WITH NOFORMAT, NOINIT, NAME = N'AW2014-T Log Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10 GO   :CONNECT SQLTest\SQL2017Sub /* Noted: if the SQLTest\SQL2017Sub is higher than the publisher, then backup/restore will work, otherwise you cannot use backup/restore to init subscriber db. */ USE [master] RESTORE DATABASE [AW2014_Repl] FROM  DISK = N'\\sqlnode4\c$\Backups\UserDB\AW2014-ForSubscriber.bak' WITH  FILE = 1, MOVE N'AdventureWorks2014_Data' TO N'C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017Sub\MSSQL\DATA\AW2014_ReplOLTP_Data.mdf', MOVE N'AdventureWorks2014_Log' TO N'C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017Sub\MSSQL\DATA\AW2014_ReplOLTP_Log.ldf', NORECOVERY,  NOUNLOAD,  STATS = 5   RESTORE LOG [AW2014_Repl] FROM  DISK = N'\\sqlnode4\c$\Backups\UserDB\AW2014-ForSubscriber.trn' WITH  FILE = 1,  NOUNLOAD,  STATS = 5 GO       -- SECTION 5 ---- CREATE THE SUBSCRIPTION USING THE SYNC_TYPE = 'INITIALIZE WITH BACKUP' -- execute the script to create subscription on the publisher SQLNode4 -- Sometimes you may use @sync_type = N'replication support only' :CONNECT SQLNODE4 use [AW2014] exec sp_addsubscription @publication = N'AW2014_Pub', @subscriber = N'SQLTest\SQL2017Sub', @destination_db = N'AW2014_Repl', @subscription_type = N'Push', @sync_type = N'initialize with backup', --INITIALIZE WITH BACKUP @article = N'all', @update_mode = N'read only', @subscriber_type = 0, @backupdevicetype ='disk',  --REQUIRED @backupdevicename = N'\\SQLNode4\C$\Backups\UserDB\AW2014-ForSubscriber.bak'  --REQUIRED   :CONNECT SQLNODE4 use [AW2014] exec sp_addpushsubscription_agent @publication = N'AW2014_Pub', @subscriber = N'SQLTest\SQL2017Sub', @subscriber_db = N'AW2014_Repl', @job_login = null, @job_password = null, @subscriber_security_mode = 1, @frequency_type = 64, @frequency_interval = 0, @frequency_relative_interval = 0, @frequency_recurrence_factor = 0, @frequency_subday = 0, @frequency_subday_interval = 0, @active_start_time_of_day = 0, @active_end_time_of_day = 235959, @active_start_date = 20180212, @active_end_date = 99991231, @enabled_for_syncmgr = N'False', @dts_package_location = N'Distributor' GO   ----------------------------CONFIGURE DISTRIBUTOR IN AG COMPLETED------------------------------   ----------STOP HERE AT THIS POINT, TEST DISTRIBUTOR IN AG AND STANDALONE PUBLISHER-------------   /* Distributor in AG (SQLNode1 & SQLNode2) SQLNode4 is the standalone publisher SQLTest\SQL2017Sub is the subscriber */ --update table :Connect SQLNODE4 update AW2014.person.address_test set addressline1 = '1972 Napa Ct.' where addressid = 1   --watch result in publisher and subscriber :Connect SQLNODE4 select top 5 * from AW2014.person.address_test Go   :Connect SQLTest\SQL2017Sub select top 5 * from AW2014_Repl.person.address_test Go   -- check if the transaction and command is replicated to the secondary distributor replica :connect sqlnode1 use distributiondb select * from [dbo].[MSrepl_transactions] with(nolock) select * from [dbo].[MSrepl_commands] with(nolock) --where xact_seqno = 0x0000005300001A08001C :connect sqlnode2 use distributiondb select * from [dbo].[MSrepl_transactions] with(nolock) select * from [dbo].[MSrepl_commands] with(nolock) --where xact_seqno = 0x0000005300001A08001C   -- Failover Distributor AG  and Test the above table update again -- 1. Failover Ditributor AG :Connect SQLNODE1 :Connect SQLNODE2 ALTER AVAILABILITY GROUP [DistributionDB_AG] FAILOVER; GO   -- 2. Watch jobs on Distributor are disabled and enabled on different replicas automatically as expected (may take a min)   -- 3. Test the above table update again and watch if result is all good --update table :Connect SQLNODE4 update AW2014.person.address_test set addressline1 = '1975 Napa Ct.' where addressid = 1   --watch result in publisher and subscriber :Connect SQLNODE4 select top 5 * from AW2014.person.address_test Go   :Connect SQLTest\SQL2017Sub select top 5 * from AW2014_Repl.person.address_test Go   -- 4. Failover back to SQLNODE1 as primary Distributor replica :Connect SQLNODE2 :Connect SQLNODE1 ALTER AVAILABILITY GROUP [DistributionDB_AG] FAILOVER; GO -- 5. Wait a min or two, the jobs should be refreshed again as expected       ------------------TEST DISTRIBUTOR IN AG AND STANDALONE PUBLISHER COMPLETED-----------------------   ---------------IF THE ABOVE TEST SUCCESSFUL, CONTINUE TO CONFIGURE PUBLISHER IN AG----------------       -- SECTION 6 ---- ***CONFIGURE THE PUBLISHER DB AG AND PUBLISHER LISTENER*** -- Backup publisher database AW2014 on SQLNode4 and restore it on Secondary on SQLNode5 (with no recovery) :CONNECT SQLNODE4 BACKUP DATABASE [AW2014] TO  DISK = N'C:\Backups\UserDB\AW2014-ForAG.bak' WITH NOFORMAT, NOINIT, NAME = N'AW2014-Full Database Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10 GO   :CONNECT SQLNODE4 BACKUP LOG [AW2014] TO  DISK = N'C:\Backups\UserDB\AW2014-ForAG.trn' WITH NOFORMAT, NOINIT, NAME = N'AW2014-T Log Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10 GO   :CONNECT SQLNODE5 USE [master] RESTORE DATABASE [AW2014] FROM  DISK = N'\\SQLNode4\C$\Backups\UserDB\AW2014-ForAG.bak' WITH  FILE = 1,  NORECOVERY,  NOUNLOAD,  STATS = 5 RESTORE LOG [AW2014] FROM  DISK = N'\\SQLNode4\C$\Backups\UserDB\AW2014-ForAG.trn' WITH  FILE = 1,  NORECOVERY,  NOUNLOAD,  STATS = 5 GO       -- Create publisher AG "AW2014Pub_AG" Using GUI (Join Only). I did not script out this on-purpose to show that this part can be done with GUI, you can surely use script to create publisher database AG. Keep in mind, most of the Distributor database in AG part must be done with script, GUI doesn’t support it yet. -- 1. Primary replica: SQLNODE4  (readable) -- 2. Secondary replica: SQLNODE5  (readable) -- 3. Database: AW2014       -- Add publisher AG listener - AW2014PubAGList :CONNECT SQLNODE4 USE [master] GO ALTER AVAILABILITY GROUP [AW2014Pub_AG] ADD LISTENER N'AW2014PubAGList' ( WITH IP ((N'10.10.3.56', N'255.0.0.0') ) , PORT=1433); GO   -- Check all secondary replcas have replication feature installed -- :CONNECT SQLNODE5 USE master; GO DECLARE @installed int; EXEC @installed = sys.sp_MS_replication_installed; SELECT @installed; -- Return 1 is installed   -- SECTION 7 ---- ***CONFIGURE THE REDIRECT PUBLISHER*** -- on paimary distributor replicas SQLNode1, add publisher redirect by using publsher database ag listener AW2014PubDist :CONNECT SQLNODE1 USE distributionDB; GO EXEC sys.sp_redirect_publisher @original_publisher = 'SQLNode4', @publisher_db = 'AW2014', @redirected_publisher = 'AW2014PubAGList';    :CONNECT SQLNODE1 USE distributionDB; GO EXEC sys.sp_redirect_publisher @original_publisher = 'SQLNode5', @publisher_db = 'AW2014', @redirected_publisher = 'AW2014PubAGList';    -- check and validate all publisher replicas (must be on primary distributor replica) /* Notes: sp_validate_replica_hosts_as_publishers is an extention of sp_validate_redirected_publisher. All publisher's replica must be readable/accessible to get them verified, Otherwise, the validation will fail with error msg 21899. */ :CONNECT SQLNODE1 USE distributionDB; GO DECLARE @redirected_publisher sysname; EXEC sys.sp_validate_replica_hosts_as_publishers @original_publisher = 'SQLNode4', @publisher_db = 'AW2014', @redirected_publisher = @redirected_publisher output; select @redirected_publisher as redirect_publisher   -- alternatively, check and validate with sp_validate_redirected_publisher (must be on primary distributor replica) :CONNECT SQLNODE1 USE distributionDB; GO DECLARE @redirected_publisher sysname; EXEC sys.sp_validate_redirected_publisher @original_publisher = 'SQLNode4', @publisher_db = 'AW2014', @redirected_publisher = @redirected_publisher output; select @redirected_publisher as redirect_publisher   -- or maybe simply use query to check redirect directly, result looks good too -- this check doesn't guarentee all secondaries are accessible, only check its meta data :CONNECT SQLNODE1 select * from DistributionDB.[dbo].[MSredirected_publishers] /* -- result looks good as below, expalin the result below --   original_publisher    publisher_db    redirected_publisher ---------------------------------------------------------- SQLNode4            AW2014            AW2014PubAGList SQLNode5            AW2014            AW2014PubAGList */   -- check the distributor and publisher relationship in distribution db, the result looks good too :CONNECT SQLNODE1 sp_helpdistpublisher   -------------------------------PUBLISHER IN AG CONFIGURATION COMPLETED----------------------------------------   ---------------------------NOW, BOTH DISTRIBUTOR AND PUBLISHER ARE IN AG -------------------------------------   --------------------------------------NOW TEST THE WHOLE THING------------------------------------------------   -- Test the replication behavior with both Publisher and Distributor in AGs --   -- ***Test the above table update again and watch if result is all good*** --   --use listener to update primary publisher database :Connect AW2014PubAGList update AW2014.person.address_test set addressline1 = '1789 Napa Ct.' where addressid = 1   --check primary publisher database result :Connect AW2014PubAGList select * from AW2014.person.address_test where addressid = 1 Go   --check secondary publisher database result :Connect SQLNODE5 select * from AW2014.person.address_test where addressid = 1 Go   --check the subscriber database result :Connect SQLTest\SQL2017Sub select * from AW2014_Repl.person.address_test where addressid = 1 Go       -- ***Failover Distributor AG and Test the above table update again*** --   -- 1. Failover Ditributor AG :Connect SQLNODE1 :Connect SQLNODE2 ALTER AVAILABILITY GROUP [DistributionDB_AG] FAILOVER; GO -- 2. Watch jobs on Distributor are disabled and enabled on different replicas automatically as expected (may take a min) -- 3. Test the above table update again and watch if result is all good :Connect AW2014PubAGList --use listener to update primary publisher database update AW2014.person.address_test set addressline1 = '1790 Napa Ct.' where addressid = 1   --check primary publisher database result :Connect AW2014PubAGList select * from AW2014.person.address_test where addressid = 1 Go   --check secondary publisher database result :Connect SQLNODE5 select * from AW2014.person.address_test where addressid = 1 Go   --check the subscriber database result :Connect SQLTest\SQL2017Sub select * from AW2014_Repl.person.address_test where addressid = 1 Go       -- ***Failover Publisher AG and Test the above table update again*** --   -- 1. Failover Publisher database AG :Connect SQLNODE4 :Connect SQLNODE5 ALTER AVAILABILITY GROUP [AW2014Pub_AG] FAILOVER; GO   -- 2. Test the above table update again and watch if result is all good :Connect AW2014PubAGList --use listener to update primary publisher database update AW2014.person.address_test set addressline1 = '1791 Napa Ct.' where addressid = 1   --check primary publisher database result :Connect AW2014PubAGList select * from AW2014.person.address_test where addressid = 1 Go   --check new secondary publisher database result :Connect SQLNODE4 select * from AW2014.person.address_test where addressid = 1 Go   --check the subscriber database result --it may take a while, eventually you will see logreader parse new log and distrib send tx/cmd over to the subscriber :Connect SQLTest\SQL2017Sub select * from AW2014_Repl.person.address_test where addressid = 1 Go   -- Default behavior and how TF1448 works – refer to https://docs.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-traceon-trace-flags-transact-sql?view=sql-server-2017       -- Failover back to SQLNODE1 as primary Distributor replica :Connect SQLNODE2 :Connect SQLNODE1 ALTER AVAILABILITY GROUP [DistributionDB_AG] FAILOVER; GO -- Wait a min or two, the jobs should be refreshed again as expected   -- Failover back to SQLNODE4 as primary Publisher replica :Connect SQLNODE5 :Connect SQLNODE4 ALTER AVAILABILITY GROUP [AW2014Pub_AG] FAILOVER; GO   -- ***Final test with both Distributor and Publisher database failover together*** -- --update table :Connect AW2014PubAGList update AW2014.person.address_test set addressline1 = '1789 Napa Ct.' where addressid = 1   --check primary publisher database result :Connect AW2014PubAGList select * from AW2014.person.address_test where addressid = 1 Go   --check secondary publisher database result :Connect SQLNODE5 select * from AW2014.person.address_test where addressid = 1 Go   --check the subscriber database result --it may take a while, refresh and eventually the tx/cmd is transmitted over to the subscriber through logread/distrib :Connect SQLTest\SQL2017Sub select * from AW2014_Repl.person.address_test where addressid = 1 Go   ------------------------------------END - CONFIGURATION TEST COMPLETED HERE - END-----------------------------------------------   -------------------------------------------------------------CLEAN-UP SCRIPT--------------------------------------------------------------- -- Removing distributor and publisher -- This workflow is about removing publisher and distributor when distribution database is in AG. -- Publisher workflow -- 1) On publisher, drop all the subscriptions and publications for this publisher -- 2) And then call "sp_dropdistributor". :CONNECT SQLNODE4 sp_dropdistributor   :CONNECT SQLNODE5 sp_dropdistributor   -- Distributors cleanup workflow -- 1) Assuming SQLNode1 is the current primary of distribution database AG. SQLNode2 is the secondary replica. -- 2) On SQLNode2, call "sp_dropdistpublisher '[Publiser]'" with parameter @no_checks = 1 :CONNECT SQLNODE2 exec sp_dropdistpublisher 'SQLNODE4', @no_checks = 1 exec sp_dropdistpublisher 'SQLNODE5', @no_checks = 1   --3) On distributor primary replica, call "sp_dropdistpublisher '[Publiser]'". :CONNECT SQLNODE1 exec sp_dropdistpublisher 'SQLNODE4' exec sp_dropdistpublisher 'SQLNODE5'   --4)    Delete the Distributor AG. :CONNECT SQLNODE1 USE [master] GO DROP AVAILABILITY GROUP [DistributionDB_AG]; GO   -- 5) On the distributor secondary replica, change the "distribution" database to read_write mode by restoring the database with recovery. -- Call "sp_dropdistributiondb 'distribution'" with parameter @former_ag_secondary=1 to drop distribution database -- and to retain the snapshot directory. This will remove all the dangling jobs on this replica. :CONNECT SQLNODE2 RESTORE DATABASE DistributionDB WITH RECOVERY EXEC sp_dropdistributiondb 'DistributionDB', @former_ag_secondary=1   -- 6) On the distributor primary replica, call "sp_dropdistributiondb 'distribution'" to drop distribution database. :CONNECT SQLNODE1 EXEC sp_dropdistributiondb 'DistributionDB'   -- 7) If there are no other distribution databases in AG, sp_dropdistributor can be done on all distributor replicas. :CONNECT SQLNODE1 EXEC sp_dropdistributor go   :CONNECT SQLNODE2 EXEC sp_dropdistributor   -- CHECK AND DELETE THE FOLLOWINGS MANUALLY -- Any Replication SQL Agent jobs left on all servers -- Any Replication related linked servers on all servers -- Any Mirroring Endpoints for Replication AlwaysOn AG that is not in used -- Drop any databases if it is part of setup -- Drop any database backup files if it is part of setup   -- Drop publisher AG :CONNECT SQLNODE4 USE [master] GO DROP AVAILABILITY GROUP [AW2014Pub_AG]; GO -- Drop any endpoints for publisher AG on SQLNode4 and SQLNode5 -- Drop the AW2014 database on SQLNode5 only (secondary) -- Drop the AW2014_Repl database on subscriber SQLTest\SQL2017Sub -- Delete all backup files for AG and Replication setup in \\sqlnode4\c$\Backups\UserDB\

TDE has been out there since SQL Server 2008 and it is widely used to protect data/log/backup files at rest. When I talked with DBAs sometimes, even with some very experienced DBAs, I still feel there is some confusing around the terms, such as Service Master Key (SMK), Database Master Key (DMK), Certificate and Database Encryption Key (DEK) etc… especially in the area how we backup, maintain those keys, when we should restore, re-create them. This topic is focusing on how to add a TDE encrypted database to AlwaysOn AG using one of the most commonly used TDE methods: SMK -> DMK -> Certificate -> DEK -> User Database. I added decent amount of comments in the script below, hopefully that can be helpful.

The following scripts are tested in my lab, feel free to test it in your testing environment.

Lab Environment:

• SQL Server 2016 AlwaysOn AG - 2 nodes:
• Primary - SQL2016N1
• Secondary - SQL2016N2
• Assuming you already had OS Cluster configured and SQL installed. And AlwaysOn AG feature enabled in SQL Server configuration manager.

Basic knowledge:

1. SMK - The SMK is automatically generated the first time the SQL Server instance is started and is used to encrypt a linked server password, credentials, and the database master key. The SMK is encrypted by using the local computer key using the Windows Data Protection API (DPAPI). The DPAPI uses a key that is derived from the Windows credentials of the SQL Server service account and the computer's credentials. The service master key can only be decrypted by the service account under which it was created or by a principal that has access to the machine's credentials. SMK is stored in master database.
2. DMK - The database master key is a symmetric key that is used to protect the private keys of certificates and asymmetric keys that are present in the database. It can also be used to encrypt data, but it has length limitations that make it less practical for data than using a symmetric key. Noticed here, a DMK can be decrypted by SMK automatically or it can be decrypted by passwords using the OPEN MASTER KEY statement and an known/good password to open it explicitly, I said passwords, it is a plural, meaning a DMK can be decrypted by more than 1 passwords, so, if you don't know its original password when someone created it initially, don't worry, you can simply add a new password to it, as long as the SQL Server instance is running and the DMK can be opened by the SMK. A copy of DMK is stored in master database, and also it stored in the user database that has TDE enabled.
3. Certificate - The certificate is encrypted by the DMK, you can consider that a certificate is equivalent to an asymmetric key.
4. DEK - The DEK is encrypted by the certificate, and it is the actual key to encrypt the contents of the user database, the encrypted DEK is stored in the boot page of the user database, you can tell the boot page itself is not encrypted, right? Yes, that is correct. The boot page of a user database is not encrypted, the rest of pages are encrypted by the DEK, so, in order to access the rest of pages in the database, the DEK in the boot page must be decrypted first, which requires the certificate.

On the primary:

--create testing database
Create database DB_TDE_Test

--check if the DMK exist in master database
SELECT * FROM 
sys.symmetric_keys
WHERE name = '##MS_DatabaseMasterKey##'

--if not exist, create one
Use master
Go
CREATE MASTER KEY ENCRYPTION BY PASSWORD ='verycomplicateDMKpassword1234!!!'

--notice the DMK name is ##MS_DatabaseMasterKey## and its keyid is always 101
--if you just created on, this DMK is by default encrypted by the SMK and the password,
--that means you can decrypted it automatically when the SMK present on the server, 
--or you can decrypted it by the password
SELECT * FROM 
sys.symmetric_keys
WHERE name = '##MS_DatabaseMasterKey##'

--now you need to create a certificate, which is encrypted by the DMK
CREATE CERTIFICATE TDE_Test_Cert WITH SUBJECT = 'TDE Certificate'

--check if the certificate is created in master database
--A certificate is equivalent to an asymmetric key
select * from sys.certificates

--now it is time to create DEK that is encrypted by the certificate
--DEK is stored in the user database
Use DB_TDE_Test

Create DATABASE ENCRYPTION KEY
WITH ALGORITHM = AES_256
ENCRYPTION BY SERVER CERTIFICATE TDE_Test_Cert

--you will see the warning message:
--Warning: The certificate used for encrypting the database encryption key has not been backed up. 
--You should immediately back up the certificate and the private key associated with the certificate. 
--If the certificate ever becomes unavailable or if you must restore or attach the database on another server, 
--you must have backups of both the certificate and the private key or you will not be able to open the database.

--backup SMK
Use master
Go
--backup SMK and keep it somewhere safe.
--backup SMK is always recommended, just in case when you server is crashed and you might need to restore it on new hardware.
--Restoring SMK is very rare, you don't need to have the same SMK on each nodes.
BACKUP SERVICE MASTER KEY   TO FILE = 'c:\temp\service_master_key'   ENCRYPTION BY PASSWORD = 'SMK_Backup_password123!!!';  --backup DMK
--backup DMK and keep it somewhere safe.
--backup DMK is always recommended, it is similar as SMK, just in case, usually you don't need to restore it.
--Actually based on the DMK creation mechanism, even you restore it, it will be still a different DMK
--But we don't need to worry about that, because DMK is just used to encrypt the certificate, 
--as long as the DMK can be opened by the SMK, and the certificate can be decrypted and so the DEK can be decrypted,
--then our database can be accessed eventually. Confusing!!!! still?
Backup master key to file = 'c:\temp\database_master_key' encryption by password = 'DMK_Backup_password123!!!'


--backup certificate (***very important***)
--backup certificate is critical, make sure you always do so, the backup of the certificate will have a pair of files, one is the certificate, 
--the other one is the private key file that is encrypted by the given password.
--make sure you keep the password somewhere safe since it will be required when you need to create the same certificate on the other node using the backup file pair.
--if you forgot the certificate's backup password, you can simply back it up again using different password as long as the certificate can be opened by the DMK. 
BACKUP CERTIFICATE TDE_Test_Cert TO FILE = 'c:\temp\TDE_Test_Cert' 
WITH PRIVATE KEY ( 
					FILE = 'c:\temp\TDE_Test_Cert_Key' ,
					ENCRYPTION BY PASSWORD = 'TDE_Cert_Backup_password123!!!'
				 );


--enable TDE, now the database is TDE encrypted using the 
ALTER DATABASE DB_TDE_Test SET ENCRYPTION ON


SELECT * FROM sys.dm_database_encryption_keys;  
GO  

--check if the database is encrypted/progress (encryption_state = 3 encrypted; =2 in progress)
USE master 
GO 
SELECT db_name(database_id) [TDE Encrypted DB Name], c.name as CertName, encryptor_thumbprint , dek.*     FROM sys.dm_database_encryption_keys dek     INNER JOIN sys.certificates c on dek.encryptor_thumbprint = c.thumbprint

On the secondary:

--copy all the backups from the primary node c:\temp\ to the secondary, that way, you don't need to change the file path in the code below
--or, you can modify the code below to reflect the path that matches your testing environment

Use master
Go

--check if DMK exists in master database on the secondary
--if yes, you don't need to create a new one, just use it
SELECT * FROM 
sys.symmetric_keys
WHERE name = '##MS_DatabaseMasterKey##'

--if the DMK doesn't exist in the master database on secondary, you can either restore the DMK from the backup from the primary, or you can create a new one
--fundamentally, based on the DMK generation mechanism the DMK will be different regardless of your restoring it or creating a new one.
--here I use restore on purpose, just wanted to show you what the each password means below:
--Decryption by password: this is the backup password that you used when you backup the DMK from primary, this will allow you to restore the DMK backup
--Encryption by password: this is a new password that will be used to encrypt/decrypt the DMK once it is restored, this password will allow you to open/decrypt the DMK
--Noticed the above two passwords are for different purpose.
Restore master key from file = 'c:\temp\database_master_key' 
Decryption by password ='DMK_Backup_password123!!!' --backup password
Encryption by password = 'verycomplicateDMKpassword1234!!!New'  --new DMK encryption password
/* optional
--or create a new DMK
Use master
Go
CREATE MASTER KEY ENCRYPTION BY PASSWORD ='verycomplicateDMKpassword1234!!!New'
*/

--Now, try to open the DMK with the new password, it should open successfully, now, you can alter it to be encrypted by SMK
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'verycomplicateDMKpassword1234!!!New'

--Alter the DMK to be encrypted by the SMK
--The step is usually missed by some customers, what it does is to add the DMK to be encrypted by the SMK
--so that when the SQL Server service starts, the DMK will be automatically decrypted without explicitly opening it.
--this is required for TDE working properly
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY

--you can check if the DMK in the master database is now encrypted by the SMK (is_master_key_encrypted_by_server = 1)
select is_master_key_encrypted_by_server, * from sys.databases
where database_id = db_id(N'master')


--perfect! now, let's create certificate on the secondary using the backup pair that we made from the primary node earlier
--***Very important***, once you execute the below statement, the same certificate TDE_Test_Cert will be created from the backup pair, 
--noticed that it is now encrypted by the new DMK on the secondary node
CREATE CERTIFICATE TDE_Test_Cert FROM FILE = 'c:\temp\TDE_Test_Cert'
WITH PRIVATE KEY (FILE = 'c:\temp\TDE_Test_Cert_Key' ,
DECRYPTION BY PASSWORD = 'TDE_Cert_Backup_password123!!!');  --the backup password


--check the DMK and the certificate TDE_Test_Cert is created on the secondary
SELECT * FROM 
sys.symmetric_keys
WHERE name = '##MS_DatabaseMasterKey##'

select * from sys.certificates

On the primary:

--Now it is ready to make a backup of the database [DB_TDE_Test] on the primary (SQL2016N1) and restore it to the secondary (SQL2016N2)

--Go to the primary SQL2016N1
BACKUP DATABASE [DB_TDE_Test] TO  DISK = N'C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\Backup\DB_TDE_Test_full.bak' 
WITH NOFORMAT, NOINIT,  NAME = N'DB_TDE_Test-Full Database Backup', 
SKIP, NOREWIND, NOUNLOAD,  STATS = 10
GO

BACKUP LOG [DB_TDE_Test] TO  DISK = N'C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\Backup\DB_TDE_Test_tlog.trn' 
WITH NOFORMAT, NOINIT,  NAME = N'DB_TDE_Test-Full Database Backup', 
SKIP, NOREWIND, NOUNLOAD,  STATS = 10
GO

On the secondary:

--Go to the secondary SQL2016N2, restore database with norecovery
--the restore should be successful, since we have all the keys/certificates ready on the secondary
USE [master]
RESTORE DATABASE [DB_TDE_Test] 
FROM  DISK = N'\\sql2016n1\C$\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\Backup\DB_TDE_Test_full.bak' 
WITH  FILE = 1,  NORECOVERY,  NOUNLOAD,  STATS = 5

--apply t-log with norecovery on the secondary
RESTORE LOG [DB_TDE_Test] 
FROM  DISK = N'\\sql2016n1\C$\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\Backup\DB_TDE_Test_tlog.trn' 
WITH  FILE = 1,  NORECOVERY,  NOUNLOAD,  STATS = 5

Now, you can add the database to AG, you cannot use GUI, you have to use t-sql here

On the primary:

--add the database to a AG on the primary (SQL2016N1)
USE master 
go 
ALTER AVAILABILITY GROUP [AW2016AG] ADD DATABASE [DB_TDE_Test]

On the secondary:

--go to the secondary (SQL2016N2), turn on HADR for the database
USE master 
go 
ALTER DATABASE [DB_TDE_Test] SET HADR AVAILABILITY GROUP = [AW2016AG];

At this point, the database DB_TDE_Test is added to the AG successfully!

Now, let's clean it up after the test:

----clean up script after the test----

--on the primary replica (SQL2016N1)
use master;
go
ALTER AVAILABILITY GROUP [AW2016AG]
REMOVE DATABASE [DB_TDE_Test];
--drop the test database and its backup history
EXEC msdb.dbo.sp_delete_database_backuphistory @database_name = N'DB_TDE_Test'
GO
USE [master]
GO
ALTER DATABASE [DB_TDE_Test] SET  SINGLE_USER WITH ROLLBACK IMMEDIATE
GO
USE [master]
GO
DROP DATABASE [DB_TDE_Test]
GO
--drop the certificate
DROP CERTIFICATE TDE_Test_Cert 
--drop the DMK
DROP MASTER KEY;
--check if they are cleaned up
select * from sys.certificates;
SELECT * FROM sys.symmetric_keys;

--on the secondary replica (SQL2016N2)
DROP DATABASE [DB_TDE_Test]
--drop the certificate
DROP CERTIFICATE TDE_Test_Cert 
--drop the DMK
DROP MASTER KEY;
--check if they are cleaned up
select * from sys.certificates;
SELECT * FROM sys.symmetric_keys;

Thanks for reading! If you have any questions, please feel free to post in the comments area and I will try my best to get back to you asap.]]> https://docs.microsoft.com/archive/blogs/sqlhenryweng/how-to-setup-microsoft-r-server-one-box-configuration-to-support-ad-account-via-ldap Fri, 03 Nov 2017 18:46:25 GMT https://blogs.msdn.microsoft.com/sqlhenryweng/?p=345 One of my customers was asking how to setup Microsoft R Server, basically there are two options:

1. One-box Configuration
2. Enterprise Configuration

In this step-by-step POC tutorial, I will show you how to setup One-box configuration to support AD account via LDAP, it covers:

1. Setup One-box configuration on Server side
2. Configure On-box configuration to support AD via LDAP
3. Setup Client side to connect to R Server and test it

One-box Configuration Lab Environment:

• Microsoft R Server 9.1.0/SQLLite3.7 - SQL2016N2
• RStudio + Microsoft R Client 3.4.1 - SQL2014Client
• DC: SQL2014DC.Features2014DC.local
• All same password for test only: Corp123!

Steps tested in the Lab environment:

R Server configuration (SQL2016N2):

1. Install R 9.1.0 in en_microsoft_r_server_910_for_windows_x64_10324119.zip (From <https://docs.microsoft.com/en-us/machine-learning-server/install/r-server-install-windows> )
2. You can find the setup log (naming convention like this: Microsoft_R_Server_20171018143405.log) in %temp% to make sure installation are completed successfully
3. Connect and validate R Server installation locally

• R Server runs on demand as a background process, as Microsoft R Engine in Task Manager. Server startup occurs when a client application like R Tools for Visual Studio or Rgui.exe connects to the server.
• As a verification step, connect to the server and execute a few ScaleR functions to validate the installation.

          1) Go to C:\Program Files\Microsoft\R Server\R_SERVER\bin\x64.
           2) Double-click Rgui.exe to start the R Console application.
           3) At the command line, type search() to show preloaded objects, including the RevoScaleR package.
           4) Type print(Revo.version) to show the software version.
           5) Type rxSummary(~., iris) to return summary statistics on the built-in iris sample dataset. The rxSummary function is from  RevoScaleR.

      4.  At this point, we know the R server installation are verified locally, let's configure the server, firstly, start Admin Utility -

          CD C:\Program Files\Microsoft\R Server\R_SERVER\o16n\Microsoft.RServer.Utils.AdminUtil\

          dotnet Microsoft.RServer.Utils.AdminUtil.dll

          From <https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-use-admin-utility>

          Note: Local 'admin' account might be sufficient when trying to operationalize with a one-box configuration since everything is running within the trust boundary, it is insufficient for enterprise configurations.

      5.  Configure Server/Configure R Server for Operationalization

           A. One-box (web+compute nodes

           B. Set admin password: Corp123! ("admin" is the default admin account for One-box configuration, once you configure to use AD account, this admin account will not be used any more)

      6.  Run Diagnostic Tests to test the configuration - https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-run-diagnostics

           A. Test configuration for a 'health report' of the configuration including a code execution test. Result:

• Review the test results. If any issues arise, a raw report appears. You can also investigate the log files and attempt to resolve the issues.
• After making your corrections, restart the component in question. It may take a few minutes for a component to restart.
• Rerun the diagnostic test to make sure all is running smoothly now.

Client Configuration (SQL2014Client):

      Microsoft R Client overview:

• R Client allows you to work with production data locally using the full set of ScaleR functions, but there are some constraints. On its own, the data to be processed must fit in local memory, and processing is capped at two threads for RevoScaleR functions. +
• To benefit from disk scalability, performance and speed, push the compute context using rxSetComputeContext() to a production instance of Microsoft R Server (or R Server) such as SQL Server Machine Learning Services and Machine Learning Server for Hadoop. Learn more about its compatibility.+
• You can offload heavy processing to Machine Learning Server or test your analytics during their developmentYou by running your code remotely using remoteLogin() or remoteLoginAAD() from the mrsdeploy package.

           From <https://docs.microsoft.com/en-us/machine-learning-server/r-client/install-on-windows>

     Client Install:

1. Rstudio - https://www.rstudio.com/products/rstudio/download/

           Or Visual Studio 2015 + R Add-on - https://docs.microsoft.com/en-us/visualstudio/rtvs/installation

      2.  Download and install Microsoft R Client at https://aka.ms/rclient/

      3.  Taking Rstudio as example, set up Rstudio to use Microsoft R, go to Tools -> Global Options -> General -> R version to point to the Microsoft R Client: [64-bit] C:\Program Files\Microsoft\R Client\R_SERVER

           Reference: https://support.rstudio.com/hc/en-us/articles/200486138-Using-Different-Versions-of-R

      4.  Now you should be able to remotelogin the R Server SQL2016N2 and test it Test remote connection in R Studio from the client machine SQL2014Client:

           Refer to https://docs.microsoft.com/en-us/machine-learning-server/operationalize/how-to-connect-log-in-with-mrsdeploy#authentication

> # EXAMPLE: LOGIN, CREATE REMOTE R SESSION, GO TO REMOTE PROMPT

> remoteLogin("https://SQL2016N2:12800")

# here it will prompt you to enter the username: admin and its password you defined

REMOTE> x <- 10 # Assign 10 to "x" in remote session

REMOTE> ls() # List objects in remote session

[1] "x"

REMOTE> pause() # Pause remote interaction. Switch to local

> y <- 10 # Assign 10 to "y" in local session

> ls() # List objects in local session

[1] "y"

> putLocalObject(c("y")) # Loads local "y" into remote R session's workspace

> resume() # Resume remote interaction and move to remote command line

REMOTE> ls() # List the objects now in the remote session

[1] "x" "y"

REMOTE> exit # Destroy remote session and logout

>

The below flow chart shows you how the client/server interacted with local and remote R Sessions using pause and resume:

From <https://docs.microsoft.com/en-us/machine-learning-server/operationalize/how-to-connect-log-in-with-mrsdeploy>

Now, the client can remotelogin the R server and write R code, train a model, score a model and publish a model as a web service, it is all good, but we have give out the admin password to all R Contributors, for POC it may be ok, but in real world, we need this to support AD account, not just admin/password pair. We also need ability to categorize the permissions in different groups. Below will explain you how to do this. At high level, Microsoft R Server 9.1 supports three security roles:

1. Administrator/Owner - this role will be the owner of the R Server who have full control of the R server, can manage any service
2. Contributor - this role will be the contributor of the R Server who can publish web services, such as R programmers, data scientists etc
3. Reader - this role will be the reader of the R Server who can consumes the web services, such as application developers etc

Now, let's continue to configure this One-box R Server to support AD account via LDAP.

1. Firstly, in order to configure the R Server SQL2016N2 to use AD/LDAP, you need to add this role to the server:

      2.  After the feature/role installed, you need to setup it via the Setup Wizard below: More details in - https://blogs.msdn.microsoft.com/microsoftrservertigerteam/2017/04/10/step-by-step-guide-to-setup-ldaps-on-windows-server/

• You don't have to setup SSL/TLS at this moment, because it will need appropriate certificate, I will put together another blog to configure LDAP-S, in this demo, it it just LDAP.

    3.  Now, back to AD, For demo purpose, I created the following Roles/Groups in AD (features2014dc.local) to use the R Server, you can refer to more details in https://blogs.msdn.microsoft.com/mlserver/2017/04/10/role-based-access-control-with-mrs-9-1-0/, in this demo, I created below AD groups and users:

Group: MRSAdmins - this group will be the owner of the R Server who have full control of the R server

Member: RAdmin1

Group: Rprogrammers - this group will be the contributor of the R Server who can publish web services

Member: RProgrammer1, RProgrammer2, …

Group: AppDevelopers - this group will be the reader of the R Server who consumes the web services

Member: AppDev1, AppDev2 …

        After you created the groups and demo users in AD, it will look like this:

   4.  Once this is completed, you need to setup R Server Role Configurations, you can reference to https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-roles (this is for Enterprise Configuration, but it is similar), or just simply follow the below steps.

   5.  Now we will need to make some changes in appsettings.json file for the web node, go to the folder (by default) C:\Program Files\Microsoft\R Server\R_SERVER\o16n\Microsoft.RServer.WebNode\

   6.  Find the file appsettings.json, make a backup (I made a backup file called appsettings-backup.json_for_admin as the above screenshot) of it before you making changes, just in case, so you don't need to re-configure previously completed steps, you can easily revert back to admin/password mode if you want to. Open the json file in notepad and find the section and make changes as below as the highlighted section according to your environment, I will explain it here:

• Host - it is the IP of your DC
• QueryUserDn - is a domain account that has permission to query Active Directory using LDAP, I am thinking a service account will be preferred here, so you don't need to update this when a user leaves the company.
• QueryUserPassword - the domain account's password, which can be encrypted, if so, change to "QueryUserPasswordEncrypted": true
• SearchBase - can be the parent directory where you have the groups/users created in the above step 3, in this example, all users and groups created for R Server are under the CN=Users directory.
• In the Authorization section, add the AD groups indicating the different R Server roles.

"LDAP": {

"Enabled": true,

"Description": "Enable this section if you want to enable authentication via LDAP",

"Host": "<your_host_ip>",

"Port": 389,

"UseLDAPS": false,

"QueryUserDn": "CN=RDeployAmin,CN=Users,DC=FEATURES2014DC,DC=LOCAL",

"QueryUserPassword": "P@$$w0rd!",

"QueryUserPasswordEncrypted": false,

"SearchBase": "CN=Users,DC=FEATURES2014DC,DC=LOCAL",

"SearchFilter": "cn={0}",

"UniqueUserIdentifierAttributeName": "userPrincipalName",

"DisplayNameAttributeName": "name",

"EmailAttributeName": "mail"

},

"Authorization": {

"Owner": [ "MRSAdmins" ],

"Contributor": [ "RProgrammers" ],

"Reader": [ "App developers" ]

},

  7.  Now, go back to Admin Utility, we will need to stop web node and start web node to make the change affected.

      8.  Now, we can do Diagnostic Test, since it is not using AD Authentication, the original Admin would be disabled automatically, so you will need to enter the AD account to validate it, for example, you can enter the AD account: RAdmin1/Corp123! - which is the owner role of the R Server, it should work.

      9.  Local Diagnostic test passed. Great! Now, let’s test it from the Client machine SQL2014Client, in RStudio:

          Tested Owner - RAdmin1/Corp123! - it is in the MRSAdmins group in AD

          Tested Contributor - RProgrammer1/Corp123! - it is in Rprogrammers group in AD

Now you know how to make Microsoft R Server with One-box configuration to support AD account via LDAP.

References:

Role Based Access Control With MRS 9.1.0 - https://blogs.msdn.microsoft.com/mlserver/2017/04/10/role-based-access-control-with-mrs-9-1-0/

How to publish and manage R web services in Machine Learning Server with mrsdeploy - Owner/Contributors (Administrators or R Programmers) can publish R script as web service - https://docs.microsoft.com/en-us/machine-learning-server/operationalize/how-to-deploy-web-service-publish-manage-in-r

App Developers - usually are the MSR Server Readers role, who can consume the service:

https://docs.microsoft.com/en-us/machine-learning-server/operationalize/how-to-consume-web-service-interact-in-r

https://docs.microsoft.com/en-us/machine-learning-server/operationalize/how-to-consume-web-service-asynchronously-batch

Thanks for reading, next, I will put up together a tutorial to configure One-box to support LDAP-S, which is LDAP over SSL/TLS.]]> https://docs.microsoft.com/archive/blogs/sqlhenryweng/using-unbind-to-downgrade-bound-r-server-9-1-0 Thu, 27 Jul 2017 15:38:31 GMT https://blogs.msdn.microsoft.com/sqlhenryweng/?p=55 One of my customers recently had a situation that they upgraded their Microsoft R Server to the latest version 9.1.0, and bound it to SQL Server 2016. Then they want to downgrade to SQL Server 2016 built-in MS R version 8.0.3. According to the document - https://docs.microsoft.com/en-us/sql/advanced-analytics/r/use-sqlbindr-exe-to-upgrade-an-instance-of-sql-server, you can surely upgrade it, but how do we downgrade. Here is what I tested in my lab step by step:

Current environment: SQL Server 2016 SP1 CU3 + Microsoft R Server 9.1.0 (bound)
Goal: downgrade R server 9.1.0 to lower version, which is 8.0.3 (the built-in MSR version in SQL Server 2016 SP1) in this test

1. Unbind the R Server 9.1.0 with the SQL Server instance

cd “C:\Program Files\Microsoft\R Server\Setup”
sqlbindr.exe /list
sqlbindr.exe /unbind mssql13.mssqlserver

2. If I run the R validation script in SSMS, it failed as expected as the R Server is unbound with SQL Server instance.

3. Since unbinding removes all versions of MRS on the instance, users need to run SQL Setup repair to restore MRS on the instance.

4. During the SQL repair, the SQL Server instance will be restarted, so it requires downtime. Once the repair completes, now you have the SQL Server 2016 SP1 build-in R 8.0.3 back.

PG put a lot of effort to make the process easier, that is it! Hope this helps in case you are having similar issue.
Next time, I will put together a blog for how to in-place upgrade SQL Server 2012/2014 to SQL 2016 with R Services.

~heweng