sudo su
cd /root

openssl genrsa -out privateKey.pem 2048
openssl req -new -x509 -key privateKey.pem -out publicCert.pem -days 3650 -nodes -subj "/CN=LOCALHOST"
openssl pkcs12 -export -out merged.pfx -inkey privateKey.pem -in publicCert.pem -passout pass:

dotnet new console -o InstallCert
cd InstallCert
vi Program.cs

dotnet restore
dotnet run

/root/.dotnet/corefx/cryptography/x509stores/root/D03B70561C0E61F188AC1A6EFF47C18B8E7767E1.pfx

"JWTSigningCertificate": {
 "Enabled": true,
 "Description": "Enable this section if you want to sign the access token with a certificate instead of a randomly generated key",
 "StoreName": "Root",
 "StoreLocation": "CurrentUser",
 "SubjectName": "CN=LOCALHOST"
 }

"JWTKey": "not set"

"JWTKey": "eyJEIjoiSkp..........WdTRJZ009In0="

UPDATE :

mkdir -p /home/webnode_usr/.dotnet/corefx/cryptography/x509stores/root
cp /root/.dotnet/corefx/cryptography/x509stores/root/D03B70561C0E61F188AC1A6EFF47C18B8E7767E1.pfx /home/webnode_usr/.dotnet/corefx/cryptography/x509stores/root
chmod 777 /home/webnode_usr/.dotnet/corefx/cryptography/x509stores/root

REFERENCES:

Authentication options for R Server when operationalizing analytics

Access Token Management for API Requests

Connection Security (SSL/TLS)

1. OneBox Configuration for Windows
2. OneBox Configuration for Linux
3. OneBox Configuration on Edge Node of R Server HDInsight Cluster
4. Dynamic Scaling Enterprise Configuration on Windows with Azure SQL DB as WebNode Backend and Azure Active Directory Authentication
5. Dynamic Scaling Enterprise Configuration on Windows with SQL Always On cluster as WebNode Backend and Active Directory Authentication
6. Dynamic Scaling Enteprise Configuration on Linux with Azure SQL DB as WebNode Backend and Azure Active Directory Authentication

Currently the rx function APIs are only for use in a SQL Server compute context. They are a convenient wrapper of the suite of package management functions like install.packages() in base R.

Custom and secure package management is governed by scopes

• Shared scope - this scope allows allowed users to install & uninstall packages in per database shared location which in turn can be used by other users of the database on SQL server
• Private scope - this scope allows allowed users to install & uninstall packages in per database per user private location which can only be used by the single user of the database on SQL server

and database roles

• rpkgs-users - allows users to use shared packages installed by users belong to rpkgs-shared role
• rpkgs-private - allows all permissions as rpkgs-users role and also allows users to install, remove and use private packages
• rpkgs-shared - allows all permissions as rpkgs-private role and also allows users to install, remove shared packages
• db_owner - allows all permissions as rpkgs-shared role and also allows users to install & remove shared and private packages for other users for management

By default, any user can use the packages in the shared scope. The beauty is that, each rpkgs-private role user can also install his/her own version of this package in his/her private scope. And when this happens, the shared-scope package will be masked, meaning that when this user calls library(<package name>), only his/her installed private-scope version of this package will be loaded.

We will walk through a simple example to mimic a mock yet typical scenario where data scientists collaborate in a database.

Example:

First we create a database called RevoTestDB. Then we need to

1. Set up SQL Server R Services
2. Enable package management

Step 2 will create the additional database roles rpkgs-private, rpkgs-shared and rpkgs-users which are specific for package management as shown on the left. Then we run the T-SQL script below to create the following database users, each taking a specific database role, with their level of permissions in descending order:

The script also grants other roles and permissions needed for these users to use the database as required in the set up SQL Server R Services step.

To install the package 'scales' in shared scope for anyone to use, the senior data scientist can run the following script, and by default he/she installed the lastest version (0.4.1) of 'scales' from MRAN:

Then any user can use the package by either passing the package name to transformPackages in an rx fucntion or loading the package in an rxExec call. For example,

Now say the junior data scientist wants to install an older version of scales (0.4.0) for himself/herself's own use, he/she can, for example, download the .zip file to his local working directory and run the following script to install scales 0.4.0 from the downloaded .zip file by specifying repos = NULL in rxInstallPackages and verify that's actually the version that's loaded. Other users will not be affected by the junior data scientist's installation and will still be using scales 0.4.1.

The data scientist lead can install/uninstall any packages on behalf of any user. And when he/she uninstalls a package, the dependencies are automatically taken care of by specifying dependencies = TRUE in rxRemovePackages.

The output will be something like this:

To change a SQL Server instance's system package installation manually, refer to Install Additional R Packages on SQL Server.

Zipkin is a tracing tool for distributed systems to log events related to each individual activity (also known as job, transaction, or trace, depending on the context) in a way that allows the path of the activity through the system to be reconstructed. It offers correlation across multiple services within a customer’s application. With Application Insights integration, Zipkin now gains the ability to log data to an Application Insights store and to trace activities through multiple services including non-logging systems using impersonation. It provides application performance management features from Application Insights along with tracing capabilities provided by Zipkin.

Application Insights users would also be able to leverage the distributed tracing offered through Zipkin by instrumenting their services using existing libraries. To use the Application Insights back-end store, configure your Zipkin server instance to use the Application Insights plug-in. This integration makes monitoring and debugging your overall end-to-end applications much easier.

Azure-based systems may already have one or more associated Application Insights instances. If so, you can use any one of them. Otherwise, you can create a new Application Insights instance by logging in to the Azure Portal, and then navigate to the Application Insights blade, either by typing “Application Insights” into the search bar or by choosing “Application Insights” from the service list.

Prerequisites

1. Oracle client must be installed and the client's bin folder must be added to PATH environment variable.
2. Recommend using separate sqlnet.ora and tnsnames.ora files so as to separate regular connections from wallet connection. Normally you can find these files under $ORACLE_HOME/network/admin folder or something similar.
3. TNS_ADMIN environment variable should point to the approproate folder with sqlnet.ora and tnsnames.ora.

Setting up wallet

mkdir -p C:\Oracle\admin\mywallet

mkstore -wrl C:\Oracle\admin\mywallet -create

For a comprehensive description of Oracle wallet see here.

SQLNET.WALLET_OVERRIDE = TRUE WALLET_LOCATION=( SOURCE=(METHOD=FILE) (METHOD_DATA=(DIRECTORY=C:\Oracle\admin\mywallet)) )

mkstore -wrl C:\oracle\admin\mywallet -createCredential <TNS alias> <myuser> <mypassword>

(TNS alias is the alias from tnsnames.ora file)

Now we can test the wallet using sqlplus:

sqlplus /@<TNS alias>

This should log you in on sqlplus/SQL prompt without asking for username or password.

sConnectionStr <- "Driver={Oracle in OraClient12Home1};DBQ=<TNS alias>;" checkDBversion <- RxOdbcData(sqlQuery="SELECT BANNER FROM V$VERSION", connectionString = sConnectionStr, rowBuffering=FALSE) rxGetInfo(checkDBversion, numRows = 1)

Connection string: Driver={Oracle in OraClient12Home1};DBQ=<TNS alias>; Data Source: ODBC Data (1 row starting with row 1): BANNER 1 Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

References:

• Download and install RStudio Server Pro from here (OR) RStudio Server Open Source License from here.

wget https://download2.rstudio.org/rstudio-server-rhel-1.0.143-x86_64.rpm
sudo yum install --nogpgcheck rstudio-server-rhel-1.0.143-x86_64.rpm
sudo rstudio-server verify-installation
sudo rstudio-server version

wget https://download2.rstudio.org/rstudio-server-rhel-pro-1.0.143-x86_64.rpm
sudo yum install --nogpgcheck rstudio-server-rhel-pro-1.0.143-x86_64.rpm
sudo rstudio-server verify-installation
sudo rstudio-server version

• We need to set some environment variables at the start of R session. This can be achieved using Renviron file. Append the following lines to /opt/cloudera/parcels/MRS/lib64/R/etc/Renviron

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/cloudera/parcels/MRS/lib64/R/lib
R_LIBS=/opt/cloudera/parcels/MRS/lib64/R/library
MRS_PARCEL_PATH=/opt/cloudera/parcels/MRS

• Create libjvm.so symlink

sudo ln -s /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so /opt/cloudera/parcels/MRS/hadoop/libjvm.so

• Copy RevoHadoopEnvVars.site file from home directory into hadoop directory. (NOTE: If .RevoHadoopEnvVars.site file is not present in the home directory , just run R command once - this will generate site file in home directory)

sudo cp ~/.RevoHadoopEnvVars.site /opt/cloudera/parcels/MRS/hadoop
sudo mv /opt/cloudera/parcels/MRS/hadoop/.RevoHadoopEnvVars.site /opt/cloudera/parcels/MRS/hadoop/RevoHadoopEnvVars.site

• Restart RStudio Server

sudo rstudio-server stop
sudo rstudio-server restart

• RStudio Server will be available in the following url : http://<nodename>:8787. (Make sure port 8787 is open)

LOCAL COMPUTE CONTEXT

LOCAL COMPUTE CONTEXT ON HDFS DATA

HADOOP COMPUTE CONTEXT

SPARK COMPUTE CONTEXT

Introduction

Using Microsoft R Server with sparklyr

• A Hadoop cluster with Spark and valid installation of Microsoft R Server
• Microsoft R Server is configured for use with Hadoop and Spark (Instructions here)
• Microsoft R Server SampleData loaded into HDFS
• gcc and g++ installed on the edgenode that the example will be run on
• Write permissions to the R Library Directory
• Read/Write permissions to HDFS directory /user/RevoShare
• An internet connection or the ability to download and manually install sparklyr and h2o

Installation of sparklyr package

Example One: Load and Partition Data in sparklyr, Train and Predict in MRS

1. Create a connection to Spark using rxSparkConnect(), specifying a sparklyr interop; using sparklyr and its interfaces to connect to Spark.
2. Call rxGetSparklyrConnection() on the compute context to get a sparklyr connection object.
3. We will use dplyr to load mtcars into a Spark DataFrame via the sparklyr connection object.
4. Partition the data in-Spark into a training and scoring set using dplyr.
5. After partitioning, we will register the training set DataFrame in Spark to a Hive table.
6. Train a model in ScaleR using rxLinMod() on an RxHiveData() object.
7. With that trained model, well will run a toy prediction using rxPredict() on the test partition.
8. After prediction, take the root mean square to determine accuracy.

Example Two: Load Data with MRS, Partition and Train a Model with sparklyr

1. Create a connection to Spark using rxSparkConnect(), specifying a sparklyr interop; using sparklyr and its interfaces to connect to Spark.
2. Call rxGetSparklyrConnection() on the compute context to get a sparklyr connection object.
3. Use Microsoft R Server to load data from many sources
4. Partition the data in-Spark into a training and scoring set using dplyr.
5. After partitioning, we will register the training set DataFrame in Spark to a Hive table.
6. Train a model using sparklyr to call Spark ML algorithms
7. Take a summary of the trained model to see estimates and errors

Example Three: Connect to Spark and Load Data with MRS, Cache Data with dplyr, Train a Model and Predict with H2O, Gather Metrics with MRS

1. Create a connection to Spark using rxSparkConnect(), specifying a sparklyr interop; using sparklyr and its interfaces to connect to Spark.
2. Call rxGetSparklyrConnection() on the compute context to get a sparklyr connection object.
3. Use Microsoft R Server to load Training and Test Data from HDFS
4. Represent the Data as Hive Tables, and Cache the Tables in Spark
5. Cast the data to h2o data frames for analysis
6. Train a model using h2o's built in GLM algorithm
7. Print the model data
8. Run a prediction on the test data with h2o.predict
9. Take the ROC and Area Under the Curve (AUC) to see how our model did

Conclusion

SQL Saturday is round the corner again! SQL Saturday is a great event to meet the community, gather feedback and meet some great friends that we made during our last few SQL Saturdays. This time the Tiger team will be presenting a session on how to use R-Services with SQL Server to build an analytical workflow for banking solutions. We will talk about how our customer Jack Henry & Associates, an S&P 400 company that supports more than 11,300 financial institutions with core processing services, is leveraging the power of SQL Server and R to make drive intelligent insights into their data warehousing software. Post the session, I will share a complete solution which you can deploy on our Data Science Virtual Machine on Azure and tinker around with the demo solution at your free time.

Our Corporate Vice President, Joseph Sirosh, had demonstrated this solution along with Jack Henry & Associates at Ignite. In this session, I will talk about the nuts and bolts on how to build a scalable predictive engine with SQL Server and using the enhancements shipped in SQL Server 2016. After this session, you will be able to build your very own scalable predictive engine on SQL Server 2016!

Title: Building 1 million predictions per second using SQL-R [Session Link]

When: 15th April, 2017 | 2:45PM – 3:45PM PST

Where: Building 92, 15010 NE 36th St, Redmond (Room: Utopia)

Abstract: Using the power of OLTP and data transformation in SQL 2016 and advanced analytics in Microsoft R Server, various industries really push the boundary of processing higher number of transaction per second (tps) for different use cases. In this talk, we will walk through the use case of predicting loan charge off (loan default) rate, architecture configuration that enable this use case, and rich visual dashboard that allow customer to do what-if analysis. Attend this session to find out how SQL + R allows you to build an "intelligent datawarehouse".

My colleague, Pedro Lopes, will be presenting a session on “Enhancements that will make your SQL database roar”.

Looking forward to meet the community this Saturday!

The slide deck used for my presentation can be found on SlideShare. The PowerBI dashboard and the demo scripts can be downloaded from the tigertoolbox repo on GitHub.

How to enable logging

Log file locations

Local Compute Context

Distributed compute contexts

RxHadoopMR and RxSpark

Client (when not on edge node):

On Edge node:

Worker node logs:

Teradata

SQL Server