What are Recommended Solutions to work with RStudio and HDInsight Spark Cluster

Christoph Kiefer 141 Reputation points
2020-07-01T20:37:32.087+00:00

Dear All

We are currently implementing in-house / on-prem machine learning solutions in R (RStudio). We are in the process of moving our data to the cloud by the means of a sophisticated ingestion process through Apache Nifi. Currently the data lands on an Azure Blob storage as an Avro file with a corresponding Hive table around it. We also created an HDInsight Spark cluster 3.6 to query the data.

We realized that the HDI cluster of type rserver (ML Services) does not have ESP support and that it will be retired by the end of the year. ESP support is crucial for us as we work in a large, international company with strict security / privacy rules.

Having said that, I wonder what are the best working Azure cloud solutions to work with R / RStudio?

Best regards
Christoph

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
198 questions
Azure R Server for HDInsight
Azure R Server for HDInsight
An Azure service that provides predictive analytics, machine learning, and statistical modeling for big data.
13 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 76,836 Reputation points Microsoft Employee
    2020-07-09T10:29:00.113+00:00

    Hello @ChristophKiefer-9885,

    You can install R-Studio as a custom component.

    Important note that we will not be able to provide support on functioning of a custom component.

    Here is a 3rd party content which suggests that solution - Installing R-studio on Azure HDInsight.

    Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

    Hope this helps. Do let us know if you any further queries.

    Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.