load data into SQL Server via Pyspark.
Hi everyone, I don't know what else to do. I tested it with several cluster versions. Need help. thanks. Error return in the description below: org.apache.spark.SparkException: Job aborted due to stage failure: Task 31 in stage 1.0 failed 4 times,…
Notebook execution is in Failed state in SynapseNotebook
When Synapse pipeline fails because of Notebook, this error can be seen in Monitor tab of Synapse Studio. However, in log analytics it doesn't show the error. Is there a way to get error details form Log analytics? LogAnalytics error: …
Microsoft.Data.Analysis DataFrame: Is there a best practice for normalizing the length of columns?
Hi, I'm building a DataFrame with DataFrameColumns that have various lengths. Is there a best practice/idiom to normalize the length of all columns before building the DataFrame? Chris
Using Sql Command in C# Visual Studio
Why does the SQL Command not Delete the Data from the Database, in the properties i have already set the sql command text on the Delete Comand. Is the sql Command used rightly? public Form1() { InitializeComponent(); sqlConnection1.Open(); } …
ML.net Model Builder onnx-Model input preprocessing
Hi. After I've trained an object detection model with Model Builder through Azure; is there a way to find out which preprocessing (like scaling the input image to 600x800) needs to be performed when the onnx-model is used for inference (like in a c++…
Deploy ASP.NET Core Web API with Simba Spark Driver for Databricks to Azure App Serivce
Hi all, I have a .NET Core Web API which creates a connection to the SQL endpoint of Azure Databricks with the official Simba ODBC driver. Locally everything works fine. I can see my System DSN entry and I use this driver to create the connection and…
Microsoft.ML and Xamarin Forms (Android)
Hello! I'm trying to use Microsoft.ML on Xamarin Forms (Android) application. It compiles well and runs on x64 platform, but when I deploy it on a device (Arm64) and it tries to call Fit() of IEstimator<ITransformer> interface, it crashes with…
Hadoop and Big Data
I recently started studying Big Data in my university. But i am having troubles in understanding how is Hadoop and Big Data related? Anyone can explain this?
Compiler error "consume excess memory due to the indexing expression"
Hi, I am trying to construct a Bayesian network that contains 8 parents(which are iKorean, iOthrserv, iMay, dYrsserv, iSept80, iVietnam and iFeb55) pointing to 1 child(which is iRvetserv), and while compiling, I got the error like this: " [1] This…
Training Bayesian network with Infer.NET but doesn't know any prior and conditional probability
Hi, I am new to Infer.NET. I know this question might be asked a few years ago but some posts' links were broken so I just ask here again. Let assume that if I have the whole dataset for training and I would like to construct a Bayesian network in…
Give hints to train a category model in ML.NET?
I have quite good data for training a model to predict a category label based on many features. But it's it possible, or even necessary, to "hint" or give a list of categories for each record that I know for sure it can NOT be? Would it be…
Max of Column B based on Column A
I have a table with company names and their IDs. Some companies could have multiple IDs, so I want to create another table with the company names and maximum of the IDs (basically the latest created ID). I have been able to get the company names in a…
how i cane solve these error one report in porter appear these We couldn't connect to the Analysis Services server.
how i cane solve these error one report in porter appear these We couldn't connect to the Analysis Services server. Make sure you've entered the connection string correctly. Request IDab071746-2b75-80f4-4fb7-0177c5f4369b TimeTue Nov 30 2021…
ML.NET unsupervised learning for image classification
I want to know if it is possible to do image classification using ML.NET. If possible, I want to see an example. I am making a WindowsForm application in which you can specify the path to the folder with images. My task is to classify this set of…
How to get prediction interval from ML.NET model builder
I am trying to predict Candidate joining the company based on parameters like Location, Skills and Designation , I am using ML.NET model builder to build a model and predict number of days required by candidate to join the company , I want to get the…
Azure synapse Notebook error creating session Apache spark pool
I experiencing that error starting a session on notebook There is no running application on that pool. The day before all worked correctly. No changes has been performed on permission or spark pool I've also tried to remove and upload…
How to reduce interaction for training model?
After reading this topic https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/load-data-model-builder it seems to me that training a model is a rather manual task and I'm not talking about data preparing. You have to manually select…
Saving Spark SQL result in a new dataframe
Hey, A simple question. Wondering how to save query results into a dataframe from magic command in Spark notebook in Synapse. This allows me to save query result directly into a new data frame. dataframe = spark.sql("<query>") However…
ML.NET ImageClassification with GPU
I 'll get a very low accuracy(about 3%) This is not the case with CPU(about 88%) with RTX 3070ti What may be the reason for this?
Query Timeout on Azure Synapse serverless pool
Hey microsoft , Is there any default query timeout on serverless pool ? when im using serverless pool view as my source to powerbi , some time im not getting any error , and some times im getting timeout error at source side . i didnt figure out the…