Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,094 questions
How to connect and send and get HTTPS requests from client machine to azure_ML compute instance's localhost?
Emre Tekinalp
5
Reputation points
Hello everybody,
I'm trying to figure out and understand what the proper process is for the following setup:
I have an electron app written in node.js, js on my local machine (the client). This can be a mac or windows or linux, it does not matter.
When I launch that electron app I want to connect to an azure_ML compute instance (or clusters later but first I want to make it work with compute instances).
On this azure_ML compute instance I have already a REST API call setup which works and gives me a localhost with port 1234 for example.
Now from my client machine's electron app I want to send https requests via node.js to the azure_ML compute instance's localhost, let it process the request (GET or POST requests) and send it back once it is processed to the electron app on my local machine.
Ideally I want to send to a custom IP address only accessible by the client only, something like: https://myCustomerXYZ.OnAzure:1234 (it does not need to have the port number if I can get a custom client specific link that I can http request to as well)
I was able to create an SSH key, the public on the azure_ML compute instance and the private key on the client's local machine. BUT I was not able to send https requests to the azure machine (I used also stdin without any luck).
SO with all that said and laid out, what is the correct, secure and proper azure way I have to do to make this work? The ultimate goal is to make this work on production level for multiple clients over the world.Are there any step by step tutorials that are recommended?
I was searching and checking for couple of days multiple sources (from webapp to application gateway to api manager to ssh keys etc) and there are many ways but I'm not sure what the proper approach is? And is there someone who achieved something like this I described above?
Any help or direction or guide is much appreciated.
Thank you very much for your time,
Emre
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 11%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
Ramr-msft 14,521 Reputation points2023-08-29T01:57:54.38+00:00 @Emre Tekinalp Thanks for the question, Please try the following steps.
- Create a virtual network in Azure and connect it to your local network using a VPN or ExpressRoute. This will allow you to securely connect to the Azure resources from your local machine.
Create an Azure Machine Learning Compute instance and deploy your REST API on it. You can use Azure Machine Learning Studio or Azure Machine Learning CLI to create and deploy the instance.
Configure the Azure Machine Learning Compute instance to use HTTPS. You can use a self-signed certificate or a certificate from a trusted certificate authority.
Create an Azure Application Gateway and configure it to use the Azure Machine Learning Compute instance as a backend pool. The Application Gateway will act as a reverse proxy and allow you to access the Azure Machine Learning Compute instance from your local machine using a custom URL.
Configure the Azure Application Gateway to use HTTPS. You can use a self-signed certificate or a certificate from a trusted certificate authority.
Update your local machine's hosts file to map the custom URL to the IP address of the Azure Application Gateway.
Use Node.js to send HTTPS requests to the custom URL. You can use the
httpsmodule in Node.js to send HTTPS requests. -
Emre Tekinalp 5 Reputation points
2023-08-29T03:59:30.31+00:00 Thank you so much Ramr-msft!
You cannot imagine how much I appreciate your response! This is a great guideline! I just stumbled upon virtual networks and express Route VPNs this morning! It will be a mountain to climb since I'm still going over the fundamental courses in azure but I'm so in for the climb!
Once I have this working I will reply back to share the results and challenges along the way!Best Regards,
Emre
-
Emre Tekinalp 5 Reputation points
2023-08-30T15:31:22.28+00:00 Hello ramr-msft,
I am still in the process of setting this up and figuring out. I was able to create a p2s VPN connection to connect my local machine to the virtual network in azure. Now I'm struggling to make the next part happen:
- Configure the Azure Machine Learning Compute instance to use HTTPS. You can use a self-signed certificate or a certificate from a trusted certificate authority.
I was searching and also asking bing chat but even bing chat does not know what to do there. Can you elaborate on that part more how to configure a az ml ci to use https? What are the exact steps to make this work?
Thank you so much,
Emre
-
Ramr-msft 14,521 Reputation points
2023-09-04T03:24:07.8333333+00:00 @Emre Tekinalp Thanks for the details, Here are the Repo of samples for using the az ml cli that can help.
To use HTTPS with Azure Machine Learning CLI, you can use the following command:
az configure --defaults cli_ml_config.connection.scheme=httpsThis will set the default scheme to HTTPS for the CLI configuration. You can also set other defaults like the workspace and resource group using the
az configurecommand. For more information, please refer to this. -
Emre Tekinalp 5 Reputation points
2023-09-05T07:28:46.4333333+00:00 thank you for your response I appreciate it, I'm still stuck because I don't really understand.
I looked into the examples but don't know how to use them or do with them in my setup.
Here is my setup:I have an azure compute instance in azure ML workspace inside there I have git cloned
https://github.com/AUTOMATIC1111/stable-diffusion-webui.gitand after running a conda setup to get all the packages and install everything it sets up the FAST API which I am calling by running
source webui.shin a notebook terminal
Then this opens up a localhost:7861 port where I can access under localhost:7861/docs the FASTAPI docs and endpoints/commands.
I would be able to send curl commands inside the notebook.
NOW having said that I don't want to get into the azure portal and notebook and run curls etc instead I want to send a node.js http requests from my on prem local machine which looks like this:
// inside main.js const postData = JSON.stringify({ "prompt": prompt, "negative_prompt": negativePrompt, "alwayson_scripts": { "controlnet": { "args": controlNetUnits } }, "sampler_index": samplerIndex, "denoising_strength": denoise, "seed": seed, "batch_size": numSamples, "steps": steps, "cfg_scale": cfgScale, "width": width, "height": height, "enable_hr": enableHighres, "hr_scale": 2, "hr_upscaler": "REALESRGANBLA", "hr_sampler_name": samplerIndex, "hr_prompt": prompt, "hr_negative_prompt": negativePrompt }); const postDataBytes = encoder.encode(postData); const options = { hostname: 'localhost', port: <CALL THE LOCALHOST:7861 INSIDE AZURE ML COMPUTE INSTANCE>, path: '/sdapi/v1/txt2img', method: 'POST', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json', 'Content-Length': postDataBytes.length } }; const requestResults = await postRequest(options, postData); // Send the HTTP POST request // this is inside the postRequest function now .... const req = httpRequest(options, (res) => { let data = ''; res.on('data', (chunk) => { data += chunk; console.log(chunk); }); res.on('end', () => { console.log(`Response Status Code: ${res.statusCode}`); console.log(`Response Headers: ${JSON.stringify(res.headers)}`); console.log(`Response Body: ${data}`); try { const parsedResponse = JSON.parse(data); console.log('Response:', parsedResponse); resolve(parsedResponse); } catch (error) { console.error('Error parsing JSON response:', error); console.log('Response:', data); // Handle the error or reject the promise here } req.end(); }); // Close the connection after the response has ended req.end(); }); req.on('error', (error) => { console.error(`Error making HTTP request: ${error}`); reject(error); }); // Send the POST data req.write(postData);Now I tried to SSH into the VM machine and used ssh2 for node.js and it connects to the VM through my local machine but the server port always crashes with internal server errors after random calls which makes it super instable.
I don't understand how to make this work with the endpoints. I don't want to run a web application because it will be super expensive since we will need those azure ML compute instances or compute clusters and on top there is no selection in the web app creation for GPU machines it's only CPU which costs like hundreds of dollars per month and I want to be able to pay only when the ML compute instance is running and not 24/7.
So all I want is to send my node.js http request from my on prem laptop above with that payload to the azure ML compute instance which has already the FAST API running once I run "source webui.sh" in a ssh call.
I checked multiple times the documentations but there is no solution to my problem directly. There are online endpoints for ML compute instances but they all require an interference Model and docker container. I don't have a Model and I don't want to use docker container.
All I want is to use the compute instance in azure ML where I have already git cloned and setup a running localhost:7861 port where I want to send from my local on prem client laptop the above node.js http request call and get the generated image response back.
I tried to reach out to many forums and people but no one could really help since everything is super vague and broad.
Can you please tell me the exact steps, guides, tutorials, links, youtube videos, articles that I have to follow or look into in the correct order to make this happen?
Thank you for your time I really appreciate it.
Best Regards,
Emre
Sign in to comment