Jupyter Notebook not running as fast as expected?

Question

Hi,

I have recently started using Azure because I need to run a very demanding piece of code on Jupyter Notebook. I am attempting to run the notebooks by creating a workspace via Azure Machine Learning

I have obtained quotas for High Performance Computing but still, the time required to run my code is the same as it is with my laptop. My laptop isn't the greatest so I suspect there's either something wrong with my connectivity or the way I set up the notebook. Is it enough to just connect to a compute instance or do I need to set up an environment at the very top?

EDIT: The device I am using is a Standard_HB60rs with 60 cores, 223.52GB RAM and 700GB Storage which costs $2.28/hour. I am importing a package that is used for simulations of quantum mechanical systems called qutip.

Installing Qutip

conda install numpy scipy cython matplotlib nose jupyter notebook spyder

conda config --append channels conda-forge

conda install qutip

The installation seems to be working as I have been able to run the next cells which use the package.

Cell 1 where I import all necessary packages

%matplotlib notebook

import copy
import numpy as np
import math
import matplotlib.pyplot as plt
from qutip.qip.noise import RandomNoise

pi = np.pi

from qutip.qip.device import Processor
from qutip.operators import sigmaz, sigmay, sigmax, destroy
from qutip.states import basis
from qutip.metrics import fidelity
from qutip.qip.operations import rx, ry, rz, hadamard_transform

hbar = 1.0545718 * (10**-34) #in Js
h = hbar * 2 * np.pi #in Js
eV = 1.60217662 * (10**-19)
h_eV = 6.58 * (10**-22)*(10**6) #h in eVs

Cell 2

processor = Processor(N=1) #this command builds a 2-level quantum mechanical system

parameters for some of the functions that will be used

Delta = (2*(109)) * (h) #in Joules
Delta_2 = ((Delta/(h)) * 2 * pi)/(1013) #in Hz units

processor.add_control((0.5*Delta_2*sigmax()), targets=0, label="sigmaz") #Hamiltonian of the system

Cell 3. Here, I drive my system from its ground state to a superposed state.

w_min = (pi/2/Delta_2)*(10**-2)
w_max = (pi/2/Delta_2)
step = 0.01

tpoints = np.linspace(w_min,w_max,123)

basis0 = basis(2,0)

final_states = []

for t in tpoints:
tlist = np.linspace(0,t,1000)
processor.pulses[0].coeff = np.array([1 for t in tlist])
processor.pulses[0].tlist = tlist
result = processor.run_state(init_state=basis0)
final_states.append(result.states[-1])

The Physics of the matter isn't the matter here. I know the code does what it is supposed to do; the remaining of the code includes various loops which do similar computations. The code in #cell 3 would usually take between 10-20 seconds to run on my laptop. I was surprised to see that it would take the same time on Azure.

Hope this is a bit more clear

Thank you!

Answer 1

Thanks for sharing detailed information. When using Azure ML Notebooks, you simply need to add compute instance, and depending on your workload you can request quota increase. However, various factors can affect performance. For example, writing large files of data on the OS disk of the compute instance or installing certain extensions can cause slow down performance. However, If you continue to experience unusually slow performance, I encourage you to contact Azure support for further analysis on the issue. Hope this helps.