question

alexandertikhomirov-7179 avatar image
0 Votes"
alexandertikhomirov-7179 asked jeffhebrank commented

error using concurrent.futures (ThreadPoolExecutor) in Synapse Apache Spark notebook

I have an unpredictable error

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
at [Source: (String)""; line: 1, column: 0]

using concurrent.futures.ThreadPoolExecutor approach to parallel tasks in my python code for apache spark notebook

for example using simple example. but sometimes it works.

167097-image.png


 import concurrent.futures
 import time
    
 def subjob(i):
     print(i)
    
    
 def main(p):
     with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
         futures = {executor.submit(subjob, i): i for i in p}
         for future in concurrent.futures.as_completed(futures):
             print("outer job done\n")
    
 if __name__ == "__main__":
     p = [1,2]
     main(p)


azure-synapse-analytics
image.png (32.4 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered jeffhebrank commented

Hi @alexandertikhomirov-7179,

Thank you for posting query in Microsoft Q&A Platform.

I used same code which shared you. In my case, I am not seeing errors. Please check below screenshot.
167835-image.png

Please check my Spark pool configurations.
167864-image.png

Could you please cross verify your Spark Pool configurations with above and see if that helps?

Hope this helps. Please let us know if any further queries.


Please consider hitting Accept Answer. Accepted answers helps community as well.


image.png (21.2 KiB)
image.png (24.6 KiB)
· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I have the same config
167874-image.png

and during first run I have again this error
167933-image.png
when clicking again a few times error disappeared, but no output
167853-image.png

only running [25] time I have an expected output result
167854-image.png


0 Votes 0 ·
image.png (30.1 KiB)
image.png (12.5 KiB)
image.png (6.3 KiB)
image.png (6.7 KiB)
ShaikMaheer-MSFT avatar image ShaikMaheer-MSFT alexandertikhomirov-7179 ·

Hi @alexandertikhomirov-7179,

Yes. Looks strange behavior to me. I escalated this internally to see if any insights I can get. Will share updates once I here back. Thank you.


Please consider hitting Accept Answer. Accepted answers helps community as well.

0 Votes 0 ·

The problem was with using print statement with multithreading.

answer from MS support

Issue:
You are having the below error:

• com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (String)''; line: 1, column: 0]

Cause:
Caused by python interpreter in Livy. For original python interpret, Livy uses "exec" to run code, and get output from redirected stderr and stdout. Then we would meet such issue(depends on which is finished first: pool shutdown and stdout flush).

For Ipython interpreter, we replace plain python interpreter with ipython, and all output are bounded to the corresponding statement, so ipython should help in this case.

So, this is caused by default Python Interpretor used and IPython interpreter should solve this issue.

Solution:
We are working to fix this issue, if everything goes as expected there is a new Azure Synapse deployment for Ipython start going to production after mid-February region by region, this should fix the issue when enabling the IPython feature.


0 Votes 0 ·
ShaikMaheer-MSFT avatar image ShaikMaheer-MSFT alexandertikhomirov-7179 ·

Awesome. Thank you for sharing details here.

0 Votes 0 ·

Hi @alexandertikhomirov-7179,

Could you please consider marking it as accepted answer? Accepted answers helps community as well. Thank you.

0 Votes 0 ·
jeffhebrank avatar image jeffhebrank alexandertikhomirov-7179 ·

Can you comment on the status of this fix? It says Mid-February the fix would start rolling out, however I have started seeing this error much more frequently in my own synapse workspace over the past month.

0 Votes 0 ·