Error using model.fit() in Synapse Apache Spark notebook

Abdelmonim Klaii 20

I have an unpredictable error.

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (String)""; line: 1, column: 0]

I would like to train a time series model using SARIMAX with four exogenous variables.

When I use model.fit(), this error appears.

model=sm.tsa.statespace.SARIMAX(train['PUISSANCE'],order=(5,1,5),exog=train[['JOUR_SEMAINE', 'Jour_F', 'HEURE_CREUSE', 'HEURE_POINTE']])
results = model.fit()
results.summary()

Please let me know what might be wrong.

PRADEEPCHEEKATLA 90,246 Reputation points

2023-07-18T06:11:10.24+00:00
@Abdelmonim Klaii - Thanks for the question and using MS Q&A platform.

The error message you provided suggests that there is no content to map due to end-of-input. This error can occur when the input data is empty or when there is a problem with the input data format.

In your case, it is possible that the train dataframe is empty or that there is a problem with the format of the train dataframe. To verify if the train dataframe is empty, you can try printing it using the show() method:

train.show()

If the train dataframe is not empty, you can try converting it to a Pandas dataframe before passing it to the SARIMAX model:

train_pd = train.toPandas() model=sm.tsa.statespace.SARIMAX(train_pd['PUISSANCE'],order=(5,1,5),exog=train_pd[['JOUR_SEMAINE', 'Jour_F', 'HEURE_CREUSE', 'HEURE_POINTE']]) results = model.fit() results.summary()

If the problem persists, you can try checking the data types of the columns in the train dataframe to make sure they are compatible with the SARIMAX model. You can use the printSchema() method to check the data types:

train.printSchema()

I hope this helps! Let me know if you have any further questions.
Abdelmonim Klaii 20 Reputation points

2023-07-18T10:25:52.2033333+00:00

Thank you @PRADEEPCHEEKATLA for your support,

1/ I attempted to use train.head() and verified that the train dataframe is not empty.

2/ The train table is a dataframe.

3/ The columns in the dataframe consist of integer and float data types, which are compatible with the SARIMAX model.(The same code work perfectly in jupyter anaconda).

Note please once this error occurs, it is systematically repeated every time code is executed, even for simple instructions such as print('hello world').
PRADEEPCHEEKATLA 90,246 Reputation points

2023-07-24T08:52:06.59+00:00

@Abdelmonim Klaii - I agree that this issue looks strange and I wasn't able to reproduce this issue. If you have a support plan could you please file a support ticket for deeper investigation and do share the SR# with us?
PRADEEPCHEEKATLA 90,246 Reputation points

2023-07-27T06:27:34.5433333+00:00

@Abdelmonim Klaii - Did you get a chance to open a support ticket?
Ryan Schaub 0 Reputation points

2024-02-01T17:41:46.0866667+00:00

@Abdelmonim Klaii Were there any follow-ups or resolutions to this issue? I am having the same exact issue using tsa's SARIMAX.
Sebastian Bönig 10 Reputation points

2024-06-07T11:49:02.1733333+00:00

Any news on that topic? The problem still presits.

3 answers

Sebastian Bönig 10 Reputation points

2024-06-10T14:11:00.5933333+00:00
I found that using the parameter disp=False worked for me on a Microsoft Fabric notebook

model.fit(disp=False)
Please sign in to rate this answer.

2 people found this answer helpful.
Alexey Kazakov 0 Reputation points

2024-11-16T14:00:17.0766667+00:00

Yes, that worked
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Venkateswara Rao Sanaka 0 Reputation points Microsoft Employee

2024-03-27T14:20:47.92+00:00

On what version of Spark 3.3 are you facing the issue?

Just run the below command, and share the output

print(sc.version)

I can't reproduce the issue on 3.3.1.5.2-117504963
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Venkateswara Rao Sanaka 0 Reputation points Microsoft Employee

2024-03-27T14:24:33.61+00:00

If you are getting the below exception, please rerun the cell until it succeedes.

com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (String)""; line: 1, column: 0]
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Error using model.fit() in Synapse Apache Spark notebook

3 answers

Your answer