Synapse : Query spark table with sql serverless return "Invalid object name"

David Simon 1

Hello,

I don't now why when I try to build spark table which contain decimal(32,16) the table is created but unqueryable.
When I remove this field, the rest of the other fields are string and now I can query the table into the SQL edior.

Can you explain me why I have this problem ?

PRADEEPCHEEKATLA-MSFT 85,586 Reputation points Microsoft Employee

2021-08-24T05:43:06.187+00:00

Hello @David Simon ,

Welcome to the Microsoft Q&A platform.

To understand your query, could you please share the code snippet and the screenshot of the query?
Ronen Ariely 15,186 Reputation points

2021-08-24T05:56:20.553+00:00

In addition to the information which PRADEEPCHEEKATLA-MSFT ask, please provide the queries to create the table with that decimal column

Thanks

1 answer

David Simon 1

Hey this is the code

Ronen Ariely 15,186 Reputation points

2021-08-25T08:37:00.57+00:00

Hello,

The request was to share code which we can use to reproduce the issue and not images which are useless

David Simon 1

file_name = ".parquet"
entity_name ="fact_finance_budget"
path_raw =""
path_raw_file ="{}/{}".format(path_raw,file_name)
path_ods =""
path_ods_file ="{}/{}".format(path_ods,entity_name)
ods_database_name = "ods"
file_format_ods ="PARQUET"

spark.read.parquet(path_raw_file)\
    .createOrReplaceTempView("temp_dataset")

%%sql  
Create or REPLACE TEMPORARY VIEW temp_dataset_view as 
SELECT
    uuid() ods_id,
    AccountNumber_BK,
    Activity_BK,
    cast( AmountCHF as decimal(12,2)) as  AmountCHF,
    -- cast( AmountCHF as varchar(100)) as  AmountCHF,
    Company_BK,
    CostCenter_BK,
    Date_BK,
    DocumentDescription,
    DocumentNumber_BK,
    Project_BK,
    Specificity_BK
from temp_dataset

spark.read.table("temp_dataset_view")\
    .write\
    .format("parquet")\
    .mode("overwrite")\
    .saveAsTable("{}.{}".format(ods_database_name,entity_name))

David Simon 1 Reputation point

2021-08-25T11:00:20.433+00:00

But with out data how can you test ?
PRADEEPCHEEKATLA-MSFT 85,586 Reputation points Microsoft Employee

2021-08-26T07:57:23.63+00:00

Hello @David Simon ,

Could you please share the screenshot of the query spark table with sql serverless return "Invalid object name"?

Share via

Synapse : Query spark table with sql serverless return "Invalid object name"

1 answer