I have one doubt though. I am trying to do this in Azure Synapse. Is that the problem?
Yes. Azure Synapse supports most T-SQL types but there are a few exceptions. Below is an excerpt from the Query Language documentation that explains why the rowversion (a.k.a. timestamp) is not recognized (added emphasis in bold):
all Transact-SQL types except cursor, hierarchyid, ntext, text, and image, rowversion, Spatial Types, sql_variant, and xml
You can use timestamp as a column name but with an alternate data type (e.g. binary(8)). The implication is data loaded as part of an ETL process will need to be converted for ingestion and. of course, the value will not automatically change with row updates like rowversion type values do in SQL Server.