What you have would work fine in regular SQL Server, thanks to something known as ownership chaining. That is, when a module accesses an object that has the same owner as the module, there is no permission check at all. However, it appears that this is not implemented in Synapse Dedicated Pool. I also found an old thread which suggested that this may come in the future: https://learn.microsoft.com/en-us/answers/questions/528220/idea-suggestion-is-it-possible-to-support-ownershi. But that was three years ago...
Unfortunately, I cannot think of a solution that would meet your requirements. (But I should add that I normally do not work with Synapse, but I hang around with the regular SQL Server.)
You could submit a suggestion for this feature on https://feedback.azure.com/d365community/forum/9b9ba8e4-0825-ec11-b6e6-000d3a4f07b8
Here is an example of ownership chaining at work. The below runs without error on Azure SQL Database, but fails with a permission error on Synapse Dedicated Pool:
CREATE TABLE testtable(a int NOT NULL)
go
CREATE PROCEDURE test_sp @a int AS
INSERT testtable(a) VALUES(@a)
go
CREATE USER nisse WITHOUT LOGIN
GRANT EXECUTE ON test_sp TO nisse
go
EXECUTE AS USER = 'nisse'
go
EXEC test_sp 122
go
REVERT
go
SELECT a FROM testtable
go
DROP PROCEDURE test_sp
go
DROP TABLE testtable
go
DROP USER nisse