Hey there Handian Sudianto
Thats a good question and thanks for using QandA platform
I guess yes, there can be downtime, but it depends on the specific changes you're making and your configs.
Usually increasing the DTU generally doesn't require downtime because it's a measure of the performance level, and Azure SQL Database can dynamically adjust resources based on demand. even if there is a downtime it will be very breif.
On the other hand, increasing the Data Max Size, can potentially cause downtime if the new size exceeds the current provisioned capacity. This is because Azure SQL Database might need to perform storage operations to accommodate the increased size,
Follow this upgrade during an non peak time and I suggest you to do that, to experience lesser downtime.
https://learn.microsoft.com/en-us/azure/azure-sql/database/elastic-pool-overview?view=azuresql
If this helps kindly accept the answer thanks much.