Hello @Chuyu Ou ,
Welcome to the Microsoft Q&A platform.
You can create a notebook without running a cluster.
Before you can do any work in a notebook, you must first attach the notebook to a cluster.
Note: To execute any cell you need cluster is up and running.
For more details, refer Azure Databricks - Use notebooks
Hope this helps. Do let us know if you any further queries.
------------
Please don’t forget to Accept Answer
and Up-Vote
wherever the information provided helps you, this can be beneficial to other community members.