To use the Microsoft Graph PowerShell module, you don't necessarily need to register a new Azure App. However, the specifics depend on what you're trying to achieve:
If you just install the Microsoft Graph PowerShell module and use the Connect-MgGraph cmdlet, it will prompt you for login. By default, it uses a public client (an already registered app by Microsoft) for the authentication.
This is great for quick tasks.
If you need more control, like specific permissions, application permissions, or using client credentials, you'll want to register an Azure App.
Once registered, you can grant the necessary permissions to the app in Azure AD and then use the Connect-MgGraph with the -ClientId, -TenantId, and possibly -ClientSecret or -CertificateThumbprint parameters.
This approach is more suitable for production scripts, automation, or when you need specific permissions that the default experience doesn't provide.