Posting this to help anyone else who comes across this.
Microsoft support have advised that this behaviour is both expected and by design. Here's a link to the official article.
I have fed back to Microsoft that i fundamentally disagree with the statements in this article that "App registration doesn't pose a security issue to customers or their data"
and that
"Copilot Studio handles the app registration behind the scenes, ensuring that the agent has the necessary credentials and permissions to interact with Omnichannel, selected channels, and skills. The customer can focus on designing and publishing the agent, without worrying about the technical details of app registration."
App registrations are a common exfiltration and persistence tactic of threat actors and it's only a matter of time before CoPilot is exploited to deliver these. It's why as an industry (Microsoft even recommends this in their Secure Score) app registrations by end users should be disabled or restricted to a subset of users. One could argue that a CoPilot Studio user is a power-user of sorts but overall this just seems to lack fine grained control that the organisation should have.
I did ask Microsoft the obvious question "is there anything we can do?". Their only solution is to remove the "Microsoft Copilot with Graph-grounded chat" license from those users.
if you're wondering what that does:
"The main areas in which Microsoft Copilot with graph-grounded chat can increase an organisation’s efficiency are:
- Email organisation – The feature uses AI to summarise email threads and suggest appropriate responses to the company’s clients.
- Creation and management of documents – The tool can generate, summarise and manage all documents created with apps included in the Microsoft365 package.
- Meeting summaries – The tool generates meeting minutes along with follow-up actions. This is available for those who are late or unable to attend the meeting, allowing them to catch up faster.
Graph-grounded chat provides contextual assistance to its users, enabling them to perform their tasks in different Microsoft 365 apps faster and more efficiently. This Copilot feature also makes creating impactful business presentations in PowerPoint or visualising data in Excel tables easier."
Basically all the stuff an organisation may buy CoPilot for.
So in short either do not buy CoPilot Studio until Microsoft have addressed these design flaws or have App registration monitoring in place so you can investigate these registrations and act accordingly post the incident and hope nothing bad happened in between