A cloud-based service included in Microsoft 365, delivering scalable messaging and collaboration features with simplified management and automatic updates.
If the goal is to remove the object completely, then yes, delete from AD and it will be removed from Exchange as well.
$Rooms = import-csv rooms.csv
$Rooms | % {Remove-RemoteMailbox -Identity $_.Names}
Test with a few first, before running against the entire list
https://learn.microsoft.com/en-us/powershell/module/exchange/remove-remotemailbox?view=exchange-ps