I want to provide an answer to my own question.
I went into 'Resource Groups' and found that a Resource Group had been created based on the name of the static web app (so the app was called 'foo' and the newly created Resource Group was called 'foo-rg') so this is good because I can now remove the web app.
On further investigation the 'foo-rg' appeared in the swa-cli.config.json. When I changed that to the resource-group I wanted to use and repeated the deployment everything was as I expected it to be.
Hope this is useful to somebody.