Like you, I had no desire to see the news feed upon startup OR when I open a new tab.
First, navigate to Edge's settings using the three dots on the upper right hand side of the window
Next, navigate to "Start, home, and new tabs"
In the "Start, home, and new tabs" section, I personally have it set to a couple sites/tools I use regularly for work, but you can set it to "Open the new tab page" which I'll get to below.
Edge handles new tabs separately and you are required to "personalize" the new tab layout. To do this, while still on the "Start, home, and new tabs" settings page, scroll down to "New tab page" then click "Customize"
Now, on the top right, click the cog icon and select "Custom"
Here, you can remove all "features" and content
Now, your new tabs should open up to something that resembles this, and if you set your startup page to "Open the new tab page" it should show this upon startup as well. :)
Hope this helps everyone!