Windows Server Failover Clustering (WSFC) is different from SQL Server failover clustered instance. SQL Server runs on top of the WSFC for it to be highly available. This means you can have an extra node in a WSFC but not have SQL Server running on top of it. This also means failover of the SQL Server instance is not possible on this node even though it is part of the WSFC.
if I should add the new servers to the actual failover cluster without any SQL components installed and then run the SQL Server installer and chose add a node to existing cluster when the failover cluster is set up?
Yes. You need a maintenance window, just in case something goes wrong.
First, add the servers as a nodes in the WSFC. Next, remove the old node from the SQL Server failover clustered instance using the maintenance option to remove it. Then, add the new node to the SQL Server failover clustered instance.
Repeat until you removed all of the old nodes from the SQL Server failover clustered instance. This will require failing over to the new node.
You can end up having a 4-node WSFC with a combination of the old hardware and new hardware, with SQL Server running only on the 2 new nodes. Once you're happy with it, evict the old servers from the WSFC.
Treat this as a project. Get a project manager involved to coordinate everything between the different teams.