A Microsoft extension to the ANSI SQL language that includes procedural programming, local variables, and various support functions.
This is working as designed. In your query that gets the error, you give the table t the alias t2. That means everywhere in your query t will now be known as t2. So when it sees the reference to t in the WHERE clause, it no longer refers to anything, so you get the error.
The other two statements that work you rename something else as t2 (in one case the derived table (select * from t) and in the other it's the cte. So you didn't rename the table t in the delete from t, so the query still knows the reference to table t.
If you wanted two references to the table t, one called t and the other t2, you could do
delete from t from t cross join t t2 where t.i = t2.i;
Then you would have two references to t, one called t and the other t2. And that is syntactically correct.
Tom