"I have checked the column and all it has in it is nulls or integers."
You might have checked the data visually, in that case you have garbage in the data which doesn't show up visually.
The trick is to find those rows, and those rows only. Below show you how you can do that. In your WHERE clause, use IS NOT NULL so you don't return the rows which are NULL and also use TRY_CAST to find those that aren't convertible to int:
DROP TABLE IF EXISTS #t
CREATE TABLE #t(c1 int identity, c2 nvarchar(10))
INSERT INTO #t(c2) VALUES ('1')
INSERT INTO #t(c2) VALUES ('123')
INSERT INTO #t(c2) VALUES (NULL)
INSERT INTO #t(c2) VALUES ('12' + CHAR(9)) --Note that CHAR(9) is TAB
--It looks like data is fine!
SELECT * FROM #t
--But conversion fails because of TAB
SELECT c1, CAST(c2 AS int)
FROM #t
--How to find the failed rows, and those only?
SELECT c1, c2
FROM #t
WHERE c2 IS NOT NULL
AND TRY_CAST(c2 AS int) IS NULL