OK, use the NA() function instead of the " " in your formula. Once you see your table filling up with the ugly #N/A values, select all cells in the table, then on the Home ribbon click Conditional Formatting > new Rule > use a formula to determine ...
In the formula box enter
=isna(A1)
Replace A1 with the top left cell of your selection and make sure that there are no $ signs in the cell reference. Then click the Format button and set the font color equal to the cell fill color, e.g. if the cells are white, select a white font.
That will make the #N/A values invisible to your users, but the cells are suitable for a XY chart.
Let me know how you get on.