Hi Chris
I do not recognize that specific on-screen indicator . . .
Click your Start Button, then just type mobility
Open the Mobility Center from the resulting list, is there an option to turn off the Caps Lock indicator there?
If not, is there specific software installed on your PC for your keyboard, if so, open that and work through the settings to see if there is an option to turn off that on-screen indicator . . .