Hello JAMES, Clinton,
Thank you for posting in Q&A forum.
Did you mean you want to set "Security Group" via item-level targeting below?
If so, you can add the user objects or computer objects into this security group.
And if you link this GPO to an OU, you also need to add the user objects or computer objects in this security group to the OU.
I hope the information above is helpful.
If you have any question or concern, please feel free to let us know.
Best Regards,
Daisy Zhou