gpedit.msc is contained under windows\system32 and from memory it only comes with Vista Business they don't think Vista Home or Premium users need gpedit.
and about deleting you simply right click and delete it or just left click and use the Delete key on the keyboard not the backspace key. double clicking the key simply allows you to edit the value contained by the key