The solution is both simple and not simple.
The standard approach is to calculate the border path. Then walk the border checking each pixel (or sample) to see it contrast is required. You may want a minimum length of path failure to determine if the border is required.
This very similar to a fill, but checking just the border.