I know this is an old post, but I was having the exact same issue, but for some reason after reading YuZhou-MSFT's first reply it dawned on me that the problem might be with my mouse settings.
So I checked my Logitech settings, turned off smooth scrolling there, and now my scrolling doesn't have that annoying slow-down anymore!
Just a crisp, clean stop!