Correct, delegates define a contract (e.g. a function that accepts an int and returns a string). The function implementation is interchangeable. In your example a method/class that executes a delegate provided by consumer code is defining the contract, and it's the consumers responsibility to provide an implementation.
This means that the code entity that our library author has made is "Open to extension, since we're extending the library behaviour with no need to change the library implementation, as it's Closed to modification.
This principle can be applied to many places where a contract can be defined, such as the contract provided by an interface or base class.
You can imagine that designing an entire system by a loosely coupled set of contracts would make individual code entities much easier to change the implementation of without breaking the contract. It can also hinder readability having too many abstractions, so there's definitely some discretion required.