The process is pretty much the same as with on-premises, with the only difference being that you have to manually "touch" objects before they appear in newly created address lists. Refer to the articles below for more info:
https://learn.microsoft.com/en-us/exchange/address-books/address-lists/address-lists