Is it possible to read from azure service bus Queue(or) topic from a specific offset in java ?

@Vishva In service bus once the message is received (depending the mode of receive of the message) the message will be deleted from service bus.
There is no concept of offset in service bus as we have in azure event hub. If you want mutiple consumer to read the same message then you can use topic subscription model. Individual application will read the message from their respective subscription and the message will be deleted (based on receive mode) only from that subscription.

Lets say I consume 20k messages asynchronously and my processing failed. I want my code to read the same messages again when I retrigger the job. Is this possible to do with supported java libraries ?
If your consumer application failed at 20K th message then the previous message are never available as those will be already deleted. Once your application restart the next available message will be consumed.

If you are using the legacy Java SDK you can refer Receive messages from a queue for more details what are the different mode (PeekLock and ReceiveAndDelete) and the usage.
If you are using the new azure messaging servicebus package then you can refer to this section.

Please refer to Service Bus libraries for Java for more details.