There are size and time limits (httpRequestTimeout) as to what Data Factory will accept for any response, regardless of how many records are returned. The first thing to keep in mind is if you have a very large API response, ADF may not read in all of it.
The built-in paging support has some caveats, in that the API you're calling must emit which are detailed at:
https://learn.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support
There are some solutions in another thread - https://social.msdn.microsoft.com/Forums/en-US/ab9faec5-8e08-467c-86b4-0072b871985b/unable-to-use-the-pagination-rules-in-rest-sources-in-adf?forum=AzureDataFactory
But these may not work for you depending on the format of the API you're calling