Max pages best practice

Question

Is there a best practice or suggested way for handling large documents (> 2000 page limit)? I’d like to handle these documents in batches to avoid data loss. I know that there is a page range parameter but without knowing the total number of pages ahead of time I can’t take advantage of it as reliably.

Answer 1

Hi there Seth Fenster

Thats a good question and thanks fore using QandA [platform

sure, one possible workaround is to implement a batch processing mechanism that dynamically adjusts the batch size based on the document's size.

Begin by dividing the large document into smaller, manageable batches.

Process each batch sequentially, monitoring the progress and handling any errors or exceptions encountered along the way.

Dynamically adjust the batch size based on the processing performance and resource availability. For example, if processing a batch takes longer than expected or consumes excessive resources, consider reducing the batch size to improve efficiency.

Implement t error handling mechanisms to handle any failures gracefully.

Ensure that processed data is persisted securely, either locally or in a reliable storage solution, to prevent data loss in case of failures or interruptions.

If this helps kindly accept the answer thanks much.