I'm trying to process an excel file containing roughly 500k rows of data and converting it into a csv. To do this, I've used ExcelConnector to get the content of the document and have used the Create CSV Table function in putting the data into a csv file. I've enabled pagination and have set the max items setting to 100k but after running the pipeline, 3 csv files were generated, each containing around 100k rows of data. So, more than 100k rows of data are missing.
Can the number of pages be configured? If yes, how to do it? And, am I missing a step here?
Thanks in advance for the response/ideas.