This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 50%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKN%3C/text%3E%3C/svg%3E)
Hi
I have long running task which is reading large file(3GB,5GB) excel,txt from server and dump to SQL. So I have created a Worker service and deployed that to Kubernetes Cluster and POD. I have also applied Horizontal Pod scaling but my concern is if file is half way and if POD is not able to load that and due to scaling if new POD replica is created then how I can manage the file which is already processed half way and start from their only.
Can you help me with best solution.
Thanks
Keyur
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 98%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Hello @Keyur Narkhi ,
I am trying to understand your current application design .
what exactly that excel file contains ? Is that a set of records (rows,columns)?
Does that application read each row at one time and insert into SQL?
Let's say if the excel file has got 100 records , after reading 50 records and inserting to SQL , if the POD crashes and spins up the new POD - you are asking - how the new POD can continue reading from 51st record instead of from start ?
Regards,
Shiva.
Hi Thanks for the reply.
I have bunch rows with columns in excel. I am reading the file and then using bulkinsert dumipng to database. so I might need to change and use the batch process to perform small number of records to insert and when load increase how newly created POD handle the state ? that is my concern.