Quarterly data load in SSIS package

Question

I just started using SSIS tool and i need quick help to load data quarterly

Here`s my scenario:
I came up with a query ( source Database: DB2 ) which will extract data from 2010-01-01 to 2021-12-31,(12 years of data) however the data volume is too high ( around 300 M), so i would like to split the data source query to load data into quarter wise.
I tried year wise and still i am getting more volume of data which my SSIS server is not able to handle.
I have created a year loop to loop it through, in that created a script task into it followed by dataflow task.
for example,
select * from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-12-31'

This i would like to loop it as, ( 4 times load, 1 for each quester)

select * from tab1 where start_date >= '2010-01-01' and end_Date <= '2010-03-31'
select * from tab1 where start_date >= '2010-04-01' and end_Date <= '2010-06-30'
select * from tab1 where start_date >= '2010-07-01' and end_Date <= '2010-09-30'
select * from tab1 where start_date >= '2010-10-01' and end_Date <= '2010-12-31'

year wise perfectly working fine, however i am not getting any idea how do i need to load the data into quarter wise.
Any help is greatly appriciated.

Answer 1

Hi @Woodcook Dominick ,

The Conditional Split can route data rows to different outputs depending on whatever criteria of the data that you wish.

It works rather like the SWITCH block in a procedural language.

Here is a blog you may take a reference to.

ssis-basics-using-the-conditional-split

In addition, you may also use OFFSET FETCH feature as a solution to load large volumes of data.

See here: sql-offset-fetch-feature-loading-large-volumes-of-data-using-limited-resources-with-ssis

Regards，

Zoe

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.