Issue loading 2 tables out of 40

Astrid o 1 Reputation point
2020-12-07T16:01:00.247+00:00

Hi,

I have a dynamics SSIS package, that gets a list of tables and procs from another table on the database and check the size of each table to see what tables to load first.

45737-ssis1.png

my problem is that out of the 40+ tables, it doesn't load 2 big tables. The biggest table contains 187,511,514 rows and the other table has 21,444,505 rows (basically it doesn't load the two biggest tables.

I imagine that has to do with the buffer or something, any ideas on what I can do?

SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
2,509 questions
{count} votes

3 answers

Sort by: Most helpful
  1. Monalv-MSFT 5,896 Reputation points
    2020-12-08T03:27:41.877+00:00

    Hi @Astrid o ,

    my problem is that out of the 40+ tables, it doesn't load 2 big tables. The biggest table contains 187,511,514 rows and the other table has 21,444,505 rows (basically it doesn't load the two biggest tables.

    May I know which were the largest rows that you can load the tables?

    We can use Conditional Split Transformation in Data Flow Task to divide these two big tables into several small tables with proper rows.

    Best Regards,
    Mona

    ----------

    If the answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
    Hot issues in November--What can I do if my transaction log is full?
    Hot issues in November--How to convert Profiler trace into a SQL Server table?


  2. Tom Phillips 17,721 Reputation points
    2021-01-12T15:57:58.82+00:00

    Most likely you have a logic problem in your code. Since you are using tons of conditions, it is impossible to guess. It is likely not skipping the big tables. You are using %5 to split your list and not accounting for something.

    The first thing to do is add some logging of what exactly each "Package*" is processing. That will give you a clue as to if the tables are even reaching that point.

    0 comments No comments

  3. Astrid o 1 Reputation point
    2021-01-29T13:52:22.807+00:00

    I changed the expression task for a sql task and didnt have any more issue.

    0 comments No comments