Share via

Issue with Data Load for oracle source using Azure Data Factory

Anmol Ganju 176 Reputation points
2023-12-26T07:56:56.8633333+00:00

Hi,

Getting below issue while loading the data from oracle source does anyone have encountered this before:
User's image

Qurey prepared for data load:

SELECT LOT_NUMBER,LOT_NAME,CHANGED_BY,CHANGED_ON,DESCRIPTION,GROUP_NAME,DATE_CREATED,PRODUCT,PRODUCT_GRADE,PRODUCT_VERSION,CLOSED,LOT_COMMENT,SUPPLIER,EXT_LINK,REPORT_DATE,CUSTOMER,REPORT_NUMBER,REPRINT_FLAG,REPRINT_DATE,PRODUCTION_DATE,EXPIRY_DATE,QUANTITY,ORDER_NUMBER,ORIGINAL_LOT,PARENT_LOT,SAMPLING_POINT,FULL_TESTING_REQD,IN_SPEC,PRIMARY_IN_SPEC,TEMPLATE,SAMPLE_PLAN,X_COMP_TRGT_WGT_1,X_COMP_TRGT_WGT_2,X_COMP_TRGT_WGT_3,X_COMP_TRGT_WGT_4,X_COMP_TRGT_WGT_5,X_LOT_NAME,X_VAL_LOT,X_ALT_TEMPLATE,IS_TIME_ZERO,X_BATCH_PURPOSE,X_CANCELED,X_DATE_RECEIVED,X_EXT_COA_DATA,X_FORCE_FULL_TESTING,X_INCLUDE_IN_TREND,X_INTERFACE_LOGIN,X_ITEM_CODE,X_NUM_CONTAINERS,X_RECONTROL,X_RELEASE_DATE,X_RETEST_DATE,X_SAMPLE_PLAN,X_SPEC_CLASS,X_SPEC_CODE,X_SUPPLIER_LOT,X_SUPP_EXP_DATE,X_TEST_STATUS,X_UNIT,X_VENDOR,X_PLANT,X_IS_RELEASE_LOT,FULL_CHANGED_BY,SIGNED,X_HISTORICAL,X_INV_REF,X_FP_LOT,X_BULK_ID,X_PROD_SPEC_REF,X_REVIEWED,X_LOCATION,X_PROD_FAMILY,X_CLEAN_EVENT,NULL AS STR_MFR,X_PROD_SPEC_REF FROM LABOWN.LOT WHERE CHANGED_ON>= TO_DATE('2023-12-20T14:22:25','YYYY-MM-DD"T"HH24:MI:SS"Z"') AND CHANGED_ON<= TO_DATE('2023-12-23T08:50:16','YYYY-MM-DD"T"HH24:MI:SS"Z"') AND X_PLANT IN ('1488')

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,656 questions
Accepted answer
  1. Amira Bedhiafi 34,491 Reputation points Volunteer Moderator
    2023-12-26T12:05:52.7733333+00:00

    You can check this link https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-parquet

    Your issue is occuring during the creation or manipulation of Parquet files.

    The error message indicates that a field (presumably in your Oracle source or the Parquet file) has more than one value where only one was expected. Verify the data schema in both the Oracle database and the Parquet file format to ensure they align correctly.

    Also, review your ADF configuration for the copy activity. Ensure that you're using the correct linked services, datasets, and that any mapping or schema-related settings are correct.

    1 person found this answer helpful.
    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Aswin 472 Reputation points
    2023-12-26T11:03:28.6633333+00:00

    An error occurred when invoking java, message: java.lang.IllegalStateException:field 74 (X_PROD_SPEC_REF) can not have more than one value: [1]\ntotal entry:6

    • Based on the error message you provided, it seems there is an issue with the data in the X_PROD_SPEC_REF field. There are duplicate column names 'X_PROD_SPEC_REF' in the structure. You cannot add same key twice.
    • To solve this, remove the duplicate field in the query and execute the pipeline.

    Image of the query used in copy activity with duplicate fields:

    image of query with duplicate fields

    1 person found this answer helpful.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.