Azure ML v2 mode=upload output to an ADLS Gen1 datastore fails with "MSCONCAT 0x83090a70 (gen1-to-gen2 symlink)" for files larger than ~8 MB

Question

I have an Azure Machine Learning v2 pipeline (SDK azure-ai-ml) whose step writes an output file to an ADLS Gen1 datastore (datastore type AzureDataLakeGen1). The output is declared as:

Output(

My training script just writes the file locally to the managed-output mount; Azure ML's runtime then uploads it to the datastore.

Symptom: a small output (~7.8 MB) uploads fine, but a larger output (~8.3 MB and up) fails during the upload finalize with:

Error Code: ScriptExecution.WriteStreams.Unexpected
Native Error: An error occurred when streaming data to output target.
azure_adls_gen1 service: IllegalArgumentException:
  "MSCONCAT failed with error 0x83090a70 (Bad Request. The specified operation is not valid
   for gen1-to-gen2 symlink.)"

The target folder was migrated from ADLS Gen1 to Gen2 and is exposed as a gen1-to-gen2 symlink. The storage service rejects the WebHDFS op=MSCONCAT call on such paths. The upload is done by the Azure ML common runtime (rslex), not by my code.

What I've confirmed

It is not a permissions problem: the same job, using the same datastore identity, successfully writes the smaller file to the same folder. Only the larger file fails.

The difference appears to be a single-block upload threshold (~8 MiB). Below it, rslex writes the file in one stream (I see a "custom writer" copy with no concat) and it succeeds. Above it, rslex splits the file into blocks and finalizes with op=MSCONCAT, which the migrated-symlink path rejects.

Writing the same large payload to a freshly created (non-migrated) folder succeeds — only the migrated gen1→gen2 symlink folders reject MSCONCAT.

What I've tried (no luck)

mode=rw_mount instead of mode=upload — the FUSE flush also finalizes with MSCONCAT and fails the same way (surfaces as OSError [Errno 5]).

Forcing rslex to use its sequential (non-parallel) ADLS Gen1 copier via the env var DATASET_ADLSGEN1_USE_ALTERNATIVE_COPIER=false:

Setting it in the environment image had no effect — the uploader still logged alternative=true (the uploader runs in a separate common-runtime image and doesn't inherit the training image's env vars).

  Setting it via the job's **`environment_variables`** *did* reach the uploader (it logged `alternative=false`), **but the write still failed with the same MSCONCAT error** — so the sequential copier *also* uses MSCONCAT for multi-block uploads.
  

So both copier paths issue MSCONCAT once the file spans multiple blocks, and I can't shrink the file below the threshold.

Questions

Is there a supported way to make the managed ADLS Gen1 upload avoid the MSCONCAT finalize — i.e. write the file as a single stream / append-only — for files above the single-block threshold? (The smaller file already succeeds this way.)

Is the single-block / single-PUT size threshold configurable (e.g. an azureml-dataprep-rslex setting such as a stream-copier block size) so that a ~10–50 MB file is written as one block and no concat is issued? If so, can it be passed via the job's environment_variables?

Is MSCONCAT-on-a-gen1→gen2-symlink a known limitation, and is a runtime fix planned (e.g. detect symlink targets and fall back to an append-only finalize)?

Is it expected that the non-alternative copier (DATASET_ADLSGEN1_USE_ALTERNATIVE_COPIER=false) still issues MSCONCAT for multi-block uploads?

If none of the above is possible, what is the recommended pattern for writing >8 MB artifacts from an Azure ML job to an ADLS Gen1 datastore whose folders have been migrated to gen2 symlinks?

Minimal repro

An AzureDataLakeGen1 datastore whose target folder is a gen1→gen2 migrated symlink.

A v2 command job with a uri_file output, mode=upload, whose command writes a >8 MiB file to the output mount, e.g. dd if=/dev/urandom of=${{outputs.out}} bs=1M count=25.

Job fails at upload finalize with MSCONCAT failed with error 0x83090a70 ... gen1-to-gen2 symlink. A <8 MiB payload to the same folder succeeds.

Environment

azure-ai-ml (v2), pipeline / command job, mode=upload, uri_file output.

Datastore type: AzureDataLakeGen1.

• Azure ML common runtime with azureml-dataprep-rslex ~2.25.

Linux compute cluster.I have an Azure Machine Learning v2 pipeline (SDK azure-ai-ml) whose step writes an output file to an ADLS Gen1 datastore (datastore type AzureDataLakeGen1). The output is declared as:

Output(

My training script just writes the file locally to the managed-output mount; Azure ML's runtime then uploads it to the datastore.

Symptom: a small output (~7.8 MB) uploads fine, but a larger output (~8.3 MB and up) fails during the upload finalize with:

Error Code: ScriptExecution.WriteStreams.Unexpected
Native Error: An error occurred when streaming data to output target.
azure_adls_gen1 service: IllegalArgumentException:
  "MSCONCAT failed with error 0x83090a70 (Bad Request. The specified operation is not valid
   for gen1-to-gen2 symlink.)"

The target folder was migrated from ADLS Gen1 to Gen2 and is exposed as a gen1-to-gen2 symlink. The storage service rejects the WebHDFS op=MSCONCAT call on such paths. The upload is done by the Azure ML common runtime (rslex), not by my code.

What I've confirmed

It is not a permissions problem: the same job, using the same datastore identity, successfully writes the smaller file to the same folder. Only the larger file fails.

  The difference appears to be a **single-block upload threshold (~8 MiB)**. Below it, rslex writes the file in one stream (I see a "custom writer" copy with **no** concat) and it succeeds. Above it, rslex splits the file into blocks and finalizes with **`op=MSCONCAT`**, which the migrated-symlink path rejects.
  
     Writing the same large payload to a **freshly created (non-migrated) folder succeeds** — only the migrated gen1→gen2 symlink folders reject MSCONCAT.
     
     ### What I've tried (no luck)

        **`mode=rw_mount`** instead of `mode=upload` — the FUSE flush also finalizes with MSCONCAT and fails the same way (surfaces as `OSError [Errno 5]`).
        
           Forcing rslex to use its **sequential (non-parallel) ADLS Gen1 copier** via the env var `DATASET_ADLSGEN1_USE_ALTERNATIVE_COPIER=false`:
           
                 Setting it in the **environment image** had no effect — the uploader still logged `alternative=true` (the uploader runs in a separate common-runtime image and doesn't inherit the training image's env vars).
                 
                       Setting it via the job's **`environment_variables`** *did* reach the uploader (it logged `alternative=false`), **but the write still failed with the same MSCONCAT error** — so the sequential copier *also* uses MSCONCAT for multi-block uploads.
                       
                       So both copier paths issue MSCONCAT once the file spans multiple blocks, and I can't shrink the file below the threshold.
                       
                       ### Questions

                          Is there a **supported way to make the managed ADLS Gen1 upload avoid the `MSCONCAT` finalize** — i.e. write the file as a single stream / append-only — for files above the single-block threshold? (The smaller file already succeeds this way.)
                          
                             Is the **single-block / single-PUT size threshold configurable** (e.g. an `azureml-dataprep-rslex` setting such as a stream-copier block size) so that a ~10–50 MB file is written as one block and no concat is issued? If so, can it be passed via the job's `environment_variables`?
                             
                                Is **MSCONCAT-on-a-gen1→gen2-symlink a known limitation**, and is a runtime fix planned (e.g. detect symlink targets and fall back to an append-only finalize)?
                                
                                   Is it **expected** that the non-alternative copier (`DATASET_ADLSGEN1_USE_ALTERNATIVE_COPIER=false`) still issues MSCONCAT for multi-block uploads?
                                   
                                      If none of the above is possible, what is the **recommended pattern** for writing >8 MB artifacts from an Azure ML job to an ADLS Gen1 datastore whose folders have been migrated to gen2 symlinks?
                                      
                                      ### Minimal repro

                                         An `AzureDataLakeGen1` datastore whose target folder is a gen1→gen2 migrated symlink.
                                         
                                            A v2 command job with a `uri_file` output, `mode=upload`, whose command writes a >8 MiB file to the output mount, e.g. `dd if=/dev/urandom of=${{outputs.out}} bs=1M count=25`.
                                            
                                               Job fails at upload finalize with `MSCONCAT failed with error 0x83090a70 ... gen1-to-gen2 symlink`. A <8 MiB payload to the same folder succeeds.
                                               
                                               ### Environment

                                                  `azure-ai-ml` (v2), pipeline / command job, `mode=upload`, `uri_file` output.
                                                  
                                                     Datastore type: `AzureDataLakeGen1`.
                                                     
                                                        Azure ML common runtime with `azureml-dataprep-rslex` ~2.25.
                                                        
                                                           Linux compute cluster.