An Azure service for ingesting, preparing, and transforming data at scale.
Data Flow source test connection to Cosmos DB fails with InvalidSubscriptionId error via Managed VNet Private Endpoint
Yeshwanth Reddy Sangam
0
Reputation points Microsoft Employee
Azure Data Factory Data Flow fails to connect to Azure Cosmos DB (SQL API) when using Managed Virtual Network with a Managed Private Endpoint. The Copy Activity works correctly with the same linked service and private endpoint configuration.
Error Details:
- Status code: 400
- Error: {"error":{"code":"InvalidSubscriptionId","message":"The provided subscription identifier 'resourceGroups' is malformed or invalid."}}
- This error occurs when testing connection from a Data Flow source, and during Data Flow debug/execution.
- Tested after publishing as well
Behavior:
- Linked service "Test Connection" → SUCCEEDS ✓
- Copy Activity using same linked service → SUCCEEDS ✓
- Data Flow source using same linked service → FAILS ✗ (400 InvalidSubscriptionId)
This appears to be a bug in how the Data Flow Spark runtime resolves Cosmos DB resource IDs through Managed VNet private endpoints. The error suggests the resource path is being incorrectly parsed, with "resourceGroups" being interpreted as the subscription identifier.
Azure Data Factory
Azure Data Factory
Sign in to answer