How to Properly Ingest Strings with Commas into Kusto Table in single column?

Question

Certainly! When posting a question to a community forum, it's crucial to be clear and concise, ensuring to provide all the necessary details without overwhelming the reader. Here’s a refined version for the Azure community:

---

Title: How to Properly Ingest Strings with Commas into Kusto Table without Truncation?

Body:

Hello Azure Community,

I am facing an issue while ingesting string data containing commas into a Kusto table using C#. The string gets truncated at the comma when ingested. Below is a simplified version of the code I am using:

csharpCopy code
string measureValue = "GAIT AID REQUIRED,NON-STERIODAL INJECTIONS,KNEE BRACES,ORTHOTICS";
string measureValueForVarchar = $"\"{measureValue.Replace("\"", "\"\"")}\"";
string kustoQuery = $".ingest inline into table {tableName} <| ... , {measureValueForVarchar}";

After ingestion, when querying the table, the measureValueForVarchar field shows only the first part of the string, i.e., "GAIT AID REQUIRED", instead of the complete string.

I have tried various methods to format the string to ensure that the commas are not interpreted as delimiters, including enclosing the string in double quotes and replacing embedded double quotes. However, the string still gets truncated at the comma.

csharpCopy code
// Example of the ingested string
"measure_value_varchar": "GAIT AID REQUIRED

Could someone please assist me in addressing this issue, ensuring that the entire string, including commas, gets ingested properly without truncation?

Thank you in advance for your time and assistance!

Answer 1

Hi Kiwitech,

The data content to ingest is parsed as CSV, unless otherwise modified by the ingestion properties. According to the Microsoft documentation:

This command is intended for manual ad-hoc testing. For production use, we recommended using the methods described in ingest from storage or ingest from query, which are better for bulk delivery of large amounts of data.

Another way is using the SDK. Look at this blog of @Sander van de Velde | MVP for more information and an example:
https://sandervandevelde.wordpress.com/2023/03/23/programmatically-ingest-data-into-azure-data-explorer/

Kind regards,

Wilko

---

Please do not forget to "Accept the answer” wherever the information provided helps you, this can be beneficial to other community members. If you have extra questions about this answer, please click "Comment".