Use the sample app generator wizard to create code to ingest and query your data
The sample app generator wizard is a tool that allows you to create a working app to ingest and query your data in your preferred programming language. The generated code references the relevant SDK frameworks and is tailored to your cluster and data. The app is a self-contained, executable example that demonstrates how to connect to a cluster using authentication, administer, and ingest and query data using the SDK.
You can use the generated code as a baseline to write your own apps, altering the code as you go, or you can copy sections of code into your own apps. The code includes comments with tips, best practices, links to reference material, and recommended TODO changes when adapting the code to your needs.
Prerequisites
- A Microsoft account or an Azure Active Directory user identity. An Azure subscription isn't required.
- An Azure Data Explorer cluster and database. You can create a free cluster or create a full cluster. To decide which is best for you, check the feature comparison.
- Sign in to the Azure Data Explorer web UI and add a connection to your cluster.
Note
To enable access between a cluster and a storage account without public access (restricted to private endpoint/service endpoint), see Create a Managed Private Endpoint.
Access the wizard
The sample app generator wizard guides you through the ingestion process.
To access the wizard from the Azure Data Explorer web UI, use one of the following methods:
Select Data in the left pane. Within the Data Management page under Quick actions, select Generate Sample App.
Select Data in the left pane. Within the Data Management page, select SDKs tab. Within SDKs sub-tab, select an SDK.
Sample app generator wizard
The sample app generator wizard guides you through the ingestion process.
On the Destination tab, specify the destination cluster, database, and table and then select the programming language for your app.
Note
This section describes the wizard in general. The options you select depend on what data format you are ingesting, what kind of data source you are ingesting from, and whether you are ingesting into a new or existing table.
For sample scenarios, see:
- Ingest into a new table from Azure Blob Storage in CSV format
- Ingest into an existing table from a local file in JSON format
The wizard guides you through the following options:
- Ingest into an existing table
- Ingest into a new table
- Ingest data from:
- Blob Storage
- A local file
Schema mapping
The service automatically generates schema and ingestion properties, which you can change. You can use an existing mapping structure or create a new one, depending on if you're ingesting to a new or existing table.
In the Schema tab, do the following actions:
- Confirm the autogenerated compression type.
- Choose the format of your data. Different formats will allow you to make further changes.
- Change mapping in the Editor window.
File formats
The ingestion wizard supports ingesting from source data in all data formats supported by Azure Data Explorer for ingestion.
Editor window
In the Editor window of the Schema tab, you can adjust data table columns as necessary.
The changes you can make in a table depend on the following parameters:
- Table type is new or existing
- Mapping type is new or existing
| Table type | Mapping type | Available adjustments |
|---|---|---|
| New table | New mapping | Change data type, Rename column, New column, Delete column, Update column, Sort ascending, Sort descending |
| Existing table | New mapping | New column (on which you can then change data type, rename, and update), Update column, Sort ascending, Sort descending |
| Existing mapping | Sort ascending, Sort descending |
Note
When adding a new column or updating a column, you can change mapping transformations. For more information, see Mapping transformations
Note
At any time, you can open the command editor above the Editor pane. In the command editor, you can view and copy the automatic commands generated from your inputs.
Mapping transformations
Some data format mappings (Parquet, JSON, and Avro) support simple ingest-time transformations. To apply mapping transformations, create or update a column in the Editor window.
Mapping transformations can be performed on a column of Type string or datetime, with the Source having data type int or long. Supported mapping transformations are:
- DateTimeFromUnixSeconds
- DateTimeFromUnixMilliseconds
- DateTimeFromUnixMicroseconds
- DateTimeFromUnixNanoseconds
For more information, see mapping transformations.
Code generation
Once you have completed schema mapping and column manipulations, the sample app generator wizard will start the code generation process.
Initial data exploration
After code generation, the sample app project will be available for download as a ZIP file. Detailed instructions for running the project with the selected SDK will be shown both on the summary tab and inside the sample project file. In addition, the wizard gives you options to use Quick commands for initial exploration of your data.
Note
You may need to run the downloaded app for the Quick commands to work.