ADX Extents

Question

Below script creates a table "t1",

1. How do I create extents for this table?
2. what is the purpose of creating extents?

.create table t1 ( Timestamp:datetime, From:string, TravelMode:string, Tier:string)
 .ingest inline into table t1 <|  
2010-01-01,NY,Flight,Premium
2010-01-02,Boston,Train,Economy
2010-01-02,NY,Flight,Business
2010-01-02,Boston,Bus,Business
2010-01-03,Philadelphia,Train,Premium
2010-01-04,Texas,Bus,Economy
2010-01-05,Texas,Bus,Economy
2010-01-05,NJ,Train,Business
2010-01-06,NJ,Bus,Premium
2010-01-07,NJ,Bus,Economy
2010-01-08,NJ,Flight,Premium
2010-01-08,Newark,Bus,Business
2010-01-08,Newark,Flight,Economy
2010-01-08,Michigan,Bus,Economy
2010-01-09,Michigan,Train,Economy
2010-01-10,NY,Bus,Business
2010-01-11,NY,Flight,Economy
2010-01-12,NY,Train,Premium
2010-01-13,Birmingham,Bus,Business

Answer 1

Hello @Bexy Morgan ,

Extents (aka data shards) are a fundamental part of ADX, it tells you how data is stored for fast retrieval.

Please read this overview which is written in a pretty clear way.

One of the most important abilities of extents we use in our projects is 'tags'.

If we ingest batches of data (like files), we tag all records in this batch with the same unique tag (like the filename)

.ingest ... with @'{"tags":"["filename:abc123"]"}'

Now we can check if ingestion was successful because we can check for the tag.

We can also use tags to prevent double ingestion of the same data:

.set-or-append arraytestTarget with (ingestIfNotExists = '["onlyonce1"]', tags = '["ingest-by:onlyonce1"]') <| arraytest | where d == 'dev2'

I hope this gives some pointers on how to use extents. Please take some time to check the documentation.

---

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.