Update Policy Function

Question

Update Policy Function

Bexy Morgan 260

Using update policy function, ingest data from source tables SourceTableA,SourceTableB,SourceTableC into Target table.

.create table SourceTableA (PK:string,A1:string,A2:int)
.create table SourceTableB (PK:string,B1:string,B2:int)
.create table SourceTableC (PK:string,C1:string)

.ingest inline into table ["SourceTableA"] <|
"ABC","Q",88
"DEF","W",11

.ingest inline into table ["SourceTableB"] <|
"ABC","Al",100
"DEF","Md",1111
"FGH","Bd",9824

.ingest inline into table ["SourceTableC"] <|
"ABC","Quad"
"FGH","Tex"


.create table Target (PK:string,A1:string,A2:int,B1:string,B2:int,C1:string)

Sander van de Velde | MVP 37,066 Reputation points MVP

2023-10-17T16:04:38.58+00:00

Hello Bexy,

It seems you left out the actual policy functions and update policies in this code example.

Note the update policies are only triggered by new lines AFTER the policy is set up.

Already existing data in the source tables will not be 'transferred' by the update policy. You need to handle those yourself.
Bexy Morgan 260 Reputation points

2023-10-17T16:28:16.8333333+00:00

No Sander, I understand how it works, but I'm dealing with three distinct tables, each sharing certain columns. Every table incorporates several 'extend' operators, resulting in three completely new output tables. My goal is to combine these three resulting output tables into a single output table using the update policy function
Sander van de Velde | MVP 37,066 Reputation points MVP

2023-10-17T16:43:22.91+00:00

Hello Bexy,

I expect this to be doable.

A (target) table can have multiple update policies.

Just give your target table three separate update policies, each for another source table with a custom update policy function.

All three functions receive messages from their source table and need to honor the target table format.

Is this the scenario you are looking for?
Bexy Morgan 260 Reputation points

2023-10-17T17:00:27.07+00:00

Yes Sander, it is the scenario

1 answer

Your answer

Sander van de Velde | MVP 37,066 Reputation points MVP

2023-10-17T16:04:38.58+00:00

Hello Bexy,

It seems you left out the actual policy functions and update policies in this code example.

Note the update policies are only triggered by new lines AFTER the policy is set up.

Already existing data in the source tables will not be 'transferred' by the update policy. You need to handle those yourself.
Bexy Morgan 260 Reputation points

2023-10-17T16:28:16.8333333+00:00

No Sander, I understand how it works, but I'm dealing with three distinct tables, each sharing certain columns. Every table incorporates several 'extend' operators, resulting in three completely new output tables. My goal is to combine these three resulting output tables into a single output table using the update policy function
Sander van de Velde | MVP 37,066 Reputation points MVP

2023-10-17T16:43:22.91+00:00

Hello Bexy,

I expect this to be doable.

A (target) table can have multiple update policies.

Just give your target table three separate update policies, each for another source table with a custom update policy function.

All three functions receive messages from their source table and need to honor the target table format.

Is this the scenario you are looking for?
Bexy Morgan 260 Reputation points

2023-10-17T17:00:27.07+00:00

Yes Sander, it is the scenario

Answer 1

Sander van de Velde | MVP 37,066 MVP

Hello Bexy,

You want to ingest data from three different (source) tables into one (target) table.

A (target) table can have multiple update policies.

Follow this blog post regarding adding a (one) update policy.

Just give your target table three separate update policies, each for another source table with a custom update policy function.

So you probably need three functions.

All (three) functions receive messages from their source table and need to honor the target table format.

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

Anonymous

Hi @Sander van de Velde ,my apologies to ask you again.

I think my question will be clear now. How should the write the update policy here to ingest the data from the below 3 functions sta( ), stb( ), stc( ) into Target table.

.create table SourceTableA (PK:string,A1:string,A2:int)
.create table SourceTableB (PK:string,B1:string,B2:int)
.create table SourceTableC (PK:string,C1:string)

.ingest inline into table ["SourceTableA"] <|
"ABC","Q",88
"DEF","W",11

.ingest inline into table ["SourceTableB"] <|
"ABC","Al",100
"DEF","Md",1111
"FGH","Bd",9824

.ingest inline into table ["SourceTableC"] <|
"ABC","Quad"
"FGH","Tex"


.create function
 with (docstring = 'STA', folder = 'UpdatePolicyFunctions')
     sta()  
{
SourceTableA
| extend extraAcolumn1 = A2+2
|extend extraAcolumn2 = strcat("anonymous",A1)
}

.create function
 with (docstring = 'STB', folder = 'UpdatePolicyFunctions')
     stb()  
{
SourceTableB
| extend extraBcolumn = 1 + B2
}


.create function
 with (docstring = 'STC', folder = 'UpdatePolicyFunctions')
     stc()  
{
SourceTableC
| extend extraCcolumn = strcat("anonymous", C1) 
}

.create table Target (PK:string,A1:string,A2:int,B1:string,B2:int,C1:string,extraAcolumn1:int,extraAcolumn2:string,
extraBcolumn:int,extraCcolumn:string)

Target

Share via

Update Policy Function

1 answer

Your answer