how to remove duplicate json values in Azure logic apps need workflow chart

Question

how to remove duplicate json values in Azure logic apps need workflow chart

SURULIRAJAN V 5

[
{
  "books": [
    {
      "title": "The Great Gatsby",
      "author": "F. Scott Fitzgerald",
      "editor": "Maxwell Perkins",
      "id": "ISBN-123456789",
      "language": "English"
    },
    {
      "title": "The Great Gatsby",
      "author": "F. Scott Fitzgerald",
      "editor": "Maxwell Perkins",
      "id": "ISBN-123456789",
      "language": "English"
    },
    {
      "title": "To Kill a Mockingbird",
      "author": "Harper Lee",
      "editor": "Tay Hohoff",
      "id": "ISBN-987654321",
      "language": "English"
    },
    {
      "title": "1984",
      "author": "George Orwell",
      "editor": "T.S. Eliot",
      "id": "ISBN-567890123",
      "language": "English"
    },
    {
      "title": "One Hundred Years of Solitude",
      "author": "Gabriel García Márquez",
      "editor": "Salvador García",
      "id": "ISBN-234567890",
      "language": "Spanish"
    }
  ]
}
]

This is my sample data . i need to avoid duplicate values and store it in azure blob storage using azure logic apps. tell me workflow in brief

Nitish Sharma 441 Reputation points

2023-08-17T14:07:27.3333333+00:00
Hi @SURULIRAJAN V

You can try below steps:

Create a trigger like Recurrence

Initialize Variable which hold the unique book list.

Foreach loop to iterate over each book in the input JSON array.

Inside the loop, use a Condition action to check whether the current book already exists in the variable or not.

If book doesn't exist then add it to variable.

If book exist then do nothing.

When the loop is done, save that variable to Blob.

Let me know if this helps.
SURULIRAJAN V 5 Reputation points

2023-08-18T11:09:14.2533333+00:00

i need working chart like this
Nitish Sharma 441 Reputation points

2023-08-22T16:07:39.3733333+00:00

What working chart is needed?

Are you able to create it based on the steps shared above, you will be able to find all these in LogicApps desginer.

2 answers

Your answer

Nitish Sharma 441 Reputation points

2023-08-17T14:07:27.3333333+00:00

Hi @SURULIRAJAN V

You can try below steps:

Create a trigger like Recurrence

Initialize Variable which hold the unique book list.

Foreach loop to iterate over each book in the input JSON array.

Inside the loop, use a Condition action to check whether the current book already exists in the variable or not.

If book doesn't exist then add it to variable.

If book exist then do nothing.

When the loop is done, save that variable to Blob.

Let me know if this helps.
SURULIRAJAN V 5 Reputation points

2023-08-18T11:09:14.2533333+00:00

i need working chart like this
Nitish Sharma 441 Reputation points

2023-08-22T16:07:39.3733333+00:00

What working chart is needed?

Are you able to create it based on the steps shared above, you will be able to find all these in LogicApps desginer.

Answer 1

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more

Answer 2

Hello, @SURULIRAJAN V - Thanks for reaching out. IMO, it's a lot easier to do this kind of task (checking duplicates in JSON data) in Azure Functions and then incorporate it into your Logic Apps workflow. Alternative to the Azure Functions can be the use of Inline Code action but note that it'll require that you have an Integration Account linked to your Logic Apps.

With your request for a working chart, I think that you meant a screenshot of the workflow in Designer mode so please see below for the approach that uses the Azure Functions running Nodejs in Consumption SKU:

For the code that checks the duplicates and removes them, I've taken a snippet from a post on StackOverflow which checks the duplicates based on the value of the name property in the JSON.
With small edits for checking on the value of the id property, the code for the HTTP-triggered functions looks like this:
A quick test of the function above with the exact JSON in your example results in the following output:

[
  {
    "title": "The Great Gatsby",
    "author": "F. Scott Fitzgerald",
    "editor": "Maxwell Perkins",
    "id": "ISBN-123456789",
    "language": "English"
  },
  {
    "title": "To Kill a Mockingbird",
    "author": "Harper Lee",
    "editor": "Tay Hohoff",
    "id": "ISBN-987654321",
    "language": "English"
  },
  {
    "title": "1984",
    "author": "George Orwell",
    "editor": "T.S. Eliot",
    "id": "ISBN-567890123",
    "language": "English"
  },
  {
    "title": "One Hundred Years of Solitude",
    "author": "Gabriel García Márquez",
    "editor": "Salvador García",
    "id": "ISBN-234567890",
    "language": "Spanish"
  }
]

With the Function App created and tested successfully, it's now just a matter of incorporating it into the Logic Apps workflow. Here, I created a workflow that (a) triggers an HTTP request with a payload of the same JSON data, (b) removes duplicates by running our Azure Functions, (c) saves the filtered JSON into files container of the Blob Storage as output.json file:
The workflow above results in the following:

I hope this answer helps you get up and running. Please let me know if you have any questions.

Share via

how to remove duplicate json values in Azure logic apps need workflow chart

2 answers

Your answer