Extract JSON Data values and identify the differences

Hellothere8028 821 Reputation points
2020-12-05T12:30:50.723+00:00

Hi All,

I have a Health coding work which is done by two people(the coder does it first and then the auditor checks it).. So two people do it for every transaction (transaction is denoted by a pair of Billid and ctextid).. The work done by the auditor is final and correct...The Health coding done by them is stored in the JSON format..I need to extract specific values from the json format such as cCode,dCode,hcpc ,mod and then identify the deleted and added for each of them..Deleted is basically which is not there in the Auditor Json but is there in the coder json for a transaction...Added is basically what is not there in the coder json but is there in the auditor json...Also the earliest created (time) for a transaction denotes the work done by coder and the later created(time) denotes the work done by auditor...Please find attached the file ..Can you please help me with this.. Please find the DDL below:

Input table:
create table ##input1
(Billid int,
Ctextid int,
info varchar(max),
codedby varchar(30),
created datetime2
)
insert into ##input1 values
('7111','89321','[
{
""cCode"": ""7832"",
""Date"": ""08/26/2020"",
""ID"": ""511"",
""description"": ""holos"",
""dgoses"": [
{
""description"": ""disease"",
""dCode"": ""Y564"",
""CodeAllId"": ""8921"",
""messages"": [
""""
],
""sequenceNumber"": 1
},
{
""description"": ""acute pain"",
""dCode"": ""U3321"",
""CodeAllId"": ""33213"",
""messages"": [
""""
],
""sequenceNumber"": 2
},
{
""description"": ""height"",
""dCode"": ""U1111"",
""CodeAllId"": ""33278"",
""messages"": [
""""
],
""sequenceNumber"": 3
},
{
""description"": ""PIDEMIA "",
""dCode"": ""H8811"",
""CodeAllId"": ""90000"",
""messages"": [
""""
],
""sequenceNumber"": 4
}
],
""familyPlan"": """",
""hcpc"": 5,
""id"": """",
""isEPS"": false,
""mod"": ""67"",
""originalUnitAmount"": ""8888"",
""type"": ""CHARGE"",
""unitAmount"": ""9000"",
""vId"": ""90001""
},
{
""cCode"": ""900114"",
""Date"": ""08/26/2020"",
""ID"": ""523"",
""description"": ""heart valve"",
""dgoses"": [
{
""description"": ""Fever"",
""dCode"": ""J8923"",
""CodeAllId"": ""892138"",
""messages"": [
""""
],
""sequenceNumber"": 1
}
],
""familyPlan"": """",
""hcpc"": 1,
""id"": """",
""mod"": ""26"",
""originalUnitAmount"": ""19039"",
""type"": ""CHARGE"",
""unitAmount"": ""1039"",
""vId"": ""5113""
}
]','cmc5','2020-11-11 13:39:25.922 +0000'),
('7111','89321','[
{
""cCode"": ""78832"",
""Date"": ""08/26/2020"",
""ID"": ""511"",
""description"": ""holos"",
""dgoses"": [
{
""description"": ""disease"",
""dCode"": ""Y564"",
""CodeAllId"": ""8921"",
""messages"": [
""""
],
""sequenceNumber"": 1
},
{
""description"": ""acute pain"",
""dCode"": ""U3321"",
""CodeAllId"": ""33213"",
""messages"": [
""""
],
""sequenceNumber"": 2
},
{
""description"": ""height"",
""dCode"": ""U41111"",
""CodeAllId"": ""33278"",
""messages"": [
""""
],
""sequenceNumber"": 3
},
{
""description"": ""PIDEMIA "",
""dCode"": ""H8811"",
""CodeAllId"": ""90000"",
""messages"": [
""""
],
""sequenceNumber"": 4
}
],
""familyPlan"": """",
""hcpc"": 8,
""id"": """",
""isEPS"": false,
""mod"": ""67"",
""originalUnitAmount"": ""8888"",
""type"": ""CHARGE"",
""unitAmount"": ""9000"",
""vId"": ""90001""
},
{
""cCode"": ""900114"",
""Date"": ""08/26/2020"",
""ID"": ""523"",
""description"": ""heart valve"",
""dgoses"": [
{
""description"": ""Fever"",
""dCode"": ""J8923"",
""CodeAllId"": ""892138"",
""messages"": [
""""
],
""sequenceNumber"": 1
}
],
""familyPlan"": """",
""hcpc"": 1,
""id"": """",
""mod"": ""126"",
""originalUnitAmount"": ""19039"",
""type"": ""CHARGE"",
""unitAmount"": ""1039"",
""vId"": ""5113""
}
]','cmc6','2020-11-11 13:45:20.383 +0000')

Output table

Create table ##output1
(Billid int,
codedby varchar(30),
ctextid int,
cCodecoder varchar(1000),
cCodeAuditor varchar(1000),
deletedcCode varchar(1000),
addedcCode varchar(1000),
dCodeCoder varchar(1000),
dCodeAuditor varchar(1000),
deleteddCode varchar(1000),
addeddCode varchar(1000),
hcpccoder varchar(1000),
hcpcauditor varchar(1000),
deletedhcpc varchar(1000),
addedhcpc varchar(1000),
modcoder varchar(1000),
modauditor varchar(1000),
deletedmod varchar(1000),
addedmod varchar(1000)
)

Insert into ##output1 values
('7111','cmc5','89321','7832,900114','78832,900114','7832','78832','Y564,U3321,U1111,H8811,J8923','Y564,U3321,U41111,H8811,J8923','U1111','U41111','5,1','8,1','5','8','67,26','67126','26','126')

Thanks,
Arun

Developer technologies | Transact-SQL
Developer technologies | Transact-SQL
A Microsoft extension to the ANSI SQL language that includes procedural programming, local variables, and various support functions.
{count} votes

Answer accepted by question author
  1. Viorel 125.8K Reputation points
    2020-12-05T21:54:34.543+00:00

    In addition, check a possible initial script, for cCode and dCode:

    drop table if exists ##input1
    drop table if exists ##output1
    
    create table ##input1
    (
        Billid int,
        Ctextid int,
        info varchar(max),
        codedby varchar(30),
        created datetime2
    )
    
    insert into ##input1 values
    ('7111','89321','
    [
      {
        "cCode": "7832",
        "Date": "08/26/2020",
        "ID": "511",
        "description": "holos",
        "dgoses": [
          {
            "description": "disease",
            "dCode": "Y564",
            "CodeAllId": "8921",
            "messages": [
              ""
            ],
            "sequenceNumber": 1
          },
          {
            "description": "acute pain",
            "dCode": "U3321",
            "CodeAllId": "33213",
            "messages": [
              ""
            ],
            "sequenceNumber": 2
          },
          {
            "description": "height",
            "dCode": "U1111",
            "CodeAllId": "33278",
            "messages": [
              ""
            ],
            "sequenceNumber": 3
          },
          {
            "description": "PIDEMIA ",
            "dCode": "H8811",
            "CodeAllId": "90000",
            "messages": [
              ""
            ],
            "sequenceNumber": 4
          }
        ],
        "familyPlan": "",
        "hcpc": 5,
        "id": "",
        "isEPS": false,
        "mod": "67",
        "originalUnitAmount": "8888",
        "type": "CHARGE",
        "unitAmount": "9000",
        "vId": "90001"
      },
      {
        "cCode": "900114",
        "Date": "08/26/2020",
        "ID": "523",
        "description": "heart valve",
        "dgoses": [
          {
            "description": "Fever",
            "dCode": "J8923",
            "CodeAllId": "892138",
            "messages": [
              ""
            ],
            "sequenceNumber": 1
          }
        ],
        "familyPlan": "",
        "hcpc": 1,
        "id": "",
        "mod": "26",
        "originalUnitAmount": "19039",
        "type": "CHARGE",
        "unitAmount": "1039",
        "vId": "5113"
      }
    ]
    ','cmc5','2020-11-11 13:39:25.922'),
    
    ('7111','89321','
    [
      {
        "cCode": "78832",
        "Date": "08/26/2020",
        "ID": "511",
        "description": "holos",
        "dgoses": [
          {
            "description": "disease",
            "dCode": "Y564",
            "CodeAllId": "8921",
            "messages": [
              ""
            ],
            "sequenceNumber": 1
          },
          {
            "description": "acute pain",
            "dCode": "U3321",
            "CodeAllId": "33213",
            "messages": [
              ""
            ],
            "sequenceNumber": 2
          },
          {
            "description": "height",
            "dCode": "U41111",
            "CodeAllId": "33278",
            "messages": [
              ""
            ],
            "sequenceNumber": 3
          },
          {
            "description": "PIDEMIA ",
            "dCode": "H8811",
            "CodeAllId": "90000",
            "messages": [
              ""
            ],
            "sequenceNumber": 4
          }
        ],
        "familyPlan": "",
        "hcpc": 8,
        "id": "",
        "isEPS": false,
        "mod": "67",
        "originalUnitAmount": "8888",
        "type": "CHARGE",
        "unitAmount": "9000",
        "vId": "90001"
      },
      {
        "cCode": "900114",
        "Date": "08/26/2020",
        "ID": "523",
        "description": "heart valve",
        "dgoses": [
          {
            "description": "Fever",
            "dCode": "J8923",
            "CodeAllId": "892138",
            "messages": [
              ""
            ],
            "sequenceNumber": 1
          }
        ],
        "familyPlan": "",
        "hcpc": 1,
        "id": "",
        "mod": "126",
        "originalUnitAmount": "19039",
        "type": "CHARGE",
        "unitAmount": "1039",
        "vId": "5113"
      }
    ]
    ','cmc6','2020-11-11 13:45:20.383')
    
    
    create table ##output1
    (Billid int,
    codedby varchar(30),
    ctextid int,
    cCodecoder varchar(1000),
    cCodeAuditor varchar(1000),
    deletedcCode varchar(1000),
    addedcCode varchar(1000),
    dCodeCoder varchar(1000),
    dCodeAuditor varchar(1000),
    deleteddCode varchar(1000),
    addeddCode varchar(1000),
    hcpccoder varchar(1000),
    hcpcauditor varchar(1000),
    deletedhcpc varchar(1000),
    addedhcpc varchar(1000),
    modcoder varchar(1000),
    modauditor varchar(1000),
    deletedmod varchar(1000),
    addedmod varchar(1000)
    )
    
    Insert into ##output1 values
    ('7111','cmc5','89321','7832,900114','78832,900114','7832','78832','Y564,U3321,U1111,H8811,J8923','Y564,U3321,U41111,H8811,J8923','U1111','U41111','5,1','8,1','5','8','67,26','67126','26','126')
    
    
    select *
    from ##input1
    
    select *
    from ##output1
    
    ---
    
    ;
    with I as
    (
        select *, 
            dense_rank() over (order by Billid, Ctextid) as tid,
            dense_rank() over (partition by Billid, Ctextid order by Created) as n
        from ##input1
    ),
    D as
    (
        select I.*, mk.[key] as mk, m.*, dk.[key] as dk, d.*
        from I
        cross apply openjson(info) mk
        cross apply openjson(mk.value) with
            (
                cCode nvarchar(max) '$.cCode',
                dgoses nvarchar(max) '$.dgoses' as json
            ) m
        cross apply openjson(dgoses) dk
        cross apply openjson(dk.value) with
            (
                dCode nvarchar(max) '$.dCode'
            ) d
    ),
    C as
    (
        select * from D where n = 1
    ),
    A as
    (
        select * from D where n = 2
    )
    select 
        Billid, 
        codedby, 
        Ctextid,
    
        (
            select string_agg(cCode, ',') within group (order by mk)
            from 
            (
                select distinct cCode, mk
                from C
                where tid = t.tid
            ) d
        ) as cCodeCoder,
        (
            select string_agg(cCode, ',') within group (order by mk)
            from 
            (
                select distinct cCode, mk
                from A
                where tid = t.tid
            ) d
        ) as cCodeAuditor,
        (
            select string_agg(cCode, ',')
            from
            (
                select cCode
                from C
                where tid = t.tid
                except
                select cCode
                from A 
                where tid = t.tid
            ) d
        ) as deletedcCode,
        (
            select string_agg(cCode, ',')
            from
            (
                select cCode
                from A
                where tid = t.tid
                except
                select cCode
                from C
                where tid = t.tid
            ) d
        ) as addedcCode,
    
        (
            select string_agg(dCode, ',') within group (order by mk, dk)
            from 
            (
                select distinct dCode, mk, dk
                from C
                where tid = t.tid
            ) d
        ) as dCodeCoder,
        (
            select string_agg(dCode, ',') within group (order by mk, dk)
            from 
            (
                select distinct dCode, mk, dk
                from A
                where tid = t.tid
            ) d
        ) as dCodeAuditor,
        (
            select string_agg(dCode, ',')
            from
            (
                select dCode
                from C
                where tid = t.tid
                except
                select dCode
                from A 
                where tid = t.tid
            ) d
        ) as deleteddCode,
        (
            select string_agg(dCode, ',')
            from
            (
                select dCode
                from A
                where tid = t.tid
                except
                select dCode
                from C
                where tid = t.tid
            ) d
        ) as addeddCode
    
    from I as t
    where n = 1
    

    Extend it for other data, or find a more compact approach.

    0 comments No comments

3 additional answers

Sort by: Most helpful
  1. Erland Sommarskog 129.6K Reputation points MVP Volunteer Moderator
    2020-12-05T19:53:37.507+00:00

    First: you are on the RTM version of SQL 2019. You should apply Cumulative Update 8, the most recent CU8, at first possible occasion. It has nothing to do with your question, but there are bug fixes in in the CUs, not the least related to scalar user-defined functions. After applying the CU, you should run sp_refreshsqlmodule on all scalar user-defined functions.

    Now to your problem. I found it to be too big and complex to be interested in trying to compose a solution from start to end. Also, I will have to admit that I don't really like the format of the desired output. But to get you started, here is an example of how you can extract the data to relational format:

    SELECT *FROM ##input1  
    CROSS APPLY OPENJSON(info) with (cCode varchar(10),  
                                     dgoses nvarchar(MAX) '$.dgoses' AS JSON)   
    CROSS APPLY OPENJSON(dgoses) with (dCode varchar(10),  
                                       sequenceNumber int)  
    

    I think OPENJSON will do all you need, but it is possible that you will find JSON_VALUE or JSON_QUERY useful as well. If you have never seen these functions before, I recommend this link:
    https://learn.microsoft.com/en-us/sql/relational-databases/json/convert-json-data-to-rows-and-columns-with-openjson-sql-server?view=sql-server-ver15.

    I would also recommend you to approach this in steps. First focus to wrestle out the data into tables. Then work on making the comparisons between the coder and the auditor, but still produce that data in relational format. (That is, normalised tables with one value per cell.) Finally, compose the desired output table. You are of course welcome to ask for help along the way.

    And, oh, one more thing: the JSON data you have is flawed. You need to do this:

    UPDATE #input1 SET info = replace(info, '""', '"')  
    
    1 person found this answer helpful.

  2. EchoLiu-MSFT 14,626 Reputation points
    2020-12-11T11:28:09.847+00:00

    Hi @Hellothere8028 ,

    Sorry to reply you so late!
    When you post a question, it is recommended that you post a minimal example, the complete data is not easy to read because of its complexity.ErlandSommarskog has provided you with steps and examples, have you tried it?
    I tested the code of Viorel-1 and it works,can help you extract the specific value you need.
    47339-image.png

    You can first try according to the steps of ErlandSommarskog, you can come to the community for help if you encounter any problems in the process.

    Regards
    Echo


    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
    Hot issues November--What can I do if my transaction log is full?
    Hot issues November--How to convert Profiler trace into a SQL Server table


  3. Hellothere8028 821 Reputation points
    2020-12-25T12:03:27.597+00:00

    Thanks @Viorel : Really Appreciate your help!!

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.