Can OpenAI classifciation be prompted to detect duplicate data?

Dan Holmes 21

I am trying to get a sense of OpenAI's ability to detect duplicate data. For instance if i have the following two sets of attributes that describe a company:

company name, address city, address state, address zip

Can i create a prompt that would provide the likelihood (as a number 1-100) that they are the same organization?

I have tried a few variations including supplying examples. It will generate data but i haven't convinced it to return a number nor am i even sure i have prompted it well enough to understand what i am after.

YutongTie-MSFT 51,766 Reputation points

2023-03-21T02:00:30.42+00:00

Hello @Dan Holmes

There is something I tried let me know is the yellow part you want or not, if yes, I can share my code sample to you. If not, could you please share more details about what you want?

Given the following two sets of attributes for different companies:

Company A: {company name: ABC Inc, address city: New York, address state: NY, address zip: 10001} Company B: {company name: XYZ Corp, address city: San Francisco, address state: CA, address zip: 94107}
Dan Holmes 21 Reputation points

2023-03-21T11:41:30.3466667+00:00

Thank you. That is what i was after.

The end goal is to be able to ask openai if a certain set of entity attributes matches an existing entity. For this case consider a CRM and i want to keep my data clean by not creating duplicate entries.

I would like to send openai, "Search through my company list and show me the companies that are likely matched sorted by descending probability".

That would mean openai would need to be able to read my company list. How do i do that? Are there existing examples ?

Share via

Can OpenAI classifciation be prompted to detect duplicate data?

Your answer