Share via

OpenAI GPT 3.5 save/recall system content

Jelle Holtkamp 45 Reputation points
2023-09-06T14:37:55.74+00:00

I constructed a rather detailed (large) instruction set for use in the System Role Content property when using the OpenAI chat service. I'm wondering 2 things: does this property count as tokens processed? I.e. is this part billed? And if so, how do make it so that I only have to provide this information once and have the chat maintain the instructions in future prompts? I am using the Rest API and would prefer to do any configuration using powershell/REST api if possible.

Azure OpenAI in Foundry Models

Answer accepted by question author

  1. Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
    2023-09-13T18:00:30.47+00:00

    Hi @Jelle Holtkamp ,

    I have checked internally on your second ask; it is not possible as this not how LLM's work as it cannot "remembers" the initial conversation. You can however work towards creating prompts with Chain of Thought Prompting to lower the possibility of inaccurate outcomes.

    Please let me know if you have any other questions.

    Thanks

    Saurabh

    Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

    Was this answer helpful?

    1 person found this answer helpful.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Saurabh Sharma 23,866 Reputation points Microsoft Employee Moderator
    2023-09-07T00:29:29.6733333+00:00

    Hi @Jelle Holtkamp ,

    Thanks for using Microsoft Q&A!!

    Yes, the system role content property counts as tokens processed and is billed accordingly. The number of tokens used by the system role content property depends on the length of your instruction set. Basically, all content sent to the OpenAI service "counts" as tokens sent and responses requested (e.g. completions) also "count".

    I am checking on your other ask internally if this is possible in GPT 3.5 version and get back to you.

    Thanks

    Saurabh

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.