OpenAI GPT 3.5 save/recall system content

Question

I constructed a rather detailed (large) instruction set for use in the System Role Content property when using the OpenAI chat service. I'm wondering 2 things: does this property count as tokens processed? I.e. is this part billed? And if so, how do make it so that I only have to provide this information once and have the chat maintain the instructions in future prompts? I am using the Rest API and would prefer to do any configuration using powershell/REST api if possible.

Answer 1

Hi @Jelle Holtkamp ,

I have checked internally on your second ask; it is not possible as this not how LLM's work as it cannot "remembers" the initial conversation. You can however work towards creating prompts with Chain of Thought Prompting to lower the possibility of inaccurate outcomes.

Please let me know if you have any other questions.

Thanks

Saurabh

---

Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.

Answer 2

Hi @Jelle Holtkamp ,

Thanks for using Microsoft Q&A!!

Yes, the system role content property counts as tokens processed and is billed accordingly. The number of tokens used by the system role content property depends on the length of your instruction set. Basically, all content sent to the OpenAI service "counts" as tokens sent and responses requested (e.g. completions) also "count".

I am checking on your other ask internally if this is possible in GPT 3.5 version and get back to you.

Thanks

Saurabh