Share via

Best Way To Give OpenAI Assistant Regular Access To Most Recent Error Logs?

John Moto 46 Reputation points
2024-05-28T18:38:24.06+00:00

I am looking at regularly giving an OpenAI Assistant access to information about our servers and services, such as error logs, Application Insights, and more. Since this information changes every day, I would have to regularly give the Assistant the most recent information (probably daily). I cannot find good information about the best way to do this.

It looks like there are three primary ways this could be done:

  1. Create one Assistant. Every morning, delete previous-day CSV files and upload new CSV files.
  2. Create one Assistant. Use function calling to allow it to download logs from our server by users as-needed.
  3. Create a new Assistant every morning. Upload CSV files with the most recent information and force our system to use the new Assistant instead of the previous day's one.

Or is there another option I don't know about yet?

I thought I wanted to upload the log files to the Assistant directly, so the Assistant itself would always have access to them, which should allow any user creating a thread to not have to make additional calls to get the data. This is what I have tried:User's image

That works, but I noticed that the input tokens start jumping to the tens of thousands within a couple questions about that data. Isn't that going to cost a lot very quickly?

On the other hand, the issue with function calling is that each user would have to ask for the data to be loaded before they would have access to it, resulting in at least one function/API call per user to get each type of data in order for the Assistant to utilize it. And wouldn't that still generate a ton of input token usage after it brings back the data from the function call?

Along with the "best way" to do this, which method would be most cost effective?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,094 questions
Accepted answer
  1. AshokPeddakotla-MSFT 35,971 Reputation points Moderator
    2024-05-29T02:39:35.77+00:00

    John Moto Greetings!

    Please below answer for your queries.

    The best method for you will depend on your specific needs and requirements.

    • Creating one Assistant and uploading new CSV files every morning is a good option if you want to keep the same Assistant and have it access the most recent information. However, keep in mind that this method may result in a large number of input tokens being used, especially if the CSV files are large. This could become costly very quickly.
    • Using function calling to allow the Assistant to download logs from your server as-needed is a good option if you want to reduce the number of input tokens being used. However, this method may result in each user having to ask for the data to be loaded before they can access it, resulting in at least one function/API call per user to get each type of data in order for the Assistant to utilize it.
    • Creating a new Assistant every morning and uploading CSV files with the most recent information is also a good option if you want to ensure that the Assistant always has access to the most recent information. However, this method may result in a large number of input tokens being used, especially if you create a new Assistant every day.

    In terms of cost-effectiveness, using function calling to allow the Assistant to download logs from your server as-needed may be the most cost-effective option, as it reduces the number of input tokens being used.

    On the other hand, the issue with function calling is that each user would have to ask for the data to be loaded before they would have access to it, resulting in at least one function/API call per user to get each type of data in order for the Assistant to utilize it. And wouldn't that still generate a ton of input token usage after it brings back the data from the function call?

    Yes, you are correct that using function calling to allow the Assistant to download logs from your server as-needed may result in each user having to ask for the data to be loaded before they can access it, resulting in at least one function/API call per user to get each type of data in order for the Assistant to utilize it. This could result in a large number of input tokens being used, especially if you have a large number of users.

    However, keep in mind that the number of input tokens used will depend on the size of the data being downloaded and the number of users accessing it. If the data is small and only a few users are accessing it, then the number of input tokens used may not be significant. Additionally, you can optimize the function/API calls to minimize the number of input tokens used.

    Overall, using function calling to allow the Assistant to download logs from your server as-needed may still be a cost-effective option, especially if you have a large number of users and the data being downloaded is small.

    I hope this helps! Let me know if you have any further questions

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.