Function Calling With Vand.io and OpenAI Models

November 21, 2023

Augmenting an AI Model with RAG and External Tools has been challenging with the most active development around Retrieval Augmented Generation (RAG) and very little attention given to external tools.

This is starting to change as ChatGPT Plus has (had?) plugins and now “GPTs” which provides augmentation in the form of RAG, two powerful built-in tools (a code interpreter and a web browser) as well as one “Action” or what they used to call a plugin. This is a “function call” under the hood in OpenAI parlance and enables the model to access an external tool (API) for additional information or skills.

For those looking to leverage AI models in their own applications OpenAI created it’s Assistant API which includes RAG and it’s own proprietary tools. While the Assistant API has made many things easier it’s still challenging for many to use and many have found it’s easy to run up large bills.

For those looking for an easier tool and would like the flexibilty of using other models (e.g. LLama based models) and have transparency and control over all interactions with the model there are now other options.

AIAPI - What is it?

AIAPI is a baby step to provide an out-of-the-box library and chat client that provides everything necessary to augment your AI model with external tools and is not tied to any AI model or service. It’s built on top of the vand-python library and the excellent simplaichat library where in inherits several key features including:

  • Easy to use sessions that string together all messages (assistant, user, and function).
  • Save and reload sessions
  • Include your own functions / tools in your application to augment the AI model’s capabilities.
  • Simple access to thousands of tools found on https://vand.io.

Together these are similar to OpenAI’s Assistant API but with several advantages. Here’s a quick look at how these tools compare.

Concept Comparison

Item Assistant API AIAPI
Model Object (define a model, parameters, etc.) Assistant AIChat
Messages Messages Messages
Log of Chat messages Threads Sessions
Executing a model call Run NA
Function Calling Included OpenAI tools or write your own Included Vand tools or write your own

Usage Comparison

Here’s a side-by-side look at how you might use AIAPI or OpenAI’s Assistant API. See a complete example of AIAPI usage on GitHub. See a [cookbook(https://cookbook.openai.com/examples/assistants_api_overview_python)] on using the Assistant API.

Action Assistant API AIAPI

Create an Assistant

 
assistant = client.beta.assistants.create(
    name="Math Tutor",
    instructions="You are a helpful assistant.",
    tools=[{"type": "code_interpreter"}],
    model="gpt-4-1106-preview"
)

assistant = AIChat(
    system="You are a helpful assistant.", 
    model="gpt-3.5-turbo-1106",
    functions=["getWeather"]
)

Create a Thread

 
thread = client.beta.threads.create()
NA. A session (thread) is automatically created.
Add a Message to a Thread 
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="I need to solve the equation " \
       "`3x + 11 = 14`. Can you help me?"
)

message = assistant("New User Message")
Run the Thread 
run = client.beta.threads.runs.retrieve(
  thread_id=thread.id,
  run_id=run.id
)

NA. The session is automaticaly “run” when the new message is added in the step above.
Display the Assistant's Response 
messages = client.beta.threads.messages.list(
  thread_id=thread.id
)
The latest response (assistant message) would be in the message variable if you invoked with message = assistant("New User Message"). To get the entier session (thread) use assistant.pprint_session().
Save the Thread NA. Thread is automatically saved with it's ThreadID Save to local file with: 
assistant.save_session(
    output_path="weather_chat.json", 
    format="json"
)
Load a Thread Need to know the `ThreadID`.

get https://api.openai.com/v1/threads/{thread_id}

 Load from local file with:

assistant.load_session("weather_chat.json")

Learn more about AIAPI and download it and try it out yourself on GitHub.