Augmenting an AI Model with RAG and External Tools has been challenging with the most active development around Retrieval Augmented Generation (RAG) and very little attention given to external tools. This is starting to change as ChatGPT Plus has (had?) plugins and now “GPTs” which provides augmentation in the form of RAG, two powerful built-in tools (a code interpreter and a web browser) as well as one “Action” or what they used to call a plugin.
Augmenting language models with tools dramatically increases their usefulness, accuracy, and capabilities.
OpenAI’s GPT models have had the ability to leverage external tools via Function Calls and with the improved function calling announced with gpt-3.5-turbo-1106 they are more reliable.
Vand is a directory of tools and a simple Python package (vand-python) that makes it easy to find and augment your model with tools via with OpenAI’s models.
As amazing as Large language models (LLMs) are they have some major challenges. They often “hallucinate” or make assertions that sound believable but aren’t actually true. This leads to mistakes in math, challenges combining multiple skills, and some commonsense reasoning chains. Furthermore, many of the impressive capabilities of LLMs are only present in very large models. This large number of parameters and the need for data makes LLMs difficult and expensive to train, run, and keep updated.