How Can You Tell an LLM to Call an External API?

Large language models (LLMs) like GPT have advanced natural language understanding and generation capabilities. While these models shine in processing and generating text, practical applications often require them to interact with external data sources or services. One way to achieve this is instructing an LLM to call an external API. This article explores how to design and implement such interactions effectively.

What Does It Mean for an LLM to Call an API?

An LLM, in its base form, cannot execute code or directly make network requests like calling an API. Instead, it generates text based on patterns learned during training. For an LLM to “call” an API, it generally means the model outputs instructions or parameters that an external program can interpret to make the actual API request.

The interaction between the LLM and the external system involves two parts:

1. Text generation by the LLM: The model generates the desired API call syntax, parameters, or a structured command.
2. Execution by an external system: A separate program reads the LLM's output, performs the API call, and potentially sends the results back for further processing.

How to Prompt an LLM for an API Call

To guide the model in generating API call requests, prompts should be clear and structured. Here’s how you can approach it:

1. Define the API and Its Parameters in the Prompt

Provide detailed information about the API, such as endpoint URLs, expected parameters, and response formats. This primes the model to produce usable requests.

Example prompt snippet:

HtmlCopy code

2. Use Structured Output Formats

Encourage the model to respond in machine-readable formats such as JSON or XML. This reduces ambiguity and eases parsing.

Example expected output from LLM:

JsonCopy code

3. Specify the Nature of the API Call

Clarify if the HTTP method should be GET or POST and whether headers or authentication tokens are required. This helps the model generate complete calls.

Example:

HtmlCopy code

Integrating LLM Outputs with External Systems

Since the model only generates text, a system must interpret its outputs and handle the actual API calls.

1. Parse the Model’s Response

Once the LLM returns a structured response, parse it using your preferred programming language. For example, parse JSON strings into dictionaries or objects.

2. Perform the API Request

Use the parsed data to construct an HTTP request using libraries such as requests in Python or fetch in JavaScript. Send the request and capture the API response.

3. Optional: Feed API Response Back to the LLM

Applications often require the LLM to analyze the API response or generate follow-up content. Pass the API response to the model as additional context in new prompts.

Practical Example: Weather Query Assistant

Imagine creating a chatbot that provides weather updates. The LLM generates API calls to a weather service based on user requests.

1. User: "What’s the weather in London tomorrow?"
2. LLM output:

JsonCopy code

3. External code makes the API call and receives weather data.
4. The bot combines the API data with text generation to reply: "The forecast for London on June 2nd is sunny with a high of 22°C."

Handling Complex API Interactions

Some APIs require multiple sequential calls or dynamic parameters based on previous results.

Chain of Calls

Prompt the LLM to output multiple API call batches, specifying the order or dependence between them.

Error Responses

The external system should detect API errors or unexpected results and optionally prompt the LLM to retry or generate alternative queries.

Tools for API Integration with LLMs

Several frameworks facilitate API calling from language models. Some provide built-in support for structured outputs or enable defining functions that the model can invoke. Keeping API descriptions concise and unambiguous makes integration smoother.

LLMs do not directly execute code but excel in generating structured instructions suitable for external API calls. Design prompts to provide clear API details and request structured outputs. Use an external interpreter to handle the actual API communication. This collaborative approach unlocks powerful applications where language models and real-time external data combine seamlessly.