Package com.composum.ai.backend.base.service.chat

Interface GPTChatCompletionService

All Known Implementing Classes:

GPTChatCompletionServiceImpl

public interface GPTChatCompletionService

Raw abstraction of the ChatGPT chat interface, with only the details that are needed.

This does deliberately not use the OpenAI API classes because we want to be able to switch to a different API implementation, and hide their complexity from the rest of the code. If we need special parameters, we will add new methods with more specific function.

Method Summary

Modifier and Type	Method	Description
int	countTokens(String text)	Counts the number of tokens for the text for the normally used model.
List<float[]>	getEmbeddings(List<String> texts, GPTConfiguration configuration)	Calculates embeddings for the given list of texts.
String	getEmbeddingsModel()	Returns the model used for getEmbeddings(List, GPTConfiguration).
String	getSingleChatCompletion(GPTChatRequest request)	The simplest case: give some messages and get a single response.
GPTChatMessagesTemplate	getTemplate(String templateName)	Retrieves a (usually cached) chat template with that name.
String	htmlToMarkdown(String html)	Helper for preprocessing HTML so that it can easily read by ChatGPT.
boolean	isEnabled()	Whether ChatGPT completion is enabled.
boolean	isEnabled(GPTConfiguration gptConfig)	Checks whether isEnabled() and whether gptConfig enables executing GPT calls.
boolean	isVisionEnabled()	Returns true if vision is enabled.
String	markdownToHtml(String markdown)	Opposite of htmlToMarkdown(String).
String	shorten(String text, int maxTokens)	Helper method to shorten texts by taking out the middle if too long.
void	streamingChatCompletion(GPTChatRequest request, GPTCompletionCallback callback)	Give some messages and receive the streaming response via callback, to reduce waiting time.
void	streamingChatCompletionWithToolCalls(GPTChatRequest request, GPTCompletionCallback callback)	Give some messages and receive the streaming response via callback, to reduce waiting time.

Method Detail

getSingleChatCompletion

@Nullable
String getSingleChatCompletion(@Nonnull
                               GPTChatRequest request)
                        throws GPTException

The simplest case: give some messages and get a single response. If the response can be more than a few words, do consider using streamingChatCompletion(GPTChatRequest, GPTCompletionCallback) instead, to give the user some feedback while waiting.

Throws:

GPTException

streamingChatCompletion

void streamingChatCompletion(@Nonnull
                             GPTChatRequest request,
                             @Nonnull
                             GPTCompletionCallback callback)
                      throws GPTException

Give some messages and receive the streaming response via callback, to reduce waiting time. It possibly waits if a rate limit is reached, but otherwise returns immediately after scheduling an asynchronous call.

Throws:

GPTException

streamingChatCompletionWithToolCalls

void streamingChatCompletionWithToolCalls(@Nonnull
                                          GPTChatRequest request,
                                          @Nonnull
                                          GPTCompletionCallback callback)
                                   throws GPTException

Give some messages and receive the streaming response via callback, to reduce waiting time. This implementation also performs tool calls if tools are given in GPTChatRequest.getConfiguration(). It possibly waits if a rate limit is reached, but otherwise returns immediately after scheduling an asynchronous call.

Throws:

GPTException

getTemplate

@Nonnull
GPTChatMessagesTemplate getTemplate(@Nonnull
                                    String templateName)
                             throws GPTException

Retrieves a (usually cached) chat template with that name. Mostly for backend internal use. The templates are retrieved from the bundle resources at "chattemplates/", and are cached.

Parameters:

templateName - the name of the template to retrieve, e.g. "singleTranslation" .

Throws:

GPTException

shorten

@Nonnull
String shorten(@Nullable
               String text,
               int maxTokens)
        throws GPTException

Helper method to shorten texts by taking out the middle if too long. In texts longer than this many tokens we replace the middle with " ... (truncated) ... " since ChatGPT can only process a limited number of words / tokens and in the introduction or summary there is probably the most condensed information about the text. The output has then maxTokens tokens, including the ... marker.

Parameters:

text - the text to shorten

maxTokens - the maximum number of tokens in the output

Throws:

GPTException

htmlToMarkdown

@Nonnull
String htmlToMarkdown(@Nullable
                      String html)

Helper for preprocessing HTML so that it can easily read by ChatGPT.

markdownToHtml

String markdownToHtml(String markdown)

Opposite of htmlToMarkdown(String).

countTokens

int countTokens(@Nullable
                String text)

Counts the number of tokens for the text for the normally used model. Caution: message boundaries need some tokens and slicing text might create a token or two, too, so do not exactly rely on that.

isEnabled

boolean isEnabled()

Whether ChatGPT completion is enabled. If not, calling the methods that access ChatGPT throws an IllegalStateException.

isEnabled

boolean isEnabled(GPTConfiguration gptConfig)

Checks whether isEnabled() and whether gptConfig enables executing GPT calls. (That is currently whether there is an api key either globally or in the gptConfig).

isVisionEnabled

boolean isVisionEnabled()

Returns true if vision is enabled.

getEmbeddings

@Nonnull
List<float[]> getEmbeddings(List<String> texts,
                            GPTConfiguration configuration)
                     throws GPTException

Calculates embeddings for the given list of texts.

Throws:

GPTException

getEmbeddingsModel

String getEmbeddingsModel()

Returns the model used for getEmbeddings(List, GPTConfiguration).