View source for 2024:LLM Connector (Repository Option)

{{AutoVersion}}

<blockquote>{{#lst:Glossary|LLM Connector}}</blockquote>

== About ==
The only configurable property on the '''''LLM Connector''''' is '''''Service Providers'''''. This property is a collection of '''''LLM Providers''''' which contain properties that establish a connection an [https://en.wikipedia.org/wiki/Large_language_model LLM] [https://en.wikipedia.org/wiki/API API].
<div style="padding-left: 1.5em;">
=== Azure Provider ===
[[image: 2024_LLM-Connector_00_01.png]]

The main property of concern here is '''''Deployments'''''. This property is a collection one or more '''''Deployments''''' specific to the Azure LLM API.
<div style="padding-left: 1.5em;">
==== Chat Service ====
[[image: 2024_LLM-Connector_00_02.png]]

''More information coming soon...''

==== Embeddings Service ====
[[image: 2024_LLM-Connector_00_03.png]]

''More information coming soon...''
</div>

=== OpenAI Provider ===

{|class="fyi-box"
|
'''FYI'''
|
In order to connect Grooper to OpenAI webservices, you will need an OpenAI API key.
# Create or sign in to an OpenAI account at https://platform.openai.com.
# Navigate to the "API keys" page (https://platform.openai.com/api-keys)
# Select the "Create new secret key" button. You may optionally name the key.
# Make sure to save this somewhere safe and do not share it with anyone.
|}

{|class="attn-box"
|
&#9888;
|
'''''BE AWARE:''''' You ''must'' have a payment method in your OpenAI account to use LLM Constructs (such as '''''AI Extract''''') in Grooper. If you do not have a payment method, Grooper cannot return a list of models when configuring an LLM Construct's '''''Model''''' property.
|}

[[image: 2024_LLM-Connector_00_04.png]]
<div style="padding-left: 1.5em;">
<big>URL</big><br>
This property is the URL at which [https://en.wikipedia.org/wiki/OpenAI OpenAI] services are hosted. This property will typically be left at the default setting, as most users will use OpenAI's hosted service. If you are hosting a [https://en.wikipedia.org/wiki/ChatGPT ChatGPT] clone, you would use the URL to your local clone.

<big>Authorization</big><br>
This property controls how the API key is handed to the requested service.
* '''None''': No API key required.
* '''Bearer''': "Bearer" authorization is a type of authentication scheme used in [https://en.wikipedia.org/wiki/Basic_access_authentication HTTP authentication] protocols. In this scheme, a "bearer token" is sent in the [https://en.wikipedia.org/wiki/HTTP HTTP] header to authenticate requests to protected resources. The token acts as a key, and the server validates it to allow access. The format of the authorization header is typically:<pre>Authorization: Bearer <token></pre>
: This method is widely used in modern web services and APIs, particularly in [https://en.wikipedia.org/wiki/OAuth#OAuth_2.0 OAuth 2.0].
* '''API Key''': This is a "Microsoft-style" API key authentication. "Microsoft-style API key authentication" refers to an authentication method where clients include an API key in the HTTP request header to access Microsoft services or APIs. This key acts as a unique identifier for the client and authorizes their access to the requested resources. The typical format of the authorization header is:<pre>Ocp-Apim-Subscription-Key: <api-key></pre>
: This method is simple to implement and is used to track and control how the API is used.

<big>API Key</big><br>
This property is the API key to use when accessing web services. This is the main property to consider when connecting to the OpenAI API. [https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key Obtain an API key from OpenAI] and insert it here.

<big>Use System Messages</big><br>
This property specifies whether to use "system" messages for contextual information, or use "user" messages for all interaction. This is off by default with the assumption you may choose to connect to LLMs like [https://en.wikipedia.org/wiki/Mistral_AI Mistral] or [https://en.wikipedia.org/wiki/Llama_(language_model) Llama], as they do not use system messages.
In the context of large language models, such as those used in conversational AI systems, "system messages" and "user messages" serve distinct roles:
* '''System Messages''': These are instructions or context provided by the system to guide the behavior and responses of the language model. They often include information about the conversation's tone, style, and specific tasks the model should perform. For example, a system message might specify that the model should respond in a formal tone or prioritize certain types of information.
* '''User Messages''': These are inputs or queries provided by the user. They represent the questions, commands, or prompts that the user wants the model to respond to. The model processes these messages to generate appropriate and relevant responses based on the given instructions and context.
Together, system and user messages facilitate dynamic and contextually appropriate interactions between the user and the language model.
</div>
</div>

==== OpenAI compatible services ====

Grooper can connect to any LLM service that adheres to the OpenAI API standard using the '''''OpenAI''''' provider.

We have confirmed the following services will integrate using the '''''OpenAI''''' provider:
* Various models using [https://groq.com/ Groq]
* Various models using [https://openrouter.ai/ OpenRouter]
* Various models hosted locally using [https://lmstudio.ai/ LMStudio]
* Various models hosted locally using [https://ollama.com/ Ollama]

'''''BE AWARE:''''' While all OpenAI is fully compatible with all LLM constructs in Grooper, these services may only have partial compatibility using the '''''OpenAI''''' provider.

== How To ==
<div style="padding-left: 1.5em;">
=== Establish an Open AI Provider ===
# Click on the '''Grooper''' '''Root''' node.
# Click the ellipsis button for the '''''Options''''' property.
# In the "Options" window click the "Add" button.
# Choose "LLM Connector" from the drop-down menu.

[[image: 2024_AI-Extract_01_01.png]]


# <li value=5> Click the ellipsis button for the "Service Providers" property of the newly added '''''LLM Connector'''''.</li>
# Click the "Add" button in the "Service Providers" window.
# Choose the '''''Open AI Provider''''' option from the drop-down menu.

[[image: 2024_AI-Extract_01_02.png]]


# <li value=8>Enter your API Key into the '''''API Key''''' property.</li>
#* [https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key Obtain an API key from OpenAI]

[[image: 2024_AI-Extract_01_03.png]]
</div>