Grooper and AI

From Grooper Wiki
(Redirected from GrooperAI)

Grooper's groundbreaking AI-based integrations make getting good data from your documents a reality with less setup than ever before.

Artificial intelligence and machine learning technology has always been a core part of the Grooper platform. It all started with ESP Auto Separation's capability to separate loose pages into discrete documents using trained examples. As Grooper has progressed, we've integrated with other AI technologies to improve our product.

Grooper not only offers Azure's machine-learning based OCR engine as an option, but we improve its results with supplementary data from traditional OCR engines. This gives you the "best of both worlds": Highly accurate character recognition and position data for both machine printed and hand written text with minimal setup.

In version 2024, Grooper added a slew of features incorporating cutting edge AI technologies into the platform. These AI based features accelerate Grooper's design and development. The end result is easier deployments, results with less setup and, for the first time in Grooper, a document search and retrieval mechanism.

In version 2025, we introduced AI Assistants, a conversational AI that is knowledgeable about your documents and more. With AI Assistants, users can ask a chatbot persona questions about resources it is connected to, including documents, databases and web services.


Grooper's AI integrations and features include:

  • Azure OCR - A machine learning based OCR engine offered by Microsoft Azure.
  • Large language model (LLM) based data extraction at scale
  • A robust and speedy document search and retrieval mechanism
  • Document assistants users can chat with about their documents


In this article, you will find:

Relevant article links

Here, you will find links to all currently available articles pertaining to Grooper AI functionality.

Azure OCR

  • Azure OCR - An OCR Engine option for OCR Profiles that utilizes Microsoft Azure's Read API. Azure's Read engine is an AI-based text recognition software that uses a convolutional neural network (CNN) to recognize text. Compared to traditional OCR engines, it yields superior results, especially for handwritten text and poor quality images. Furthermore, Grooper supplements Azure's results with those from a traditional OCR engine in areas where traditional OCR is better than the Read engine.

LLM connectivity and constructs

LLM Connectivity

LLM Connector: A "Repository Option" set on the Grooper Root node. This provides connectivity to LLMs offered by OpenAI and Microsoft Azure. There are currently two connection options:

  • OpenAI - This provider connects Grooper to LLMs offered by the OpenAI API or compatible APIs.
    • Compatible APIs must use "chat/completions" and "embeddings" endpoints similar to OpenAI API to interoperate with Grooper's LLM features.
  • Azure - This provider connects Grooper to LLMs offered by Microsoft Azure in their "Model Catalog" (including Azure OpenAI models).

LLM-enabled extraction capabilities

  • Ask AI: An LLM-based "Value Extractor" specialized for natural language responses. This extractor returns results by passing the document's text and a natural language prompt to a chat completion service (a chatbot). The chatbot uses the document's text and the prompt to respond with an answer. The chatbot's answer is the extractor's result.
  • AI Schema Extractor: An LLM-based "Value Extractor" specialized for structured JSON responses. The AI Schema Extractor enables advanced, schema-driven data extraction from unstructured or semi-structured documents by leveraging generative AI. It is designed for scenarios where precise, reliable, and repeatable extraction of structured data is required, such as tables, line items, or multi-field records.
  • AI Extract: An LLM-based "Fill Method" for large-scale data extraction. Fill Methods are configured on Data Models and Data Sections. They perform a secondary extraction after the child Data Elements extractors and extract methods execute. Or, they can act as the primary extraction mechanism if no extractors or extract methods are configured. AI Extract extracts the Data Model using an LLM chat completion service. In many cases, the only configuration required is Data Elements added to the model. AI Extract fills in the Data Model based on what its Data Elements are named.
  • LLM-enabled Data Section Extract Methods
    • AI Collection Reader: An LLM-based Section Extract Method for multi-instance section extraction. The AI Collection Reader extends the capabilities of AI Section Reader to multi-instance Data Sections, which represent repeating records inside a document. Note that it is also possible to extract multi-instance Data Sections using the AI Extract fill method. The main difference is that AI Collection Reader is optimized for processing large multi-page documents which need to be processed in chunks to avoid exceeding the context length large language models (LLMs).
    • AI Section Reader: An LLM-based Section Extract Method for single-instance section extraction. The AI Section Reader provides advanced extraction of single-instance Data Sections from documents using generative AI, leveraging large language models (LLMs) such as OpenAI's GPT series. It is designed to handle complex, variable, or ambiguous document layouts where traditional extraction methods may be insufficient.
    • AI Transaction Detection: An LLM-based Section Extract method specialized for transaction sections found on certain kinds of documents. The AI Transaction Detection extract method enables Grooper to automatically segment documents into individual transactions — such as payroll reports, EOBs, or other document types which contain repeating data structures. It works by detecting consistent features (anchors) that mark the start of each transaction. It can also extract structured data from each detected transaction using generative AI, supporting both simple and highly complex document layouts.
    • Clause Detection: Clause Detection is designed to locate specified types of clauses in natural language documents and return them as section instances in Grooper. The user provides one or more sample clauses, then an embeddings model determines the portion of the document (or "chunk") most similar to the samples, and Grooper returns the paragraph(s) in the chunk. Data Sections using this Section Extract Method can then use AI Extract to return data using only the text-content of the detected paragraph.
  • AI Table Reader: An LLM-based Data Table Extract Method. The AI Table Reader enables advanced extraction of tabular data from documents using generative AI, powered by large language models (LLMs) such as OpenAI's GPT series. It is designed to interpret semi-structured or unstructured content and transform it into structured table instances, even when the table layout is ambiguous or not explicitly formatted.

LLM-enabled separation and classification capabilities

  • AI Separate: An LLM-based Separation Provider. Unlike traditional separation methods that rely on fixed rules, barcodes, or control sheets, AI Separate uses natural language understanding to evaluate the meaning and structure of page content. This enables robust separation even when documents lack consistent separators or when page layouts vary significantly.
  • LLM Classifier: An LLM-based Classify Method. Classification is performed by sending the document’s content and a list of candidate Document Types (with their descriptions) to the LLM. The model analyzes the text and selects the best match, enabling robust classification even for complex or variable documents.
  • Mark Attachments: The Mark Attachments activity assists document separation decisioning by attaching documents to "parent" documents (such as an Exhibit that should be attached to a legal document). Mark Attachments is configured by adding one or more "Attachment Rules". When an Attachment Rule's "Generative AI" option is enabled, an LLM will analyze if a document should be attached to the one before it or after it.

Other LLM-enabled capabilities

  • AI Generator: AI Generators can be used in the Search page (see AI Search and the Search page below). AI Generators use an LLM prompt to create simple text-based documents (TXT, CSV, HTML, etc) from results in the Search page. This can include simple reports, contact lists, summaries and more.
  • AI Productivity Helpers: There are several productivity helpers that use an LLM to assist Grooper Designers. AI Helpers can help users writing regular expressions, entering AI Search queries, build Data Models, and more! A full list of helpers is found on the AI Productivity Helpers page.

LLM related properties and concepts

  • Parameters - These properties adjust how an LLM gives a response. For example, "Temperature" controls the "creativity" or "randomness" of an LLMs response.
  • Quoting Method - This determines what text is fed to an LLM. Depending on the Quoting Method selected and configured, this include the whole document, a portion of the document, layout data, the document's extracted Data Model values, or some combination of this data.
  • Alignment - These properties control how Grooper highlights LLM-based results in a Document Viewer (or how it "aligns" an LLM response to what text on the document was used to give that response).
  • Prompt Engineering - This concept refers to the process of designing and refining natural language prompts to get a desired answer from an LLM chatbot.

AI Search and the Search page

AI Search - AI Search opens up the Search page to Grooper users. AI Search is Grooper's document indexing mechanism. Once an index is created and documents are added to it, users can search for documents and their data from the Search page.

Simple and advanced querying mechanisms provide users flexibility in their search criteria. Users can use the Search page to retrieve documents for research purposes or use a variety of document processing commands to do more work in Grooper.

The AI Search article covers the following topics:

AI Assistants and the Chat page

AI Assistant - AI Assistants open up the Chat page to Grooper users. From the Chat page, users can select an AI Assistant and start asking it questions about resources its connected to. These resources include document content in an AI Search index, databases connected with a Grooper Data Connection and web services that can be connected using a RAML definition.

AI Assistants use a technique called "retrieval augmented generation" (or RAG) to come up with a "retrieval plan" to determine what content from what resource is relevant to the conversation. This allows Grooper users a simple way to interact with documents and data by simply asking questions in a chat session.

OpenAI and Azure OpenAI rate limits

OpenAI rate limits

OpenAI's full rate limit documentation

Rate limits restrict how many requests or tokens you can send to the OpenAI API in a specific time frame. Limits are set at the organization and project level (not per individual user) and vary by model type. Rate limits are measured in several ways:

  • RPM - Requests per minute (the number of calls to the "chat/completions" or "embeddings" endpoints per minute)
  • RPD - Requests per day (the number of calls to the "chat/completions" or "embeddings" endpoints per day)
  • TPM - Tokens per minute (effectively how much text is sent and received per minute)
  • TPD - Tokens per data (effectively how much text is sent and received per day)
  • Usage limit (OpenAI limits the total amount an organization can spend on the API per month)

OpenAI places users in "usage tiers" based on how much they spend monthly with OpenAI and how old their account is. As usage tiers increase, so does the rate limit thresholds for each model.

Tier Qualification Usage limits
Free User must be in an allowed geography $100 / month
Tier 1 $5 paid $100 / month
Tier 2 $50 paid and 7+ days since first successful payment $500 / month
Tier 3 $100 paid and 7+ days since first successful payment $1,000 / month
Tier 4 $250 paid and 14+ days since first successful payment $5,000 / month
Tier 5 $1,000 paid and 30+ days since first successful payment $200,000 / month

Users will automatically move up in usage tiers the more they spend with OpenAI. Users may also contact OpenAI support to request an increase in usage tier.

You can view your current usage tier and rate limits in the Limits page of the OpenAI Platform.

  • For Grooper users just getting started connecting to the OpenAI API, this can cause large volumes of work to time out. Each time Grooper executes one of its LLM-based features (such as an AI Extract operation), it sends a request to OpenAI (these count toward your requests per minute/day), handing it document text and system messages (these count toward your tokens per minute/day).

Azure OpenAI rate limits

Azure AI Foundry full rate limit documentation.

Azure OpenAI models are deployed with Azure AI Foundry. This allows users to deploy OpenAI models outside of OpenAI's infrastructure as a service you manage in your Azure environment. They are still rate limited. However, in general, users are able to achieve greater throughput deploying Azure OpenAI models than using the OpenAI API. This is especially the case for new OpenAI accounts.

Quotas and limits aren't enforced at the tenant level. Instead, the highest level of quota restrictions is scoped at the Azure subscription level. Rate limits are similar to OpenAI API's rate limts:

  • RPM - Requests per minute (the number of calls to the "chat/completions" or "embeddings" endpoints per minute)
  • TPM - Tokens per minute (effectively how much text is sent and received per minute)


Several different factors affect your rate limits.

  • Global vs Data Zone deployments
    • Global deployments have larger rate limits and are better suited for high-volume workloads.
    • Example: Global Standard gpt-4.1 (Default) = 1M tokens per minute & 1K requests per minute
    • Example: Data Zone Standard gpt-4.1 (Default) = 300K tokens per minute & 300 requests per minute
  • Subscription tier
    • Enterprise and Microsoft Customer Agreement - Enterprise (MCA-E) vs Default
    • Enterprise and MCA-E agreements have larger rate limits than the default "pay-as-you-go" style agreement. The Default tier is best suited for testing and small teams, with MCA-E better suited for mid-to-large organizations and Enterprise is best for large organizations, especially those in regulated industries.
    • Example: Global Standard gpt-4.1 (Enterprise and MCA-E) = 5M tokens per minute & 5K requests per minute
    • Example: Global Standard gpt-4.1 (Default) = 1M tokens per minute & 1K requests per minute
  • Region
    • Rate limits are defined per region (e.g. South Central US, East US, West Europe, etc.)
    • However, you are not limited to a single global quota. You can deploy models in multiple regions to effectively increase your throughput (as long as your subscription supports it).
  • Model
    • Similar to OpenAI, rate limits differ depending on the model you use.
    • Example: Global Standard gpt-4.1 (Enterprise and MCA-E) = 5M tokens per minute & 5K requests per minute
    • Example: Global Standard gpt-4o (Enterprise and MCA-E) = 30M tokens per minute & 180K requests per minute

Does OpenAI or Azure AI Foundry save my data or use it for training?

For OpenAI models
Grooper integrates with OpenAI API not Chat GPT. When using the OpenAI API, your data (prompts, completions, embeddings, and fine-tuning data) is not used for training to improve OpenAI models (unless you explicitly opt in to share data with OpenAI). Your data is not available to other customers or other third parties.
All data passed to and from OpenAI (prompts, completions, embeddings, and fine-tuning data) is encrypted in transit.
Data is saved in the case of fine-tuning data for your own custom models. Fine-tuned models are available to you and no one else (without your consent). All stored fine-tuning data may be deleted at your discretion. All stored data is encrypted at rest. The OpenAI API may store logs for up to 30 days for abuse monitoring. However, they offer a "zero data retention" option for trusted customers with sensitive applications. You will need to contact the OpenAI sales team for more information on obtaining a zero data retention policy.
For Azure AI Foundry Models (including Azure OpenAI models)
Azure AI models are deployed in Azure resources under your control in your tenant. Models are deployed in Azure and operate as a service under your control. Your data (prompts, completions, embeddings, and fine-tuning data) is not available to other customers, OpenAI, or other third parties. Your data is not used for training to improve models by Microsoft, OpenAI or any other third parties with out your permission or instruction.
All data passed to and from the model service (prompts, completions, embeddings, and fine-tuning data) is encrypted in transit.
Some data is saved in certain cases, such as data saved for fine-tuning your own custom models. All stored data is encrypted at rest. All data may be deleted at your discretion. Azure will not store prompts and completions without enabling features that do so. Azure OpenAI may store logs for up to 30 days for abuse monitoring purposes, but this can be disabled for approved applications.

OpenAI and Azure account setup

For Azure OCR connectivity

Grooper can implement Azure AI Vision's Read engine for OCR. "Read" is a machine learning based OCR engine used to recognize text from images with superior accuracy, including both machine printed and handwritten text.

Azure OCR Quickstart

In Azure:

  1. If not done so already, create an Azure account.
  2. Go to your Azure portal
  3. If not done so already, create a "Subscription" and "Resource Group."
  4. Click the "Create a resource" button
  5. Search for "Computer Vision"
  6. Find "Computer Vision" in the list click the "Create" button.
  7. Follow the prompts to create the Computer Vision resource.
  8. Go to the resource.
  9. Under "Keys and endpoint", copy the API key.

In Grooper

  1. Go to an OCR Profile in Grooper.
  2. For the OCR Engine property, select Azure OCR.
  3. Paste the API key in the API Key property.
  4. Adjust the API Region property to match your Computer Vision resource's Region value
    • You can find this by going to the resource in the Azure portal and inspecting its "Location" value.
External resources

For Azure AI Search connectivity

Grooper uses Azure AI Search's API to create a document search and retrieval mechanism.

Azure AI Search Quickstart

In Azure:

  1. If not done so already, create an Azure account.
  2. Go to your Azure portal
  3. If not done so already, create a "Subscription" and "Resource Group."
  4. Click the "Create a resource" button
  5. Search for "Azure AI Search"
  6. Find "Azure AI Search" in the list click the "Create" button.
  7. Follow the prompts to create the Azure AI Search resource.
    • The pricing tier defaults to "Standard". There is a "Free" option if you're just wanting to try this feature out.
  8. Go to the resource.
  9. In the "Essentials" panel, copy the "Url" value. You will need this in Grooper.
  10. In the left-hand navigation panel, expand "Settings" and select "Keys"
  11. Copy the "Primary admin key" or "Secondary admin key." You will need this in Grooper.

In Grooper:

  1. Connect to the Azure AI Search service by adding the "AI Search" option to your Grooper Repository.
  2. From the Design page, go to the Grooper Root node.
  3. Select the Options property and open its editor.
  4. Add the AI Search option.
  5. Paste the AI Search service's URL in the URL property.
  6. Paste the AI Search service's API key in the API Key property.
  7. Full documentation on Grooper's AI Search capabilities can be found in the AI Search article.
External resources

For LLM connectivity

Grooper can integrate with OpenAI and Microsoft Azure's LLM models to implement various LLM-based functionality. Before you are able to connect Grooper to these resources, you will need accounts setup with these providers. For information on how to set up these accounts/resources, refer to the following links:

OpenAI LLM Quickstart

In OpenAI:

  1. If you have not done so already, create an OpenAI account.
  2. Go to the API keys section of the OpenAI Platform.
  3. Press the "Crate new secret key" button.
  4. Follow the prompts to create a new key.
  5. Copy the API key. You will not be able to view/copy it again.

In Grooper:

  1. Connect to the OpenAI API by adding an "LLM Connector" option to your Grooper Repository.
  2. From the Design page, go to the Grooper Root node.
  3. Select the Options property and open its editor.
  4. Add the LLM Connector option.
  5. Select the Service Providers property and open its editor.
  6. Add an OpenAI provider.
  7. Paste the OpenAI API key in the API Key property.
  8. Refer to the following articles for more information on functionality enabled by the LLM Connector:
External resources

Azure LLM Quickstart

Microsoft Azure offers access to several LLMs through their "Model Catalog". This includes Azure OpenAI models, Mistral AI models, Meta's Llama models, and more. To access a model from the catalog in Grooper, it must first be deployed in Azure from Azure AI Foundry (formerly Azure AI Studio).

This advice is intended for users wanting to deploy the Azure OpenAI models in the Azure model catalog and use them in Grooper. We will instruct you how to use Azure AI Foundry (formerly Azure AI Studio) to deploy OpenAI models. Deploying other chat completions and embeddings models in the Azure model catalog may require additional setup in Azure.

  • Please Note: Azure OpenAI integration with Grooper is different from OpenAI. Each model needs to be deployed in Azure to access it in Grooper (where the OpenAI provider allows you to select all models in Grooper).

In Azure:

Create an "Azure AI project" in Azure AI Foundry (formerly Azure AI Studio)

  1. If not done so already, create an Azure account.
  2. If not done so already, create a "Subscription" and "Resource Group" in the Azure portal
  3. Go to Azure AI Foundry (formerly Azure AI Studio).
  4. From the Home page, press the "New project" button.
  5. A dialogue will appear to create a new project. Enter a name for the project.
  6. If you don't have an Azure AI hub or "hub", select "Create a new hub"
  7. Follow the prompts to create a new hub.
    • When selecting your "Location" for the hub, be aware Azure AI services availability differs per region. Certain models may not be available in certain regions.
    • When selecting "Connect Azure AI Services or Azure OpenAI", you can either:
      • Connect to an existing Azure OpenAI or Azure AI Services resource.
      • Or, you can create one.
      • If you want access to all models from all providers (not just OpenAI) you must use Azure AI Services.

Select and deploy an LLM model from your Azure AI hub

  1. From your hub, select "Models + endpoints"
  2. Press the "Deploy model" button.
  3. Select "Deploy base model".
  4. Choose the OpenAI model of your choice.
    • Grooper's integration with Azure's Model Catalog uses either "chat completion" or "embeddings" models depending on the feature. To better narrow down your list of options, expand the "Inference tasks" dropdown and select "Chat completion" and/or "Embeddings."
    • Embeddings models are required when enabling "Vector Search" for an Indexing Behavior, when using the Clause Detection section extract method, or when using the "Semantic" Document Quoting method in Grooper.
    • If you connected with Azure AI Services, you will have access to all models in Azure's model catalog. However, non-OpenAI models (such as mistral or Llama) may have additional setup required.
  5. Press the "Confirm" button.
  6. Press the "Deploy" button.

Get the necessary information you'll need in Grooper

  1. The model will be deployed in your Azure AI project. In your project, go to "Models + endpoints".
  2. Select the OpenAI model you deployed.
  3. Copy the "Target URI". You will need to enter this in Grooper.
  4. Copy the "Key". You will need to enter this in Grooper.
    • Notice as well the "Authentication type" is "Key". This will be important when configuring Grooper too.


In Grooper:

  1. Connect to the Azure model deployment by adding an "LLM Connector" option to your Grooper Repository.
  2. From the Design page, go to the "Grooper Root" node.
  3. Select the "Options" property and open its editor.
  4. Add the "LLM Connector" option.
  5. Select the "Service Providers" property and open its editor.
  6. Add an "Azure" provider.
  7. Select the "Deployments'" property and open its editor.
  8. Add a "Chat Service", "Embeddings Service", or both.
    • "Chat Services" are required for most LLM-based features in Grooper, such as AI Extract and AI Assistants. "Embeddings Services" are required when enabling "Vector Search" for an Indexing Behavior, when using the Clause Detection section extract method, or when using the "Semantic" Document Quoting method in Grooper.
  9. In the "Model Id" property, enter the model's name (Example: "gpt-35-turbo").
  10. In the "URL" property, enter the "Target URI" from Azure
    • For Azure OpenAI model deployments this will resemble:
      • https://{your-resource-name}.openai.azure.com/openai/deployments/{model-id}/chat/completions?api-version={api-version} for Chat Service deployments
      • https://{your-resource-name}.openai.azure.com/openai/deployments/{model-id}/embeddings?api-version={api-version} for Embeddings Service deployments
    • For other models deployed in Azure AI Foundry (formerly Azure AI Studio) this will resemble:
      • https://{model-id}.{your-region}.models.ai.azure.com/v1/chat/completions for Chat Service deployments
      • https://{model-id}.{your-region}.models.ai.azure.com/v1/embeddings for Chat Service deployments
  11. Set "Authorization" to the method appropriate for the model deployment in Azure.
    • How do I know which method to choose? In Azure, under "Keys and Endpoint", you'll typically see
      • "API Key" if the model uses API key authentication (Choose "ApiKey" in Grooper).
      • "Microsoft Entra ID" if the model supports token-based authentication via Microsoft Entra ID (Choose "Bearer" in Grooper).
    • Azure OpenAI supports both API Key and Microsoft Entra ID (formerly Azure AD) authentication. Azure AI Foundry (formerly Azure AI Studio) models often lean toward token-based (Bearer in Grooper) authentication.
  12. In the "API Key" property, enter the Key copied from Azure.
  13. (Recommended) Turn on "Use System Messages" (change the property to "True").
    • When enabled, certain instructions Grooper generates and information handed to a model are sent as "system" messages instead of "user" messages. This helps an LLM distinguish between contextual information and input from a user. This is recommended for most OpenAI models, but may not be supported by all compatible services.
  14. Press "OK" buttons in each editor to confirm changes and press "Save" on the Grooper Root.
  15. Refer to the following articles for more information on functionality enabled by the LLM Connector:


External resources
Retrieved from "https://wiki.grooper.com/index.php?title=Grooper_and_AI&oldid=31330"