2023.1:Classification (Concept)
|
This article is about an older version of Grooper. Information may be out of date and UI elements may have changed. | |
| 2025 | 2023.1 |
Classification is the process of identifying and organizing documents into categorical types based on their content or layout. Classification is key for efficient document management and data extraction workflows. Grooper has different methods for classifying documents. These include methods that use machine learning and text pattern recognition. In a Grooper Batch Process, the Classify Activity will assign a Content Type to a folder Batch Folder.
About
As far as Grooper is concerned a document is a Batch Folder objects with Batch Page objects as its children. Before classification, the document (Batch Folder) is unclassified, or "blank". Grooper doesn't know what kind of document it is yet. To give an example, you can have both an invoice and a purchase order within your Batch, and Grooper won't know the two, never mind their classification, unless you perform Classification. Documents are classified by:
- Most often, the Classify activity using training data or rules set on a Content Model
- A Classify step will automate document classification in a Batch Process. During the Classify activity, Grooper will use information from the document and its pages and configurations from a Content Model (such as the Classification Method used) to assign the document a Document Type from a Content Model.
- In some cases, the Separate activity by assigning a Document Type to each new folder created
- For example, the ESP Auto Separation Separation Provider is a classification-based method of separation. It will both separate pages into document folders and classify the documents during the Separate activity.
- Manually assigning a Document Type by right-clicking a Batch Folder and using the "Apply Document Type" command.
Classification and Data Extraction
Why is Classification so important? What does it have to do with data and the Batch Process? Classification is performed before data extraction, and is actually a critical part of data extraction. Data extraction executes using configured Data Elements in a Data Model. A Data Model is part of a Content Model's hierarchy. Therefore, a document must be assigned a Content Type (specifically a Document Type of a Content Model) in order for the Extract activity to see the Data Models specifications for data extraction.
Until a document is classified, it has no Content Type assigned to it. It doesn't know which Content Model and corresponding Document Types and Data Models you're using to extract data. Without this information, Grooper will not understand which Data Elements to look for on which Document Types. Nor will it know the the extractors used to return values to the Data Elements in a Data Model.
In other words, the document must be classified (having a Document Type assigned to it) before performing the Extract activity.
Classification Methods
A document can be classified in a variety of ways, through training examples of a Document Type and matching similarity to the training data or creating extractor based rules using key words phrases or other text data (or a combination of the two). The method you choose is determined by the Classification Method property of a Content Model. There are four Classification Methods available in Grooper
|
⚠ |
In our documentation you may read about a "rules based" or "training based" classification approaches.
|
Each of the four different methods are described below. For further details, please click the links above to their respective articles that discuss each method in length.
Lexical
Lexical Classification is a particular Classification Method that relies upon a document's text. Naturally, OCR must be run beforehand, so that Grooper can read the text.
- To choose this method, simply go to your Content Model.
- Expand the Classification Method property.
- Select Lexical.
The Lexical Classification method is ideal to use for documents that can be classified by the type of verbiage they use. This is best exemplified for documents that do not have labels or obvious titles to help a user classify them. For example, something like a cover letter would make good use of Lexical Classification. While it doesn't have anything the other three methods can make use of, it does have several words that relate to employment. You can use this as a kind of key that will tell Grooper what words it needs to look for, and if they appear at a certain frequency, then that document will need to be classified as a cover letter.
Rules Based
Some documents have ways of identifying themselves — this could be through certain labels, or a title. For example, an invoice will be titled as such and have helpful labels such as "Invoice Date", or "Invoice Number."
Grooper can be trained to look for these details, and use them as rules by which to classify documents. In order to do this, we set up what's called a Positive Extractor on the Document Type. As long as the Positive Extractor returns at least one result, then a document will be classified as a certain Document Type. For more information on the Positive Extractor, see here: [1]
Visual
|
Visual Classification is different. Instead of relying on textual information, Visual Classification relies on how a document looks. More specifically, the arrangement of its pixels. |
|
|
This can be useful for classifying documents that have a table-like structure. For example: |
|
Unfortunately, the weakness of Visual Classification comes into play when you have two documents that are similar in their pixel arrangement. If two documents are similar in appearance, then they could be classified as the same document, regardless of whether or not they actually are.
Labelset-Based
Labels are a way that documents organize their information. Of course, not all documents are exactly alike. Two health insurance forms from two different companies will have the same exact information, but their labels might be different. One insurance company may refer to a client's identification as an "Insurance Number", while another may refer to it as "Information No."
|
For example, take a look at these invoices and how they label their invoice numbers. |
|
|
The information falls under the same category, but the labels are different. If we were looking to classify these documents by their company, then we would use Labelset-Based Classification to identify documents using "Invoice Number." as NormanDog, and those using "Invoice #" as Alchemical.
For more detailed information on Labelset-Based Classification, please see the Labelset section of our Labeling Behavior article.
Glossary
Activity: Activity is a property on edit_document Batch Process Steps. Activities define specific document processing operations done to a inventory_2 Batch, folder Batch Folder, or contract Batch Page. Batch Process Steps configured with specific Activities are frequently referred by the name of the Activity followed by the word "step". For example: Classify step.
Batch Folder: folder Batch Folder objects are defined as container objects within a inventory_2 Batch that are used to represent and organize both folders and pages. They can hold other Batch Folders or contract Batch Page objects as children. The Batch Folder acts as an organizational unit within a Batch, allowing for a structured approach to managing and processing a collection of documents.
- Batch Folders are frequently referred to simply as "documents".
Batch Page: contract Batch Page objects represent individual pages within a inventory_2 Batch. The Batch Page object is the most granular unit in the hierarchy of Batch Objects in Grooper.
- Batch Pages are frequently referred to simply as "pages".
Batch Process: settings Batch Process objects are crucial components in Grooper's architecture. A Batch Process orchestrates the document processing strategy and ensures each inventory_2 Batch of documents is managed systematically and efficiently.
- Batch Processes by themselves do nothing. Instead, the workflows they execute are designed by adding child edit_document Batch Process Steps.
- A Batch Process is often referred to as simply a "process".
Batch: inventory_2 Batch objects are fundamental in Grooper's architecture as they are the containers of documents that get moved through Grooper's workflow mechanisms known as settings Batch Processes.
Classification Method: A stacks Content Model's Classification Method property determines the technique used for document classification. Classification sorts folder Batch Folders into categories (called "description Document Types"). Grooper's various Classification Methods can utilize text-based pattern matching, machine learning models, or other methodologies to identify and organize documents accurately.
Classification: Classification is the process of identifying and organizing documents into categorical types based on their content or layout. Classification is key for efficient document management and data extraction workflows. Grooper has different methods for classifying documents. These include methods that use machine learning and text pattern recognition. In a Grooper Batch Process, the Classify Activity will assign a Content Type to a folder Batch Folder.
Classify: unknown_document Classify is an Activity that "classifies" folder Batch Folders in a inventory_2 Batch by assigning them a Content Type (e.g. a description Document Type) using patterns, lexical understanding, or rules as defined by a stacks Content Model.
Content Model: stacks Content Model node objects define the taxonomy of document sets in terms of the description Document Type they contain. They also house the Data Elements that appear on each collections_bookmark Content Category and Document Type within them. Content Models serve as the root of a Content Type hierarchy and are crucial for organizing the different types of documents that Grooper can recognize and process.
Content Type: Content Type refers to objects in Grooper used to classify folder Batch Folders. These include: stacks Content Models, collections_bookmark Content Categories, and description Document Types.
Data Element: Data Element refers to the objects in Grooper used to collect data from a document. These include: data_table Data Models, insert_page_break Data Sections, variables Data Fields, table Data Tables, and view_column Data Columns.
Data Model: data_table Data Model node objects serve as the top-tier structure defining the taxonomy for Data Elements and are leveraged during the Extract Activity to extract data from a folder Batch Folders. They are a hierarchy of Data Elements that sets the stage for the extraction logic and review of data collected from documents.
Document Type: description Document Type objects represent a distinct type of document, like an invoice or contract. Document Types are created as children of a stacks Content Model or a collections_bookmark Content Category and are used to classify individual folder Batch Folders. Each Document Type in the hierarchy defines the Data Elements and Behaviors that apply to Batch Folders of that specific classification.
ESP Auto Separation: ESP Auto Separation is a Separation Provider used for document separation. It is unique in that it both separates and classifies documents at the same time. It uses page-level classification training examples (among other things) to determine where to insert document folders in a inventory_2 Batch.
Extract: export_notes Extract is an Activity that retrieves information from folder Batch Folder documents, as defined by Data Elements in a data_table Data Model. This is how Grooper locates unstructured data on your documents and collects it in a structured, usable format.
Labeling Behavior: A Labeling Behavior is a Content Type Behavior designed to collect and utilize a document's field labels in a variety of ways. This includes functionality for classification, field extraction, table extraction, and section extraction.
Labelset-Based: Labelset-Based is a Classification Method that leverages the labels defined via a Labeling Behavior to classify folder Batch Folders.
Lexical: The Lexical Classification Method classifies folder Batch Folders based on the text content of trained document examples. This is achieved through the statistical analysis of word frequencies that identify description Document Types.
Machine: computer Machine node objects represent servers that have connected to the Grooper Repository. They allow for the management of Grooper Service instances and serve as a connection points for processing jobs to be executed on the server hardware. Machine objects are essential for the scaling of processing capabilities and for distributing processing loads across multiple servers.
OCR: OCR is stands for Optical Character Recognition. It allows text on paper documents to be digitized, in order to be searched or edited by other software applications. OCR converts typed or printed text from digital images of physical documents into machine readable, encoded text.
Rules-Based: The Rules-Based Classification Method employs "rules" defined on each description Document Type to classify folder Batch Folders. Positive Extractor and Negative Extractor properties are configured for each Document Type to positively or negatively associate a Batch Folder based on predefined criteria.
Note where the Positive and Negative Extractors will impact all Classification Method results, the Rules-Based method classifies using only these properties and nothing else.
Separate: insert_page_break Separate is an Activity that sorts contract Batch Pages into individual folder Batch Folders. This distinguishes "loose pages" from the documents formed by those pages. Once loose pages are separated into Batch Folder documents, they can be further processed by unknown_document Classify, export_notes Extract, output Export and other Activities that need to run on the folder (i.e. document) level.
Separation Provider: The Provider property of the Separate Activity defines the type of separation to be performed at the designated Scope.
Separation: Separation is the process of taking an unorganized inventory_2 Batch of loose contract Batch Pages and organizing them into documents represented by folder Batch Folders in Grooper. This is done so Grooper can later assign a description Document Type to each document folder in a process known as "classification".
Visual: The Visual Classification Method uses image analysis instead of text data to determine the description Document Type assigned to a folder Batch Folder during classification. Instead of using text-based extractors, an "Extract Features" IP Command in an perm_media IP Profile is used to collect image-based data from a Batch Folder's image(s). This image-based data is compared against that of previously trained document examples of each Document Type to classify the Batch Folder.