2023.1:Separation (Concept)

This article is about an older version of Grooper. Information may be out of date and UI elements may have changed.
2025	2023.1	2023	2.80

Separation is the process of taking an unorganized inventory_2 Batch of loose contract Batch Pages and organizing them into documents represented by folder Batch Folders in Grooper. This is done so Grooper can later assign a description Document Type to each document folder in a process known as "classification".

About

Let's revisit the Five Phases of Grooper.

1. Acquire
   • Either physical pages are scanned into Grooper or digital files are imported into a Batch in Grooper.
2. Condition
   • This involves running Recognize and OCR on the Batch to allow Grooper to read the text and clean up the pages if needed.
3. Organize
   • This is where you separate the pages in the Batch into individual document folders.
   • After the pages have been separated, then the document folders are classified.
4. Collect
   • Data is extracted from the documents.
5. Deliver
   • The extracted data is exported from Grooper to the destination of your choice.

What is Separation?

Imagine you have a bunch of pages and you put them in a box. This is similar to importing pages into a Batch.

Now, let's say you need to look for a specific type of document in the box. It would be difficult to just go through all of the loose pages to find the documents you are looking for. It's difficult to determine where one document ends and another begins.

So, you take those pages and you sort them into folders. Each folder contains one document which is comprised of one or more pages. Now, it is much easier to tell the documents apart!

This is how separation works. It is organizing the pages so that Grooper can identify one document from another.

Why do we need to separate?

The point of separation is so Grooper can later Classify the documents. Classification is the process of assigning Document Types to the document folders. Classification can only be applied to a document folder, not loose pages. So, even if you are only bringing in a single document into Grooper, you would need to make sure the loose pages are contained within a document folder in order to classify.

The point of document classification is to let Grooper know what to do with the documents.

• For example, Grooper won't know what data to extract from a document without a Data Model. Where does that Data Model come from? A Document Type. How does a document folder get a Document Type? Classification!
• For more information on classification, please take a look at our Classification wiki article.

When do we need to separate?

There are two ways to bring documents into Grooper. You can scan physical copies of the pages directly into a Batch or you can import digital documents into Grooper. After you have your documents in your Batch, whether or not you need to separate depends first on whether your documents are scanned or imported. If imported, whether or not they need to be separated depends on if they are "Discreet" or "Packeted" documents.

Scanned Documents

Documents scanned into Grooper always need to be separated. Documents are scanned page by page into a Batch. Separation takes those loose pages and turns them into documents by creating a folder for each document and placing their pages inside.

Imported Documents: Discreet Documents

"Discreet" documents are digital documents where each file contains one document. When a file is imported into Grooper, it is automatically put in its own document folder. If the file only contains one individual document, then it will already be in its own document folder and there is no need to separate.

Imported Documents: Packeted Documents

Sometimes when you import digital documents, each file might contain multiple documents. These are considered "Packeted" documents. In this case, Grooper will bring the file in as a document folder, but it will need to be separated so that each document is contained within its own document folder.

How do we separate?

Separation is a process that happens as part of a Batch Process. You will need to add a Separate step (a Batch Process Step with the Activity property set to Separate) to your Batch Process and configure it with a Separation Provider. A Separation Provider is a property that tells Grooper how we want to run separation on the pages. For example, if each of our documents are four pages each, we might want to tell Grooper to separate every four pages. We could also separate every time Grooper finds a title at the top of a page.

There are 8 different Separation Providers you can configure. Here we are going to give a brief explanation of the provider, but for a deeper understanding you will need to visit each of their articles individually:

• Change in Value Separation - Grooper will separate when it detects a value (that you configure) changes from one page to another, such as an invoice number.
• Control Sheet Separation - During the scanning process, you can make sure to place a Grooper Control Sheet in between each document. Grooper will separate at the point it detects a Control Sheet.
• EPI Separation - Grooper will separate based on extracted page numbers and will detect a new document when the page number resets or when a lower page number comes up in the Batch.
• ESP Auto Separation - One of the more complicated separation techniques involving Lexical training.
  • Since ESP Auto Separation uses a Content Model's training data (as well as classification rules set on its Document Types), it both separates and classifies documents during the Separate activity.
• Event-Based Separation - Grooper will separate based on an "event" such as after X number of pages or any time Grooper encounters a blank page. The following are events that can be configured for this provider:
  • Blank Page - A blank page will trigger a new folder.
  • Barcode - A scanned barcode will trigger a new folder.
  • Content Type - This Separation Event uses Lexical or Visual training examples to trigger folder creation. Whenever a page confidently matches a trained example document's first page, a new folder is created.
  • Page Count - This is for fixed page separation. A new folder is created by a set number of pages for a document.
  • Shape - A new folder is created every time a "shape feature" is detected.
• Multi Separator - This provider allows you to use multiple Separation Providers at once.
• Pattern-Based Separation - Grooper will separate based on text patterns, such as a document title or label.
• Undo Separation - This provider actually turns separated documents back into loose pages.

Text-Based vs. Scan Supported

It is important to note that the majority of Separation Providers are "Text-Based" providers. That means they require readable data from your pages to work. OCR must be performed prior to any Text-Based provider being used. The following are Text-Based providers:

• Change in Value Separation
• EPI Separation
• ESP Auto Separation
• Pattern-Based Separation

"Scan Supported" providers do not need readable data to work. That means it is possible to run these providers as early as when you are scanning in physical pages. OCR is not required prior to running a Scan Supported provider. The following are Scan Supported Providers:

• Control Sheet Separation
• Event-Based Separation