2.80:Infer Grid (Table Extract Method)
| ‼ |
LEGACY TECHNOLOGY DETECTED!! Infer Grid was replaced by Grid Layout in version 2021.
|
Infer Grid is one of three Table Extraction methods to extract data from tables on documents. It uses the positional location of row and column headers to interpret where a tabular grid would be around each value in a table and extract values from each cell in the interpreted grid.
This method extracts information by inferring a grid from the row and column header positions. This is done by assigning an X Axis Extractor to match the column headers and, a Y Axis Extractor to match row headers. A grid is created from the header positions extracted from the two extractors.
Furthermore, if table line positions can be obtained from a Line Detection or Line Removal IP Command, only the X Axis Extractor is needed. In these cases, the X Axis Extractor can be used to find the column header labels, and the grid will be created using the table lines in the documents Layout Data. The raw text data obtained from the Recognize activity will populate each cell of the grid according to where it is on the page.
Use Cases
Non-Standard Tables
The Infer Grid method excels at many cases where the table structure is not easily understood by the Row Match or Header-Value methods. This is especially true for tables with table lines present. Examine the table below.
Row Match might work, but it would be a heavy lift. First, each row's pattern is different. There are names on one, addresses on another, phone numbers on another. Every row has a different pattern. It would take some creative configuration. You could try to make a row out of the columns. It would take a series of extractors, be very effort intensive and complicated to set up.
Header-Value would also have problems. The column header labels ("Lender", "Mortgage Broker", etc), would be straightforward. But the value extractors would be tricky. It's possible a generic text segment extractor could get you close, but at least the "Address" row presents problems because it is a two line value instead of a single line. Again, it could be doable, but it would take some effort.
Infer Grid can do this job with a single extractor. All you would need to do is write an extractor to find the "X Axis"; so all the column header labels in a row.
Since table lines are present, the text falling inside each cell (obtained via the Recognize activity could be extracted to the corresponding cell in the column
Furthermore, if table lines are not present, Infer Grid can use both both the row and column header labels by using both the "Y Axis Extractor" and "X Axis Extractor" properties. We can use two extractors, one to return all the Y Axis labels and one to return the X Axis labels, and use their positions to infer the table's structure.
OMR Checkboxes
OMR stands for "Optical Mark Recognition". It is a a way to determine if a checkbox is marked or not on a document. If you think back to your grade school days and remember taking tests and filling in bubbles on an answer sheet, you already have experience with OMR! Those answer sheets are fed through a machine that reads the "checkbox state" of the boxes, either filled in (checked) or not. There are many examples of current documents where checkboxes are used to record a boolean response ("true or false" or "yes or no"), a multiple choice response, or other information. Grooper uses OMR to read those checkbox states.
The Infer Grid method is the easiest way to read checkbox states inside a table. Once the table's structure is found using the axis extractors, you can choose which columns contain checkboxes. Grooper will use Layout Data obtained from a Box Detection or Box Removal IP Command to determine if the box is filled in or left blank. Refer to the tutorial below for more information on how to configure this use.
| Marking the "Farm" and "Simulator" columns as OMR Columns in the Infer Grid Property Panel will return a value of "True" if the box is checked and "False" if it is blank. |
 |
Re-OCRing Tricky Cells
The Infer Grid method also allows you to choose a column and apply a secondary OCR profile to the cells within that column. This is useful for tables that have specialized fonts for values filled inside the cells.
For example, the OCR-A font is not easily read by most modern OCR engines. However, Google's Tesseract OCR engine has some specialized functionality for the font. A document using a column like the one to the left could process most of the document, using an OCR profile that reads conventional fonts, including the column headers such as "Date". Then, the cells inside the grid, containing dates in the OCR-A font, could be reprocessed using another OCR profile that uses the Tesseract engine.
How To
Creating a Data Table in Grooper
Before you begin
A Data Table is a Data Element used to model and extract a table's information on a document. Just like other Data Elements, such as Data Fields and Data Sections, Data Tables are created as children of a Data Model. This guide assumes you have created a Content Model with a Data Model.
We will use the table below as our example for creating a Data Table.
Using the Node Tree on the left side of Grooper Design Studio, navigate to the Data Model you wish to add the Data Table to. Data Tables can be created as children of any Data Model at any hierarchy in a Content Model.
Add a Data Table
Right click the Data Model object, mouse over "Add" and select "Data Table"
The following window will appear. Name the table whatever you would like and press "OK" when finished.
This creates a new Data Table object in the Node Tree underneath the Data Model.
Add Data Columns
Right click the Data Table object, mouse over "Add" and select "Data Column"
This brings up the following window to name the Data Column. When finished, press "OK" to create the object.
This creates a new Data Column object in the Node Tree underneath the Data Model.
Repeat Until Finished
Add as many columns as necessary to complete the table. For our example, we have a single Data Table with five Data Columns, each one named for the corresponding column on the document.
Configure Infer Grid for OMR Checkboxes
| ! | Some of the tabs in this tutorial are longer than the others. Please scroll to the bottom of each step's tab before going to the step. |
A Data Table is a Data Element used to model and extract a table's information on a document. Just like other Data Elements, such as Data Fields and Data Sections, Data Tables are created as children of a Data Model. This guide assumes you have created a Content Model with a Data Model.
We will use the table below as our example. This is a mockup of a government form using OMR checkboxes to check off whether or not certain critera listed in the "Description" column is met.
Obtain the Document's Layout Data
This method heavily relies on Layout Data in order to work. Before we can use Infer Grid to extract this table's information, we need to know the table's line positions and checkbox states.
That means we will need to do some image processing using the following IP Commands
- Line Removal or Line Detection
- Box Removal or Box Detection
You can learn more about image processing for tables visiting this article. However, it does not discuss Box Detection as it relates specifically to this use case.
The Box Detection and Box Removal commands use Optical Mark Recognition (OMR) to determine if a box is checked or not. It functions similarly to the Line Detection and Line Removal in that it also is looking for lines. After all, a box is made of lines. The Box Detection command is configured to only look at boxes of a certain size, in order to avoid "seeing" larger boxes as checkboxes. If some thing is "seen" inside the box (through Grooper's "blob detection"), it's checkbox state is "True" and "False" if not. Both the box's location on the page and its checkbox state are stored in the "LayoutData.json" file (along with lines detected from the Line Detection or Line Removal command).
Below, see the Box Removal IP Command in an IP Profile.
In this case, we are using both Line Removal and Box Removal. Be careful about the order your IP Commands are operating. Boxes are made of lines. If Line Removal runs before Box Removal, you run the risk of removing all or part of those box lines. Box Removal should always run before Line Removal in an IP Profile.
Add a Data Table
Create a Data Table with three Data Columns. The five columns for our example are "Operator Name", "Well Name", "Lease Number", "PC", and "Runs". Refer to the Creating a Data Table section above for more information on adding a Data Table to a Data Model.
Set the Extract Method
First, set the "Extract Method" property to "Infer Grid". (1) Select the Data Table object in the Node Tree (2) Select the "Extract Method" property. (3) Using the dropdown list, select "Infer Grid"
Configure the Axis Extractor
The first step when configuring Infer Grid for any table is to configure the Axis Extractors. These are extractors written to locate the column and row header label locations. Once these locations are known, infer grid can interpret a grid structure where it expects to find each cell in the table.
For our document, our table uses lines to divide its rows and columns into bounded cells. Because of this, we can get away with only using a single axis. We will use the "X Axis Extractor" property to locate the headers "Description", "Farm", and "Simulator".
These headers can be found with a simple Internal pattern (Although if your documents are more complicated, you can use a Reference to an extractor created in the Node Tree). Expand the "X Axis Extractor" property. Select the "Type" property and choose "Internal" from the dropdown list.
Select the "Pattern" property and press the ellipsis button at the end to bring up the Pattern Editor.
We will write a single pattern to match each column header. The following pattern will work just fine.
description\t
farm\t
simulator
| ! | This pattern uses tabs as anchors between column header label. Don't forget to turn on "Tab Marking" in the "Properties" tab. It is found by expanding the "Preprocessing Options" property. |
We've returned all our header labels smashed together as a single result. But Infer Grid needs individual instances of each header. Just like we did in the #Using Row Match with Named Groups|Using Row Match with Name Groups tutorial, we will use named groups to create instances. This time, we will create instances for each column label, from which Infer Grid will use their positions on the document to create a grid.
Select each label (without the tab character) in the Value Editor and make a named group out of each one. You can either right click the selection and choose "Create Group" option or use the Ctrl + G hotkey on your keyboard.
| ! | Remember to name the groups the same as their corresponding Data Column. That way the instances results will populate the correct Data Column in the Data Table. |
That will make the full regex the pattern below.
(?<Description>description)\t
(?<Farm>farm)\t
(?<Simulator>simulator)
However, notice this table actually is actually split into two tables side by side. But, our pattern only matches the headers on the left side and not the right.
If we switch over to the "Text" tab, we can easily identify the problem.
We can easily fix resolve this issue by using fuzzy matching. Switch the the "Properties" tab and change the "Mode" property from "RegEx" To "FuzzyRegEx"
Press the "OK" button to exit the Pattern Editor.
At this point, if we test extraction, we can see part of Infer Grid in action. All of the text inside the Description column's cells on the page is extracted, populating cells the Description Data Column. With the X Axis extractor we created, the Infer Grid is able to establish where the column headers are on the page. Then, it uses the line positions obtained from a Line Detection or Line Removal IP Command, to establish the table's structure, mapping out each cell according to their line boundaries.
Also notice even though this table starts on one half on the page and then continues on the second half, the data is extracted as if it were a single table.
However, we don't have any information for the "Farm" and "Simulator" columns. Those are blank or possibly picking up some errant OCR data.
Set the OMR Columns
In order to read the checkbox states, all we need to do is tell Infer Grid they are present in those columns. This is done using the "OMR Columns" property.
Select the "OMR Columns" property, and expand the dropdown menu. This will pop up a list of all the Data Columns in the Data Table. Simply check the boxes by the columns that contain checkboxes, in this case "Farm" and "Simulator".
For each cell in the selected columns, Infer Grid will look at the Layout Data obtained by a Box Detection or Box Removal IP Command to see if a mark was detected for the box. If a mark was detected, that means the box is checked and it is assigned the value "True". If not, that cell is assigned the value "False".
Press the "Test Extraction" button to see our results.
