2023:Grid Layout (Table Extract Method)

From Grooper Wiki
Revision as of 08:46, 27 August 2024 by Randallkinard (talk | contribs)

This article is about an older version of Grooper.

Information may be out of date and UI elements may have changed.

202520242023

The Grid Layout Table Extract Method uses the positional location of row and column headers to interpret where a tabular grid would be around each value in a table and extract values from each cell in the interpreted grid.


This method extracts information by inferring a grid from the row and column header positions.  This is done by assigning an X Axis Extractor to match the column headers and, a Y Axis Extractor to match row headers.  A grid is created from the header positions extracted from the two extractors. 

Furthermore, if table line positions can be obtained from a Line Detection or Line Removal IP Command, only the X Axis Extractor is needed. In these cases, the X Axis Extractor can be used to find the column header labels, and the grid will be created using the table lines in the documents Layout Data. The raw text data obtained from the Recognize activity will populate each cell of the grid according to where it is on the page.

FYI

In version 2021, Grid Layout replaced the Infer Grid table extract method. Their logic and function is largely the same. If you're looking for information on the now deprecated Infer Grid method, visit this article

You may download and import the file(s) below into your own Grooper environment (version 2023). There is a Batch with the example document(s) discussed in this tutorial, as well as a Project configured according to its instructions.
Please upload the Project to your Grooper environment before uploading the Batch. This will allow the documents within the Batch to maintain their classification status.

Use Cases

Non-Standard Tables

The Grid Layout method excels at many cases where the table structure is not easily understood by the Row Match or Header-Value methods. This is especially true for tables with table lines present. Examine the table below.

Row Match might work, but it would be a heavy lift. First, each row's pattern is different. There are names on one, addresses on another, phone numbers on another. Every row has a different pattern. It would take some creative configuration. You could try to make a row out of the columns. It would take a series of extractors, be very effort intensive and complicated to set up.

Header-Value would also have problems. The column header labels ("Lender", "Mortgage Broker", etc), would be straightforward. But the value extractors would be tricky. It's possible a generic text segment extractor could get you close, but at least the "Address" row presents problems because it is a two line value instead of a single line. Again, it could be doable, but it would take some effort.

Grid Layout can do this job with a single extractor. All you would need to do is write an extractor to find the "X Axis"; so all the column header labels in a row.

Since table lines are present, the text falling inside each cell (obtained via the Recognize activity could be extracted to the corresponding cell in the column

Furthermore, if table lines are not present, Grid Layout can use both both the row and column header labels by using both the "Y Axis Extractor" and "X Axis Extractor" properties. We can use two extractors, one to return all the Y Axis labels and one to return the X Axis labels, and use their positions to infer the table's structure.


OMR Checkboxes

OMR stands for "Optical Mark Recognition". It is a a way to determine if a checkbox is marked or not on a document. If you think back to your grade school days and remember taking tests and filling in bubbles on an answer sheet, you already have experience with OMR! Those answer sheets are fed through a machine that reads the "checkbox state" of the boxes, either filled in (checked) or not. There are many examples of current documents where checkboxes are used to record a boolean response ("true or false" or "yes or no"), a multiple choice response, or other information. Grooper uses OMR to read those checkbox states.

The Grid Layout method is the easiest way to read checkbox states inside a table. Once the table's structure is found using the axis extractors, you can choose which columns contain checkboxes. Grooper will use Layout Data obtained from a Box Detection or Box Removal IP Command to determine if the box is filled in or left blank. Refer to the tutorial below for more information on how to configure this use.

Marking the "Farm" and "Simulator" columns as OMR Columns in the Grid Layout Property Panel will return a value of "True" if the box is checked and "False" if it is blank.

Re-OCRing Tricky Cells

The Grid Layout method also allows you to choose a column and apply a secondary OCR profile to the cells within that column. This is useful for tables that have specialized fonts for values filled inside the cells.

For example, the OCR-A font is not easily read by most modern OCR engines. However, Google's Tesseract OCR engine has some specialized functionality for the font. A document using a column like the one to the left could process most of the document, using an OCR profile that reads conventional fonts, including the column headers such as "Date". Then, the cells inside the grid, containing dates in the OCR-A font, could be reprocessed using another OCR profile that uses the Tesseract engine.


How To

Configuring Grid Layout for Tables with Lines

Before you begin

A Data Table is a Data Element used to model and extract a table's information on a document. Just like other Data Elements, such as Data Fields and Data Sections, Data Tables are created as children of a Data Model. This guide assumes you have created a Content Model with a Data Model.

We will use the table below as our example for creating a Data Table.

Navigate to a Data Model

Using the Node Tree on the left side of Grooper Design Studio, navigate to the Data Model you wish to add the Data Table to. Data Tables can be created as children of any Data Model at any hierarchy in a Content Model.

Add a Data Table

Right click the Data Model object, mouse over "Add" and select "Data Table"


The following window will appear. Name the table whatever you would like and press "OK" when finished.


This creates a new Data Table object in the Node Tree underneath the Data Model.

Add Data Columns

Right click the Data Table object, mouse over "Add" and select "Data Column"


This brings up the following window to name the Data Column. When finished, press "OK" to create the object.


This creates a new Data Column object in the Node Tree underneath the Data Model.

Repeat Until Finished

Add as many columns as necessary to complete the table. For our example, we have a single Data Table with five Data Columns, each one named for the corresponding column on the document.

Configure Extract Method

With the Data Table and its child Data Columns created, it is not time to configure extraction for this table. We will beging by configuring the Extract Method for this Data Table.


  1. With the appropriate Data Columns added, seelct the parent Data Table.
  2. Click the drop-down for the Extract Method property.
  3. Select the Grid Layout option.


  1. Click the arrow to expand the sub-properties of the Extract Method property.
  2. Click the drop-down for the X Axis Extractor property.
  3. Select the Pattern Match option in the drop-down menu.


  • Click the ellipsis button to bring up the pattern editor dialog box.

Configure Pattern for X-Axis Extractor

We now need to write a pattern that will act to define the horizontal definition of our table's "grid". The easiest way to do this is to return a result that contains all the column headers of our table. Furthermore, the result returned needs to be subdivided into sub-instances that will match, exactly, the names of their respective Data Columns.


  • Set the pattern to match the headers seen on the document, being careful to put a space after each entry before returning the line.


  1. Highlight "Order Date", being careful to not also select the white space. Right-click the highlighted text ...
  2. ... select the "Create Group" option in the sub-menu. You can also use the hotkey "Ctrl+G"


  • This will put the highlighted text inside the parenthesis with ?<>. It will also place the cursor inside the <> and allow you to name the group. Be sure to replace spaces with _. The name of the group should match the name of the desired Data Column exactly.


  1. Repeat the group naming process for the remaining parts of the pattern.
  2. Click "OK" when done.

Test Table Extraction Results

The Data Table and its child Data Columns have been created. And, given that the target document has been Recognized and has layoutdata containing line information, an X-Axis (horizontal) extractor has been created. The combination of the lines on the document, and the appropriately configured horizontal extractor should be enough to return results. This will now be tested.


  1. Click the "Save" button to save changes made.
  2. Click on the "Tester" tab.


  1. Click the "Test" button.
  2. Because this document has layout dta, the Grid will be created by the lines of the page.

Configuring Grid Layout for Tables without Lines

Before you begin

We will now take a look at a very similar table as what we previously worked with. The key difference with this table is that there are no lines defining the structure of the table. We as humans can easily intuit its structure, but without the lines present Grooper will need more guidance to understand the table's structure.

If a table has lines the X-Axis, or "hoizontal", extractor of our Grid Layout approach will essentially define the upper boundary of our table and use line information to then draw the grid of our table. Without the lines, however, an X-Axis extractor will not be enough. How will Grooper understand the structure of the table without the lines?

We will configure what's referred to as a "Y-Axis", or "vertical", extractor to allow Grooper to "infer the grid" of the table by taking the bounds of the returned results of these two extractors and drawing said grind based on the intersection of the two.


Configuring Table for Grid Layout

First things first we need a Data Table with appropriate child Data Columns. We then need to set the Extract Method and set our X-Axis Extractor.


  1. The "Table without Lines" setup is very similar to the setup of the "Table with Lines".
  2. The Extract Method is set to Grid Layout.
  3. The X Axis Extractor property is set to Pattern Match.

Configure Pattern for X-Axis Extractor

Next we will edit the pattern of our X-Axis Extractor to find the column headers of our table and return the results in named sub-groups that match our Data Columns.


  • A pattern targeting headers of the table is the same, with named groups to match the 'Data Columns.

Create Extractor for Y-Axis Extractor

We will now configur our Y-Axis Extractor. This will require a little more logic than a local pattern on the property will allow, so we will leverage a Data Type.


  1. In order to "define the grid" of the table without lines, an extractor that helps define the structure of the table is made. IF an X Axis Extractor sets the horizontal dimensions, an extractor on the Y will define the vertical.
  2. A reg-ex pattern is used to find a value that will occur on every row of the table. In this case the pattern is: [0-9]{1,2}/[0-9]{1,2}/[0-9]{4}
  3. The Collation is set to Array because the Y Axis Extractor needs to return one result. This one result will be divided into sub-instances that will define the vertical structure of the table.


  • Upon inspecting the result, we can observe the desired output. A single result with an array of sub-elements.

Complete Table Configuration and Test Results

With the Y-Axis extractor made, we now need to plug it into our Data Table and test the results.


  1. To finish setting up our Grid Layout for a table without lines, select the Data Table...
  2. ... set the Y Axis Extractor to Reference ...
  3. point the Extractor property to the "Array" Data Type ...
  4. ... set the Header Column property to the "Order Date" column. The values of this column are being leveraged to define the structure of the grid, so this property is set to allow the values to be included in extraction.


  1. Click on the "Tester" tab ...
  2. ... click the "Test" button to test extraction ...
  3. ... and return results.
  4. However, becasuse there are no lines in the layout data file the grid is created based on the bounds of the results from the X and Y axis extractors. Thus demonstrating a shortcoming of this extraction technique.

Configure Grid Layout for OMR Check Boxes

A Data Table is a Data Element used to model and extract a table's information on a document. Just like other Data Elements, such as Data Fields and Data Sections, Data Tables are created as children of a Data Model. This guide assumes you have created a Content Model with a Data Model.

We will use the table below as our example. This is a mockup of a government form using OMR checkboxes to check off whether or not certain critera listed in the "Description" column is met.

Obtain the Document's Layout Data

This method heavily relies on Layout Data in order to work. Before we can use Grid Layout to extract this table's information, we need to know the table's line positions and checkbox states.

That means we will need to do some image processing using the following IP Commands

  1. Line Removal or Line Detection
  2. Box Removal or Box Detection

You can learn more about image processing for tables visiting this article. However, it does not discuss Box Detection as it relates specifically to this use case.

The Box Detection and Box Removal commands use Optical Mark Recognition (OMR) to determine if a box is checked or not. It functions similarly to the Line Detection and Line Removal in that it also is looking for lines. After all, a box is made of lines. The Box Detection command is configured to only look at boxes of a certain size, in order to avoid "seeing" larger boxes as checkboxes. If some thing is "seen" inside the box (through Grooper's "blob detection"), it's checkbox state is "True" and "False" if not. Both the box's location on the page and its checkbox state are stored in the "LayoutData.json" file (along with lines detected from the Line Detection or Line Removal command).

Below, see the Box Removal IP Command in an IP Profile.





In this case, we are using both Line Removal and Box Removal. Be careful about the order your IP Commands are operating. Boxes are made of lines. If Line Removal runs before Box Removal, you run the risk of removing all or part of those box lines. Box Removal should always run before Line Removal in an IP Profile.

Add a Data Table

Create a Data Table with three Data Columns. The five columns for our example are "Operator Name", "Well Name", "Lease Number", "PC", and "Runs". Refer to the Configuring Grid Layout for Tables with Lines section above for more information on adding a Data Table to a Data Model.

Set the Extract Method

First, set the "Extract Method" property to "Grid Layout". (1) Select the Data Table object in the Node Tree (2) Select the "Extract Method" property. (3) Using the dropdown list, select "Grid Layout"

Configure the Axis Extractor

The first step when configuring Grid Layout for any table is to configure the Axis Extractors. These are extractors written to locate the column and row header label locations. Once these locations are known, Grid Layout can interpret a grid structure where it expects to find each cell in the table.

For our document, our table uses lines to divide its rows and columns into bounded cells. Because of this, we can get away with only using a single axis. We will use the "Y Axis Extractor" property to locate the headers "Description", "Farm", and "Simulator".


These headers can be found with a simple Internal pattern (Although if your documents are more complicated, you can use a Reference to an extractor created in the Node Tree). Expand the "X Axis Extractor" property. Select the "Type" property and choose "Internal" from the dropdown list.


Select the "X-Axis Extractor" property and press the ellipsis button at the end to bring up the Pattern Editor.


We will write a single pattern to match each column header. The following pattern will work just fine. 

description 
farm 
simulator

⚠

Take note of the spaces after "description" and "farm".



We've returned all our header labels smashed together as a single result. But Grid Layout needs individual instances of each header. Just like we did in the #Using Row Match with Named Groups|Using Row Match with Name Groups tutorial, we will use named groups to create instances. This time, we will create instances for each column label, from which Grid Layout will use their positions on the document to create a grid.

Select each label (without the tab character) in the Value Editor and make a named group out of each one. You can either right click the selection and choose "Create Group" option or use the Ctrl + G hotkey on your keyboard.

⚠

Remember to name the groups the same as their corresponding Data Column. That way the instances results will populate the correct Data Column in the Data Table. Be sure to NOT include the white spaces in the named groups.

That will make the full regex the pattern below. 

(?<Description>description) 
(?<Farm>farm) 
(?<Simulator>simulator)



Press the "OK" button to exit the Pattern Editor.

At this point, if we test extraction, we can see part of Grid Layout in action. All of the text inside the Description column's cells on the page is extracted, populating cells the Description Data Column. With the X Axis extractor we created, the Grid Layout is able to establish where the column headers are on the page. Then, it uses the line positions obtained from a Line Detection or Line Removal IP Command, to establish the table's structure, mapping out each cell according to their line boundaries.

Also notice even though this table starts on one half on the page and then continues on the second half, the data is extracted as if it were a single table.


However, we don't have any information for the "Farm" and "Simulator" columns. Those are blank or possibly picking up some errant OCR data.

Set the OMR Columns

In order to read the checkbox states, all we need to do is tell Grid Layout they are present in those columns. This is done using the "OMR Columns" property.


Select the "OMR Columns" property, and expand the dropdown menu. This will pop up a list of all the Data Columns in the Data Table. Simply check the boxes by the columns that contain checkboxes, in this case "Farm" and "Simulator".


For each cell in the selected columns, Grid Layout will look at the Layout Data obtained by a Box Detection or Box Removal IP Command to see if a mark was detected for the box. If a mark was detected, that means the box is checked and it is assigned the value "True". If not, that cell is assigned the value "False".


Press the "Test Extraction" button to see our results.

Glossary

Array: Array is a Collation Provider option for pin Data Type extractors. Array matches a list of values arranged in horizontal, vertical, or text-flow order, combining instances that qualify into a single result.

Batch: inventory_2 Batch nodes are fundamental in Grooper's architecture. They are containers of documents that are moved through workflow mechanisms called settings Batch Processes. Documents and their pages are represented in Batches by a hierarchy of folder Batch Folders and contract Batch Pages.

Box: Box is a connection option for cloud CMIS Connections. It Grooper to the Box content management system for import and export operations.

Content Model: stacks Content Model nodes define a classification taxonomy for document sets in Grooper. This taxonomy is defined by the collections_bookmark Content Categories and description Document Types they contain. Content Models serve as the root of a Content Type hierarchy, which defines Data Element inheritance and Behavior inheritance. Content Models are crucial for organizing documents for data extraction and more.

Data Column: view_column Data Columns represent columns in a table extracted from a document. They are added as child nodes of a table Data Table. They define the type of data each column holds along with its data extraction properties.

  • Data Columns are frequently referred to simply as "columns".
  • In the context of reviewing data in a Data Viewer, a single Data Column instance in a single Data Table row, is most frequently called a "cell".

Data Element: Data Elements are a class of node types used to collect data from a document. These include: data_table Data Models, insert_page_break Data Sections, variables Data Fields, table Data Tables, and view_column Data Columns.

Data Field: variables Data Fields represent a single value targeted for data extraction on a document. Data Fields are created as child nodes of a data_table Data Model and/or insert_page_break Data Sections.

  • Data Fields are frequently referred to simply as "fields".

Data Model: data_table Data Models are leveraged during the Extract activity to collect data from documents (folder Batch Folders). Data Models are the root of a Data Element hierarchy. The Data Model and its child Data Elements define a schema for data present on a document. The Data Model's configuration (and its child Data Elements' configuration) define data extraction logic and settings for how data is reviewed in a Data Viewer.

Data Section: A insert_page_break Data Section is a container for Data Elements in a data_table Data Model. variables They can contain Data Fields, table Data Tables, and even Data Sections as child nodes and add hierarchy to a Data Model. They serve two main purposes:

  1. They can simply act as organizational buckets for Data Elements in larger Data Models.
  2. By configuring its "Extract Method", a Data Section can subdivide larger and more complex documents into smaller parts to assist in extraction.
    • "Single Instance" sections define a division (or "record") that appears only once on a document.
    • "Multi-Instance" sections define collection of repeating divisions (or "records").

Data Table: A table Data Table is a Data Element specialized in extracting tabular data from documents (i.e. data formatted in rows and columns).

  • The Data Table itself defines the "Table Extract Method". This is configured to determine the logic used to locate and return the table's rows.
  • The table's columns are defined by adding view_column Data Column nodes to the Data Table (as its children).

Data Type: pin Data Types are nodes used to extract text data from a document. Data Types have more capabilities than quick_reference_all Value Readers. Data Types can collect results from multiple extractor sources, including a locally defined extractor, child extractor nodes, and referenced extractor nodes. Data Types can also collate results using Collation Providers to combine, sift and manipulate results further.

Extract: export_notes Extract is an Activity that retrieves information from folder Batch Folder documents, as defined by Data Elements in a data_table Data Model. This is how Grooper locates unstructured data on your documents and collects it in a structured, usable format.

Grid Layout: The Grid Layout Table Extract Method uses the positional location of row and column headers to interpret where a tabular grid would be around each value in a table and extract values from each cell in the interpreted grid.

IP Command: IP Commands specify an image processing (IP) operation (such as image cleanup, format conversion or feature detection) and are used to construct image IP Steps in an IP Profile. IP Commands are configured using an IP Step's Command property.

IP Profile: perm_media IP Profiles are a step-by-step list of image processing operations (IP Commands). They are used for several image processing related operations, but primarily for:

  1. Permanently enhancing an image during the Image Processing activity (usually to get rid of defects in a scanned image, such as skewing or borders).
  2. Cleaning up an image in-memory during the Recognize activity without altering the image to improve OCR accuracy.
  3. Computer vision operations that collect layout data (table line locations, OMR checkboxes, barcode value and more) utilized in data extraction.

Layout Data: Layout Data refers to visual information Grooper certain IP Commands collect, such as lines, checkboxes, barcodes, and detected shapes. This data is stored in a "Grooper.Layout.json" file attached to contract Batch Pages. Layout data is used by certain extractors and other features that rely on the presence of that data to function.

Line Removal: Line Removal is an IP Command that locates and removes horizontal and vertical lines from documents. The detected line locations are stored as part of page's layout data.

Node Tree: The Node Tree is the hierarchical list of Grooper node objects found in the left panel in the Design Page. It is the basis for navigation and creation in the Design Page.

OCR: OCR is stands for Optical Character Recognition. It allows text on paper documents to be digitized, in order to be searched or edited by other software applications. OCR converts typed or printed text from digital images of physical documents into machine readable, encoded text.

Pattern Match: Pattern Match is a Value Extractor that extracts values from a document that match a specified regular expression, providing data collection following a known format or pattern.

Project: package_2 Projects are the primary containers for configuration nodes within Grooper. The Project is where various processing objects such as stacks Content Models, settings Batch Processes, profile objects are stored. This makes resources easier to manage, easier to save, and simplifies how node references are made in a Grooper Repository.

Recognize: format_letter_spacing_wide Recognize is an Activity that obtains machine-readable text from contract Batch Pages and folder Batch Folders. When properly configured with an library_booksOCR Profile, Recognize will selectively perform OCR for images and native-text extraction for digital text in PDFs. Recognize can also reference an perm_mediaIP Profile to collect "layout data" like lines, checkboxes, and barcodes. Other Activities then use this machine-readable text and layout data for document analysis and data extraction.

Reference: Reference is a Value Extractor used to reference an Extractor Node. This allows users to create re-usable extractors and use the more complex pin Data Type and input Field Class extractors throughout Grooper.

Row Match: The Row Match Table Extract Method uses regular expression pattern matching to determine a tables structure based on the pattern of each row and extract cell data from each column.

Table Extract Method: A Table Extract Method defines the settings and logic for a table Data Table to perform extraction. It is set by configuring the Extract Method property of the Data Table.

Table Extraction: "Table Extraction" refers to Grooper's ability to extract data from cells in tables on documents. This is accomplished by configuring the table Data Table and its child view_column Data Column elements in a data_table Data Model.

Retrieved from "https://wiki.grooper.com/index.php?title=2023:Grid_Layout_(Table_Extract_Method)&oldid=25196"