2023:Labeling Behavior (Behavior)

WIP	This article is a work-in-progress or created as a placeholder for testing purposes. This article is subject to change and/or expansion. It may be incomplete, inaccurate, or stop abruptly. This tag will be removed upon draft completion.

The Labeling Behavior is a Content Type Behavior designed to collect and utilize a document's field labels in a variety of ways. This includes functionality for classification and data extraction.

Previous Versions
Grooper 2021

The Labeling Behavior functionality allows Grooper users to quickly onboard new Document Types for structured and semi-structured forms, utilizing labels as a thumbprint for classification and data extraction purposes. Once the Labeling Behavior is enabled, labels are identified and collected using the "Labels" tab of Document Types. These "Label Sets" can then be used for the following purposes:

• Document classification - Using the Labelset-Based Classification Method
• Field based data extraction - Primarily using the Labeled Value Extractor Type
• Tabular data extraction - Primarily using a Data Table object's Tabular Layout Extract Method
• Sectional data extraction - Primarily using a Data Section object's Transaction Detection Extract Method

FYI	The Labeling Behavior and its functionality discussed in this article are often referred to as "Label Set Behavior" or simply "Label Sets".

About

Labels serve an important function on documents. They give the reader critical context to understand where data is located and what it means. How do you know the difference between the date on an invoice document indicating when the invoice was sent and the date indicating when you should pay the invoice? It's the labels. The labels are what distinguishes one type of date from another. For example, "Invoice Date" for the date the invoice was sent and "Due Date" for the date you need to pay by.

Labels can be a way of classifying documents as well. What does one individual label tell you about a document? Well, maybe not much. However, if you take them all together, they can tell you quite a bit about the kind of document you're looking at. For example, a W-4 employee withholding form is going to use different labels than an employee healthcare enrollment form. These are two very different documents collecting very different information. The labels used to collect this information are thus different as well.

Furthermore, you can even tell the difference between two very closely related documents using labels as well. For example, two different invoices from two different vendors may share some similarity in the labels they use to detail information. But there will be some differences as well. These differences can be useful identifiers to distinguish one from the other. Put all together, labels can act as a thumbprint Grooper can use to classify a document as one Document Type or another.

Even though these two invoices share some labels (highlighted in blue), there are others that are unique to each one (highlighted in yellow). This awareness of how one kind of invoice from one vendor uses labels differently from another can give you a method of classifying these documents using their label sets.

The Labeling Behavior is built on these concepts, collecting and utilizing labels for Document Types in a Content Model for classification and data extraction purposes.

As a Behavior, the Labeling Behavior is enabled on a Content Type object in Grooper. ⚠ While you can enable Labeling Behavior on any Content Type, in almost all cases, you will want to enable this Behavior on the Content Model. Typically, you want to collect and use label sets for multiple Document Types in the Content Model, not just one Document Type individually. Enabling the Behavior on the Content Model will enable the Labeling Behavior for all child Document Types, allowing you to collect and utilize labels for all Document Types. Here, we have selected a Content Model in the Node Tree. To add a Behavior, select the Behaviors property and press the ellipsis button at the end.
This will bring up a dialogue window to add various behaviors to the Content Model, including the Labeling Behavior Add the Labeling Behavior using the "Add" button. Select Labeling Behavior from the listed options.
Once added, you will see a Labeling Behavior item added to the Behaviors list. Selecting the Labeling Behavior in the list, you will see property configuration options in the right panel. The configuration options in the property panel pertain to fuzzy matching collected labels as well as constrained and vertical wrapping capabilities to target stacked labels. By default, Grooper presumes you will want to use some fuzzy matching and enable constrained and vertical wrapping. These defaults work well for most use cases. However, you can adjust these properties here as needed. Press the "OK" button to finish adding the Labeling Behavior and exit this window.
Now on the Content Model tab you should see a Behavior is set. Save your changes
Once the Labeling Behavior is enabled, the next big step is collecting label sets for the various Document Types in your Content Model. With the Labeling Behavior enabled, you will now see a "Labels" tab present for the Content Model. This tab is also now present for each individual Document Type as well. Label sets are collected in this tab for each Document Type in the Content Model. Each Document Type has its own set of labels used to define information on the document. For example, the "Factura" Document Type in this Content Model uses the label "PO Number" to call out the purchase order number on this invoice document. A different Document Type, corresponding to a different invoice format, might use a different label such as "Purchase Order Number" or "PO #". Ultimately, this is the data we want to collect using the Content Model's Data Model. We use the "Labels" tab to collect labels corresponding to the various Data Elements (Data Fields, Data Tables, and Data Sections) of the Data Model. This provides a user interface to enter a label identifying the value you wish to collect for the Data Elements. For example, the label "PO Number" identifies the purchase order number for this invoice. Therefore, the label "PO Number" is collected for the "Purchase Order Number" Data Field in the Data Model. For more information on collecting label sets for the Document Types in your Content Model see the How To section of this article. ⚠ In Grooper 2023, for labels to show up in the Labels tab of the Content Model, a Label Set aware Value Extractor (such as Labeled Value or Tabular Layout) must be set on your Data Fields and Data Tables.
Once label sets are collected for each Document Type, they can be used for classification and data extraction purposes. For example, labels were used in this case to: Classify the documents, assigning each document the appropriate Document Type. Extract all the Data Fields seen here, collecting field based data from the document.
Extract the "Line Items" Data Table, collecting the tabular data seen here. For more information on how to use labels for these purposes, see the How To section of this article.

How To

The Labeling Behavior (often referred to as "Label Set Behavior" or just "Label Sets") are well suited for structured and semi-structured document sets. Label Sets are particularly useful for situations where you have multiple variations for one kind of document or another. While the information you want to extract from the document set may be the same from variation to variation, how the data is laid out and labeled may be very different from one variation of the document to another. Label Sets allow you to quickly onboard new Document Types to capture new form structures.

We will use invoices for the document set in the following tutorials. In a perfect world, you'd create a Content Model with a single "Invoice" Document Type whose Data Model would successfully extract all Data Elements for all invoices from all vendors every time no matter what. This is often not the case. You may find you need to add multiple Document Types to account for variations of an invoice from multiple vendors. Label Sets give you a method of quickly adding to Document Types to model new variations. In our case, we will presume we need to create one Document Type for each vendor. We will start with five Document Types for invoices from five vendors. Factura Lasku Envoy Rechnung Arve

You may download and import the file below into your own Grooper environment (version 2021). This contains the Batch(es) with the example document(s) discussed in this tutorial and the Content Model(s) configured according to the instructions. Media:Labeling Behavior (v2021).zip

Collect Label Sets

Navigate to the Labels UICollect Field LabelsCollect Table and Column LabelsAuto Map LabelsCollect Custom Labels

Navigate to the Labels UI

Collecting labels for the Document Types in your Content Model will be the first thing you want to do after enabling the Labeling Behavior. Labels for each Data Element in the Document Type's Data Model are defined using the "Labels" tab of the Content Model. Select the desired Content Model. Navigate to the "Labels" tab. With a Batch selected in the "Test Batch" window panel, select a document folder.
Right click on the folder. Click "Assign Document Type.."
When the "Assign Document Type" window pops up, click the hamburger button on the right side of the Content Type property. From the drop down, find the Content Model you are using and select the correct Document Type for the document.
Click "Execute" to assign the Document Type.
FYI If you haven't added a Document Type for the selected document folder yet, you can click the "+" (plus sign) button at the top of the Labels UI to both create a Document Type and assign it to the document that is currently selected.
Now that the Document Type has been set on the document, we have a couple of elements showing in our "LABELS" UI. However, we are not seeing any Data Fields or the Data Table objects. The eye icon at the top of the "Labels" tab hides all fields that do not have a label set on them by default. Click the eye button to show these fields.
Now all objects are showing in the "LABELS" UI.

Click me to return to the top

Collect Field Labels

Now that this document has been classified (assigned a Document Type from our Content Model), we can collect labels for its Document Type. This can be done in one of three ways:

1. Lassoing the label in the "Document Viewer".
2. Double-clicking the label in the Document Viewer.
3. Typing the label in manually.

‼	Going forward, this tutorial presumes you have obtained machine readable text from these documents, either OCR'd text or native text, via the Recognize activity.

Generally the quickest way is by simply lassoing the label in the "Document Viewer". Select the Data Element whose label you wish to collect. Here, we are selecting the "Invoice Number" Data Field. Click the "Rubberband Label" button. With your cursor, lasso around the text label on the document.
Upon lassoing the label in the Document Viewer, the OCR'd or native text behind the selected region will be used to populate the Data Element's label. At this point, the label for the "Invoice Number" Data Field is now "Invoice Number" because that's the text data we selected. Whatever text characters you lasso with your cursor will be assigned as the label.
If you choose, you may also manually enter a label for a Data Element by simply typing it into the text box. Here we've selected the "Purchase Order Number" Data Field and entered "PO Number". This will correspond to the label "PO Number" on the document itself.
If you type in the label incorrectly with typos and the label is unable to match anything on the document, an error icon will appear next to the label rather than a check mark. ⚠ An error icon will appear any time whatever is in the label field does not match with any OCRed data on the document. This can be due to a typo on the label or OCR error.
Continue lassoing or manually entering labels until all are collected. Next, we will focus on collecting labels from tables and table columns (the Data Table and Data Column elements in a Data Model). The process is essentially the same, but bears some extra explanation.

Click me to return to the top

Collect Table and Column Labels

Table and column labels can be used for tabular data extraction as well, setting a Data Table object to use the Tabular Layout Extract Method. When collecting labels for this method of table extraction, keep in mind you must collect the individual column headers, and may optionally collect both the full row of column header labels as well. While it is optional, it is generally regarded as best practice to capture the full row of column header labels. This will generally increase the accuracy of your column label extraction. We will do both in this tutorial. We will collect the full row of column header labels for the Data Table object's label. We will collect each individual column header label for each individual Data Column object's label. This may seem like you are duplicating your efforts but it is often critical to do both in order for the Tabular Layout Extract Method to map the table's structure and ultimately collect the table's data. In particular if you are dealing with OCR text data containing inaccurate character recognition data, establishing the full header row for the table will boost the fuzzy matching capabilities of the Labeling Behavior.
To collect the Data Table's label, select the Data Table object in the Labels tab. Here, we've selected the Data Table named "Line Items". Lasso the entire header row for the table. You may notice there are more columns on this table than we are collecting. As it is on the document, the table has six columns. But we're only collecting four, the "Quantity", "Description", "Unit Price", and "Line Total" Data Columns. Generally, you should collect the whole row of column headers, even if there are extra columns whose data you are not collecting.
Next, collect each child Data Column's header label. Here, we've selected the "Quantity" Data Column. Lasso the individual column header for the selected Data Column. Here, the stacked label, "Qty. Ord.".
Continue collecting labels for the remaining Data Columns. We have four Data Columns for this Data Table. Therefore, we collect four header labels from the document.

Click me to return to the top

Auto Map Labels

As you add labels for each Document Type, you may find some documents have labels in common. For example, there are only so many ways to label an invoice number. It might be "Invoice Number", "Invoice No", "Invoice #" or even just "Invoice". Some invoices are going to use one label, others another. When collecting labels for multiple Document Types you can use the "Auto Map" feature to automatically add labels you've previously collected on another Document Type. So far, we've only collected labels for one Document Type, the "Factura" Document Type. Now, we're collecting labels for the "Lasku" Document Type. Press the "Auto Map" button to automatically assign previously collected labels,
Grooper will search the document's text for labels matching those previously collected on other Document Types. For example, we collected the label "Remit To:" for the "Remit Address" Data Field for the "Factura" Document Type. The "Auto Map" feature found a match for this label on the document and assigned the "Lasku" Document Type's "Remit Address" Data Field the same label. If a match is not found, the Data Element's label is left blank. For example, the label for the "Invoice Amount" Data Field for the "Factura" Document Type was "Amount due". This label was nowhere to be found on this document. The invoice amount is labeled "Total" on the "Lasku" documents. So, the label is left blank for you to collect. As you keep collecting labels for more and more Document Types, the Auto Map feature will pick up more and more labels, allowing you to quickly onboard new Document Types.
Be aware, you may still need to validate the auto mapped values and make adjustments. For example, the label "Date" is very generic. This label does actually correspond to the invoice date on the "Lasku" Document Type in this case. However, that could label some other date on another Document Type. Even on this document, the label "Date" is returning the "Date" portion of "Ship Date" and other instances where "Date" is found in the text. As a side note, there are ways to make simple labels like "Date" more specific to the data they pertain to using "Custom Labels". More on that in the next tab. You can also make minor adjustments to the mapped labels. The mapped label for the "Purchase Order Number" Data Field was "PO Number" (as it was collected for the "Factura" Document Type), but it is more specifically "PO Number:" on the "Lasku" documents. We can just add the colon at the end of the label manually.

Click me to return to the top

Collect Custom Labels

It's important to keep in mind labels are collected for corresponding Data Elements in a Data Model. You collect one label per Data Element (Data Field, Data Section, Data Table or Data Column). What if you want to collect a label that is distinct from a Data Element, one that doesn't necessarily have to do with a value collected by your Data Model? And why would you even want to?

That's what "Custom Labels" are for. Custom labels serve two primary functions:

1. Providing additional labels for classification purposes.
2. Providing context labels when a Data Element's label matches multiple points on a document

Custom Labels may only be added to Data Model, Data Section or Data Table objects' labels. Put another way, any Data Element in the Data Model's hierarchy that can have child Data Elements can have custom labels. When used for classification purposes, custom labels are typically added to the Data Model itself. First select the Data Element in the Data Model's hierarchy to which you wish to add the label. In our case, we're selecting the Data Model itself. Click the "Add Label" icon at the top of the "Label" tab.
When the small window pops up, enter a name for your custom label and click "Add Custom".
This will add a new label tab, named whatever you named it in the previous step. Here, we just named it "Custom 01". Collect the custom label by either lassoing the text using the Document Viewer or manually typing in the label. For example, the word "Invoice" might be a useful label for classification purposes. This label isn't used to collect anything in our Data Model, but might be helpful to identify this and other invoices from the Factura Technology Corp as "Factura" Document Types. Collecting the label "Invoice" as a Custom Label will allow us to use it as a feature of this Document Type for classification.
Now check your custom label for accuracy and OCR errors.
You may add more Custom Labels to the selected Data Element by repeating the process described above. Click the "Add Labels" icon again. Name your label and click "Add Custom".

Custom Labels as Context Labels

Some labels are more specific than others. The label "Invoice Date" is more specific than the label "Date". If you see the label "Invoice Date" you know the date you're looking at is the date the invoice was generated. The label "Date" may refer to the invoice's generation date or it could be part of another label like "Due Date". However, some invoice formats will label the invoice date as simply "Date". For example, the label "Date" on this "Factura" Document Type does indeed correspond to the invoice date for the "Invoice Date" Data Field. However, this label pops up as part of other labels too, such as the "Date" in "Due Date" or "Order Date". This can present a challenge for data extraction. The possibilities for false-positive results tend to crop up the more generic the label used to identify a desired value. There are three separate date values identified by the word "Date" (in full or in part) on this document.

This is the second reason Custom Labels are typically added for a Document Type, to provide extra context for generic labels, especially when they produce multiple results on a document, leading to false-positive data extraction.

There are two steps to adding and using a Custom Label for this purpose:

1. Add the Custom Label.
2. Marry the Custom Label with the Data Element's label.

We will refer to this type of a Custom Label as a "Context Label" from here out.

The only "trick" to this is adding the Context Label to the appropriate level of the Data Model's hierarchy. Remember, a Custom Label may only be added to a Data Model, Data Section or Data Table object. We cannot add a Custom Label to a Data Field, such as the "Invoice Number" Data Field. To add a Context Label a Data Field can use, we must add the Custom Label to its direct parent Data Element. In the case of the "Invoice Date" Data Field its direct parent Data Element is the Data Model itself.
Click the "Add Label" icon at the top of the "Labels" tab. Type in a name for your label and click "Add Custom"
We are going to lasso these two labels "Date" and "Page" together to make the custom label.
We can see that the label was collected appropriately. This custom label will provide the simple label "Date" some extra context. Which of the three results for the label "Date" do we want to accept? The one falling within this zone.
Now that we've added the label, we need to marry the Custom Label with the Data Field its giving extra context to. This is done with the Parent property of a Data Field label. Click on the check mark icon next to the "Invoice Date" Data Field label.
Click on the hamburger icon next to the Parent property. Using the drop down list, select the Custom Label you wish to use for the Context Label.
Click "SAVE" to save your changes.
Notice with this Context Label added we only return a single result for the "Invoice Date" Data Field's label "Date". This is the label we want to associate with this Data Field. The other two results do not fall within the Context Label, and are no longer returned.

Click me to return to the top

Label Sets & Classification

About Labelset-Based Classification

Label Sets can be used for classifying documents using the Labelset-Based Classification Method. For structured and semi-structured forms labels end up being a way of identifying a document. Without the field data entered, the labels are really what define the document. You know what kind of document you're looking at based on what kind of information is presented and in the case of Labelset-Based classification how that data is labeled. Even when those labels are very similar from one variant to the next, they end up being a thumbprint of that variant. For example, you might use Labelset-Based classification to create Document Types for different variations of invoices from different vendors. The information presented on each variant from each vendor will be more or less the same, and some labels will be more commonly used by different vendors (such as "Invoice Number"). However, if there is enough variation in the set of labels, you can easily differentiate an invoice from one vendor verses another just based on the variation in labels.

Take these four "documents". Each one is collecting the same information: A person's name Their social security number Their birthday Their phone number Their address So we might have five Data Fields in our Data Model, one for each piece of information. We'd also collect one label for each Data Field as well. While the data we want from these documents is the same, there is some variation in the labels used for each different document type. If we wanted to distinguish these four documents from each other by classifying using the Labelset-Based Classification Method. This is all done measuring the similarity between the collected label sets for each Document Type. How is Document Type "B" different from Document Type "A"? It uses the label SSN: instead of Social Security Number:. How is Document Type "C" different from Document Type "A"? It uses the labels SSN: instead of Social Security Number: and DOB: instead of Date of Birth:. How is Document Type "D" different from Document Type "A"? It uses the labels SSN: instead of Social Security Number:, DOB: instead of Date of Birth:, and Phone #: instead of Phone Number.

Using the Labelset-Based Classification Method unclassified documents are classified by assigning the document the Document Type whose labels are most similar. The basic concept is "similarity" is determined by how many labels are shared between the unclassified document and the label sets collected for the Document Types in your Content Model. The unclassified document is assigned the Document Type with the highest degree of similarity between matched labels and the Document Types' label sets.

The similarity calculation is very straightforward. Grooper searches for labels collected for every Document Type and measures the total character difference between all the labels matched on the document. If each of these five labels is collected for each Document Type's Label Set, you'd have the following character totals for the set. Document Type "A" - 63 total label characters. Document Type "B" - 44 total label characters. Document Type "C" - 34 total label characters. Document Type "D" - 29 total label characters. How similar is Document Type "A" to Document Type "B"? "A" uses the label Social Security Number: instead of SSN: However, there is a match for the remaining four labels. The remaining four labels, Name:, Date of Birth, Phone Number: and Address: are comprised of 40 characters. The similarity score is the percentage of matched label characters divided by the total characters in the Document Type's label set. 40 matched label characters / 44 total label characters = 0.9091 "A" is roughly 91% similar to "B" How similar is Document Type "A" to Document Type "C"? "A" uses the label Social Security Number: instead of SSN: and Date of Birth instead of DOB: However, there is a match for the remaining three labels. The remaining three labels, Name:, Phone Number: and Address: are comprised of 26 characters. The similarity score is the percentage of matched label characters divided by the total characters in the Document Type's label set. 26 matched label characters / 34 total label characters = 0.7647 "A" is roughly 76% similar to "B" How similar is Document Type "A" to Document Type "D"? Figure out what labels from "A" match "D", and do the math.

If we ran one of these "documents" into Grooper, we can see these results very clearly. The document shares all five labels in common with the "A" Document Type. Grooper searches for labels matching the label sets for all Document Types in the Content Model and creates a similarity score for each one. You can see the math described above play out here. Matching all labels in the "A" Document Type's label set, the document is considered 100% similar. Less so for the other Document Types because while they share some labels (like Name:), some are different (like Social Security Number: versus SSN: Upon classification, the document folder is assigned the Document Type with the highest similarity score. In this case the "A" Document Type.