2.80:Flow Collation (Concept)

Flow collation methods allow Data Type extractors to parse data using the the flow of text within a document.

This is particularly useful when processing natural language. The "Flow" property is available to the following Collation Providers:

• Combine
• Key-Value Pair
• Key-Value List
• Array
• Ordered Array

About

When extracting data from a document's text, there are three important relationships to consider:

Syntactical

• Data always has a syntax to it that indicates what that piece of data is. Specific characters in a specific order give data its syntax. For example, dates have a certain syntax that makes it obvious what you're reading is a date. When you see 12\25\2020, you instantly know this collection of numbers and slashes is a date. This is because the syntax of two numbers followed by a slash followed by two numbers followed by four numbers is a standard month, date, year format that makes it clear you're looking at a date. Without the slashes, it's less clear. "12252020" could be a date, but it could also just be a string of eight numbers.

Semantic

• Words themselves follow a syntax. The alphabet "a" through "z" are the characters in that syntax. However, the characters "turtle" and "uttelr" are two different things. One is a semi-aquatic reptile. The other is non-sense. The characters "turtle" mean something. They have semantic value. You can use the semantic relationships between pieces of text to target the data you want to return.
• For example, "Date: 12252020"
  • Without the slashes, "12252020" may be a date or just a bunch of numbers. However, it's clearly a date if you see the word "Date" in front of it. You're using the semantic value of the word "date" to understand that string of digits.

Spatial

• Spatial relationships refer to how the layout of text on a document informs the meaning of specific data elements. How a label is positioned next to a value provides the context for understanding that value. Its position next to the value uses a spatial relationship. For example, documents may call out data elements horizontally or vertically.

Horizontal	Vertical
Date: 12252020	Date: 12252020

Using Flow to Find Multiple Values in a Text Block

Understanding these relationships are important to understanding how to target and return the values you want from your text. Flow based methods use a document's text flow as its spatial relationship. English reads left to right from the top of the page to the bottom. While you probably don't even think about it when you're reading, this spatial relationship of characters and words is critical to understanding what you're reading.

Take this intro paragraph from the Wikipedia entry on Linnaeus's two-toed sloths:

If we, as readers, want to know where these sloths live, it's relatively easy, right? We just read along until we find the words "found in" and the region "South America".

Flow based collation methods work much the same way. We could set up a Data Type in Grooper using Key-Value Pair collation in Flow Layout mode. The Key extractor would locate the phrase "found in" on the document. The Value extractor would locate the region, "South America". Using Flow Layout, the Data Type reads through the text much like you would as an English reader. From the point the Key "found in" is located, the Data Type steps through the characters going left to right and down the page looking for the Value "South America".

In the example above, the value was right next to the key, but the beauty of the Key-Value Pair Collation's flow layout is it can be set up to scrawl past multiple characters until it finds the value it's looking for.

For more information on how to set up a Key-Value Pair using Flow Layout, see the How To section of this article.

The Array, Ordered Array, and Key-Value List Collation Providers can also utilize this Flow Layout.

Using Flow to Combine

The Flow method can also return the text between two or more values in a text flow. The The Combine, Array, and Ordered Array Collation Providers all have a "Combine Method" property. Setting this property to "Flow" will combine all the characters between the returned values in the text flow.

This method is useful to return larger sections of text where you can anchor off individual values within the full document.

Use Cases

Flow Collation methods are part of Grooper's natural language processing solution. Unstructured documents, such as contracts, use language to define data, such as the terms in the contract. Natural language presents several challenges for data extraction, including the fact that that data may exist in various ways in a paragraph flow that is not easily predictable.

What you can predict, however, is that the text will follow a normal lexical flow. For documents in English, you know text will always read left to right and top to bottom. Flow methods utilize this structure for various data extraction purposes.

Version Differences

There are no version differences to point out at this time.