Data Extractor (Concept) - Revision history

Dgreenwood: /* The Reference extractor */

2025-08-28T16:42:09Z

The Reference extractor

← Older revision		Revision as of 11:42, 28 August 2025
Line 92:		Line 92:

	==== The Reference extractor ====		==== The Reference extractor ====
	The Reference extractor is unique among the extractor types. It allows users to reference an extractor node (such as a {{IconName\|Data Type}} Data Type or {{IconName\|Value Reader}} Value Reader).		The Reference extractor is unique among the extractor types. It allows users to reference the results of an extractor node (such as a {{IconName\|Data Type}} Data Type or {{IconName\|Value Reader}} Value Reader).
	* [[Reference]]		* [[Reference]]
	</div>		</div>

Dgreenwood: /* LLM-based extractors */

2025-08-28T16:24:18Z

LLM-based extractors

← Older revision		Revision as of 11:24, 28 August 2025
Line 70:		Line 70:
	==== LLM-based extractors ====		==== LLM-based extractors ====

	These extractors use generative AI to return results. The document ~~content~~ and other prompts defined by the user is fed to a large language model (LLM) for analysis. To utilize these extractors, you must add an "[[LLM Connector]]" Repository Option to the Grooper Root.		These extractors use generative AI to return results. The document text and other prompts defined by the user are fed to a large language model (LLM) for analysis. To utilize these extractors, you must add an "[[LLM Connector]]" Repository Option to the Grooper Root.

	* [[Ask AI]]		* [[Ask AI]]

Dgreenwood: /* Extractor Nodes */

2025-08-27T16:50:05Z

Extractor Nodes

← Older revision		Revision as of 11:50, 27 August 2025
Line 97:		Line 97:

	== Extractor Nodes==		== Extractor Nodes==
			<blockquote>{{#lst:Glossary\|Extractor Nodes}}</blockquote>
	<section begin="Extractor Nodes" />		<section begin="Extractor Nodes" />
	=== Types of Extractor Nodes ===		=== Types of Extractor Nodes ===
Line 107:		Line 108:

	Extractor Nodes are tools to extract/return document data. But they don't do anything by themselves. They are used by extractor properties on other nodes in Grooper.		Extractor Nodes are tools to extract/return document data. But they don't do anything by themselves. They are used by extractor properties on other nodes in Grooper.
	* Example: [[Data Element]]s, such as [[Data Field]]s, are ultimately collect document data. It is ~~the~~ Data Field's "Value Extractor" ~~configuraiton~~ that does this. ~~However, you~~ may configure ~~a Data Field's Value Extractor~~ with an Extractor Node to do so.		* Example: When {{IconName\|Extract}} runs on a document, [[Data Element]]s (such as {{IconName\|Data Field}} [[Data Field]]s) are ultimately used collect document data.
	* Example: ~~The~~ [[Pattern-Based Separation]] provider's ~~"Value Extractor"~~ is ultimately what identifies patterns to separate Batch Pages into Batch Folders. However, you may configure ~~the Value Extractor~~ with an Extractor Node to do so.		*: ''It is a Data Field's "Value Extractor" property that does this. You may configure this property with an Extractor Node to do so.''
	* Example: When [[Classify]] runs on a document, a [[Document Type]]'s "Positive Extractor" will be used to assign a Batch Folder the Document Type if it returns a value. You may configure the Positive Extractor with an Extractor Node to do so.		* Example: When executed in a {{IconName\|Separate}} [[Separate]] step, the [[Pattern-Based Separation]] provider's is ultimately what identifies patterns to separate Batch Pages into Batch Folders.
			*: ''It is its "Value Extractor" property that does this. However, you may configure this property with an Extractor Node to do so.
			* Example: When {{IconName\|Classify}} [[Classify]] runs on a document, a {{IconName\|Document Type}} [[Document Type]]'s "Positive Extractor" property will be used to assign a Batch Folder the Document Type if it returns a value.
			*: ''You may configure the Positive Extractor with an Extractor Node to do so.''
	* And so on and so on for any extractor property for any node in Grooper.		* And so on and so on for any extractor property for any node in Grooper.

Dgreenwood: /* Extractor Nodes */

2025-08-27T16:36:18Z

Extractor Nodes

← Older revision		Revision as of 11:36, 27 August 2025
Line 98:		Line 98:
	== Extractor Nodes==		== Extractor Nodes==
	<section begin="Extractor Nodes" />		<section begin="Extractor Nodes" />
			=== Types of Extractor Nodes ===
	There are three types of Extractor Nodes in '''Grooper''':		There are three types of Extractor Nodes in '''Grooper''':
	: {{ValueReaderIcon}} '''[[Value Reader]]'''		: {{ValueReaderIcon}} '''[[Value Reader]]'''
	: {{DataTypeIcon}} '''[[Data Type]]'''		: {{DataTypeIcon}} '''[[Data Type]]'''
	: {{FieldClassIcon}} '''[[Field Class]]''' ~~''Uncommonly used~~ in ~~current~~ Grooper ~~implementations''~~		: {{FieldClassIcon}} '''[[Field Class]]'''
	All three of these node types perform a similar function. They ~~are configured to~~ return data from documents. However, they differ in their configuration and ~~data extraction purpose~~.		:*<li class="attn-bullet"> Advances in large-language models (LLMs) have largely made Field Classes obsolete. LLM-based extraction methods in Grooper (such as [[AI Extract]]) can achieve similar results with nowhere near the amount of set up.
			All three of these node types perform a similar function. They return data from documents. However, they differ in their configuration and utility.

	Extractor ~~nodes~~ are tools to extract/return data. ~~Ultimately~~, "[[Data ~~Element~~]]s" are ~~what collects~~ data. ~~They~~ may ''~~use'~~' extractor ~~nodes to help collect data~~ in ~~a '''Data Model'''~~.		Extractor Nodes are tools to extract/return document data. But they don't do anything by themselves. They are used by extractor properties on other nodes in Grooper.
			* Example: [[Data Element]]s, such as [[Data Field]]s, are ultimately collect document data. It is the Data Field's "Value Extractor" configuraiton that does this. However, you may configure a Data Field's Value Extractor with an Extractor Node to do so.
			* Example: The [[Pattern-Based Separation]] provider's "Value Extractor" is ultimately what identifies patterns to separate Batch Pages into Batch Folders. However, you may configure the Value Extractor with an Extractor Node to do so.
			* Example: When [[Classify]] runs on a document, a [[Document Type]]'s "Positive Extractor" will be used to assign a Batch Folder the Document Type if it returns a value. You may configure the Positive Extractor with an Extractor Node to do so.
			* And so on and so on for any extractor property for any node in Grooper.

	To that end, ~~extractor nodes~~ serve three purposes:
			To that end, Extractor Nodes serve three purposes:
	# To be re-usable units of extraction		# To be re-usable units of extraction
	# To collate data		# To collate data
	# To leverage machine learning algorithms to target data in the flow of text		# To leverage machine learning algorithms to target data in the flow of text
			#*<li class="attn-bullet"> Advances in large-language models (LLMs) have largely made Field Classes obsolete. LLM-based extraction methods in Grooper (such as [[AI Extract]]) can achieve similar results with nowhere near the amount of set up.
	<div style="padding-left: 1.5em">		<div style="padding-left: 1.5em">

	=== Re-usability ===		=== Re-usability ===
	Extractor nodes are meant to be referenced either by other extractor nodes or, importantly, by Data Elements such as '''Data Fields''' in a '''Data Model'''.		Extractor nodes are meant to be referenced either by other extractor nodes or, importantly, by Data Elements such as '''Data Fields''' in a '''Data Model'''.
Line 121:		Line 130:
	=== Machine learning ===		=== Machine learning ===
	Many documents contain important pieces of information buried within the flow of text, like a legal document. These types of documents and the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" extractor known as a '''Field Class''' to leverage machine learning algorithms to target important information.		Many documents contain important pieces of information buried within the flow of text, like a legal document. These types of documents and the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" extractor known as a '''Field Class''' to leverage machine learning algorithms to target important information.
			*<li class="attn-bullet"> Advances in large-language models (LLMs) have largely made Field Classes obsolete. LLM-based extraction methods in Grooper (such as [[AI Extract]]) can achieve similar results with nowhere near the amount of set up.

	=== Extractor Nodes vs Value Extractors ===		=== Extractor Nodes vs Value Extractors ===
Line 130:		Line 140:
	* Example" "Field Classes" cannot function without its "Value Extractor" and "Feature Extractor" properties configured, both of which specify a Value Extractor.		* Example" "Field Classes" cannot function without its "Value Extractor" and "Feature Extractor" properties configured, both of which specify a Value Extractor.

	However, ~~extractor nodes~~ are used when you need to ''reference'' them for their designated strengths:		However, Extractor Nodes are used when you need to ''reference'' them for their designated strengths:
	* re-usability		* re-usability
	* collation		* collation
	* machine learning		* machine learning
			**<li class="attn-bullet"> Advances in large-language models (LLMs) have largely made Field Classes obsolete. LLM-based extraction methods in Grooper (such as [[AI Extract]]) can achieve similar results with nowhere near the amount of set up.

	=== Related Node Types ===		=== Related Node Types ===

Dgreenwood: /* Value Extractors */

2025-08-27T16:22:04Z

Value Extractors

← Older revision		Revision as of 11:22, 27 August 2025
Line 12:		Line 12:

	== Value Extractors ==		== Value Extractors ==
			<blockquote>{{#lst:Glossary\|Value Extractor}}</blockquote>

	"Value Extractors" are the primitive operators in Grooper that perform "data extraction". They read data values from the text (and sometimes visual content) of a page or document. This is the foundation of Grooper's capability to locate and collect document data, such as dates, numbers, entity names, barcodes, checkboxes, text labels, paragraphs, and more.		"Value Extractors" are the primitive operators in Grooper that perform "data extraction". They read data values from the text (and sometimes visual content) of a page or document. This is the foundation of Grooper's capability to locate and collect document data, such as dates, numbers, entity names, barcodes, checkboxes, text labels, paragraphs, and more.

Dgreenwood at 15:53, 27 August 2025

2025-08-27T15:53:51Z

@@ Line 4: / Line 4: @@
 Data extractors (or simply "extractors") are used in a variety of ways, including (but not limited to):
 Extractors are highly configurable in terms of how data is targeted, how it is ordered and sorted, how text is pre-processed, what tolerance the extractor has for "fuzzy" results, how results are post-processed and more.  However, at their core, extractors are simply tools used to parse text data from a larger text source (e.g. a single field from a whole document).
-== Extractor Types ==
+== Value Extractors ==
-{{#lst:Extractor Type (Property)|Extractor Types}}
+"Value Extractors" are the primitive operators in Grooper that perform "data extraction". They read data values from the text (and sometimes visual content) of a page or document. This is the foundation of Grooper's capability to locate and collect document data, such as dates, numbers, entity names, barcodes, checkboxes, text labels, paragraphs, and more.
 == Extractor Nodes==

Dgreenwood: /* Extractor Nodes */

2025-08-06T20:04:46Z

Extractor Nodes

← Older revision		Revision as of 15:04, 6 August 2025
Line 19:		Line 19:
	: {{ValueReaderIcon}} '''[[Value Reader]]'''		: {{ValueReaderIcon}} '''[[Value Reader]]'''
	: {{DataTypeIcon}} '''[[Data Type]]'''		: {{DataTypeIcon}} '''[[Data Type]]'''
	: {{FieldClassIcon}} '''[[Field Class]]'''		: {{FieldClassIcon}} '''[[Field Class]]''' ''Uncommonly used in current Grooper implementations''
	All three of these node types perform a similar function. They are configured to return data from documents. However, they differ in their configuration and data extraction purpose.		All three of these node types perform a similar function. They are configured to return data from documents. However, they differ in their configuration and data extraction purpose.

Line 40:		Line 40:
	Many documents contain important pieces of information buried within the flow of text, like a legal document. These types of documents and the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" extractor known as a '''Field Class''' to leverage machine learning algorithms to target important information.		Many documents contain important pieces of information buried within the flow of text, like a legal document. These types of documents and the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" extractor known as a '''Field Class''' to leverage machine learning algorithms to target important information.

	=== Extractor ~~nodes~~ vs Value Extractors ===		=== Extractor Nodes vs Value Extractors ===
	Extractor nodes should not be confused with "Value Extractors". There are ''many'' places in '''Grooper''' where extraction logic can be applied for one purpose or another. In these cases a Value Extractor is chosen to define the logic required to return a desired value~~. In fact, the extractor nodes themselves each leverage specific Value Extractors to define their logic~~.		Extractor nodes should not be confused with "Value Extractors". There are ''many'' places in '''Grooper''' where extraction logic can be applied for one purpose or another. In these cases a Value Extractor is chosen to define the logic required to return a desired value.

	Value ~~Extractor examples:~~		In fact, the Extractor Nodes themselves will leverage specific Value Extractors to define their logic.
	* "~~Pattern-Match~~" ~~uses regular expressions to return results.~~		* Example: "Value Readers" are configured using a single property "Extractor". This property specifies a single Value Extractor which determines how data is extracted. Value Readers are essentially an encapsulation of a single Value Extractor configuration that can be reused by multiple other extraction elements and properties, such as Data Fields and Data Types.
	* "~~Labeled OMR~~" ~~uses~~ a ~~regex and computer vision to return results for checkboxes~~.		* Example: "Data Types" have several properties that can be configured with Value Extractors, including its "Local Extractor", "Input Filter", and "Exclusion Extractor" properties.
	* Other Value ~~Extractors may use~~ a ~~combination of~~ Value ~~Extractors~~ that ~~work together to return results in specific ways~~.		* Example" "Field Classes" cannot function without its "Value Extractor" and "Feature Extractor" properties configured, both of which specify a Value Extractor.
	** The Labeled Value is a Value Extractor that ~~defines its own set of~~ Value Extractors~~: One for both~~ its ~~Label~~ Extractor and Value Extractor properties.

	However, extractor nodes are used when you need to ''reference'' them for their designated strengths:		However, extractor nodes are used when you need to ''reference'' them for their designated strengths:

Dgreenwood: /* Extractor Nodes */

2025-08-06T19:51:18Z

Extractor Nodes

← Older revision		Revision as of 14:51, 6 August 2025
Line 15:		Line 15:

	== Extractor Nodes==		== Extractor Nodes==
	<section begin="Extractor ~~Objects~~" />		<section begin="Extractor Nodes" />
	There are three types of ~~"extractor nodes"~~ in '''Grooper''':		There are three types of Extractor Nodes in '''Grooper''':
	: {{ValueReaderIcon}} '''[[Value Reader]]'''		: {{ValueReaderIcon}} '''[[Value Reader]]'''
	: {{DataTypeIcon}} '''[[Data Type]]'''		: {{DataTypeIcon}} '''[[Data Type]]'''
Line 72:		Line 72:
	</div>		</div>
	</div>		</div>
	<section end="Extractor ~~Objects~~" />		<section end="Extractor Nodes" />

Dgreenwood: /* Extractor Nodes */

2025-04-28T16:33:29Z

Extractor Nodes

← Older revision		Revision as of 11:33, 28 April 2025
Line 16:		Line 16:
	== Extractor Nodes==		== Extractor Nodes==
	<section begin="Extractor Objects" />		<section begin="Extractor Objects" />
	There are three types of "~~Extractor Objects~~" in '''Grooper''':		There are three types of "extractor nodes" in '''Grooper''':
	: {{ValueReaderIcon}} '''[[Value Reader]]'''		: {{ValueReaderIcon}} '''[[Value Reader]]'''
	: {{DataTypeIcon}} '''[[Data Type]]'''		: {{DataTypeIcon}} '''[[Data Type]]'''
	: {{FieldClassIcon}} '''[[Field Class]]'''		: {{FieldClassIcon}} '''[[Field Class]]'''
	All three of these ~~objects~~ perform a similar function. They ~~are objects that~~ are configured to return data from documents. However, they differ in their configuration and data extraction purpose.		All three of these node types perform a similar function. They are configured to return data from documents. However, they differ in their configuration and data extraction purpose.

	"Extractor ~~Objects"~~ are tools to extract/return data. Ultimately, "Data ~~Elements~~" are what collects data. They may ''use'' extractor ~~objects~~ to help collect data in a '''Data Model'''.		Extractor nodes are tools to extract/return data. Ultimately, "[[Data Element]]s" are what collects data. They may ''use'' extractor nodes to help collect data in a '''Data Model'''.

	To that end, extractor ~~objects~~ serve three purposes:		To that end, extractor nodes serve three purposes:
	# To be re-usable units of extraction		# To be re-usable units of extraction
	# To collate data		# To collate data
	# To leverage machine learning algorithms to target data in the flow of text		# To leverage machine learning algorithms to target data in the flow of text
	<div style="padding-left: 1.5em">		<div style="padding-left: 1.5em">
	=== Re-~~Usability~~ ===		=== Re-usability ===
	"Extractor ~~Objects"~~ are meant to be referenced either by other ~~"Extractor Objects"~~, ~~or more~~ importantly, by "Data Elements~~". For example, an individual '''Data Field''' can be configured on its own to collect a date value,~~ such as ~~the "Received Date" on an invoice. However, what if another~~ '''Data ~~Field~~''' ~~is collectig~~ a different date format, like the "Due Date" on the same invoice? In this case you would create one "Extractor Object", like a '''Value Reader''', to collect any and all date formats. You could then have each '''Data ~~Field''' reference that ''one~~'' '~~''Value Reader''' and further configure each individual '''Data Field''' to differentiate their specific date value~~.		Extractor nodes are meant to be referenced either by other extractor nodes or, importantly, by Data Elements such as '''Data Fields''' in a '''Data Model'''.

	~~=== Data Collation ===~~		For example, an individual '''Data Field''' can be configured on its own to collect a date value, such as the "Received Date" on an invoice. However, what if another '''Data Field''' is collecting a different date format, like the "Due Date" on the same invoice? In this case you would create one extractor node, like a '''Value Reader''', to collect any and all date formats. You could then have each '''Data Field''' reference that ''single'' '''Value Reader''' and further configure each individual '''Data Field''' to differentiate their specific date value.
	~~Another~~ example ~~would be configuring a~~ '''Data ~~Type~~''' to ~~target entire rows of information within~~ a ~~table of data~~. ~~Several~~ '''~~Value Reader~~''' "~~Extractor Objects~~" ~~could be made as children of~~ the '''~~Data Type~~''', each ~~targeting a specific value within the table row. The parent~~ '''Data ~~Type~~''' ~~would then collate the results of its child~~ '''~~Value Reader~~''' ~~"Extractor Objects" into one result. A~~ '''~~Data Table~~'~~'' would then reference the~~ '''Data ~~Type~~''' to ~~collect the appropriate rows of information~~.

	=== ~~Machine Learning~~ ===		=== Data collation ===
	~~Many documents contain important pieces~~ of information ~~buried~~ within the ~~flow of text~~, ~~like~~ a ~~legal document~~. ~~These types~~ of ~~documents and~~ the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" "Extractor Object" known as a '''~~Field Class~~''' to ~~leverage machine learning algorithms to target important~~ information.		Another example would be configuring a '''Data Type''' to target entire rows of information within a table of data. Several '''Value Reader''' nodes could be made as children of the '''Data Type''', each targeting a specific value within the table row. The parent '''Data Type''' would then collate the results of its child '''Value Reader''' nodes into one result. A '''Data Table''' would then reference the '''Data Type''' to collect the appropriate rows of information.

	=== ~~Extractor Objects vs Value Extractors~~ ===		=== Machine learning ===
	"~~Extractor Objects~~" ~~should not be confused with "Value Extractors". There are ''many~~'' ~~places in~~ '''~~Grooper~~'~~'' where extraction logic can be applied for one purpose or another. In these cases a "Value Extractor" is chosen to define the logic required~~ to ~~return a desired value. In fact, the "Extractor Objects" themselves each~~ leverage ~~specific "Value Extractors"~~ to ~~define their logic~~.		Many documents contain important pieces of information buried within the flow of text, like a legal document. These types of documents and the data they contain require an entirely different approach to extracting data than a highly structured document like an invoice. For these situations you can use a "trainable" extractor known as a '''Field Class''' to leverage machine learning algorithms to target important information.

	~~"Value~~ Extractor~~" examples:~~		=== Extractor nodes vs Value Extractors ===
	* ''Pattern-Match'' uses regular expressions to return results.		Extractor nodes should not be confused with "Value Extractors". There are ''many'' places in '''Grooper''' where extraction logic can be applied for one purpose or another. In these cases a Value Extractor is chosen to define the logic required to return a desired value. In fact, the extractor nodes themselves each leverage specific Value Extractors to define their logic.
	* ''Labeled OMR'' uses a regex and computer vision to return results for checkboxes.
	* Other "Value Extractors~~" may use a combination of~~ "Value Extractors" ~~that work together to return results in specific ways~~.
	** The ''~~Labeled Value~~'' ~~"Value Extractor" defines a "Value Extractor" for both its '~~'''~~'Label Extractor''''' and ''~~'''Value Extractor~~''''' properties~~.

	However, ~~"Extractor Objects"~~ are used when you need to ''reference'' them for their designated strengths:		Value Extractor examples:
	* re-~~usbaility~~		* "Pattern-Match" uses regular expressions to return results.
			* "Labeled OMR" uses a regex and computer vision to return results for checkboxes.
			* Other Value Extractors may use a combination of Value Extractors that work together to return results in specific ways.
			** The Labeled Value is a Value Extractor that defines its own set of Value Extractors: One for both its Label Extractor and Value Extractor properties.

			However, extractor nodes are used when you need to ''reference'' them for their designated strengths:
			* re-usability
	* collation		* collation
	* machine learning		* machine learning

	=== Related ~~Objects~~ ===		=== Related Node Types ===
	<div style="padding-left: 1.5em">		<div style="padding-left: 1.5em">
	==== Value Reader ====		==== Value Reader ====
Line 60:		Line 62:
	{{#lst:Glossary\|Data Type}}		{{#lst:Glossary\|Data Type}}

	* For example, if you're extracting a date that could appear in multiple formats within a document, you'd use various ~~"Extractor Objects"~~ (each capturing a different format) as children of a '''Data Type'''.		* For example, if you're extracting a date that could appear in multiple formats within a document, you'd use various extractor nodes (each capturing a different format) as children of a '''Data Type'''.

	The '''Data Type''' also defines how to collate results from one or more extractors into a referenceable output. The simplest type of collation (''Individual'') would just return all individual extractors' results as a list of results.		The '''Data Type''' also defines how to collate results from one or more extractors into a referenceable output. The simplest type of collation (''Individual'') would just return all individual extractors' results as a list of results.

Dgreenwood: /* Extractor Objects */

2025-04-28T16:24:17Z

Extractor Objects

← Older revision		Revision as of 11:24, 28 April 2025
Line 14:		Line 14:
	{{#lst:Extractor Type (Property)\|Extractor Types}}		{{#lst:Extractor Type (Property)\|Extractor Types}}

	== Extractor ~~Objects~~ ==		== Extractor Nodes==
	<section begin="Extractor ~~Nodes~~" />		<section begin="Extractor Objects" />
	There are three types of "Extractor Objects" in '''Grooper''':		There are three types of "Extractor Objects" in '''Grooper''':
	: {{ValueReaderIcon}} '''[[Value Reader]]'''		: {{ValueReaderIcon}} '''[[Value Reader]]'''