Data Validation in Grooper: Difference between revisions

From Grooper Wiki
No edit summary
// via Wikitext Extension for VSCode
 
(34 intermediate revisions by 2 users not shown)
Line 1: Line 1:
"Field validation" refers to the various ways Grooper can warn users a '''Data Field''' or '''Data Column''' cell value may be inaccurate (e.g. invalid).  Invalid fields are set to an error state, making it more obvious for reviewers something is wrong.  The '''Data Viewer''' may also be configured such that the '''Review''' task cannot be completed until the error is resolved.
{{AutoVersion}}


== Ways to validate data in Grooper ==
{|class="download-box"
|
[[File:Asset 22@4x.png]]
|
You may download the ZIP(s) below and upload it into your own Grooper environment (version 2025). The first contains one or more '''Batches''' of sample documents. The second contains one or more '''Projects''' with resources used in examples throughout this article.
* [[Media:2025 Data-Validation Batches.zip]]
* [[Media:2025 Data-Validation Projects.zip]]
|}


There are several ways data in a '''Data Model''' is validated in Grooper:
== Introduction ==
* The '''Data Field/Data Column's''' '''''Value Type''''' configuration validates data based on what type of information is expected (e.g. decimal values, datetime values, string values, or other types of values)
'''Data Validation''' in Grooper ensures values extracted into a [[Data Field]] are correct, complete, and consistent before moving downstream. Validation can:
* "Required" fields are considered invalid if no value is collected.
* Prevent invalid formats (dates, numbers, emails)
* Is Valid expressions  
* Enforce required values
* Calculated Value expressions (when using "Validate" mode or "SetIfEmpty" mode and a value is present)
* Check values against business rules
* Verify against external systems (e.g., databases)
Grooper validates using properties on the [[Data Field]], expressions, external [[Lookup]]s, and rule-driven actions. Validation occurs when fields are set, formatted, re-validated in design/test, and during Batch processing.
 
The different methods of data validation in Grooper discussed in this article include:
* Value Types
* Required Fields
* Calculated Value expressions
* Is Valid expressions
* Is Required expressions
* Database Lookups
* Database Lookups
* Data Rules
* Data Rules and Data Actions
 
== Value Types ==
The "Value Type" determines how a Data Field's value is parsed, formatted, and validated. It controls the primitive type (text, number, date/time, boolean), array support, and formatting rules. Invalid values will fail parsing/validation and can be flagged by the field.
 
'''Examples:'''
* Validate a "Invoice Date" as a date only. Non-date text will be considered invalid.
* Validate a "Unit Price" as numeric. Non-numeric text will be considered invalid.
 
=== How to set the Value Type ===
# On the Grooper Design Page, navigate to a Data Field or Data Column.
# Set "Value Type" property to the desired type (e.g., Date/Time, Numeric, String).
# Expand out the "Value Type" sub properties and edit the Format Specifier if desired.
#* The format specifier will change the extracted value to a specified format for that specific Value Type. For example, making sure all dates are returned in the MM/DD/YYYY format.
# (Optional) Set "Processing Mode" to '''Validate''' for validation-only or '''Both''' to format and validate.
# (Optional) For multi-values, set "Value Delimiter" (e.g., <code>;</code>) to validate each item.
# Save and test. Enter sample values; invalid inputs will not parse or will show error state.
 
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cml5cxvhm0047yw0jkiwhmsgr?embed_v=2&utm_source=embed" loading="lazy" title="01 What are Value Types?" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cml5cy9b4003fx90idyi6uias?embed_v=2&utm_source=embed" loading="lazy" title="02 Format Specifiers(Copy)" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
=== Value Types properties overview ===
 
This section lists the core properties that define a '''Value Type''' (Storage Type) used by a [[Data Field]]. Each item includes its name, definition, remarks, and typical purpose/use case.
 
'''"Processing Mode"'''
*'''Definition:''' Controls how values are processed by the Value Type (Format, Filter, Both, Validate).
*'''Remarks:''' Determines whether values are reformatted, filtered, both, or only validated. Use '''Format''' to reformat without filtering, '''Filter''' to filter without formatting, '''Both''' to apply both operations, and '''Validate''' for validation-only scenarios.
*'''Purpose/Use case:''' Choose the level of processing needed for your field. For example, use '''Validate''' when you only need to confirm values without altering their presentation; use '''Both''' to standardize display while ensuring values meet constraints.
 
'''"Value Delimiter"'''
*'''Definition:''' The delimiter used to separate multiple values stored in a single field.
*'''Remarks:''' Enables array-style storage when non-empty. Common choices include <code>,</code> or <code>;</code>, or custom strings such as <code>|</code>. Grooper splits and joins using this delimiter and will parse/format each item independently.
*'''Purpose/Use case:''' Allow multi-value lists (e.g., multiple dates, codes, IDs) in one Data Field while retaining strict per-item validation and formatting.
 
'''"Windows Type"'''
*'''Definition:''' The underlying .NET type (e.g., string, decimal, DateTime) used at runtime by the Value Type.
*'''Remarks:''' Controls parsing and formatting behavior. For arrays, the runtime element type is the item type within the collection.
*'''Purpose/Use case:''' Ensure the Data Field uses the correct primitive type for both validation and formatting (e.g., numbers vs. dates).
 
'''"CMIS Type"'''
*'''Definition:''' The Value Type’s CMIS-compatible property type.
*'''Remarks:''' Used where CMIS integration or property typing is required.
*'''Purpose/Use case:''' Maintain compatibility with CMIS repositories when exporting or syncing metadata.
 
'''"SQL Type"'''
*'''Definition:''' The database type string associated with this Value Type.
*'''Remarks:''' Useful for mapping to relational storage or validating schema alignment when persisting Data Field values.
*'''Purpose/Use case:''' Provide appropriate database typing for storage or integration.
 
'''"Name"'''
*'''Definition:''' The display name of the Value Type.
*'''Remarks:''' Reflects the Storage Type’s friendly name and indicates array status in UI (e.g., "Date Array" when multi-value).
*'''Purpose/Use case:''' Inform users which type (and array status) the field is using during configuration and review.
 
'''"Default Value"'''
*'''Definition:''' The default, typed value for the Value Type.
*'''Remarks:''' Used for testing, formatting demonstrators, or to initialize parsing/format checks (e.g., format validation on edit).
*'''Purpose/Use case:''' Provide a baseline value for format preview or validation diagnostics.
 
'''"Is Quoted"'''
*'''Definition:''' Indicates whether values should be quoted when substituted (e.g., in SQL or external queries).
*'''Remarks:''' Typically <code>false</code> for numeric types, <code>true</code> for string-like types when embedded in queries.
*'''Purpose/Use case:''' Prevent syntax errors or injection issues when building dynamic queries (e.g., Database Lookups).
 
'''"Is Formattable"'''
*'''Definition:''' Specifies if the Value Type supports custom formatting.
*'''Remarks:''' When <code>true</code>, format strings and culture are applied to produce display-appropriate values.
*'''Purpose/Use case:''' Standardize how values appear to users (e.g., currency, dates, number precision).
 
'''"Format String"'''
*'''Definition:''' The format specifier applied when formatting values.
*'''Remarks:''' Validated against the type; invalid specifiers are flagged during property validation. Used only for types that support formatting.
*'''Purpose/Use case:''' Control precision, style, or presentation (e.g., <code>c</code> for currency, date masks).
 
'''"Choice Values"'''
*'''Definition:''' The list of allowed or suggested values exposed by the Value Type (if it is a choice type).
*'''Remarks:''' Used when the type itself provides a choice list (e.g., Booleans, predefined enumerations). Each value is validated against the type.
*'''Purpose/Use case:''' Limit input to a defined set of values and enable dropdowns/autocomplete consistent with the type’s constraints.
 
'''"Is Choice"'''
*'''Definition:''' Indicates whether the Value Type enforces a discrete set of allowed values.
*'''Remarks:''' If <code>true</code>, input is restricted to the provided choices. Otherwise, a field’s Lexicon or list settings may supply choices separately.
*'''Purpose/Use case:''' Enforce controlled vocabularies or simple yes/no style inputs at the type level.
 
'''"Validation Notes"'''
*'''Definition:''' Optional notes shown when validation fails, describing format/rules.
*'''Remarks:''' Included with error messages to guide users when a typed value is invalid (e.g., acceptable range or format).
*'''Purpose/Use case:''' Provide clear guidance during Review when correcting invalid entries.
 
'''"Runtime Type"'''
*'''Definition:''' The effective CLR type used at runtime; array-aware when "Value Delimiter" is set.
*'''Remarks:''' Equals "Array Type" for multi-value fields, otherwise equals "Windows Type".
*'''Purpose/Use case:''' Helps ensure expression engines, lookups, and formatters treat values with the correct runtime type.
 
'''"Array Type"'''
*'''Definition:''' The array type corresponding to the Value Type (e.g., <code>string[]</code>, <code>DateTime[]</code>).
*'''Remarks:''' Created by <code>WindowsType.MakeArrayType()</code> when "Value Delimiter" is set.
*'''Purpose/Use case:''' Provide strongly typed multi-value support across parsing/formatting pipelines.
 
'''"Is Array"'''
*'''Definition:''' Indicates whether the Value Type treats field content as an array.
*'''Remarks:''' True when "Value Delimiter" is non-empty. Parsing/formatting operate per item; validation iterates over each value.
*'''Purpose/Use case:''' Enable multi-entry fields (e.g., multiple codes) with true typed validation for each element.
 
'''"Join Delimiter"'''
*'''Definition:''' The delimiter used when joining formatted items back into a single string.
*'''Remarks:''' Derived from "Value Delimiter"; supports escaped control characters (e.g., <code>\r</code>, <code>\n</code>, <code>\t</code>).
*'''Purpose/Use case:''' Control the final presentation of multi-value text after formatting.
 
'''"Split Delimiter"'''
*'''Definition:''' The delimiter used to split incoming text into separate items.
*'''Remarks:''' Trims spaces around the delimiter and supports escaped control characters. Newlines are also treated as separators when splitting.
*'''Purpose/Use case:''' Ensure reliable parsing of multi-item inputs, even with varied line endings or spaced delimiters.
 
=== Practical guidance ===
* Use "Processing Mode" = '''Validate''' when you only want to test correctness without changing user-entered formatting; use '''Both''' for consistent display and enforced rules.
* For multi-value fields, set "Value Delimiter" and rely on "Is Array" behavior to validate each entry independently.
* For database queries in [[Lookup]]s, ensure "Is Quoted" matches the underlying type to prevent SQL issues.
* Use "Format String" with "Is Formattable" types (e.g., Date/Time, Numeric) to present values clearly in Review and exports.
* When publishing choice-driven fields, prefer "Is Choice" with "Choice Values" or use a [[Lexicon]] on the Data Field if choices are field-specific rather than type-level.
 
== Required fields ==
The "Required" property enforces that a Data Field must contain a value by the end of extraction/validation. Empty values will be flagged for review.
 
When to use:
* Key identifiers (e.g., "Invoice Number")
* Compliance-critical fields (e.g., "Tax ID")
* Downstream-required fields (e.g., "Customer ID")
 
'''Example'''
* Require "Vendor ID" on invoices. If empty after extraction, the field is flagged to ensure correction during Review.
 
'''Steps'''
# Navigate to the target Data Field or Data Column.
# Locate and enable the "Required" property in the property grid.
# Save and test extraction. Empty fields will show an error state until a value is assigned.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cml6oj32g003sy80i2ziewtdr?embed_v=2&utm_source=embed" loading="lazy" title="03 Required Fields" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
== Calculated Value expressions ==
A "Calculated Value" expression validates, derives, or normalizes a value. Use it to compute results or to re-check consistency with "Calculate Mode"/tolerance. Calculation can be paired with validation to detect mismatches.
 
Calculated Value can be used to:
 
* Normalize presentation (e.g., format phone numbers)
* Derive values (e.g., "Amount" = "Quantity" × "Unit Price")
* Cross-field consistency checks
 
'''Example'''
* Validate line item totals:
<code>
@Calculated_Total == (@Quantity * @Unit_Price)
</code>
* Auto-calculate "Calculated Total":
<code>
@Quantity * @Unit_Price
</code>
 
In our case, we are specifically looking at using Calculated Value for validation.
 
'''How to configure'''
# Navigate to to the target [[Data Field]] (e.g., "Subtotal").
# Make sure the Value Type property is set to a valid Value Type, such as Decimal. You cannot calculate string values.
# Locate the Calculated Value property in the property grid in the Expressions category.
# Write an expression referencing other fields or Data Columns (e.g., <code>Line_Items.SumOf("Extended Price")</code>).
# Make sure the Calculate Mode property is set to "Validate".
# You can adjust the Calculate Tolerance property for numeric rounding differences.
# Save and test. Mismatches will flag the field.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmlfapxou01ka1l0ix1lpdo8k?embed_v=2&utm_source=embed" loading="lazy" title="04 Using Calculated Value for Validation" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
 
== Is valid expressions ==
 
An '''Is Valid''' expression is a custom validation rule you can add to a [[Data Field]] to decide whether the field's value is acceptable. The expression is written as a Visual Basic (VB.NET) expression and must evaluate to '''True''' or '''False'''. If it evaluates to '''False''', the field is marked invalid and will show an error during review.
 
Is Valid expressions are useful when simple validation (such as "Value Type" conversion or "Required") is not enough. They let you enforce business rules, compare values across fields, and validate values against document context.
 
=== When to use Is Valid ===
 
Use an "Is Valid" expression when you need validation that depends on logic rather than formatting. Common scenarios include:
 
* '''Range checks''' (values must be within a minimum/maximum).
* '''Cross-field validation''' (one field must match or relate to another field).
* '''Date rules''' (dates cannot be in the future, or must be within a specific window).
* '''Pattern or character rules''' beyond basic parsing (for example, disallow certain characters).
* '''Consistency checks''' against calculated totals (without automatically setting the value).
 
If your goal is to compute an expected value and optionally populate the field, consider using "Calculated Value" instead of (or in addition to) "Is Valid".
 
=== How it works ===
 
# A [[Data Field]] gets a value (typically from a Value Extractor, or entered by a user).
# Grooper validates the value type (for example, whether it can be parsed as a date or number based on the field's "Value Type").
# If the field has an "Is Valid" expression, Grooper evaluates it.
#* If the expression returns '''True''', validation passes.
#* If the expression returns '''False''', the field is set to an error state.
#* If "Validate Message" is configured, Grooper displays that message to the user when validation fails.
 
'''Tip:''' Use "Validate Message" to provide clear instructions to operators about what needs to be corrected.
 
 
=== Configure Is Valid on a Data Field ===
 
# In Grooper Design, select the [[Data Model]] (or a [[Data Section]]) containing your [[Data Field]].
# Click the [[Data Field]] you want to validate.
# In the property grid, locate the "Is Valid" property in the Expressions category.
# Set the "Is Valid" property to a VB.NET expression that returns a Boolean (True/False).
# (Optional) Set the "Validate Message" property to a VB.NET expression that returns a string. This message is shown when "Is Valid" evaluates to False.
# Save your changes.
# Test the validation by testing extraction on a Test Batch and reviewing the field's status.
 
=== Examples ===
 
==== Example 1: equality of two Fields ====
 
The goal in this first example is to determine if the value in one field is the same as the value in another field. So, in our case, we are going to make sure that the total on an invoice below the line items table matches the total quoted at the top of the Invoice.
 
# Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Please Pay Data Field.
# Scroll through the property grid and locate the Is Valid property.
# Click the "..." to the right of the property to open the editor.
# Type in your VB.NET code for your Is Valid expression. In this example our expression is: <code>Please_Pay = Total_Due</code>
# Click "OK" on the Is Valid editor. 
# (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: <code>"Please Pay does not equal Total Due"</code>
# Save your changes to the Data Field and test your extraction at the Data Model level.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmnen8xtb000exq0k76wxb3dz?embed_v=2&utm_source=embed" loading="lazy" title="05 IsValid - Equality of Two Fields" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
==== Example 2: sum of a Data Column ====
 
While the general instructions remain the same for every Is Valid use, the expression you choose to use changes how the validation works. Let's look at a second example. In this example we are going to tell Grooper to sum up the Extended Price column in our Line Items table and make sure that total is equal to the extracted value in the Subtotal Data Field.
 
# Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Subtotal Data Field.
# Scroll through the property grid and locate the Is Valid property.
# Click the "..." to the right of the property to open the editor.
# Type in your VB.NET code for your Is Valid expression. In this example our expression is: <code>Subtotal = SumFieldInstance("Line Items\Extended Price")</code>
# Click "OK" on the Is Valid editor. 
# (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: <code>"Subtotal does not equal sum of Extended Price"</code>
# Save your changes to the Data Field and test your extraction at the Data Model level.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmnena53m002mwo0inc5f5nyr?embed_v=2&utm_source=embed" loading="lazy" title="06 Is Valid - Sum of a Data Column" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
==== Example 3: Date Validation ====
 
Is Valid expressions can also do more than just compare two fields to see if they are equal. In this example we're going to use an expression to validate whether or not a date is valid. A date will be considered valid if it takes place in the past. If you receive an invoice with a date in the future, that Invoice would not be valid.
 
# Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Invoice Date Data Field.
# Scroll through the property grid and locate the Is Valid property.
# Click the "..." to the right of the property to open the editor.
# Type in your VB.NET code for your Is Valid expression. In this example our expression is: <code>Invoice_Date < Now</code>
# Click "OK" on the Is Valid editor. 
# (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: <code>"Date is in the future"</code>
# Save your changes to the Data Field and test your extraction at the Data Model level.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmnena0ha0010wo0iw06c3etk?embed_v=2&utm_source=embed" loading="lazy" title="07 Is Valid - Date Validation" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
== Is Required expressions ==
 
An '''Is Required''' expression is a conditional rule you can add to a Data Field to decide whether the field must contain a value at runtime. Unlike a static required setting, an Is Required expression can evaluate the current document's data and context (for that specific field instance) and then determine if the field should be treated as required.
 
You would use an Is Required expression when a field is only mandatory some of the time, such as when another field has a particular value, when a total is above a threshold, or when a document meets specific business conditions.
 
In data validation, Is Required expressions help Grooper enforce “requiredness” rules that are dependent on the extracted data, improving data quality without forcing operators to fill out fields that do not apply.
 
=== How Is Required differs from "Required" (Required Fields) ===
 
The "Required" property is a '''static''' rule:
* If "Required" is enabled, the Data Field must always have a value.
* If the field is empty, the field is set to an error state with a required-value message.
 
The "Is Required" property is a '''dynamic''' rule:
* If "Is Required" is set, Grooper evaluates the expression for each field instance.
* If the expression evaluates to '''True''' and the field is empty, the field is treated as required and becomes invalid until a value is provided.
* If the expression evaluates to '''False''', the field is allowed to remain empty without error.
 
=== How Is Required differs from "Is Valid" and "Calculated Value" ===
 
"Is Required" answers: '''Must this field have a value?'''
* It is primarily an “empty value” rule (required vs optional).
* It is used when the main question is whether blank is acceptable.
 
"Is Valid" answers: '''Is the current value acceptable?'''
* It is evaluated when there is a value present.
* It is used for business-rule validation such as ranges, formats beyond "Value Type", or cross-field checks.
 
"Calculated Value" answers: '''What should the value be?'''
* It computes an expected value (and can optionally set the field's value depending on "Calculate Mode").
* It is best for totals, derived values, and consistency checks where Grooper can calculate what the value should be.
 
In practice:
* Use "Is Required" when the field may be blank sometimes.
* Use "Is Valid" when the field has a value but you need to validate correctness.
* Use "Calculated Value" when the value should be derived from other data (and possibly auto-filled).
 
 
=== How Is Required is used during validation ===
 
During validation, Grooper checks whether a Data Field is empty:
* If the field is empty and "Required" is enabled, the field is invalid.
* If the field is empty and an "Is Required" expression exists, Grooper evaluates the expression:
** If it returns '''True''', the field becomes invalid until a value is entered.
** If it returns '''False''', the field is allowed to remain blank.
 
This allows the Data Model to enforce mandatory data only when it applies, reducing unnecessary errors in Review while still ensuring required information is captured when needed.
 
=== How To ===
 
# Navigate a Data Field and locate the Is Required property in the Field's property grid.
# Click the "..." to the right of the Is Required property.
# When the Is Required editor pops up, begin typing in your Is Required expression in the text box.
## In our example we are using the following expression: <code>PO_Number <> ""</code>
# Click "OK" on the editor when finished.
# Save your changes to the Data Field.
# Test your extraction to make sure the Is Require expression is having the desired outcome.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmnj28rjf04aiwn0j9kgp2u5d?embed_v=2&utm_source=embed" loading="lazy" title="08 Validation Using Is Required" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
 
 
== Database Lookups (validation) ==
 
A '''Database Lookup''' is a type of [[Lookup Specification]] that runs an SQL query against an external database (through a [[Data Connection]]). In a validation scenario, the goal is usually to confirm that one or more extracted values in your [[Data Model]] match something in a database.
 
Database Lookups are useful for validation when “being the right format” is not enough. For example, an invoice number might be formatted correctly and still be invalid because it does not exist for the vendor in your accounting system. A Database Lookup can validate that the value (or combination of values) exists, and optionally return supporting data to confirm the document is consistent.
 
==== Why use Database Lookups for validation ====
 
Database Lookups are the best choice when your validation rule depends on an external source of truth:
 
* Validating IDs (Vendor IDs, Customer IDs, Student IDs, Employee IDs)
* Validating document keys (Purchase Order Numbers, account numbers, policy numbers)
* Validating multi-field relationships (Vendor + Invoice Number, Customer + Account + Location)
* Reducing downstream errors by preventing invalid values from being accepted during Review
 
They are especially valuable when the data is frequently changing and cannot be maintained reliably as a static list (for example, active vendors or current product catalogs).
 
==== How Database Lookups validate data ====
 
Database Lookups validate data by using your extracted [[Data Field]] values as '''query criteria'''.


=== Value Types ===
* You build an SQL query in the lookup's "SQL Query" property.
* In that query, you reference Grooper values using <code>@VariableName</code> syntax (most commonly, <code>@FieldName</code>).
* When the lookup runs, Grooper substitutes the current field values into the query and executes it.
* The lookup is considered a '''hit''' when the query returns exactly '''one''' row.
* If the query returns:
** '''0 rows''' (no match), the lookup can treat that as a validation failure (based on the lookup's Miss Disposition settings).
** '''More than 1 row''' (ambiguous match), the lookup can treat that as a validation failure (based on the Conflict Disposition settings).


=== Required Fields ===
In other words, for validation-only use, the most common pattern is:
* '''Return 1 row = valid'''
* '''Return 0 rows = invalid'''
* '''Return multiple rows = needs attention / invalid'''


=== Is Valid expressions ===
==== How To ====


=== Calculated Value expressions ===
Before you can use Database Lookups for validation, you need:


=== Database Lookups ===
# A configured [[Data Connection]] that can connect to your database.
#* In the Data Connection, set the "Connection Settings" property to the correct provider and supply server/authentication settings.
# A Data Model with the Data Fields you want to validate.


{|class="attn-box"
For the following example, it is assumed you already have a Data Connection configured and you have your Data Model set up.
 
# The Database Lookup can be configured on any container Data Element. Select one, such as the Data Model.
# Click the "..." to the right of the Lookups property in the property grid.
# When the Lookups window pops up, click the "+" icon and select "Database Lookup" from the drop down.
# In the property grid on the right, set the Database Connection property to your configured Data Connection.
# Click the "..." to the right of the SQL Query property.
# Type in your SQL query, using the intellisense to help if desired. The following SQL Query is used in the example below:
<pre>SELECT PO_Total AS PO_Total FROM PO_Totals WHERE PO_Number = @ P_O_Number</pre>
#<li value=7> When finished, click "OK" on the SQL Query editor.
# For validation purposes, leave the Miss Disposition and Conflict Disposition properties at their defaults.
# Click "OK" on the Lookups window.
# Save your changes to the Data Element.
# Test your extraction.
#* If the lookup was successful, you will not see any errors.
#* If the lookup returned no results and you have left the Miss Disposition to defaults, an error will indicate the issue.
#* If the lookup found multiple results and you have left the Conflict Disposition to defaults, an error will indicate the issue. 
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmnoxdjls01js000jld7ojgb8?embed_v=2&utm_source=embed" loading="lazy" title="09 Validation Using Database Lookups" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
 
==== Two-Way Matching ====
 
Grooper can perform Two-Way Matching using Data Lookups.
 
Two way matching is a process commonly used by Accounts Payable to compare the information on an invoice received from a vendor to the original PO. They then can discuss any discrepancies with the vendor. Grooper can actually perform this two way matching validation using lookups.
 
You will need a PO SQL Database to match against. Usually Two-Way Matching is performed on line items in a Data Table, with that in mind, you may need to set up your Lookup on the Data Table rather than a Data Model or Data Section to have access to all objects needed for the lookup.
 
# Click on the Data Table node on the Design Page.
# Locate the Lookups Property and click the "..." icon.
# Add a new Database Lookup and set the Database Connection to an established Data Connection.
# Click the "..." to the right of the SQL Query property.
# Type in your SQL Query into the editor and click "OK". The following Query is used in the example below:
<pre>SELECT * FROM POData WHERE (PO_Number = @#P_O_Number) AND (Item_Number = @Item_Number) AND (Quantity = @Quantity) AND (Unit_Price = @Unit_Price) AND (Extended_Price = @Extended_Price)</pre>
#<li value=6> Since we only want to validate the data and don't want anything overwritten, change the Field Population property from Overwrite to None.
# Click "OK" on the Lookups window.
# Test your extraction. Anything that doesn't match the PO information in the Database will be flagged.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmlr1dnni004gzz0irp5otgfp?embed_v=2&utm_source=embed" loading="lazy" title="08 Two-Way Matching" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
 
== Data Rules and Data Actions (validation) ==
 
A '''[[Data Rule]]''' is a configurable rule that runs against extracted data and can apply conditional logic to a Data Model, Data Section, or Data Table. A Data Rule is made up of two main parts:
 
* A "Scope" (where the rule runs)
* An optional "Trigger" (the condition that determines what happens)
 
A '''[[Data Action]]''' is the operation a Data Rule performs when its "Trigger" evaluates to '''True''' (the "TrueAction") or '''False''' (the "FalseAction"). Data Actions are the “do something” part of the rule. They are commonly used to:
 
* Normalize or standardize values (for example, trimming whitespace or converting text to uppercase)
* Copy or move values between fields
* Assign calculated values
* Run lookups
* Apply structured, multi-step logic by nesting actions and child rules
 
=== Purpose of Data Rules ===
 
The main purpose of Data Rules is to let you apply consistent, repeatable logic to extracted data after extraction. This helps keep extractors focused on '''finding data''' while Data Rules handle '''what to do with the data once it is found'''.
 
Data Rules can run at different levels:
 
* If the "Scope" is a [[Data Model]], the rule runs once per document (on the [[Document Instance]]).
* If the "Scope" is a [[Data Section]], the rule runs once per section instance (on each [[Section Instance]]).
* If the "Scope" is a [[Data Table]], the rule runs once per row (on each [[Table Row Instance]]).
 
=== How Data Rules support validation ===
 
Data Rules themselves are '''not''' validation rules in the same way a Data Field's "Required", "Is Required", "Is Valid", or "Calculated Value" expressions are. Those field-level properties directly determine whether a field is valid and what error message is shown.
 
Instead, Data Rules support validation by adding conditional logic that can:
 
* '''Require data only when it applies''' (conditional requiredness)
* '''Prepare or normalize values''' so other validation methods work reliably
* '''Drive consistency between fields''' by setting values based on other values
* '''Trigger additional logic when conditions are met''' (for example, apply different rules per document scenario)
 
This is especially useful when validation depends on multiple fields, repeating lists (tables/sections), or “if/then” business rules that would be difficult to express on a single field alone.
 
{|class="fyi-box"
|-
|
|
&#9888;
'''FYI'''
|
|
'''''BE AWARE: Setting "Is Valid" expressions on a lookup field can be problematic. Set "Is Valid" expressions on target fields (or other fields in the Data Model) instead.'''''
When a Data Rule changes values, Grooper can re-validate fields that now have values so they can clear previous error states.
|}
 
=== How Data Rules do not validate, but help validation ===
 
A helpful way to think of Data Rules is:
 
* [[Data Field]] validation answers: '''“Is this value valid?”'''
* Data Rules answer: '''“If this is true, what should I do next?”'''
 
So a Data Rule typically:
# Checks a condition using "Trigger"
# Applies a "TrueAction" or "FalseAction"
# Optionally runs child Data Rules (nested steps)
 
This lets you build validation support logic such as:
* “If this invoice is taxable, then Sales Tax must be present.”
* “If Country is United States, then State must be present.”
* “If Line Items exist, then Total must equal the sum of Line Totals (or set Total to the sum).
 
=== When to choose Data Rules for validation workflows ===
 
Use Data Rules when:
 
* You need '''if/then''' logic across multiple fields
* The same rule must be applied repeatedly across a [[Data Table]] row set or [[Data Section]] instances
* You need to '''normalize''' extracted text before other validation rules run
* You want a centralized place to manage multi-step validation support logic (instead of scattering it across many fields)
 
For simple scenarios (one field, simple range checks, always-required fields), prefer field-level settings like "Required", "Is Required", "Is Valid", and "Calculated Value".
 
=== How To ===
 
Now that we understand what Data Rules are, let's learn how to configure them in Grooper.
 
==== Triggers and TrueActions ====
 
A '''TrueAction''' is the “then” part of a [[Data Rule]].
 
* If the "Trigger" expression returns '''True''', Grooper executes the rule's "TrueAction".
* If the "Trigger" is blank, the trigger is treated as always true, so the "TrueAction" runs every time the rule runs.
* After the "TrueAction" runs, any child Data Rules under that rule are executed (in order).
* If the "Trigger" returns '''False''', the "TrueAction" does not run (and the rule may run a "FalseAction" instead, if configured).
 
<big>Why it matters for validation</big>
 
TrueAction is commonly used to apply conditional “if/then” logic that supports validation, such as:
* requiring a value only in certain situations
* normalizing a value before field validation occurs
* setting or clearing values based on business rules
 
<big>How To set up a True Action on a Data Rule</big>
 
# In Grooper Design, create or select a current Data Rule in your Project.
# Set the Data Rule's "Scope" to the Data Element the rule should run on:
#* Data Model = runs once per document
#* Data Section = runs once per section instance
#* Data Table = runs once per table row
# (Optional) Set the rule's "Trigger" to a Boolean expression that determines when the rule should run.
#* If left blank, the trigger is treated as always true (the rule always runs).
# Set the Data Rule's "TrueAction" property:
#* Click the "TrueAction" property.
#* Choose the Data Action type you want to run.
#* Configure the selected Data Action's properties (such as source/target fields, transformation options, lookup settings, etc.).
# Save the rule and test it by running extraction and reviewing results.


Grooper's validation logic has an order of operations that can make it seem like '''''Is Valid''''' expressions on lookup fields do not work.  This is not really the case.  Instead, the lookup's validation is taking priority.


Lookup validation presumes an error should be thrown if the lookup ''misses''. In more nuanced scenarios, the results of the lookup need to be used to validate something based on some condition. '''''Is Valid''''' expressions are useful for validating data using conditional logic.  However, they should not be configured on lookup fields.
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmmkogfi70036yv0jg7u7n9h5?embed_v=2&utm_source=embed" loading="lazy" title="01 Data Rules - True Action" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>


==== False Actions ====


For example, imagine a '''''Database Lookup''''' validating duplicate invoices. The lookup checks to see if an invoice number is found in a database (lookup field = "InvoiceNo") and if it is sets the value of a Grooper field to "True" (target field = "Duplicate"). An '''''Is Valid''''' expression could be used to error a field when "Duplicate" is "True". If set on "InvoiceNo", no error will appear. The lookup is ''successful''. No field is errored because the lookup did ''not'' miss. (Furthermore, no further validation is executed on the lookup field. Because lookups are always executed on field edit, it is most efficient to do things this way). Instead, the '''''Is Valid''''' expression should be configured on the ''target field'' "Duplicate". This will properly error a field in '''Review'''.
A '''False Action''' is the [[Data Action]] a [[Data Rule]] runs when the rule's "Trigger" evaluates to '''False'''. It is the “else” part of the rule.
 
* If the "Trigger" returns '''True''', Grooper runs the rule's "TrueAction".
* If the "Trigger" returns '''False''', Grooper runs the rule's "FalseAction" (if one is configured).
* If no "FalseAction" is set, nothing happens when the trigger is false (the rule simply does not apply its action).
 
<big> Why would you use a False Action? </big>
 
False Actions are useful when you need to handle both outcomes of a condition, for example:
 
* '''Clear or reset data''' when a condition is not met
* '''Skip population''' and keep existing values when a condition is not met
* '''Apply a different cleanup/normalization step''' depending on the document scenario
* '''Enforce “either/or” logic''' (do one thing if true, a different thing if false)
 
<big> How to set up a False Action on a Data Rule </big>
 
# In Grooper Design, create or select the Data Rule you want to configure.
# Set the rule's "Scope" to the Data Element the rule applies to (such as a Data Model, Data Section, or Data Table).
# Enter a Boolean expression in the rule's "Trigger" property.
#* The "FalseAction" property is only shown when "Trigger" is not empty.
# Click the "FalseAction" property.
# Choose the Data Action type you want to run when the trigger is false.
# Configure the selected Data Action's properties as needed.
# Save and test by running extraction and reviewing results.
 
{|class="fyi-box"
|-
|
'''FYI'''
|
* If the "Trigger" is blank, the rule is treated as always true, so "FalseAction" does not apply.
* When the "Trigger" is false, Grooper runs "FalseAction" and skips running child [[Data Rule]]s under that rule.
|}
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmmm7wzys00ef0u0j4goczb1f?embed_v=2&utm_source=embed" loading="lazy" title="02 Data Rules - False Action" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
==== Require Value Data Action (Validation) ====
 
The '''Require Value''' Data Action is used to enforce that one or more selected Data Elements have data. When a required value is missing, the action marks the item as invalid and records an issue. This is most useful for '''conditional requiredness''' (required only when a rule's "Trigger" is true).
 
<big> What it validates </big>
 
Require Value checks the items you select in "RequiredElements" and fails validation when:
 
* A required [[Data Field]] exists but is blank
* A required [[Data Field]] instance does not exist (for multi-instance situations)
* A required [[Data Section]] or [[Data Table]] instance does not exist
* A required collection (multi-instance [[Data Section]] or [[Data Table]]) exists but has no child instances
 
When it finds a problem, it:
* Sets an error on the field (for fields), or adds an error to the container (for sections/tables)
* Adds an issue to the rule result (optionally categorized and with metadata)
 
<big> Why use Require Value instead of a Data Field's "Required" property </big>
 
Use Require Value when the requirement is '''conditional''' or when you need to require:
 
* Multiple fields together based on a single trigger
* A section/table to exist (not just a single field)
* Different required sets in different scenarios (via different rules/triggers)
 
If a field should always be required, use the Data Field "Required" property instead (simpler and always enforced).
 
<big>How to set it up (step-by-step) </big>
 
# In Grooper Design, create or select a Data Rule in your Project.
# Set the rule's "Scope" to where the rule should run:
#* Data Model (validate once per document)
#* Data Section (validate once per section instance)
#* Data Table (validate once per table row)
# (Optional, but common) Set the rule's "Trigger" to the condition that makes values required.
# Set the rule's "TrueAction" to '''Require Value'''.
# Configure the Require Value action:
#* Set "RequiredElements" to the Data Fields (and/or Data Sections / Data Tables) that must be present when the rule applies.
#* (Optional) Set "LogMessage" to a friendlier message to show when the requirement fails.
#* (Optional) Set "Category" to group the issue (useful for reporting/routing).
#* (Optional) Set "Metadata" to attach key/value details (one per line, <code>key=value</code>).
# Save and test by running extraction and validating in Review.
 
<big> Tips </big>
 
* Keep the "Scope" as narrow as possible:
* Use '''Data Table''' scope to validate per-row requirements.
* Use '''Data Section''' scope to validate per-instance requirements.
* Use '''Data Model''' scope for document-level rules that coordinate multiple areas.
* Use a clear "LogMessage" so Review users know what to do next (for example, “PO Number is required when Payment Method is PO.”).
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmmumbaab04x2430je72cru37?embed_v=2&utm_source=embed" loading="lazy" title="03 Required Value Data Action" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
==== Calculate Value Data Rule (Validation) ====
 
The '''Calculate Value''' Data Action computes a value using a "Value Expression" and writes it to a target Data Field (or to a custom variable). By itself, Calculate Value does not “validate” a value. It supports validation by:
 
* populating missing values so required/format validation can pass
* standardizing values so "Value Type" and "Is Valid" validations work consistently
* calculating an “expected” value that another validation step can compare against
 
<big>How to set it up (step-by-step) </big>
 
# In Grooper Design, create or select a Data Rule in your Project.
# Set the rule's "Scope" to where the rule should run:
#* Data Model (validate once per document)
#* Data Section (validate once per section instance)
#* Data Table (validate once per table row)
# (Optional, but common) Set the rule's "Trigger" to the condition that makes values required.
# Set the rule's "TrueAction" to '''Calculate Value'''.
# Configure the Calculate Value action:
#* Set "Target Field" to the Data Field you want populated.
#* Open the "Value Expression" editor and type in an expression for the calculation you want Grooper to run.
# Save and test by running extraction and validating in Review.
 
{|class="important-box"
|
'''!!'''
|
Note that there is no Calculate Mode property on the Data Rule. Calculate Value on a Data Rule cannot validate by itself. It can calculate and then populate Data Fields and then other methods of validation can be applied.  
|}
|}


=== Data Rules ===
<big> Tips </big>
 
* For validation (not population), use a Data Field "Calculated Value" with "Calculate Mode" set to Validate, and use Data Rules to decide when that validation should apply.
* Use Calculate Value in Data Rules when you need conditional logic (if/then) around calculation or normalization.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmmusv4o600sbvm0jzh1gfozc?embed_v=2&utm_source=embed" loading="lazy" title="04 Calculate Value with Data Rules" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>
 
==== Data Lookup Data Action (Validation) ====
 
The '''Data Lookup''' Data Action runs a configured [[Lookup Specification]] and uses the lookup result to either:
* '''validate''' that the current data matches an external source (hit / miss / multiple hits), and/or
* '''populate''' fields (or a collection) with values returned by the lookup.
 
For validation-focused rules, you typically configure the lookup so it behaves like: “If the criteria fields don't return exactly one matching record, treat it as a problem.”
 
===== What you need first (prerequisites) =====
 
# A Data Connection established to connect to the target Database for the lookup.
 
===== How it works in a Data Rule =====
 
When a Data Rule executes:
 
* If "Target Collection" is blank, the action runs the lookup in the current context and applies standard lookup behavior (validation and/or population) based on the Lookup Specification settings.
* If "Target Collection" is set to a Data Table or multi-instance Data Section, the action runs the lookup and copies each returned row into the collection (optionally clearing existing rows first).
 
So, “validation” mainly comes from what the Lookup Specification does when the query returns 0 rows or multiple rows.
 
===== Step-by-step: Use Data Lookup for validation (single record) =====
 
# Create/select a Data Rule.
# Set the Data Rule's "Scope" to the container that holds the criteria fields:
#* Data Model for document-level validation
#* Data Section for per-section-instance validation
#* Data Table for per-row validation
# (Optional) Set "Trigger" so the lookup only runs when you have enough data to validate. For example:
<pre>
Not String.IsNullOrEmpty(PO_Number)
</pre>
# Set "TrueAction" to '''Data Lookup'''.
# In the Data Lookup action, set:
#* "Lookup" = your configured Lookup Specification (for example, a [[Database Lookup]])
#* Leave "Target Collection" blank (validation / population happens in place)
# Test using the rule tester (or run extraction + review).
 
<big>Tips for validation-focused Data Lookups</big>
 
* Use a Data Rule "Trigger" to avoid running the lookup when criteria fields are blank.
* Keep the lookup query focused and return a single row for a valid match (validation is cleanest when “exactly one row” means success).
* If you want strictly validation-only, use queries like <code>SELECT 1</code> and configure the lookup to treat misses/conflicts as errors.
* Use "Target Collection" only when you intentionally want to create rows/instances from the lookup results.
 
<div style="position: relative; box-sizing: content-box; max-height: 80vh; max-height: 80svh; width: 100%; aspect-ratio: 1.78; padding: 40px 0 40px 0;"><iframe src="https://app.supademo.com/embed/cmmuydhpo00mft70j37oy9yyu?embed_v=2&utm_source=embed" loading="lazy" title="05 Data Lookup Data Action" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"></iframe></div>

Latest revision as of 09:03, 14 April 2026

This article is about the current version of Grooper.

Note that some content may still need to be updated.

2025

You may download the ZIP(s) below and upload it into your own Grooper environment (version 2025). The first contains one or more Batches of sample documents. The second contains one or more Projects with resources used in examples throughout this article.

Introduction

Data Validation in Grooper ensures values extracted into a Data Field are correct, complete, and consistent before moving downstream. Validation can:

  • Prevent invalid formats (dates, numbers, emails)
  • Enforce required values
  • Check values against business rules
  • Verify against external systems (e.g., databases)

Grooper validates using properties on the Data Field, expressions, external Lookups, and rule-driven actions. Validation occurs when fields are set, formatted, re-validated in design/test, and during Batch processing.

The different methods of data validation in Grooper discussed in this article include:

  • Value Types
  • Required Fields
  • Calculated Value expressions
  • Is Valid expressions
  • Is Required expressions
  • Database Lookups
  • Data Rules and Data Actions

Value Types

The "Value Type" determines how a Data Field's value is parsed, formatted, and validated. It controls the primitive type (text, number, date/time, boolean), array support, and formatting rules. Invalid values will fail parsing/validation and can be flagged by the field.

Examples:

  • Validate a "Invoice Date" as a date only. Non-date text will be considered invalid.
  • Validate a "Unit Price" as numeric. Non-numeric text will be considered invalid.

How to set the Value Type

  1. On the Grooper Design Page, navigate to a Data Field or Data Column.
  2. Set "Value Type" property to the desired type (e.g., Date/Time, Numeric, String).
  3. Expand out the "Value Type" sub properties and edit the Format Specifier if desired.
    • The format specifier will change the extracted value to a specified format for that specific Value Type. For example, making sure all dates are returned in the MM/DD/YYYY format.
  4. (Optional) Set "Processing Mode" to Validate for validation-only or Both to format and validate.
  5. (Optional) For multi-values, set "Value Delimiter" (e.g., ;) to validate each item.
  6. Save and test. Enter sample values; invalid inputs will not parse or will show error state.


Value Types properties overview

This section lists the core properties that define a Value Type (Storage Type) used by a Data Field. Each item includes its name, definition, remarks, and typical purpose/use case.

"Processing Mode"

  • Definition: Controls how values are processed by the Value Type (Format, Filter, Both, Validate).
  • Remarks: Determines whether values are reformatted, filtered, both, or only validated. Use Format to reformat without filtering, Filter to filter without formatting, Both to apply both operations, and Validate for validation-only scenarios.
  • Purpose/Use case: Choose the level of processing needed for your field. For example, use Validate when you only need to confirm values without altering their presentation; use Both to standardize display while ensuring values meet constraints.

"Value Delimiter"

  • Definition: The delimiter used to separate multiple values stored in a single field.
  • Remarks: Enables array-style storage when non-empty. Common choices include , or ;, or custom strings such as |. Grooper splits and joins using this delimiter and will parse/format each item independently.
  • Purpose/Use case: Allow multi-value lists (e.g., multiple dates, codes, IDs) in one Data Field while retaining strict per-item validation and formatting.

"Windows Type"

  • Definition: The underlying .NET type (e.g., string, decimal, DateTime) used at runtime by the Value Type.
  • Remarks: Controls parsing and formatting behavior. For arrays, the runtime element type is the item type within the collection.
  • Purpose/Use case: Ensure the Data Field uses the correct primitive type for both validation and formatting (e.g., numbers vs. dates).

"CMIS Type"

  • Definition: The Value Type’s CMIS-compatible property type.
  • Remarks: Used where CMIS integration or property typing is required.
  • Purpose/Use case: Maintain compatibility with CMIS repositories when exporting or syncing metadata.

"SQL Type"

  • Definition: The database type string associated with this Value Type.
  • Remarks: Useful for mapping to relational storage or validating schema alignment when persisting Data Field values.
  • Purpose/Use case: Provide appropriate database typing for storage or integration.

"Name"

  • Definition: The display name of the Value Type.
  • Remarks: Reflects the Storage Type’s friendly name and indicates array status in UI (e.g., "Date Array" when multi-value).
  • Purpose/Use case: Inform users which type (and array status) the field is using during configuration and review.

"Default Value"

  • Definition: The default, typed value for the Value Type.
  • Remarks: Used for testing, formatting demonstrators, or to initialize parsing/format checks (e.g., format validation on edit).
  • Purpose/Use case: Provide a baseline value for format preview or validation diagnostics.

"Is Quoted"

  • Definition: Indicates whether values should be quoted when substituted (e.g., in SQL or external queries).
  • Remarks: Typically false for numeric types, true for string-like types when embedded in queries.
  • Purpose/Use case: Prevent syntax errors or injection issues when building dynamic queries (e.g., Database Lookups).

"Is Formattable"

  • Definition: Specifies if the Value Type supports custom formatting.
  • Remarks: When true, format strings and culture are applied to produce display-appropriate values.
  • Purpose/Use case: Standardize how values appear to users (e.g., currency, dates, number precision).

"Format String"

  • Definition: The format specifier applied when formatting values.
  • Remarks: Validated against the type; invalid specifiers are flagged during property validation. Used only for types that support formatting.
  • Purpose/Use case: Control precision, style, or presentation (e.g., c for currency, date masks).

"Choice Values"

  • Definition: The list of allowed or suggested values exposed by the Value Type (if it is a choice type).
  • Remarks: Used when the type itself provides a choice list (e.g., Booleans, predefined enumerations). Each value is validated against the type.
  • Purpose/Use case: Limit input to a defined set of values and enable dropdowns/autocomplete consistent with the type’s constraints.

"Is Choice"

  • Definition: Indicates whether the Value Type enforces a discrete set of allowed values.
  • Remarks: If true, input is restricted to the provided choices. Otherwise, a field’s Lexicon or list settings may supply choices separately.
  • Purpose/Use case: Enforce controlled vocabularies or simple yes/no style inputs at the type level.

"Validation Notes"

  • Definition: Optional notes shown when validation fails, describing format/rules.
  • Remarks: Included with error messages to guide users when a typed value is invalid (e.g., acceptable range or format).
  • Purpose/Use case: Provide clear guidance during Review when correcting invalid entries.

"Runtime Type"

  • Definition: The effective CLR type used at runtime; array-aware when "Value Delimiter" is set.
  • Remarks: Equals "Array Type" for multi-value fields, otherwise equals "Windows Type".
  • Purpose/Use case: Helps ensure expression engines, lookups, and formatters treat values with the correct runtime type.

"Array Type"

  • Definition: The array type corresponding to the Value Type (e.g., string[], DateTime[]).
  • Remarks: Created by WindowsType.MakeArrayType() when "Value Delimiter" is set.
  • Purpose/Use case: Provide strongly typed multi-value support across parsing/formatting pipelines.

"Is Array"

  • Definition: Indicates whether the Value Type treats field content as an array.
  • Remarks: True when "Value Delimiter" is non-empty. Parsing/formatting operate per item; validation iterates over each value.
  • Purpose/Use case: Enable multi-entry fields (e.g., multiple codes) with true typed validation for each element.

"Join Delimiter"

  • Definition: The delimiter used when joining formatted items back into a single string.
  • Remarks: Derived from "Value Delimiter"; supports escaped control characters (e.g., \r, \n, \t).
  • Purpose/Use case: Control the final presentation of multi-value text after formatting.

"Split Delimiter"

  • Definition: The delimiter used to split incoming text into separate items.
  • Remarks: Trims spaces around the delimiter and supports escaped control characters. Newlines are also treated as separators when splitting.
  • Purpose/Use case: Ensure reliable parsing of multi-item inputs, even with varied line endings or spaced delimiters.

Practical guidance

  • Use "Processing Mode" = Validate when you only want to test correctness without changing user-entered formatting; use Both for consistent display and enforced rules.
  • For multi-value fields, set "Value Delimiter" and rely on "Is Array" behavior to validate each entry independently.
  • For database queries in Lookups, ensure "Is Quoted" matches the underlying type to prevent SQL issues.
  • Use "Format String" with "Is Formattable" types (e.g., Date/Time, Numeric) to present values clearly in Review and exports.
  • When publishing choice-driven fields, prefer "Is Choice" with "Choice Values" or use a Lexicon on the Data Field if choices are field-specific rather than type-level.

Required fields

The "Required" property enforces that a Data Field must contain a value by the end of extraction/validation. Empty values will be flagged for review.

When to use:

  • Key identifiers (e.g., "Invoice Number")
  • Compliance-critical fields (e.g., "Tax ID")
  • Downstream-required fields (e.g., "Customer ID")

Example

  • Require "Vendor ID" on invoices. If empty after extraction, the field is flagged to ensure correction during Review.

Steps

  1. Navigate to the target Data Field or Data Column.
  2. Locate and enable the "Required" property in the property grid.
  3. Save and test extraction. Empty fields will show an error state until a value is assigned.

Calculated Value expressions

A "Calculated Value" expression validates, derives, or normalizes a value. Use it to compute results or to re-check consistency with "Calculate Mode"/tolerance. Calculation can be paired with validation to detect mismatches.

Calculated Value can be used to:

  • Normalize presentation (e.g., format phone numbers)
  • Derive values (e.g., "Amount" = "Quantity" × "Unit Price")
  • Cross-field consistency checks

Example

  • Validate line item totals:

@Calculated_Total == (@Quantity * @Unit_Price)

  • Auto-calculate "Calculated Total":

@Quantity * @Unit_Price

In our case, we are specifically looking at using Calculated Value for validation.

How to configure

  1. Navigate to to the target Data Field (e.g., "Subtotal").
  2. Make sure the Value Type property is set to a valid Value Type, such as Decimal. You cannot calculate string values.
  3. Locate the Calculated Value property in the property grid in the Expressions category.
  4. Write an expression referencing other fields or Data Columns (e.g., Line_Items.SumOf("Extended Price")).
  5. Make sure the Calculate Mode property is set to "Validate".
  6. You can adjust the Calculate Tolerance property for numeric rounding differences.
  7. Save and test. Mismatches will flag the field.


Is valid expressions

An Is Valid expression is a custom validation rule you can add to a Data Field to decide whether the field's value is acceptable. The expression is written as a Visual Basic (VB.NET) expression and must evaluate to True or False. If it evaluates to False, the field is marked invalid and will show an error during review.

Is Valid expressions are useful when simple validation (such as "Value Type" conversion or "Required") is not enough. They let you enforce business rules, compare values across fields, and validate values against document context.

When to use Is Valid

Use an "Is Valid" expression when you need validation that depends on logic rather than formatting. Common scenarios include:

  • Range checks (values must be within a minimum/maximum).
  • Cross-field validation (one field must match or relate to another field).
  • Date rules (dates cannot be in the future, or must be within a specific window).
  • Pattern or character rules beyond basic parsing (for example, disallow certain characters).
  • Consistency checks against calculated totals (without automatically setting the value).

If your goal is to compute an expected value and optionally populate the field, consider using "Calculated Value" instead of (or in addition to) "Is Valid".

How it works

  1. A Data Field gets a value (typically from a Value Extractor, or entered by a user).
  2. Grooper validates the value type (for example, whether it can be parsed as a date or number based on the field's "Value Type").
  3. If the field has an "Is Valid" expression, Grooper evaluates it.
    • If the expression returns True, validation passes.
    • If the expression returns False, the field is set to an error state.
    • If "Validate Message" is configured, Grooper displays that message to the user when validation fails.

Tip: Use "Validate Message" to provide clear instructions to operators about what needs to be corrected.


Configure Is Valid on a Data Field

  1. In Grooper Design, select the Data Model (or a Data Section) containing your Data Field.
  2. Click the Data Field you want to validate.
  3. In the property grid, locate the "Is Valid" property in the Expressions category.
  4. Set the "Is Valid" property to a VB.NET expression that returns a Boolean (True/False).
  5. (Optional) Set the "Validate Message" property to a VB.NET expression that returns a string. This message is shown when "Is Valid" evaluates to False.
  6. Save your changes.
  7. Test the validation by testing extraction on a Test Batch and reviewing the field's status.

Examples

Example 1: equality of two Fields

The goal in this first example is to determine if the value in one field is the same as the value in another field. So, in our case, we are going to make sure that the total on an invoice below the line items table matches the total quoted at the top of the Invoice.

  1. Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Please Pay Data Field.
  2. Scroll through the property grid and locate the Is Valid property.
  3. Click the "..." to the right of the property to open the editor.
  4. Type in your VB.NET code for your Is Valid expression. In this example our expression is: Please_Pay = Total_Due
  5. Click "OK" on the Is Valid editor.
  6. (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: "Please Pay does not equal Total Due"
  7. Save your changes to the Data Field and test your extraction at the Data Model level.

Example 2: sum of a Data Column

While the general instructions remain the same for every Is Valid use, the expression you choose to use changes how the validation works. Let's look at a second example. In this example we are going to tell Grooper to sum up the Extended Price column in our Line Items table and make sure that total is equal to the extracted value in the Subtotal Data Field.

  1. Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Subtotal Data Field.
  2. Scroll through the property grid and locate the Is Valid property.
  3. Click the "..." to the right of the property to open the editor.
  4. Type in your VB.NET code for your Is Valid expression. In this example our expression is: Subtotal = SumFieldInstance("Line Items\Extended Price")
  5. Click "OK" on the Is Valid editor.
  6. (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: "Subtotal does not equal sum of Extended Price"
  7. Save your changes to the Data Field and test your extraction at the Data Model level.

Example 3: Date Validation

Is Valid expressions can also do more than just compare two fields to see if they are equal. In this example we're going to use an expression to validate whether or not a date is valid. A date will be considered valid if it takes place in the past. If you receive an invoice with a date in the future, that Invoice would not be valid.

  1. Navigate to the Data Field you want to validate. In our example we are going to set our Is Valid expression on the Invoice Date Data Field.
  2. Scroll through the property grid and locate the Is Valid property.
  3. Click the "..." to the right of the property to open the editor.
  4. Type in your VB.NET code for your Is Valid expression. In this example our expression is: Invoice_Date < Now
  5. Click "OK" on the Is Valid editor.
  6. (Optional) Insert an expression into the Validate Message property if you want an error message to appear if deemed invalid. In our example we used the following: "Date is in the future"
  7. Save your changes to the Data Field and test your extraction at the Data Model level.

Is Required expressions

An Is Required expression is a conditional rule you can add to a Data Field to decide whether the field must contain a value at runtime. Unlike a static required setting, an Is Required expression can evaluate the current document's data and context (for that specific field instance) and then determine if the field should be treated as required.

You would use an Is Required expression when a field is only mandatory some of the time, such as when another field has a particular value, when a total is above a threshold, or when a document meets specific business conditions.

In data validation, Is Required expressions help Grooper enforce “requiredness” rules that are dependent on the extracted data, improving data quality without forcing operators to fill out fields that do not apply.

How Is Required differs from "Required" (Required Fields)

The "Required" property is a static rule:

  • If "Required" is enabled, the Data Field must always have a value.
  • If the field is empty, the field is set to an error state with a required-value message.

The "Is Required" property is a dynamic rule:

  • If "Is Required" is set, Grooper evaluates the expression for each field instance.
  • If the expression evaluates to True and the field is empty, the field is treated as required and becomes invalid until a value is provided.
  • If the expression evaluates to False, the field is allowed to remain empty without error.

How Is Required differs from "Is Valid" and "Calculated Value"

"Is Required" answers: Must this field have a value?

  • It is primarily an “empty value” rule (required vs optional).
  • It is used when the main question is whether blank is acceptable.

"Is Valid" answers: Is the current value acceptable?

  • It is evaluated when there is a value present.
  • It is used for business-rule validation such as ranges, formats beyond "Value Type", or cross-field checks.

"Calculated Value" answers: What should the value be?

  • It computes an expected value (and can optionally set the field's value depending on "Calculate Mode").
  • It is best for totals, derived values, and consistency checks where Grooper can calculate what the value should be.

In practice:

  • Use "Is Required" when the field may be blank sometimes.
  • Use "Is Valid" when the field has a value but you need to validate correctness.
  • Use "Calculated Value" when the value should be derived from other data (and possibly auto-filled).


How Is Required is used during validation

During validation, Grooper checks whether a Data Field is empty:

  • If the field is empty and "Required" is enabled, the field is invalid.
  • If the field is empty and an "Is Required" expression exists, Grooper evaluates the expression:
    • If it returns True, the field becomes invalid until a value is entered.
    • If it returns False, the field is allowed to remain blank.

This allows the Data Model to enforce mandatory data only when it applies, reducing unnecessary errors in Review while still ensuring required information is captured when needed.

How To

  1. Navigate a Data Field and locate the Is Required property in the Field's property grid.
  2. Click the "..." to the right of the Is Required property.
  3. When the Is Required editor pops up, begin typing in your Is Required expression in the text box.
    1. In our example we are using the following expression: PO_Number <> ""
  4. Click "OK" on the editor when finished.
  5. Save your changes to the Data Field.
  6. Test your extraction to make sure the Is Require expression is having the desired outcome.


Database Lookups (validation)

A Database Lookup is a type of Lookup Specification that runs an SQL query against an external database (through a Data Connection). In a validation scenario, the goal is usually to confirm that one or more extracted values in your Data Model match something in a database.

Database Lookups are useful for validation when “being the right format” is not enough. For example, an invoice number might be formatted correctly and still be invalid because it does not exist for the vendor in your accounting system. A Database Lookup can validate that the value (or combination of values) exists, and optionally return supporting data to confirm the document is consistent.

Why use Database Lookups for validation

Database Lookups are the best choice when your validation rule depends on an external source of truth:

  • Validating IDs (Vendor IDs, Customer IDs, Student IDs, Employee IDs)
  • Validating document keys (Purchase Order Numbers, account numbers, policy numbers)
  • Validating multi-field relationships (Vendor + Invoice Number, Customer + Account + Location)
  • Reducing downstream errors by preventing invalid values from being accepted during Review

They are especially valuable when the data is frequently changing and cannot be maintained reliably as a static list (for example, active vendors or current product catalogs).

How Database Lookups validate data

Database Lookups validate data by using your extracted Data Field values as query criteria.

  • You build an SQL query in the lookup's "SQL Query" property.
  • In that query, you reference Grooper values using @VariableName syntax (most commonly, @FieldName).
  • When the lookup runs, Grooper substitutes the current field values into the query and executes it.
  • The lookup is considered a hit when the query returns exactly one row.
  • If the query returns:
    • 0 rows (no match), the lookup can treat that as a validation failure (based on the lookup's Miss Disposition settings).
    • More than 1 row (ambiguous match), the lookup can treat that as a validation failure (based on the Conflict Disposition settings).

In other words, for validation-only use, the most common pattern is:

  • Return 1 row = valid
  • Return 0 rows = invalid
  • Return multiple rows = needs attention / invalid

How To

Before you can use Database Lookups for validation, you need:

  1. A configured Data Connection that can connect to your database.
    • In the Data Connection, set the "Connection Settings" property to the correct provider and supply server/authentication settings.
  2. A Data Model with the Data Fields you want to validate.

For the following example, it is assumed you already have a Data Connection configured and you have your Data Model set up.

  1. The Database Lookup can be configured on any container Data Element. Select one, such as the Data Model.
  2. Click the "..." to the right of the Lookups property in the property grid.
  3. When the Lookups window pops up, click the "+" icon and select "Database Lookup" from the drop down.
  4. In the property grid on the right, set the Database Connection property to your configured Data Connection.
  5. Click the "..." to the right of the SQL Query property.
  6. Type in your SQL query, using the intellisense to help if desired. The following SQL Query is used in the example below:
SELECT PO_Total AS PO_Total FROM PO_Totals WHERE PO_Number = @ P_O_Number
  1. When finished, click "OK" on the SQL Query editor.
  2. For validation purposes, leave the Miss Disposition and Conflict Disposition properties at their defaults.
  3. Click "OK" on the Lookups window.
  4. Save your changes to the Data Element.
  5. Test your extraction.
    • If the lookup was successful, you will not see any errors.
    • If the lookup returned no results and you have left the Miss Disposition to defaults, an error will indicate the issue.
    • If the lookup found multiple results and you have left the Conflict Disposition to defaults, an error will indicate the issue.


Two-Way Matching

Grooper can perform Two-Way Matching using Data Lookups.

Two way matching is a process commonly used by Accounts Payable to compare the information on an invoice received from a vendor to the original PO. They then can discuss any discrepancies with the vendor. Grooper can actually perform this two way matching validation using lookups.

You will need a PO SQL Database to match against. Usually Two-Way Matching is performed on line items in a Data Table, with that in mind, you may need to set up your Lookup on the Data Table rather than a Data Model or Data Section to have access to all objects needed for the lookup.

  1. Click on the Data Table node on the Design Page.
  2. Locate the Lookups Property and click the "..." icon.
  3. Add a new Database Lookup and set the Database Connection to an established Data Connection.
  4. Click the "..." to the right of the SQL Query property.
  5. Type in your SQL Query into the editor and click "OK". The following Query is used in the example below:
SELECT * FROM POData WHERE (PO_Number = @#P_O_Number) AND (Item_Number = @Item_Number) AND (Quantity = @Quantity) AND (Unit_Price = @Unit_Price) AND (Extended_Price = @Extended_Price)
  1. Since we only want to validate the data and don't want anything overwritten, change the Field Population property from Overwrite to None.
  2. Click "OK" on the Lookups window.
  3. Test your extraction. Anything that doesn't match the PO information in the Database will be flagged.


Data Rules and Data Actions (validation)

A Data Rule is a configurable rule that runs against extracted data and can apply conditional logic to a Data Model, Data Section, or Data Table. A Data Rule is made up of two main parts:

  • A "Scope" (where the rule runs)
  • An optional "Trigger" (the condition that determines what happens)

A Data Action is the operation a Data Rule performs when its "Trigger" evaluates to True (the "TrueAction") or False (the "FalseAction"). Data Actions are the “do something” part of the rule. They are commonly used to:

  • Normalize or standardize values (for example, trimming whitespace or converting text to uppercase)
  • Copy or move values between fields
  • Assign calculated values
  • Run lookups
  • Apply structured, multi-step logic by nesting actions and child rules

Purpose of Data Rules

The main purpose of Data Rules is to let you apply consistent, repeatable logic to extracted data after extraction. This helps keep extractors focused on finding data while Data Rules handle what to do with the data once it is found.

Data Rules can run at different levels:

How Data Rules support validation

Data Rules themselves are not validation rules in the same way a Data Field's "Required", "Is Required", "Is Valid", or "Calculated Value" expressions are. Those field-level properties directly determine whether a field is valid and what error message is shown.

Instead, Data Rules support validation by adding conditional logic that can:

  • Require data only when it applies (conditional requiredness)
  • Prepare or normalize values so other validation methods work reliably
  • Drive consistency between fields by setting values based on other values
  • Trigger additional logic when conditions are met (for example, apply different rules per document scenario)

This is especially useful when validation depends on multiple fields, repeating lists (tables/sections), or “if/then” business rules that would be difficult to express on a single field alone.

FYI

When a Data Rule changes values, Grooper can re-validate fields that now have values so they can clear previous error states.

How Data Rules do not validate, but help validation

A helpful way to think of Data Rules is:

  • Data Field validation answers: “Is this value valid?”
  • Data Rules answer: “If this is true, what should I do next?”

So a Data Rule typically:

  1. Checks a condition using "Trigger"
  2. Applies a "TrueAction" or "FalseAction"
  3. Optionally runs child Data Rules (nested steps)

This lets you build validation support logic such as:

  • “If this invoice is taxable, then Sales Tax must be present.”
  • “If Country is United States, then State must be present.”
  • “If Line Items exist, then Total must equal the sum of Line Totals (or set Total to the sum).”

When to choose Data Rules for validation workflows

Use Data Rules when:

  • You need if/then logic across multiple fields
  • The same rule must be applied repeatedly across a Data Table row set or Data Section instances
  • You need to normalize extracted text before other validation rules run
  • You want a centralized place to manage multi-step validation support logic (instead of scattering it across many fields)

For simple scenarios (one field, simple range checks, always-required fields), prefer field-level settings like "Required", "Is Required", "Is Valid", and "Calculated Value".

How To

Now that we understand what Data Rules are, let's learn how to configure them in Grooper.

Triggers and TrueActions

A TrueAction is the “then” part of a Data Rule.

  • If the "Trigger" expression returns True, Grooper executes the rule's "TrueAction".
  • If the "Trigger" is blank, the trigger is treated as always true, so the "TrueAction" runs every time the rule runs.
  • After the "TrueAction" runs, any child Data Rules under that rule are executed (in order).
  • If the "Trigger" returns False, the "TrueAction" does not run (and the rule may run a "FalseAction" instead, if configured).

Why it matters for validation

TrueAction is commonly used to apply conditional “if/then” logic that supports validation, such as:

  • requiring a value only in certain situations
  • normalizing a value before field validation occurs
  • setting or clearing values based on business rules

How To set up a True Action on a Data Rule

  1. In Grooper Design, create or select a current Data Rule in your Project.
  2. Set the Data Rule's "Scope" to the Data Element the rule should run on:
    • Data Model = runs once per document
    • Data Section = runs once per section instance
    • Data Table = runs once per table row
  3. (Optional) Set the rule's "Trigger" to a Boolean expression that determines when the rule should run.
    • If left blank, the trigger is treated as always true (the rule always runs).
  4. Set the Data Rule's "TrueAction" property:
    • Click the "TrueAction" property.
    • Choose the Data Action type you want to run.
    • Configure the selected Data Action's properties (such as source/target fields, transformation options, lookup settings, etc.).
  5. Save the rule and test it by running extraction and reviewing results.


False Actions

A False Action is the Data Action a Data Rule runs when the rule's "Trigger" evaluates to False. It is the “else” part of the rule.

  • If the "Trigger" returns True, Grooper runs the rule's "TrueAction".
  • If the "Trigger" returns False, Grooper runs the rule's "FalseAction" (if one is configured).
  • If no "FalseAction" is set, nothing happens when the trigger is false (the rule simply does not apply its action).

Why would you use a False Action?

False Actions are useful when you need to handle both outcomes of a condition, for example:

  • Clear or reset data when a condition is not met
  • Skip population and keep existing values when a condition is not met
  • Apply a different cleanup/normalization step depending on the document scenario
  • Enforce “either/or” logic (do one thing if true, a different thing if false)

How to set up a False Action on a Data Rule

  1. In Grooper Design, create or select the Data Rule you want to configure.
  2. Set the rule's "Scope" to the Data Element the rule applies to (such as a Data Model, Data Section, or Data Table).
  3. Enter a Boolean expression in the rule's "Trigger" property.
    • The "FalseAction" property is only shown when "Trigger" is not empty.
  4. Click the "FalseAction" property.
  5. Choose the Data Action type you want to run when the trigger is false.
  6. Configure the selected Data Action's properties as needed.
  7. Save and test by running extraction and reviewing results.

FYI

  • If the "Trigger" is blank, the rule is treated as always true, so "FalseAction" does not apply.
  • When the "Trigger" is false, Grooper runs "FalseAction" and skips running child Data Rules under that rule.

Require Value Data Action (Validation)

The Require Value Data Action is used to enforce that one or more selected Data Elements have data. When a required value is missing, the action marks the item as invalid and records an issue. This is most useful for conditional requiredness (required only when a rule's "Trigger" is true).

What it validates

Require Value checks the items you select in "RequiredElements" and fails validation when:

When it finds a problem, it:

  • Sets an error on the field (for fields), or adds an error to the container (for sections/tables)
  • Adds an issue to the rule result (optionally categorized and with metadata)

Why use Require Value instead of a Data Field's "Required" property

Use Require Value when the requirement is conditional or when you need to require:

  • Multiple fields together based on a single trigger
  • A section/table to exist (not just a single field)
  • Different required sets in different scenarios (via different rules/triggers)

If a field should always be required, use the Data Field "Required" property instead (simpler and always enforced).

How to set it up (step-by-step)

  1. In Grooper Design, create or select a Data Rule in your Project.
  2. Set the rule's "Scope" to where the rule should run:
    • Data Model (validate once per document)
    • Data Section (validate once per section instance)
    • Data Table (validate once per table row)
  3. (Optional, but common) Set the rule's "Trigger" to the condition that makes values required.
  4. Set the rule's "TrueAction" to Require Value.
  5. Configure the Require Value action:
    • Set "RequiredElements" to the Data Fields (and/or Data Sections / Data Tables) that must be present when the rule applies.
    • (Optional) Set "LogMessage" to a friendlier message to show when the requirement fails.
    • (Optional) Set "Category" to group the issue (useful for reporting/routing).
    • (Optional) Set "Metadata" to attach key/value details (one per line, key=value).
  6. Save and test by running extraction and validating in Review.

Tips

  • Keep the "Scope" as narrow as possible:
  • Use Data Table scope to validate per-row requirements.
  • Use Data Section scope to validate per-instance requirements.
  • Use Data Model scope for document-level rules that coordinate multiple areas.
  • Use a clear "LogMessage" so Review users know what to do next (for example, “PO Number is required when Payment Method is PO.”).

Calculate Value Data Rule (Validation)

The Calculate Value Data Action computes a value using a "Value Expression" and writes it to a target Data Field (or to a custom variable). By itself, Calculate Value does not “validate” a value. It supports validation by:

  • populating missing values so required/format validation can pass
  • standardizing values so "Value Type" and "Is Valid" validations work consistently
  • calculating an “expected” value that another validation step can compare against

How to set it up (step-by-step)

  1. In Grooper Design, create or select a Data Rule in your Project.
  2. Set the rule's "Scope" to where the rule should run:
    • Data Model (validate once per document)
    • Data Section (validate once per section instance)
    • Data Table (validate once per table row)
  3. (Optional, but common) Set the rule's "Trigger" to the condition that makes values required.
  4. Set the rule's "TrueAction" to Calculate Value.
  5. Configure the Calculate Value action:
    • Set "Target Field" to the Data Field you want populated.
    • Open the "Value Expression" editor and type in an expression for the calculation you want Grooper to run.
  6. Save and test by running extraction and validating in Review.

!!

Note that there is no Calculate Mode property on the Data Rule. Calculate Value on a Data Rule cannot validate by itself. It can calculate and then populate Data Fields and then other methods of validation can be applied.

Tips

  • For validation (not population), use a Data Field "Calculated Value" with "Calculate Mode" set to Validate, and use Data Rules to decide when that validation should apply.
  • Use Calculate Value in Data Rules when you need conditional logic (if/then) around calculation or normalization.

Data Lookup Data Action (Validation)

The Data Lookup Data Action runs a configured Lookup Specification and uses the lookup result to either:

  • validate that the current data matches an external source (hit / miss / multiple hits), and/or
  • populate fields (or a collection) with values returned by the lookup.

For validation-focused rules, you typically configure the lookup so it behaves like: “If the criteria fields don't return exactly one matching record, treat it as a problem.”

What you need first (prerequisites)
  1. A Data Connection established to connect to the target Database for the lookup.
How it works in a Data Rule

When a Data Rule executes:

  • If "Target Collection" is blank, the action runs the lookup in the current context and applies standard lookup behavior (validation and/or population) based on the Lookup Specification settings.
  • If "Target Collection" is set to a Data Table or multi-instance Data Section, the action runs the lookup and copies each returned row into the collection (optionally clearing existing rows first).

So, “validation” mainly comes from what the Lookup Specification does when the query returns 0 rows or multiple rows.

Step-by-step: Use Data Lookup for validation (single record)
  1. Create/select a Data Rule.
  2. Set the Data Rule's "Scope" to the container that holds the criteria fields:
    • Data Model for document-level validation
    • Data Section for per-section-instance validation
    • Data Table for per-row validation
  3. (Optional) Set "Trigger" so the lookup only runs when you have enough data to validate. For example:
Not String.IsNullOrEmpty(PO_Number)
  1. Set "TrueAction" to Data Lookup.
  2. In the Data Lookup action, set:
    • "Lookup" = your configured Lookup Specification (for example, a Database Lookup)
    • Leave "Target Collection" blank (validation / population happens in place)
  3. Test using the rule tester (or run extraction + review).

Tips for validation-focused Data Lookups

  • Use a Data Rule "Trigger" to avoid running the lookup when criteria fields are blank.
  • Keep the lookup query focused and return a single row for a valid match (validation is cleanest when “exactly one row” means success).
  • If you want strictly validation-only, use queries like SELECT 1 and configure the lookup to treat misses/conflicts as errors.
  • Use "Target Collection" only when you intentionally want to create rows/instances from the lookup results.