2023:Data Rule (Node Type)

This article is about an older version of Grooper. Information may be out of date and UI elements may have changed.
2025	2023	2021

flowsheet Data Rules are used to normalize or otherwise prepare data collected in a data_table Data Model for downstream processes. Data Rules apply conditional logic to data extracted from documents (folder Batch Folders).

• Each Data Rule executes a "Data Action" which do things like computing a field's value, parse a field into other fields, perform lookups, and more.
• Data Actions can be conditionally executed based on a Data Rule's "Trigger" expression.
• A hierarchy of Data Rules can be created to execute multiple Data Actions and perform complex data transformation tasks.

WIP	This article is a work-in-progress or created as a placeholder for testing purposes. This article is subject to change and/or expansion. It may be incomplete, inaccurate, or stop abruptly. This tag will be removed upon draft completion.

Introduction

A Data Rule is a configurable rule that applies conditional logic to extracted data. Data Rules are typically used after extraction to transform values into the format your downstream systems require, or to enforce business requirements that are easier to express as a rule than as extraction logic.

Unlike extractors (which focus on finding data on the document), Data Rules focus on fixing, moving, combining, and validating the data once it has been captured into a Data Model.

What a Data Rule runs against (scope)

A Data Rule runs against instances of a selected container. The container is chosen using the rule's "Scope" property:

• If the "Scope" is set to a Data Model, the rule executes once on the root document instance.
• If the "Scope" is set to a Data Section (and the section is multi-instance), the rule executes for each section record (each Section Instance).
• If the "Scope" is set to a Data Table, the rule executes for each row (each Table Row Instance).

This is important because many Data Actions operate on fields within the current record/row, and the meaning of “source” and “target” depends on what the rule is scoped to.

Conditional execution (Trigger, True Action, False Action)

A Data Rule can optionally use conditional logic to decide what should happen:

• The "Trigger" property must be configured with a Boolean expression.
• When "Trigger" evaluates to True:
  • The "True Action" runs.
  • Any child Data Rules (nested under the current rule) also run.
• When "Trigger" evaluates to False:
  • The "False Action" runs (if configured).
  • Child Data Rules are skipped.
  • If no "False Action" is configured, the Data Rule just does not run.

If the Trigger property is blank, the Data Rule will always run.

How Data Rules differ from other conditional components

Data Rules are purpose-built for manipulating and validating extracted values inside the Data Model hierarchy (document, section record, or table row). There are other Grooper components that also use conditional logic (for example, conditional paths in a Batch Process), but those typically control workflow routing or when a step runs. Data Rules control how extracted data is modified within the document's data structure.

How to configure a Data Rule (step-by-step)

1. In Grooper Design, create or select a Data Rule (commonly stored in a Content Type's Local Resources Folder).
2. Select the Data Rule and set:
   1. "Scope" to the Data Model, Data Section, or Data Table you want to process.
   2. (Optional) "Trigger" to define when the rule should apply.
   3. "True Action" to define what happens when the trigger is met (or always, if Trigger is blank).
   4. (Optional) "False Action" to define what happens when the trigger is not met.
   5. (Optional) "Shortcut Keys" if you want a manual shortcut in review tools that support it.
3. (Optional) Add child Data Rules under the rule to build a multi-step decision tree.

Testing and troubleshooting

• If you need to test a rule interactively during Review, you can run it from a field container using the Run Rule command:
  • Alt + R
• If a rule does not appear in the list, verify:
  • The rule’s "Scope" is compatible with the selected container.
  • The rule is stored where it is accessible for the Content Type you are reviewing.
• If a rule never runs:
  • Confirm "Trigger" is either blank or evaluates to True for the current instance.
  • Temporarily simplify "Trigger" to True to isolate expression issues.
• If values change but errors do not clear:
  • Data Rules re-validate fields that now have values after the True Action runs. If an error remains, check the Data Field’s "Is Valid", "Required", "Min Confidence", and other validation settings.

Data Rule properties (overview)

The key Data Rule properties are:

• "Scope"
  • The Data Model, Data Section, or Data Table the rule applies to.
• "Trigger"
  • A Boolean expression. If blank, the rule always executes the True Action.
• "True Action"
  • The Data Action that runs when the Trigger evaluates to True (or always, if Trigger is blank).
• "False Action"
  • The Data Action that runs when the Trigger evaluates to False. This is only shown when "Trigger" is not blank.
• "Required Elements" (obsolete)
  • A legacy way to enforce conditional required elements. Prefer the Require Value Data Action.
• "Shortcut Keys"
  • Optional shortcut for use in the Data Viewer tab on the Review Page.

Data Actions

Data Actions are the building blocks used in "True Action" and "False Action". Actions can be combined and nested to create multi-step logic.

Below are the main Data Actions, in the order commonly presented.

Calculate Value

Purpose Calculates a value using an expression and applies it to a target field. Use this to standardize computed results (totals, derived values, formatted values) after extraction.

How to configure (step-by-step)

1. Create/select a Data Rule and set its "Scope" to the container where the target field exists (Data Model, Data Section, or Data Table row context).
2. For the True Action property, choose Calculate Value.
3. Set the target field (where the result should be written).
4. Configure the Calculate Value expression to produce the desired output type (string/number/date depending on the Data Field's "Value Type").

Validation usage Use Calculate Value when you want to enforce a known result:

• Calculate and set the value (then make the field read-only in the Data Field if it should never be edited), or
• Calculate and compare to an existing value (for example, flagging mismatches via Raise Issue).

Clear Item

Clear Item removes or resets the value of a specified Data Element. Use this action to clear the value of a field, remove all rows from a table, or empty a multi-instance section. For containers, all child elements are cleared; for fields, the value is set to an empty string.

Example usage: Clear the "Comments" field or remove all rows from a table.

Concat

Concat merges or combines items in a collection based on a trigger condition and parameter. This action is typically used to concatenate or aggregate data from multiple items in a collection, such as combining rows in a table or merging section instances, according to a specified rule.

Example usage: Combine adjacent table rows when a certain field matches a condition.

Copy

Copy copies data from a source element to a target element. You can copy values, containers, or collections, with options to prevent overwriting, ignore empty values, or specify a source index. This action is useful for duplicating or transferring data between fields, sections, or tables.

Example usage: Copy the value from "Address Line 1" to "Mailing Address" if the latter is empty.

Append

Append adds one or more elements to a collection, such as a table, multi-instance section, or multi-cardinality field. For each instance of the source element, an entry is added to the target collection. Optional actions can be defined to populate or copy child elements.

Example usage: Append all rows from one table to another, or add values from a multi-cardinality field to a list.

Data Lookup

Data Lookup performs a lookup operation against an external data source (such as a database) to validate or populate field values. The "Lookup" property defines the lookup specification, including criteria, target fields, and error handling. Use this action to automate field validation or population based on external data.

Example usage: Validate that a "Customer ID" exists in a database and populate related fields.

Extract From

Extract From sets a field value by running an extractor on another field. Configure the "Source Field", "Target Field", and "Extractor" properties to define the extraction logic. Use options like "Prevent Overwrite" and "Ignore Miss" to control execution and handling of missing results.

Example usage: Extract the last name from a "Full Name" field and assign it to a "Last Name" field.

Parse Value

Parse Value parses a field value into substrings using a regular expression and assigns them to sibling fields. The "Source Field" is parsed using the "Pattern" property, which should contain named groups matching the target fields.

Example usage: Split "DOE, JOHN Q" in "Full Name" into "Last Name", "First Name", and "MI" fields using a regex pattern.

Raise Issue

Raise Issue adds an issue to the issue log. The "Log Message" property defines the message, with optional "Category" and "Metadata" for classification and additional context. Use this action to flag data issues or validation errors during processing.

Example usage: Log an error if a required field is missing or a value is out of range.

Remove

Remove conditionally removes records from a collection such as a Data Section or Data Table. The "Collection" property specifies the target, and the "Trigger" property defines a Boolean expression to determine which records are removed.

Example usage: Remove all rows from a "Line Items" table where the "Amount" field is zero.

Require Value

Require Value enforces that specified data elements (fields, columns, sections, or tables) are populated with a value. The "Required Elements" property lists the elements to check. If any are missing, an error is logged and the element is marked as invalid.

Example usage: Require that "First Name" and "Last Name" fields are not blank.

Action List

Action List executes a sequence of Data Actions in a defined order. Use this to group several actions together as a single step in a Data Rule. Each action in the list is executed in sequence, allowing for complex, multi-step data processing.

Example usage: Normalize a value, validate it, and then copy it to another field, all in one step.

Execute Rule

Execute Rule invokes a Data Rule. Use this to modularize and reuse rule logic. The "Rule" property specifies which Data Rule to execute, which must apply to the same or a descendant scope.

Example usage: Invoke a Data Rule for validation or normalization.