2021:Data Rule (Node Type)
The Data Rule object allows for complex validation and manipulation of a Data Model's Data Elements (Data Fields, Data Sections, and Data Tables) in Grooper.
This allows users to create a conditional hierarchy of actions to take if certain conditions met. These conditions are configured using .NET, LINQ and/or lambda expressions. When the expression is "triggered", either evaluating to "true" or "false", certain actions can be made. These include:
- Calculate Value - This action sets the value of a Data Field or cells a Data Column, using calculate expressions to perform mathematical or concatenation operations of Data Elements.
- Clear Item - This action clears the value of a Data Element.
- Copy Item - This action copies or moves the value of a Data Element.
- Parse Value - This action uses a regular expression pattern to return part of a Data Field's value or cell in a Data Column's value.
- Raise Issue - This action adds an issue to the issue log, used for validating a Data Element. This action can also be used to flag the Data Element.
These trigger conditions and subsequent actions set on the Data Rules objects are executed through the Apply Rules activity after data is extracted from an Extract activity.
About
Some Basics About Expressions
Grooper makes use of expressions to validate extracted data and use extracted data to populate fields in a Data Model. Traditionally, this is configured on a Data Field object in a Data Model (or in the case of validating or calculating cells in a table, the Data Column object), using the Default Value, Calculated Value, or Is Valid properties.
For example, let's say we have several documents in a Batch. Each one contains W-2 wage reporting forms for various individuals and we want to do some basic tax filing calculation. In order to find someone's total income, it may not be quite as simple as pulling the listed wages from a single W-2. An individual might have multiple W-2s from multiple employers.
If an individual worked for three different employers over the course of a year, their total income would be the wages from all three W-2s added together. This is where expressions come in handy in Grooper. There is no extractible "Total Wages" field on the document. It's just three pages, each page a different W-2. There is no text data for an extractor to return that corresponds to all three W-2's wages field added together.
But we could create a Data Section in our Data Model to return the wages from each individual W-2 form, by adding a "Wages" Data Field and configuring its extraction. Then, we could create a "Total Wages" Data Field and use a Calculated Value expression to add up the results of each "Wages" Data Field in each section (each W-2 in this case).
|
Here, we have a simple Content Model set up to solve the problem described above.
|
| ||
|
| ||
|
|
Conditional Execution of Expressions and Data Rules
You can do a lot with expressions, even applying some conditional logic to their execution. If the condition is met, the expression executes. If not, it doesn't or something other expression executes. For example, in our example of documents containing W2s we make some assumptions about the document. We assume each document contains a W-2 for a single individual. Each individual should only have one social security number. It would be problematic if their were multiple social security numbers extracted from the W-2 forms. This could indicate there are multiple W-2s for multiple individuals in a single document.
To account for this, you could use a more complex Calculated Value expression to only add up the "Fed Wages" Data Fields if the social security number was the same for each document. If the condition of their only being one social security number for each W-2 is met, the expression to add up the wages would execute. If not, it wouldn't. This is basic conditional logic. If "x" condition is met, then do "y". If there's only one social security number, then add up all the wages. Otherwise, do nothing (or something else). You could go another step further and add an Is Valid expression to flag the document if the social security numbers didn't match, as well.
However, the more complex a Data Model's data hierarchy, generally the more complex these conditions tend to be. The more conditions you add for an expression to execute, the more complex the expression becomes. This can result in very cumbersome expressions that are difficult to form and manage.
This is where the Data Rule object really shines. Data Rules allow you to use Trigger expressions to determine one or multiple subsequent Actions to take if that expressions evaluates to true (or false). This is also basic conditional logic. If the trigger expression is true, do the action. Otherwise, do nothing (or a different action). Furthermore, you can more easily create a complex hierarchy of conditions by adding child Data Rules to parent Data Rules. If the trigger expression evaluates to true, the child Data Rules will execute, with their own triggers and even own child Data Rules. This allows for simpler set up, execution, and management of more complex conditional expressions as well as some actions that fall outside normal expressions you can set up in a Data Field or Data Column.
Actions
Once a Data Rule is triggered, what happens next is determined by the True Action and False Action properties. When the Trigger expression evaluates to true, the True Action is executed. When the Trigger expression returns false, the False Action is executed. This determines what action is taken once the trigger condition is met or not met.
This can be one of six choices:
- Calculate Value
- Raise Issue
- Clear Item
- Copy Item
- Parse Value
- Action List