2023:Confidence Multiplier and Output Confidence (Property)

From Grooper Wiki
Revision as of 15:17, 27 November 2023 by Rpatton (talk | contribs)
Graphic depicting the notion of Weighted Rules.

Some results carry more weight than others.

About

The Confidence Multiplier and Output Confidence properties of Data Type and Data Format extractors allow you to manually alter the confidence score of returned values.

Use of these properties is sometimes referred to as weighted rules. Its practical application allows a user to increase or decrease the confidence score of an extractor's result (or set its confidence to an assigned value). This changes the confidence of the extractor's results, making them appear more (or less) favorable. When used in combination with the Order By property set to Confidence on a parent Data Type, you can manipulate which child extractor's result the parent prioritizes.

General Usage - Confidence Multiplier

Modifying the Confidence Multiplier property of a Data Type or Data Format is done by clicking on the ellipses in the Result Options property which opens the Result Options submenu.

The Confidence Multiplier property defaults to 1 and can be changed in this submenu. The field is a double and takes floating point values.

For example, a value of 0.5 will multiply the confidence of output results by 0.5. If the output confidence was 100%, now it will be 50%. Similarly, you can increase the confidence, even above 100%. If the Confidence Multiplier property is set to 3, and an output result had a 50% confidence, it will not display as 150% confidence.

General Usage - Output Confidence

Modifying the Output Confidence property of a Data Type or Data Format is also done by clicking on the ellipses in the Result Options property which opens the Result Options submenu.

The Output Confidence property defaults to 0% and can be changed in this submenu. The default of 0% will not alter the results confidence scores. Changing this number will override whatever the result's original confidence is and replace it with this value.

For example, a value of 75% will change the confidence of output results to 75%. If the output confidence was 100%, now it will be 75%. If the output confidence was 50%, now it will be 75%. If it was 75%, it will now be (you guessed it) 75%. It doesn't matter what the original confidence was, it will be transformed to the Output Confidence value.

How To

Waterfall Classification

Setting the Classification Method property on a Content Model to Lexical or Rules-Based, one can set up Positive Extractors on Document Types. If this extractor returns a result above the Minimum Similarity set on the Content Model, the document will be assigned that Document Type during classification. By default a result from an extractor is returned at 100% confidence (unless Fuzzy RegEx is leveraged to return a result, in which case the confidence will be affected by the fuzzy algorithm.) Given this fact positive extractors are almost certain to be above the Minimum Similarity.

This extractor could be a "Waterfall Extractor", taking advantage of the Waterfall Extraction technique. However, for classification, the system is just looking for some result to be returned above the Minimum Similarity confidence threshold.

In the Waterfall Classification method, the Minimum Confidence property can be set in the Result Filter property window of a Data Type which will eliminate any results less than that confidence. This may eliminate the results of some referenced extractors which technically matched, but at a low percent.

If we happen to know that those lower confidence hits are valid and should count for classifying the document, then the Confidence Multipliers on those referenced Data Types can be set to a higher value in order to make them hit the Minimum Confidence required.

Similarly, if higher confidence hits are inappropriately classifying documents and shouldn't be returned, the Confidence Multiplier property can be reduced so that those Data Types only exceed the Minimum Confidence when they are very high confidence.

Example

A base Content Model, Batch, and Batch Process for use with this section can be found here. It is not required to download to understand this section, but can be helpful because it can be used to follow along with the content of this section. This file was exported from and meant for use in Grooper 2.9

  1. Here selected is the Content Model for this setup.
  2. The Classification Method property is set to Lexical which allows for TF-IDF training, as well as a Rulese Based approach.
    • Notice, too, its Minimum Similarity property is at its default 60%.
  2. This Data Type is the extractor supplying the Content Model's Feature Extractor property, which is what is used to train the Form Types of the Title Opinion Document Type.
  2. This Data Type is the extractor supplying the Generic Letter Document Type's positive Extractor property.
  3. It is using the the (new to Grooper 2.9) AND Collation Method.
    • Think of this as somewhat between an Array and an Ordered Array. All extraction results need to be present, but not in a specific order.
  4. The Result Options property’s sub Result Options window has the Output Confidence property set to 60%, therefore all results returned from this extractor will be returend with a confidence of 60%.
  2. This Batch Process Step is configured for ESP Auto Separation at the Batch Scope and pointed at the aforementioned Content Model. The Batch supplied with the zip file is the one being observed.
  3. In the highlighted example, with the Pages classified and the Preview button enabled, you can see that the Title Opinion Document Type similarity for page one is 89%, and the Generic Letter is coming in at the enforced 60%.
    • Were this a previous version of Grooper without this functionality, the "Rule" would have come in at a default 100%, which is obviously higher than the 89%, because confidences could not be manipulated previously. This would have resulted in the false classification of the Tile Opinion as a Generic Letter.
  4. The Positive Extractor of the Generic Letter allows the accurate classification of the letters given the "rule" is at or above the Content Model's Minimum Similarity property.

Version Differences

Prior to Grooper 2.9 the Confidence Multiplier property did not exist.

Retrieved from "https://wiki.grooper.com/index.php?title=2023:Confidence_Multiplier_and_Output_Confidence_(Property)&oldid=15854"