Confidence Multiplier and Output Confidence (Property) - Revision history

Randallkinard: // via Wikitext Extension for VSCode

2024-12-10T19:13:30Z

// via Wikitext Extension for VSCode

Randallkinard: // via Wikitext Extension for VSCode

2024-12-10T19:12:57Z

// via Wikitext Extension for VSCode

← Older revision		Revision as of 14:12, 10 December 2024
Line 1:		Line 1:
	{{~~Migrated~~}}		{{AutoVersion}}
	{{~~2023~~:~~{{PAGENAME}~~}}}
			[[File:weighted_rules_00.png\|right\|thumb\|250px\|Graphic depicting the notion of Weighted Rules.]]

			<blockquote>{{#lst:Glossary\|Confidence Multiplier and Output Confidence}}</blockquote>

			{\|class="download-box"
			\|
			[[File:Asset 22@4x.png]]
			\|
			You may download the ZIP(s) below and upload it into your own Grooper environment (version 2024). The first contains a '''Project''' with resources used in examples throughout this article. The second contains one or more '''Batches''' of sample documents.
			* [[Media:2024_Wiki_Confidence-Multiplier-and-Output_Confidence_Project.zip]]
			* [[Media:2024_Wiki_Confidence-Multiplier-and-Output_Confidence_Batch.zip]]
			\|}

			==About==
			The '''''Confidence Multiplier''''' and '''''Output Confidence''''' properties of '''[[Data Type]]''' and '''[[Value Reader]]''' extractors allow you to manually alter the confidence score of returned values.

			Use of these properties is sometimes referred to as ''weighted rules''. Its practical application allows a user to increase or decrease the confidence score of an extractor's result (or set its confidence to an assigned value). This changes the confidence of the extractor's results, making them appear more (or less) favorable. When used in combination with the '''''Order By''''' property set to ''Confidence'' on a parent '''Data Type''', you can manipulate which child extractor's result the parent prioritizes.

			===General Usage - Confidence Multiplier===
			Modifying the '''''Confidence Multiplier''''' property of a '''Data Type''' or '''Value Reader''' is done by clicking on the ellipses in the '''''Result Options''''' property which opens the '''''Result Options''''' submenu.

			The '''''Confidence Multiplier''''' property defaults to ''1'' and can be changed in this submenu. The field is a double and takes floating point values.

			For example, a value of ''0.5'' will multiply the confidence of output results by 0.5. If the output confidence was 100%, now it will be 50%. Similarly, you can increase the confidence, even above 100%. If the '''''Confidence Multiplier''''' property is set to ''3'', and an output result had a 50% confidence, it will not display as 150% confidence.

			===General Usage - Output Confidence===
			Modifying the '''''Output Confidence''''' property of a '''Data Type''' or '''Value Reader''' is also done by clicking on the ellipses in the '''''Result Options''''' property which opens the '''''Result Options''''' submenu.

			The '''''Output Confidence''''' property defaults to ''0%'' and can be changed in this submenu. The default of ''0%'' will not alter the results confidence scores. Changing this number will override whatever the result's original confidence is and replace it with this value.

			For example, a value of ''75%'' will change the confidence of output results to 75%. If the output confidence was 100%, now it will be 75%. If the output confidence was 50%, now it will be 75%. If it was 75%, it will now be (you guessed it) 75%. It doesn't matter what the original confidence was, it will be transformed to the '''''Output Confidence''''' value.

			==Waterfall Classification==
			Setting the '''''Classification Method''''' property on a '''Content Model''' to ''Lexical'' or ''Rules-Based'', one can set up ''Positive Extractors'' on '''Document Types'''. If this extractor returns a result above the '''''Minimum Similarity''''' set on the '''Content Model''', the document will be assigned that '''Document Type''' during classification. By default a result from an extractor is returned at 100% confidence (unless '''[[Fuzzy RegEx]]''' is leveraged to return a result, in which case the confidence will be affected by the fuzzy algorithm.) Given this fact positive extractors are almost certain to be above the '''''Minimum Similarity'''''.

			This extractor could be a "Waterfall Extractor", taking advantage of the Waterfall Extraction technique. However, for classification, the system is just looking for some result to be returned above the '''''Minimum Similarity''''' confidence threshold.

			In the ''Waterfall Classification'' method, the '''''Minimum Confidence''''' property can be set in the '''Result Filter''' property window of a '''Data Type''' which will eliminate any results less than that confidence. This may eliminate the results of some referenced extractors which technically matched, but at a low percent.

			If we happen to know that those lower confidence hits are valid and ''should'' count for classifying the document, then the '''Confidence Multipliers''' on those referenced '''Data Types''' can be set to a higher value in order to make them hit the '''Minimum Confidence''' required.

			Similarly, if higher confidence hits are inappropriately classifying documents and ''shouldn't'' be returned, the '''''Confidence Multiplier''''' property can be reduced so that those '''Data Types''' only exceed the '''''Minimum Confidence''''' when they are very high confidence.

			====Example====
			In the example below, we are going to use the '''Project''' and '''Batch''' that accompany the Document Classification 2023 course on Grooper University.

			# Right now we're looking at how documents are currently being classified by working in a Classify '''Batch Process Step'''.
			# We see that this Title Opinion is being misclassified as a Generic Letter.
			# Notice that the document has a similarity score of 100% for the Generic Letter '''Document Type''' and a 68% score for the Title Opinion '''Document Type'''.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 01.png\|center]]


			# If we go to the '''Content Model'''...
			# We can see that our '''''Minimum Similarity''''' property is set to 55%.
			#* Both the Generic Letter and the Title Opinion '''Document Types''' came in at above the '''''Minimim Similarity''''' percentage, but the Generic Letter won out at a higher percentage.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 02.png\|center]]


			# Let's look at the Generic Letter '''Document Type'''.
			# The '''''Positive Extractor''''' is set to a reference.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 03.png\|center]]


			# Let's look at the extractor that is being referenced.
			# We're going to scroll down to the "OUTPUT" section in the '''Data type''' "Properties" tab, and click the ellipsis button next to '''''Result Options'''''.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 04.png\|center]]


			# When the "Result Options" window pops up, we see that by default the '''''Confidence Override''''' is set to 0%.
			# If we set this property to anything other than 0%, when a document is classified, whatever '''Document Type''' is using this extractor will have a similarity score no higher than that number.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 05.png\|center]]


			# We're going to set the '''''Confidence Override''''' to 60%.
			# Click "OK" to apply the new settings.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 06.png\|center]]


			# With our settings updated, let's go back to the Classify '''Batch Process Step'''.
			# On the "Classification Tester" tab we have reclassified the documents.
			# Notice that the Title Opinion document is now being classified appropriately.
			# The Title Opinion '''Document Type''' is still coming in at 68%. However, the Generic Letter '''Document Type''' is returning with a 60% similarity score due to the '''''Confidence Override''''' property we set.

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 07.png\|center]]


			# This Generic Letter is still being classified as a Generic Letter '''Document Type'''.
			# We see that although the Generic '''Document Type''' has a 60% similarity score, it is still higher than the '''''Minimum Similiarity''''' score of 55% and it is also higher than any other '''Document Type'''

			[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 08.png\|center]]

Randallkinard at 13:23, 28 August 2024

2024-08-28T13:23:23Z

← Older revision		Revision as of 08:23, 28 August 2024
Line 1:		Line 1:
	{{~~Migrated2023~~}}		{{Migrated}}
	{{2023:{{PAGENAME}}}}		{{2023:{{PAGENAME}}}}

Randallkinard: Randallkinard moved page Confidence Multiplier and Output Confidence to Confidence Multiplier and Output Confidence (Property): new naming convention

2024-04-16T16:06:41Z

Randallkinard moved page Confidence Multiplier and Output Confidence to Confidence Multiplier and Output Confidence (Property): new naming convention

← Older revision	Revision as of 11:06, 16 April 2024
(No difference)

Dgreenwood: Removed redirect to Confidence Multiplier and Output Confidence - 2023

2023-12-22T18:47:25Z

Removed redirect to Confidence Multiplier and Output Confidence - 2023

← Older revision		Revision as of 13:47, 22 December 2023
Line 1:		Line 1:
	~~#REDIRECT [[Confidence Multiplier and Output Confidence -~~ 2023]]		{{Migrated2023}}
			{{2023:{{PAGENAME}}}}

Rpatton: Changed redirect target from Confidence Multiplier and Output Confidence - 2.90 to Confidence Multiplier and Output Confidence - 2023

2023-11-27T22:38:09Z

Changed redirect target from Confidence Multiplier and Output Confidence - 2.90 to Confidence Multiplier and Output Confidence - 2023

← Older revision		Revision as of 17:38, 27 November 2023
Line 1:		Line 1:
	#REDIRECT [[Confidence Multiplier and Output Confidence - ~~2.90~~]]		#REDIRECT [[Confidence Multiplier and Output Confidence - 2023]]

Dgreenwood: Dgreenwood moved page Confidence Multiplier and Output Confidence to Confidence Multiplier and Output Confidence - 2.90: To be consistent with current naming conventions

2022-12-09T18:19:46Z

Dgreenwood moved page Confidence Multiplier and Output Confidence to Confidence Multiplier and Output Confidence - 2.90: To be consistent with current naming conventions

New page

#REDIRECT [[Confidence Multiplier and Output Confidence - 2.90]]

← Older revision		Revision as of 14:13, 10 December 2024
Line 51:		Line 51:
	# Notice that the document has a similarity score of 100% for the Generic Letter '''Document Type''' and a 68% score for the Title Opinion '''Document Type'''.		# Notice that the document has a similarity score of 100% for the Generic Letter '''Document Type''' and a 68% score for the Title Opinion '''Document Type'''.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 01.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 01.png]]


Line 58:		Line 58:
	#* Both the Generic Letter and the Title Opinion '''Document Types''' came in at above the '''''Minimim Similarity''''' percentage, but the Generic Letter won out at a higher percentage.		#* Both the Generic Letter and the Title Opinion '''Document Types''' came in at above the '''''Minimim Similarity''''' percentage, but the Generic Letter won out at a higher percentage.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 02.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 02.png]]


Line 64:		Line 64:
	# The '''''Positive Extractor''''' is set to a reference.		# The '''''Positive Extractor''''' is set to a reference.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 03.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 03.png]]


Line 70:		Line 70:
	# We're going to scroll down to the "OUTPUT" section in the '''Data type''' "Properties" tab, and click the ellipsis button next to '''''Result Options'''''.		# We're going to scroll down to the "OUTPUT" section in the '''Data type''' "Properties" tab, and click the ellipsis button next to '''''Result Options'''''.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 04.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 04.png]]


Line 76:		Line 76:
	# If we set this property to anything other than 0%, when a document is classified, whatever '''Document Type''' is using this extractor will have a similarity score no higher than that number.		# If we set this property to anything other than 0%, when a document is classified, whatever '''Document Type''' is using this extractor will have a similarity score no higher than that number.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 05.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 05.png]]


Line 82:		Line 82:
	# Click "OK" to apply the new settings.		# Click "OK" to apply the new settings.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 06.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 06.png]]


Line 90:		Line 90:
	# The Title Opinion '''Document Type''' is still coming in at 68%. However, the Generic Letter '''Document Type''' is returning with a 60% similarity score due to the '''''Confidence Override''''' property we set.		# The Title Opinion '''Document Type''' is still coming in at 68%. However, the Generic Letter '''Document Type''' is returning with a 60% similarity score due to the '''''Confidence Override''''' property we set.

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 07.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 07.png]]


Line 96:		Line 96:
	# We see that although the Generic '''Document Type''' has a 60% similarity score, it is still higher than the '''''Minimum Similiarity''''' score of 55% and it is also higher than any other '''Document Type'''		# We see that although the Generic '''Document Type''' has a 60% similarity score, it is still higher than the '''''Minimum Similiarity''''' score of 55% and it is also higher than any other '''Document Type'''

	[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 08.png~~\|center~~]]		[[File:2023 Confidence Multiplier and Output Confidence - 2023 01 How To 01 Waterfall Classification 08.png]]