2.90:Fuzzy RegEx (Concept) - Revision history

Dgreenwood at 16:03, 5 August 2025

2025-08-05T16:03:59Z

← Older revision		Revision as of 11:03, 5 August 2025
Line 1:		Line 1:
			__NOINDEX__
	{{AutoVersion}}		{{AutoVersion}}

Randallkinard at 19:25, 23 April 2024

2024-04-23T19:25:48Z

← Older revision		Revision as of 14:25, 23 April 2024
Line 1:		Line 1:
	{{AutoVersion}}		{{AutoVersion}}
	<blockquote ~~style="font-size~~:~~14pt">~~
	''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.		<blockquote>{{#lst:Glossary\|Fuzzy RegEx}}</blockquote>
	</blockquote>

	Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.		Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.

Randallkinard: Randallkinard moved page 2.90:Fuzzy RegEx to 2.90:Fuzzy RegEx (Concept) without leaving a redirect: new naming convention

2024-04-17T16:46:23Z

Randallkinard moved page 2.90:Fuzzy RegEx to 2.90:Fuzzy RegEx (Concept) without leaving a redirect: new naming convention

← Older revision	Revision as of 11:46, 17 April 2024
(No difference)

Dgreenwood at 15:47, 29 December 2023

2023-12-29T15:47:49Z

← Older revision		Revision as of 10:47, 29 December 2023
Line 1:		Line 1:
	~~<onlyinclude>~~		{{AutoVersion}}
	<blockquote style="font-size:14pt">		<blockquote style="font-size:14pt">
	''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.		''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.
Line 9:		Line 9:

	For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.		For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.
	~~</onlyinclude>~~

	== About ==		== About ==

Dgreenwood: Dgreenwood moved page Fuzzy RegEx to 2.90:Fuzzy RegEx

2023-12-29T15:46:43Z

Dgreenwood moved page Fuzzy RegEx to 2.90:Fuzzy RegEx

← Older revision	Revision as of 10:46, 29 December 2023
(No difference)

Dgreenwood at 21:22, 9 December 2022

2022-12-09T21:22:10Z

← Older revision		Revision as of 16:22, 9 December 2022
Line 835:		Line 835:

	\|}		\|}

			[[Category:Articles]]
			[[Category:Version 2.90]]

Dgreenwood at 21:24, 23 November 2020

2020-11-23T21:24:48Z

← Older revision		Revision as of 16:24, 23 November 2020
Line 1:		Line 1:
			<onlyinclude>
	<blockquote style="font-size:14pt">		<blockquote style="font-size:14pt">
	''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.		''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.
Line 8:		Line 9:

	For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.		For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.
			</onlyinclude>

	== About ==		== About ==

Dgreenwood: /* False Positive Potential */

2020-11-20T14:03:47Z

False Positive Potential

← Older revision		Revision as of 09:03, 20 November 2020
Line 171:		Line 171:
	There's a total 16 characters in the pattern. Therefore, each swap costs 6.25% confidence. 3 swaps for a total swap cost of 18.75%. ''Fuzzy RegEx'' would match "<span style="font-family:Courier New">consecutive years</span>" for <code>consecutive days</code> with an 81.25% confidence.		There's a total 16 characters in the pattern. Therefore, each swap costs 6.25% confidence. 3 swaps for a total swap cost of 18.75%. ''Fuzzy RegEx'' would match "<span style="font-family:Courier New">consecutive years</span>" for <code>consecutive days</code> with an 81.25% confidence.

	It's important to recognize this potential to produce false positive results. Depending on your situation this can effect the accuracy of your results. You may be able to eliminate them by adjusting your '''''Minimum Similarity''' threshold, only allowing higher confident results. You may need to exempt these results through other methods, such as with an exclusion extractor. You may need human review of Grooper's automated results to ensure total accuracy. Or, you may be fine with the margin of error this produces.		It's important to recognize this potential to produce false positive results. Depending on your situation this can effect the accuracy of your results. You may be able to eliminate them by adjusting your '''''Minimum Similarity''''' threshold, only allowing higher confident results. You may need to exempt these results through other methods, such as with an exclusion extractor. You may need human review of Grooper's automated results to ensure total accuracy. Or, you may be fine with the margin of error this produces.

	Whatever your take on the situation, be aware ''Fuzzy RegEx's'' ability to capture ''more'' data comes at the potential cost of capturing ''inaccurate'' data.		Whatever your take on the situation, be aware ''Fuzzy RegEx's'' ability to capture ''more'' data comes at the potential cost of capturing ''inaccurate'' data.

Dgreenwood: /* Words of Caution */

2020-11-19T21:15:45Z

Words of Caution

← Older revision		Revision as of 16:15, 19 November 2020
Line 174:		Line 174:

	Whatever your take on the situation, be aware ''Fuzzy RegEx's'' ability to capture ''more'' data comes at the potential cost of capturing ''inaccurate'' data.		Whatever your take on the situation, be aware ''Fuzzy RegEx's'' ability to capture ''more'' data comes at the potential cost of capturing ''inaccurate'' data.

			=== Incompatibility With Infinite Quantifiers ===

			In regular expression, single characters can be pattern matched, as can ''ranges'' of characters or sets of characters. For example, <code>\d</code> will match a single digit character (0 through 9) and <code>\d{2,5}</code> will match digit characters two to five digits in length.

			There are two infinite length quantifiers:

			* The plus quantifier <code>+</code>, matching a character or character set one to any number in length.
			* The star quantifier <code>*</code>, matching a character or character set zero to any number in length.

			''Fuzzy RegEx'' looks at each character in the pattern, one after the other, compared to the text data to see if a character swap is required. If there are 20 characters in the regex pattern, ''Fuzzy RegEx'' is looking for 20 potential character swaps for every string of characters in the text data. With the <code>+</code> and <code>*</code> quantifiers, the length of the regex pattern is potentially infinite characters in length. ''Fuzzy RegEx'' would ''never stop'' looking to see if characters could be swapped.

			''Fuzzy RegEx'' is therefore incompatible with infinite quantifiers. Grooper will give you an error if the '''''Mode''''' is set to ''Fuzzy RegEx'' and these quantifiers are used in your pattern.

			=== It's Slower Than Standard RegEx ===

			While this may or may not ultimately matter, it's worth pointing out. Because ''Fuzzy RegEx'' is considering much more of the text data for potential matches than the binary "match" or "doesn't match" approach to standard regex, it requires more processing power to perform. It is therefore, slower.

			This may be the difference between a '''Data Type''' taking 2 ms to extract a value from a document compared to 20 ms. This may not seem like much, but when you're processing thousands of documents and/or using hundreds of extractors, the processing time can add up. Furthermore, depending on the complexity of the extractor, this speed for a single extractor to return a single value may be even greater.

	== How To ==		== How To ==

Dgreenwood: /* False Positive Potential */

2020-11-19T20:56:48Z

False Positive Potential

← Older revision		Revision as of 15:56, 19 November 2020
Line 154:		Line 154:
	How different are the phrases "consecutive years" and "consecutive days"? Semantically, they are very different. You'll find this type of language in legal agreements. If you agree to do something every day for a number of consecutive days, that's very different from doing something every year for a number of consecutive years.		How different are the phrases "consecutive years" and "consecutive days"? Semantically, they are very different. You'll find this type of language in legal agreements. If you agree to do something every day for a number of consecutive days, that's very different from doing something every year for a number of consecutive years.

	However, ''Fuzzy RegEx'' doesn't know what "years" and "days" ''mean'', only how similar they are based on how few character edits it takes~~, what~~ the Levenshtein distance between them is. So, what's the difference? Just three characters.		However, ''Fuzzy RegEx'' doesn't know what "years" and "days" ''mean'', only how similar they are based on how few character edits it takes. What the Levenshtein distance between them is. So, what's the difference? Just three characters.

	{\|cellpadding=10 cellspacing=5		{\|cellpadding=10 cellspacing=5