2023:Fuzzy RegEx (Concept) - Revision history

Randallkinard at 19:37, 21 November 2024

2024-11-21T19:37:30Z

← Older revision		Revision as of 14:37, 21 November 2024
Line 752:		Line 752:

	\|}		\|}


	~~== Glossary ==~~
	~~<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}~~

	~~<u><big>'''Batch'''</big></u>: {{#lst:Glossary\|Batch}}~~

	~~<u><big>'''Data Field'''</big></u>: {{#lst:Glossary\|Data Field}}~~

	~~<u><big>'''Data Type'''</big></u>: {{#lst:Glossary\|Data Type}}~~

	~~<u><big>'''Extract'''</big></u>: {{#lst:Glossary\|Extract}}~~

	~~<u><big>'''Extractor Type'''</big></u>: {{#lst:Glossary\|Extractor Type}}~~

	~~<u><big>'''Fuzzy RegEx'''</big></u>: {{#lst:Glossary\|Fuzzy RegEx}}~~

	~~<u><big>'''Image Processing'''</big></u>: {{#lst:Glossary\|Image Processing}}~~

	~~<u><big>'''Image Processing'''</big></u>: {{#lst:Glossary\|Image Processing}}~~

	~~<u><big>'''Lexicon'''</big></u>: {{#lst:Glossary\|Lexicon}}~~

	~~<u><big>'''Node Tree'''</big></u>: {{#lst:Glossary\|Node Tree}}~~

	~~<u><big>'''OCR Synthesis'''</big></u>: {{#lst:Glossary\|OCR Synthesis}}~~

	~~<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}~~

	~~<u><big>'''Pattern-Based Separation'''</big></u>: {{#lst:Glossary\|Pattern-Based Separation}}~~

	~~<u><big>'''Pattern-Based'''</big></u>: {{#lst:Glossary\|Pattern-Based}}~~

	~~<u><big>'''Project'''</big></u>: {{#lst:Glossary\|Project}}~~

	~~<u><big>'''Recognize'''</big></u>: {{#lst:Glossary\|Recognize}}~~

	~~<u><big>'''Separation'''</big></u>: {{#lst:Glossary\|Separation}}~~

	~~<u><big>'''Tab Marking'''</big></u>: {{#lst:Glossary\|Tab Marking}}~~

	~~<u><big>'''Value Reader'''</big></u>: {{#lst:Glossary\|Value Reader}}~~

	~~<u><big>'''Visual'''</big></u>: {{#lst:Glossary\|Visual}}~~

Randallkinard at 21:25, 26 August 2024

2024-08-26T21:25:10Z

@@ Line 18: / Line 18: @@
 * [[Media:2023_Wiki_Fuzzy-RegEx_Project.zip]]
 |}
 == About ==
@@ Line 795: / Line 752: @@
 |}

Randallkinard at 13:32, 10 May 2024

2024-05-10T13:32:40Z

@@ Line 18: / Line 18: @@
 * [[Media:2023_Wiki_Fuzzy-RegEx_Project.zip]]
 |}
 == About ==
 {|cellpadding=10 cellspacing=5
 |style="width:40%" valign=top|

Randallkinard at 19:26, 23 April 2024

2024-04-23T19:26:03Z

← Older revision		Revision as of 14:26, 23 April 2024
Line 1:		Line 1:
	{{AutoVersion}}		{{AutoVersion}}
	~~<section begin="glossary" />~~
	<blockquote>		<blockquote>{{#lst:Glossary\|Fuzzy RegEx}}</blockquote>
	''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.
	</blockquote>
	~~<section end="glossary" />~~
	Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.		Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.

Randallkinard: Randallkinard moved page 2023:Fuzzy RegEx to 2023:Fuzzy RegEx (Concept) without leaving a redirect: new naming convention

2024-04-17T16:46:34Z

Randallkinard moved page 2023:Fuzzy RegEx to 2023:Fuzzy RegEx (Concept) without leaving a redirect: new naming convention

← Older revision	Revision as of 11:46, 17 April 2024
(No difference)

Dsmith at 20:45, 11 March 2024

2024-03-11T20:45:30Z

← Older revision		Revision as of 15:45, 11 March 2024
Line 10:		Line 10:

	For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.		For example, a text string that is 95% similar to the regex pattern may be off by just a single character. If the '''''Minimum Similarity''''' threshold is set to ''90%'' the result would be returned, even though the pattern doesn't match the text ''exactly''.
			<br>
			<br>
			{\|class="download-box"
			\|
			[[File:Asset 22@4x.png]]
			\|
			You may download the ZIP(s) below and upload it into your own Grooper environment (version 2023). The first contains one or more '''Batches''' of sample documents. The second contains one or more '''Projects''' with resources used in examples throughout this article.
			* [[Media:2023_Wiki_Fuzzy-RegEx_Batch.zip]]
			* [[Media:2023_Wiki_Fuzzy-RegEx_Project.zip]]
			\|}
	== About ==		== About ==

Dgreenwood at 15:31, 12 January 2024

2024-01-12T15:31:31Z

← Older revision		Revision as of 10:31, 12 January 2024
Line 359:		Line 359:
	\|}		\|}

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|As well as ''decreasing'' the cost to perform a swap, you can also ''increase'' the cost.		'''FYI'''
			\|
			As well as ''decreasing'' the cost to perform a swap, you can also ''increase'' the cost.

	<code>6G=0.5</code> would ''decrease'' the cost to perform a swap from "6" to "G" by 50%.		<code>6G=0.5</code> would ''decrease'' the cost to perform a swap from "6" to "G" by 50%.
Line 387:		Line 389:
	: <code>Insert(o)=0.5</code>		: <code>Insert(o)=0.5</code>

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|You can also adjust the cost of multiple character inserts all at once by simply including more characters in the character set.		'''FYI'''
			\|
			You can also adjust the cost of multiple character inserts all at once by simply including more characters in the character set.

	For example, if you wanted to reduce the cost to insert a period (.), comma (,) or semicolon(;) by half, you could use the following fuzzy match weighting:		For example, if you wanted to reduce the cost to insert a period (.), comma (,) or semicolon(;) by half, you could use the following fuzzy match weighting:
Line 413:		Line 417:
	\|}		\|}

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|One common real-world use of the insert fuzzy match weighting is to adjust the cost to insert spaces. Space characters can often get inappropriately inserted or removed during OCR.		'''FYI'''
			\|
			One common real-world use of the insert fuzzy match weighting is to adjust the cost to insert spaces. Space characters can often get inappropriately inserted or removed during OCR.

	To adjust the insert cost of a space character, simply type a literal space between the parentheses, as seen below:		To adjust the insert cost of a space character, simply type a literal space between the parentheses, as seen below:
Line 440:		Line 446:
	: <code>Delete(o)=0.5</code>		: <code>Delete(o)=0.5</code>

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|You can also adjust the cost of multiple character deletions all at once by simply including more characters in the character set.		'''FYI'''
			\|
			You can also adjust the cost of multiple character deletions all at once by simply including more characters in the character set.

	For example, if you wanted to reduce the cost to delete a period (.), comma (,) or semicolon(;) by half, you could use the following fuzzy match weighting:		For example, if you wanted to reduce the cost to delete a period (.), comma (,) or semicolon(;) by half, you could use the following fuzzy match weighting:
Line 469:		Line 477:
	\|}		\|}

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|One common real-world use of the delete fuzzy match weighting is to adjust the cost to delete extra spaces. Space characters can often get inappropriately inserted or removed during OCR.		'''FYI'''
			\|
			One common real-world use of the delete fuzzy match weighting is to adjust the cost to delete extra spaces. Space characters can often get inappropriately inserted or removed during OCR.

	To adjust the delete cost of a space character, simply type a literal space between the parentheses, as seen below:		To adjust the delete cost of a space character, simply type a literal space between the parentheses, as seen below:
Line 699:		Line 709:
	With this fuzzy match weighting in place, the two results that mutated a <code>\t</code> character into a single space character are discarded, leaving us the desired results.		With this fuzzy match weighting in place, the two results that mutated a <code>\t</code> character into a single space character are discarded, leaving us the desired results.

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="fyi-box"
	\|~~-style="background-color:#36b0a7; color:white"~~		\|
	~~\|style="font-size:14pt"\|~~'''FYI'''\|\|Often it is the case all the large white space characters, including tab characters, should be considered immutable. Since they are used to distinguish large gaps in whitespace from a single space, very rarely (if ever) does it make sense to swap them for a single space characters.		'''FYI'''
			\|
			Often it is the case all the large white space characters, including tab characters, should be considered immutable. Since they are used to distinguish large gaps in whitespace from a single space, very rarely (if ever) does it make sense to swap them for a single space characters.

	The most common Immutable weighting is as follows:		The most common Immutable weighting is as follows:

Dgreenwood at 14:45, 12 January 2024

2024-01-12T14:45:03Z

Dgreenwood at 14:41, 12 January 2024

2024-01-12T14:41:55Z

← Older revision		Revision as of 09:41, 12 January 2024
Line 341:		Line 341:
	\|}		\|}

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="attn-box"
	\|~~-style="background-color:#f89420; color:white"~~		\|
	~~\|style="font-size:23pt"\|'''~~⚠~~'''~~		⚠
	\|		\|
	A message concerning case sensitivity:		A message concerning case sensitivity:
Line 714:		Line 714:
	</tabs>		</tabs>

	{\|~~cellpadding~~="~~10" cellspacing="5~~"		{\|class="attn-box"
	\|~~-style="background-color:#f89420; color:white"~~		\|
	~~\|style="font-size:23pt"\|'''~~⚠~~'''\|~~\|It can be easy to confuse Required Mode with Immutable Characters and vice versa. While they both throw out fuzzy matches, they do it in very different ways.		⚠
			\|
			It can be easy to confuse Required Mode with Immutable Characters and vice versa. While they both throw out fuzzy matches, they do it in very different ways.

Dgreenwood at 18:25, 9 January 2024

2024-01-09T18:25:26Z

← Older revision		Revision as of 13:25, 9 January 2024
Line 1:		Line 1:
	{{AutoVersion}}		{{AutoVersion}}
	<section begin="~~fuzzy_regex_glossary~~" />		<section begin="glossary" />
	<blockquote>		<blockquote>
	''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.		''Fuzzy RegEx'' (also referred to as "fuzzy matching" or "fuzzy mode" or even just "fuzzy") allows regular expression patterns to match text within a set percentage of similarity. This can allow Grooper users to overcome unpredictable [[OCR]] errors when extracting data from documents.
	</blockquote>		</blockquote>
	<section end="~~fuzzy_regex_glossary~~" />		<section end="glossary" />
	Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.		Typically, regular expression will either match a string of text or it won't. If you're trying to match a word and the regex pattern is even a single character off from the text data, you will not return a result.

← Older revision		Revision as of 14:37, 21 November 2024
Line 752:		Line 752:

	\|}		\|}


	~~== Glossary ==~~
	~~<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}~~

	~~<u><big>'''Batch'''</big></u>: {{#lst:Glossary\|Batch}}~~

	~~<u><big>'''Data Field'''</big></u>: {{#lst:Glossary\|Data Field}}~~

	~~<u><big>'''Data Type'''</big></u>: {{#lst:Glossary\|Data Type}}~~

	~~<u><big>'''Extract'''</big></u>: {{#lst:Glossary\|Extract}}~~

	~~<u><big>'''Extractor Type'''</big></u>: {{#lst:Glossary\|Extractor Type}}~~

	~~<u><big>'''Fuzzy RegEx'''</big></u>: {{#lst:Glossary\|Fuzzy RegEx}}~~

	~~<u><big>'''Image Processing'''</big></u>: {{#lst:Glossary\|Image Processing}}~~

	~~<u><big>'''Image Processing'''</big></u>: {{#lst:Glossary\|Image Processing}}~~

	~~<u><big>'''Lexicon'''</big></u>: {{#lst:Glossary\|Lexicon}}~~

	~~<u><big>'''Node Tree'''</big></u>: {{#lst:Glossary\|Node Tree}}~~

	~~<u><big>'''OCR Synthesis'''</big></u>: {{#lst:Glossary\|OCR Synthesis}}~~

	~~<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}~~

	~~<u><big>'''Pattern-Based Separation'''</big></u>: {{#lst:Glossary\|Pattern-Based Separation}}~~

	~~<u><big>'''Pattern-Based'''</big></u>: {{#lst:Glossary\|Pattern-Based}}~~

	~~<u><big>'''Project'''</big></u>: {{#lst:Glossary\|Project}}~~

	~~<u><big>'''Recognize'''</big></u>: {{#lst:Glossary\|Recognize}}~~

	~~<u><big>'''Separation'''</big></u>: {{#lst:Glossary\|Separation}}~~

	~~<u><big>'''Tab Marking'''</big></u>: {{#lst:Glossary\|Tab Marking}}~~

	~~<u><big>'''Value Reader'''</big></u>: {{#lst:Glossary\|Value Reader}}~~

	~~<u><big>'''Visual'''</big></u>: {{#lst:Glossary\|Visual}}~~

← Older revision		Revision as of 09:45, 12 January 2024
Line 347:		Line 347:
	A message concerning case sensitivity:		A message concerning case sensitivity:

	For the character pair, {DocumentChar}{PatternChar}, one of the characters is case sensitive where the other is ''not''.		For the character pair, <code>{DocumentChar}{PatternChar}</code>, one of the characters is case sensitive where the other is ''not''.

	: {DocumentChar} ''is'' case sensitive.		: <code>{DocumentChar}</code> ''is'' case sensitive.

	: {PatternChar} ''is not'' case sensitive.		: <code>{PatternChar}</code> ''is not'' case sensitive.

	This means <code>s5=0.5</code> would ''only'' adjust the cost to swap a lowercase "s" in the text data for a "5" in the regex pattern.		This means <code>s5=0.5</code> would ''only'' adjust the cost to swap a lowercase "s" in the text data for a "5" in the regex pattern.
Line 724:		Line 724:

	Required Mode prevents a ''pattern character'' in the regex from matching a different character in the text data.		Required Mode prevents a ''pattern character'' in the regex from matching a different character in the text data.
	~~: <li style="color:white">~~Another way of thinking about it is Required Mode makes that portion of the regex behave as if it were ''not'' fuzzy. An exact match is ''required''.~~</li>~~		*Another way of thinking about it is Required Mode makes that portion of the regex behave as if it were ''not'' fuzzy. An exact match is ''required''.

	Immutable Characters prevents a ''document character'' in the text data from matching a different character in the regex.		Immutable Characters prevents a ''document character'' in the text data from matching a different character in the regex.
	~~: <li style="color:white">~~Another way of thinking about it is Immutable Characters make the cost to swap a character in the text data infinite. Remember the normal swap weighting syntax:<~~/li~~>		* Another way of thinking about it is Immutable Characters make the cost to swap a character in the text data infinite. Remember the normal swap weighting syntax:
	~~:::~~ {Document Character}{Pattern Character}=Cost		** <code>{Document Character}{Pattern Character}=Cost</code>
	:: Immutable characters make the cost to swap infinite, or:		* Immutable characters make the cost to swap infinite, or:
	~~:::~~ {Immutable Character}{Any Pattern Character}=Infinite		** <code>{Immutable Character}{Any Pattern Character}=Infinite</code>

	\|}		\|}