2023:TF-IDF (Concept) - Revision history

Randallkinard at 15:41, 22 November 2024

2024-11-22T15:41:50Z

← Older revision		Revision as of 10:41, 22 November 2024
Line 278:		Line 278:
	[[File:2023 TF-IDF 04.png]]		[[File:2023 TF-IDF 04.png]]
	\|}		\|}

	~~== Glossary ==~~
	~~<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}~~

	~~<u><big>'''Classification Method'''</big></u>: {{#lst:Glossary\|Classification Method}}~~

	~~<u><big>'''Classification'''</big></u>: {{#lst:Glossary\|Classification}}~~

	~~<u><big>'''Content Model'''</big></u>: {{#lst:Glossary\|Content Model}}~~

	~~<u><big>'''Document Type'''</big></u>: {{#lst:Glossary\|Document Type}}~~

	~~<u><big>'''Field Class'''</big></u>: {{#lst:Glossary\|Field Class}}~~

	~~<u><big>'''Lexical'''</big></u>: {{#lst:Glossary\|Lexical}}~~

	~~<u><big>'''Machine'''</big></u>: {{#lst:Glossary\|Machine}}~~

	~~<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}~~

	~~<u><big>'''TF-IDF'''</big></u>: {{#lst:Glossary\|TF-IDF}}~~

Randallkinard at 15:36, 27 August 2024

2024-08-27T15:36:01Z

@@ Line 4: / Line 4: @@
 TF-IDF is designed to reflect how important a word is to a document or a set of documents.  The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency).  The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency).  Some words appear more generally across documents and hence are less unique identifiers.  So, their weighting is lessened.
 == Introduction: What is TF-IDF? ==
@@ Line 299: / Line 278: @@
 [[File:2023 TF-IDF 04.png]]
 |}

Randallkinard at 18:24, 10 May 2024

2024-05-10T18:24:48Z

← Older revision		Revision as of 13:24, 10 May 2024
Line 4:		Line 4:

	TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.		TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.

			== Glossary ==
			<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}

			<u><big>'''Classification Method'''</big></u>: {{#lst:Glossary\|Classification Method}}

			<u><big>'''Classification'''</big></u>: {{#lst:Glossary\|Classification}}

			<u><big>'''Content Model'''</big></u>: {{#lst:Glossary\|Content Model}}

			<u><big>'''Document Type'''</big></u>: {{#lst:Glossary\|Document Type}}

			<u><big>'''Field Class'''</big></u>: {{#lst:Glossary\|Field Class}}

			<u><big>'''Lexical'''</big></u>: {{#lst:Glossary\|Lexical}}

			<u><big>'''Machine'''</big></u>: {{#lst:Glossary\|Machine}}

			<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}

			<u><big>'''TF-IDF'''</big></u>: {{#lst:Glossary\|TF-IDF}}

	== Introduction: What is TF-IDF? ==		== Introduction: What is TF-IDF? ==

Randallkinard at 20:14, 23 April 2024

2024-04-23T20:14:42Z

← Older revision		Revision as of 15:14, 23 April 2024
Line 1:		Line 1:
	{{AutoVersion}}		{{AutoVersion}}
	~~<section begin="glossary" />~~
	<blockquote>		<blockquote>{{#lst:Glossary\|TF-IDF}}</blockquote>
	TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic intended to reflect how important a word is to a document within a collection (or document set or “corpus”). It is how Grooper uses machine learning for training-based document classification (via the [[Lexical (Classification Method)\|Lexical]] method) and data extraction (via the [[Field Class]] extractor).
	</blockquote>
	~~<section end="glossary" />~~
	TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.		TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.

	== Introduction: What is TF-IDF? ==		== Introduction: What is TF-IDF? ==

	TF-IDF stands for Term Frequency-Inverse Document Frequency, and it is a statistical measure that is used in the areas of information retrieval, machine learning, and text mining to quantify or show how important that a word or phrase is to a document within a collection (or document set or “corpus”).		TF-IDF stands for Term Frequency-Inverse Document Frequency, and it is a statistical measure that is used in the areas of information retrieval, machine learning, and text mining to quantify or show how important that a word or phrase is to a document within a collection (or document set or “corpus”).

Randallkinard: Randallkinard moved page 2023:TF-IDF to 2023:TF-IDF (Concept) without leaving a redirect: new naming convention

2024-04-18T16:38:39Z

Randallkinard moved page 2023:TF-IDF to 2023:TF-IDF (Concept) without leaving a redirect: new naming convention

← Older revision	Revision as of 11:38, 18 April 2024
(No difference)

Dgreenwood: /* Defining Term Frequency */

2024-04-04T16:32:56Z

Defining Term Frequency

← Older revision		Revision as of 11:32, 4 April 2024
Line 29:		Line 29:
	For TF-IDF, the fact that you see a term more frequently is an important feature of that document. The more time that term (also known as feature) repeats on a document, the higher that term's (or feature's) weighting value.		For TF-IDF, the fact that you see a term more frequently is an important feature of that document. The more time that term (also known as feature) repeats on a document, the higher that term's (or feature's) weighting value.

	==== ~~<span style="color:#662d91">~~Defining Term Frequency~~</span>~~ ====		==== Defining Term Frequency ====

	There are multiple ways of defining frequency, including:		There are multiple ways of defining frequency, including:

Dgreenwood at 22:52, 8 January 2024

2024-01-08T22:52:42Z

← Older revision		Revision as of 17:52, 8 January 2024
Line 1:		Line 1:
	{{AutoVersion}}		{{AutoVersion}}
	<~~blockquote style~~="~~font-size:14pt~~">		<section begin="glossary" />
			<blockquote>
	TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic intended to reflect how important a word is to a document within a collection (or document set or “corpus”). It is how Grooper uses machine learning for training-based document classification (via the [[Lexical (Classification Method)\|Lexical]] method) and data extraction (via the [[Field Class]] extractor).		TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic intended to reflect how important a word is to a document within a collection (or document set or “corpus”). It is how Grooper uses machine learning for training-based document classification (via the [[Lexical (Classification Method)\|Lexical]] method) and data extraction (via the [[Field Class]] extractor).
	</blockquote>		</blockquote>
			<section end="glossary" />
	TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.		TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.

Dgreenwood: Dgreenwood moved page TF-IDF - 2023 to 2023:TF-IDF without leaving a redirect

2024-01-03T16:21:51Z

Dgreenwood moved page TF-IDF - 2023 to 2023:TF-IDF without leaving a redirect

← Older revision	Revision as of 11:21, 3 January 2024
(No difference)

Dgreenwood at 16:21, 3 January 2024

2024-01-03T16:21:40Z

@@ Line 1: / Line 1: @@
 <blockquote style="font-size:14pt">
 TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic intended to reflect how important a word is to a document within a collection (or document set or “corpus”).  It is how Grooper uses machine learning for training-based document classification (via the [[Lexical (Classification Method)|Lexical]] method) and data extraction (via the [[Field Class]] extractor).
@@ Line 4: / Line 5: @@
 TF-IDF is designed to reflect how important a word is to a document or a set of documents.  The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency).  The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency).  Some words appear more generally across documents and hence are less unique identifiers.  So, their weighting is lessened.
 == Introduction: What is TF-IDF? ==
@@ Line 287: / Line 280: @@
 [[File:2023 TF-IDF 04.png]]
 |}

Dgreenwood at 14:29, 19 October 2023

2023-10-19T14:29:24Z

← Older revision		Revision as of 09:29, 19 October 2023
Line 4:		Line 4:

	TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.		TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.


	{\| class="wikitable" style="margin:left"		{\| class="wikitable" style="margin:left"

← Older revision		Revision as of 10:41, 22 November 2024
Line 278:		Line 278:
	[[File:2023 TF-IDF 04.png]]		[[File:2023 TF-IDF 04.png]]
	\|}		\|}

	~~== Glossary ==~~
	~~<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}~~

	~~<u><big>'''Classification Method'''</big></u>: {{#lst:Glossary\|Classification Method}}~~

	~~<u><big>'''Classification'''</big></u>: {{#lst:Glossary\|Classification}}~~

	~~<u><big>'''Content Model'''</big></u>: {{#lst:Glossary\|Content Model}}~~

	~~<u><big>'''Document Type'''</big></u>: {{#lst:Glossary\|Document Type}}~~

	~~<u><big>'''Field Class'''</big></u>: {{#lst:Glossary\|Field Class}}~~

	~~<u><big>'''Lexical'''</big></u>: {{#lst:Glossary\|Lexical}}~~

	~~<u><big>'''Machine'''</big></u>: {{#lst:Glossary\|Machine}}~~

	~~<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}~~

	~~<u><big>'''TF-IDF'''</big></u>: {{#lst:Glossary\|TF-IDF}}~~

← Older revision		Revision as of 13:24, 10 May 2024
Line 4:		Line 4:

	TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.		TF-IDF is designed to reflect how important a word is to a document or a set of documents. The importance value (or weighting) of a word increases proportionally to the number of times it appears in the document (Term Frequency). The weighting is offset by the number of documents in the set containing the word (Inverse Document Frequency). Some words appear more generally across documents and hence are less unique identifiers. So, their weighting is lessened.

			== Glossary ==
			<u><big>'''AND'''</big></u>: {{#lst:Glossary\|AND}}

			<u><big>'''Classification Method'''</big></u>: {{#lst:Glossary\|Classification Method}}

			<u><big>'''Classification'''</big></u>: {{#lst:Glossary\|Classification}}

			<u><big>'''Content Model'''</big></u>: {{#lst:Glossary\|Content Model}}

			<u><big>'''Document Type'''</big></u>: {{#lst:Glossary\|Document Type}}

			<u><big>'''Field Class'''</big></u>: {{#lst:Glossary\|Field Class}}

			<u><big>'''Lexical'''</big></u>: {{#lst:Glossary\|Lexical}}

			<u><big>'''Machine'''</big></u>: {{#lst:Glossary\|Machine}}

			<u><big>'''OCR'''</big></u>: {{#lst:Glossary\|OCR}}

			<u><big>'''TF-IDF'''</big></u>: {{#lst:Glossary\|TF-IDF}}

	== Introduction: What is TF-IDF? ==		== Introduction: What is TF-IDF? ==