Batch Processing Statistics - Revision history

Dgreenwood: /* Code Activity Stats */

2025-05-19T16:09:12Z

Code Activity Stats

← Older revision		Revision as of 11:09, 19 May 2025
Line 17:		Line 17:
	=== Folder Viewer stats ===		=== Folder Viewer stats ===

	== Code Activity ~~Stats~~ ==		== Code Activity stats ==

	=== Stats collected independent of Activity type ===		=== Stats collected independent of Activity type ===

Dgreenwood: /* Review Stats */

2025-05-19T16:09:02Z

Review Stats

← Older revision		Revision as of 11:09, 19 May 2025
Line 3:		Line 3:
	This article seeks to document the different Batch processing statistics for each Activity in Grooper. This will help you better understand the stats Grooper collects and create the kinds of queries you want in the Stats page.		This article seeks to document the different Batch processing statistics for each Activity in Grooper. This will help you better understand the stats Grooper collects and create the kinds of queries you want in the Stats page.

	== Review ~~Stats~~ ==		== Review stats ==

	=== Stats collected independent of Review Viewers used ===		=== Stats collected independent of Review Viewers used ===

	=== Scan Viewer ~~Stats~~ ===		=== Scan Viewer stats ===

	=== Data Viewer ~~Stats~~ ===		=== Data Viewer stats ===

	=== Classification Viewer ~~Stats~~ ===		=== Classification Viewer stats ===

	=== Thumbnail Viewer ~~Stats ===~~		=== Thumbnail Viewer stats ===

	~~=== Folder Viewer Stats~~ ===

			=== Folder Viewer stats ===

	== Code Activity Stats ==		== Code Activity Stats ==

Dgreenwood: /* Stats collected per Activity */

2025-05-19T16:07:55Z

Stats collected per Activity

Show changes

Dgreenwood: /* Stats collected independent of Activity type */

2025-05-19T16:01:42Z

Stats collected independent of Activity type

← Older revision		Revision as of 11:01, 19 May 2025
Line 48:		Line 48:
	*** Each thread in the Batch Process takes 5 seconds to process a single Extract task. With 10 total tasks, that's a total of 50 seconds.		*** Each thread in the Batch Process takes 5 seconds to process a single Extract task. With 10 total tasks, that's a total of 50 seconds.
	*** 10 threads @ 5 seconds per thread = 50 total seconds (each thread's processing time added together)		*** 10 threads @ 5 seconds per thread = 50 total seconds (each thread's processing time added together)

			=== Stats collected per Activity ===

			Batch Transfer
			* "Batches Transferred"

			Classify
			* "Folders Classified"
			* "Secondary Types Assigned"
			* "Default Types Assigned"

			Clip Frames
			* "Frames Extracted"

			Correct
			* "Split Count"
			* "Correction Count"
			* "Removal Count"

			Execute
			* One stat will be recorded for each command executed

			Extract
			* "Valid Documents"
			* "Invalid Documents"
			* "Fields Extracted"
			* "Fields Confident"
			* "Fields in Error"
			* "Fields With Value"
			* "Sections Extracted"
			* "Table Rows Extracted"
			* "Characters Extracted"

			Image Processing
			* "Pages Processed"
			* "Pages Modified"
			* "Pages Flagged"
			* "IP Commands Processed"
			* "PDF Pages Updated"

			Merge
			* "Pages Merged"
			* "Source Bytes"
			* "Destination Bytes"
			* "Files Created"

			Recognize
			* "Characters - OCR"
			* "Characters - Native"
			* "Characters"
			* "Layout - Barcodes"
			* "Layout - Lines"
			* "Layout - Checkboxes"
			* "Pages Processed"
			* "Pages Normalized"
			* "Masked Font Pages"
			* "Masked Font Characters"

			Redact
			* "Redactions Performed"

			Remove Level
			* "Folders Removed"

			Render
			* "Documents Rendered"
			* "Pages Rendered"
			* "Unknown File Types"
			* "Render Errors"
			* "Excluded Files"
			* "Bytes Rendered"

			Send Mail
			* "Messages Sent"

			Separate
			* "Folders Created"
			* "Pages Processed"
			* "Pages Classified"
			* "Pages Separated"
			* "Pages Deleted"
			* "Documents Classified"

			Spawn Batch
			* "Folders Moved"
			* "Folders Copied"

			Split Pages
			* "Pages Created"
			* "Grade A - High Quality"
			* "Grade B - Medium Quality"
			* "Grade C - Low Quality"
			* "Burst: PDF Page"
			* "Burst: Single Page"
			* "Burst: Rendered Image"
			* "Bytes In"
			* "Bytes Out"
			* "Image Transforms"

			Split Text
			* "Documents Created"

			Train Lexicon
			* "Features Trained"

			Translate
			* "Input Characters"
			* "Output Characters"

Dgreenwood: /* Stats collected independent of Activity type */

2025-05-19T13:59:08Z

Stats collected independent of Activity type

← Older revision		Revision as of 08:59, 19 May 2025
Line 23:		Line 23:

	Regardless of the Activity the Batch Process Step is running, every step will record the following stats:		Regardless of the Activity the Batch Process Step is running, every step will record the following stats:
	* '''Tasks Processed''' - This is the total number of tasks in the Batch Process Step that are successfully processed.
	** Note if the task errors out, it will not be recorded.		'''Tasks Processed''' - This is the total number of tasks in the Batch Process Step that are successfully processed.
	*** If there are 100 available tasks and they all complete successfully, 100 tasks are recorded.		* Note if the task errors out, it will not be recorded.
	*** If there are 100 available tasks and 10 of them error out, 90 tasks are recorded.		** If there are 100 available tasks and they all complete successfully, 100 tasks are recorded.
	** This is determined by how many tasks are available in scope.		** If there are 100 available tasks and 10 of them error out, 90 tasks are recorded.
	*** If the step is scoped to the Batch level (and the step completes successfully), there will be a single task recorded.		* This is determined by how many tasks are available in scope.
	*** If the step is scoped to the Folder level and there are 10 Batch Folders (and they all complete successfully in the step), there will be 10 tasks recorded.		** If the step is scoped to the Batch level (and the step completes successfully), there will be a single task recorded.
	*** If the step is scoped to the Page level and there are 100 Batch Pages (and they all complete successfully in the step), there will be 100 tasks recorded.		** If the step is scoped to the Folder level and there are 10 Batch Folders (and they all complete successfully in the step), there will be 10 tasks recorded.
	* '''Execution Time''' - This is the total time ~~elapsed~~ it takes the Batch Process Step to run from start to finish or "total elapsed time".		** If the step is scoped to the Page level and there are 100 Batch Pages (and they all complete successfully in the step), there will be 100 tasks recorded.
	* '''Processing Time''' - This is how much time each machine thread spends processing tasks added together.
	** This is not to be confused with "Execution Time".		'''Execution Time''' - This is the total time it takes the Batch Process Step to run from start to finish or the step's "total elapsed time".
	** Example: An Activity Processing service uses 10 threads to process tasks.
	*** There are 10 Batch Folders in a Batch at Folder level 1.		'''Processing Time''' - This is how much time each machine thread spends processing tasks added together.
	*** The Extract step is scoped to Folder level 1. So, there are 10 available tasks to process.		* This is not to be confused with "Execution Time".
	*** All threads pick up the 10 Extract tasks at roughly the same time.		* Example: An Activity Processing service uses 10 threads to process tasks.
	*** The Extract step's "Execution Time" would be ~~roughly 10~~ seconds.		** There are 10 Batch Folders in a Batch at Folder level 1.
	**** It would take roughly 10 seconds to run the Extract step from start to finish.		** The Extract step is scoped to Folder level 1. So, there are 10 available tasks to process.
	**** 10 ~~tasks~~ running concurrently @ 10 seconds per thread = A total elapsed time of 10 seconds.		** Each thread can processes one task in about 5 seconds.
	*** The Extract step's "Processing Time" would be ~~100~~ seconds ~~(or 1m 40s)~~		** All threads pick up the 10 Extract tasks at roughly the same time.
	**** Each thread in the Batch Process takes 10 seconds to process a single Extract task. With 10 total tasks, that's a total of ~~100~~ seconds.		** The Extract step's "Execution Time" would be ''5 seconds''.
	**** 10 ~~tasks~~ @ 10 seconds per thread = ~~100~~ total seconds (each thread's processing time added together)		*** It would take roughly 5 seconds to run the Extract step from start to finish.
			*** 10 threads running concurrently process 10 tasks @ 5 seconds per thread = A total elapsed time of 5 seconds.
			** The Extract step's "Processing Time" would be ''50 seconds''.
			*** Each thread in the Batch Process takes 5 seconds to process a single Extract task. With 10 total tasks, that's a total of 50 seconds.
			*** 10 threads @ 5 seconds per thread = 50 total seconds (each thread's processing time added together)

Dgreenwood: Created page with "Grooper logs a variety of processing statistics for production Batches. These stats can be seen when selecting a Batch's "Statistics" tab in the Batches Page or when returning query results from the Stats Page. Stats are logged for each step in the Batch's Batch Process, according to its Activity. Some of theses stats are obvious by their name. Some of them are less so. This article seeks to document the different Batch processing statistics for each Activity in Grooper..."

2025-05-16T22:00:58Z

Created page with "Grooper logs a variety of processing statistics for production Batches. These stats can be seen when selecting a Batch's "Statistics" tab in the Batches Page or when returning query results from the Stats Page. Stats are logged for each step in the Batch's Batch Process, according to its Activity. Some of theses stats are obvious by their name. Some of them are less so. This article seeks to document the different Batch processing statistics for each Activity in Grooper..."

New page

Grooper logs a variety of processing statistics for production Batches. These stats can be seen when selecting a Batch's "Statistics" tab in the Batches Page or when returning query results from the Stats Page. Stats are logged for each step in the Batch's Batch Process, according to its Activity. Some of theses stats are obvious by their name. Some of them are less so.

This article seeks to document the different Batch processing statistics for each Activity in Grooper. This will help you better understand the stats Grooper collects and create the kinds of queries you want in the Stats page.

== Review Stats ==

=== Stats collected independent of Review Viewers used ===

=== Scan Viewer Stats ===

=== Data Viewer Stats ===

=== Classification Viewer Stats ===

=== Thumbnail Viewer Stats ===

=== Folder Viewer Stats ===

== Code Activity Stats ==

=== Stats collected independent of Activity type ===

Regardless of the Activity the Batch Process Step is running, every step will record the following stats:
* '''Tasks Processed''' - This is the total number of tasks in the Batch Process Step that are successfully processed.
** Note if the task errors out, it will not be recorded.
*** If there are 100 available tasks and they all complete successfully, 100 tasks are recorded.
*** If there are 100 available tasks and 10 of them error out, 90 tasks are recorded.
** This is determined by how many tasks are available in scope.
*** If the step is scoped to the Batch level (and the step completes successfully), there will be a single task recorded.
*** If the step is scoped to the Folder level and there are 10 Batch Folders (and they all complete successfully in the step), there will be 10 tasks recorded.
*** If the step is scoped to the Page level and there are 100 Batch Pages (and they all complete successfully in the step), there will be 100 tasks recorded.
* '''Execution Time''' - This is the total time elapsed it takes the Batch Process Step to run from start to finish or "total elapsed time".
* '''Processing Time''' - This is how much time each machine thread spends processing tasks added together.
** This is not to be confused with "Execution Time".
** Example: An Activity Processing service uses 10 threads to process tasks.
*** There are 10 Batch Folders in a Batch at Folder level 1.
*** The Extract step is scoped to Folder level 1. So, there are 10 available tasks to process.
*** All threads pick up the 10 Extract tasks at roughly the same time.
*** The Extract step's "Execution Time" would be roughly 10 seconds.
**** It would take roughly 10 seconds to run the Extract step from start to finish.
**** 10 tasks running concurrently @ 10 seconds per thread = A total elapsed time of 10 seconds.
*** The Extract step's "Processing Time" would be 100 seconds (or 1m 40s)
**** Each thread in the Batch Process takes 10 seconds to process a single Extract task. With 10 total tasks, that's a total of 100 seconds.
**** 10 tasks @ 10 seconds per thread = 100 total seconds (each thread's processing time added together)