2023.1:Grooper Infrastructure (Concept): Difference between revisions

Revision as of 15:26, 23 July 2020

Technically speaking, what is Grooper? Grooper is a repository of information made up by 13 tables in a SQL Server Database and associated files in a Windows file share. This information is displayed to the user via an application(s) that displays a 1 to 1 relationship of what exists in that database and fileshare. Gaining an understanding of how to properly create using Grooper begins by first understanding the structure of its architecture, and why it is built the way it is. This sets a foundational groundwork for knowing how to think and interact with the application.

Grooper InfrastructureSystem Architecture

The Three Layers of Grooper

Grooper consists of three main components that constantly interact with one another:

Database
File Store
Application

The Database

This is the core component of Grooper as all aspects of the functionality and configuration of a Grooper repository are stored here as metadata in cells of tables. There are 13 tables that make up the Grooper database:

dbo.AccessControlEntry
- This table contains the information necessary for implementing Grooper’s node-level security architecture.

Column Name	Data Type	Size	Description
Id	int	10	unique number referencing the table's row
NodeId	uniqueidentifier	16	guid associated with node from Grooper node tree
PrincipalId	uniqueidentifier	16
Permissions	in	4

dbo.CustomStats
- Definition incoming ...

Column Name	Data Type	Size
SessionId	int	4
Name	Varchar	64
Value	Float	8

dbo.FileStoreEntry
- This table contains the information necessary for associating filestore objects with Grooper nodes. Note: a node with associated files will also contain those files’ filestore object locations, names, and mimetypes in the “files” column of TreeNode.

Column Name	Data Type	Size	Description
Id	int	4	sequential number assigned to object in row
FileId	uniqueidentifier	16	guid assigned as the name of the object (stored in the filestore with this number followed by .grp)
NodeId	uniqueidentifier	16	guid associated with node from grooper tree
FileStoreId	uniqueidentifier	16	guid given to the node referencing the file store

dbo.License
- This table contains information relating to Grooper licensing, including the licensing and tamper keys.

Column Name	Data Type	Size	Description
Id	int	4	unique number referencing the table’s row
SerialNumber	uniqueidentifier	16	guid of serial number
LicenseText	Varchar	1024
CustomerId	Varchar	256	number given to customer owning the license
CustomerName	varchar	256	string name given to customer owning the license
MachineId	varchar	40	unique identifier of the machine which the license was generated for
LicenseCode	varchar	50	shorthand for the type of license assigned
LicenseDesc	varchar	256	description of the type of license assigned
Quantity	int	4	amount of licenses given upon assignment
Expires	int	4
ExpirationDate	datetime	8	date and time the assigned license expires
ResetInterval	varchar	12	basis on which the assigned licenses resets
LastReset	datetime	8	date and time the license was last reset
RemainingCount	int	4	amount of licenses remaining since assigned
TamperKey	varchar	256

dbo.LicenseCheckout
- This table contains information relating to checked out licenses.
dbo.Lock
- This table contains one row per locked node; prevents overlapping access to various resources.
dbo.Log
- This table contains the Grooper log.
dbo.NodeReference
- This table contains a list of all referenced nodes, and what other nodes they are referenced by. Used to protect referenced nodes from deletion, and in determining what nodes are necessary on exporting.
dbo.ProcessingTask
- This table contains tasks submitted for activity processing as part of production batches.
dbo.ServiceInstance
- This table contains a list of installed services. Note: If an installed service does not have a reference in this table, it will not show up in Grooper Config.
dbo.SessionStats
- This table contains statistical information regarding batches in Grooper.
dbo.Setting
- This table contains only the Database version, including build number.
dbo.TreeNode
- The main Grooper table, TreeNode contains one entry for every object in the Grooper node tree. Composed of the following columns:
  - ID – The node’s unique ID. A GUID autogenerated by Grooper.
  - Row ID – Identity Column for the table. Generated by SQL server.
  - Row Version – a timestamp column, updated when the node is changed. Used by certain processes to determine if the node has been updated.
  - Name – The name of the node.
  - TypeName – The name of the object type of the node.
  - ParentID – ID of the node’s parent. The Grooper Root node has a parent ID with all zeros.
  - NodeIndex – Index of the node in it’s containing object (usually a folder)
  - Attributes – A flags attribute indicating whether the node has the following attributes: read-only, fixed contents (no children may be added), sorted (children will always show up in alphabetical order), has ACL, is a system object (may not be changed).
  - NumChildren – Number of children.
  - Properties – Column containing the JSON properties for this node.
  - NodeValues
  - Files – Files associated with this node.

Using a SQL database as the core of Grooper allows for great efficiency. Every property of every object in Grooper, as a result of being a simple entry in a table, can be loaded into memory and accessed nearly instantly. This would not be the case otherwise (if for example Grooper repositories were defined by something like a project file), as file i/o is one of the slowest aspects of modern computing. This also allows the discrete management of objects on an individual basis to allow multiple users to work in one environment and prevent work overlap by locking objects.

The File Store

The File Store in Grooper is a file share in a Windows environment. It houses the files associated with objects in Grooper that have information that would otherwise be inefficient to store in (a cell in) a database table.

The Grooper Filestore exists at a user-specified location. This may be a local or an network path, but if a filestore is given a local path, computers connecting to that repository remotely will not be able to access it. If you want to set up a repository so that other computers can connect to it, make sure you reference the filestore using a UNC path!

The filestore contains three levels of directories. A typical filestore entry will exist on disk as, e.g. 00 > 00 > 00 > [guid].grp. Each of the lowest-level folders in the filestore will have a maximum of 256 files, at which point a new folder at that level will be created. If the lowest level contains 256 folders, a new folder will be created at the level above; this gives the Grooper filestore a limit of 256 ^ 4 = 4,294,967,296 files stored on disk.

While the filestore entries are all given .grp extensions, the contents of the file are unaltered from their “actual” form. If you navigate, for example, to the .grp file associated with an pdf imported using full import, you can open it and view it with a pdf viewer. The files in the filestore are intentionally obfuscated to prevent users from interacting with them outside of Grooper, as they are essentially Grooper-internal objects.

Although the majority of files in the filestore relate to batch objects (split pages, imported documents, image processing undo objects, etc.), some files are the result of other in-grooper processes such as layout data, OCR character data, etc.

The Application

This is the most visible aspect of Grooper as it is the software you interact with that displays the currently active repository. It consists of several pieces:

Every object in the Grooper Node Tree is an object, or row, in a specific table, the dbo.TreeNode table. The GUI of Grooper is essentially wrapping information from the Grooper database, and associated files from the Grooper File Store, into a series of grids and windows that allow you to directly interact with that database, and its related Windows file share, without writing SQL queries.

@@ Line 1: / Line 1: @@
-Technically speaking, what is Grooper?  Grooper is a '''[https://en.wikipedia.org/wiki/Information_repository repository]''' of information made up by 13 tables in a '''[https://en.wikipedia.org/wiki/Microsoft_SQL_Server SQL Server]''' '''[https://en.wikipedia.org/wiki/Database Database]''' and associated files in a '''[https://en.wikipedia.org/wiki/Microsoft_Windows Windows]''' file share. This information is displayed to the user via an '''[https://en.wikipedia.org/wiki/Application_software application(s)]''' that displays a 1 to 1 relationship of what exists in that database and fileshare. Gaining an understanding of how to properly create using Grooper begins by first understanding the structure of its architecture, and why it is built the way it is. This sets a foundational groundwork for knowing how to think and interact with the application.
+Technically speaking, what is '''Grooper'''?  '''Grooper''' is a '''[https://en.wikipedia.org/wiki/Information_repository repository]''' of information made up by 13 tables in a '''[https://en.wikipedia.org/wiki/Microsoft_SQL_Server SQL Server]''' '''[https://en.wikipedia.org/wiki/Database Database]''' and associated files in a '''[https://en.wikipedia.org/wiki/Microsoft_Windows Windows]''' file share. This information is displayed to the user via an '''[https://en.wikipedia.org/wiki/Application_software application(s)]''' that displays a 1 to 1 relationship of what exists in that database and fileshare. Gaining an understanding of how to properly create using '''Grooper''' begins by first understanding the structure of its architecture, and why it is built the way it is. This sets a foundational groundwork for knowing how to think and interact with the application.
 <tabs style="margin:20px">
@@ Line 11: / Line 11: @@
 ==The Three Layers of Grooper==
-Grooper consists of three main components that constantly interact with one another:
+'''Grooper''' consists of three main components that constantly interact with one another:
 # Database
 # '''[[File Store]]'''
@@ Line 17: / Line 17: @@
 ===The Database===
-This is the core component of Grooper as all aspects of the functionality and configuration of a Grooper repository are stored here as '''[https://en.wikipedia.org/wiki/Metadata metadata]''' in '''[https://en.wikipedia.org/wiki/Table_cell cells]''' of '''[https://en.wikipedia.org/wiki/Table_(information) tables]'''. There are 13 tables that make up the Grooper database:
+This is the core component of '''Grooper''' as all aspects of the functionality and configuration of a '''Grooper''' repository are stored here as '''[https://en.wikipedia.org/wiki/Metadata metadata]''' in '''[https://en.wikipedia.org/wiki/Table_cell cells]''' of '''[https://en.wikipedia.org/wiki/Table_(information) tables]'''. There are 13 tables that make up the '''Grooper''' database:
-# dbo.AccessControlEntry
+#; dbo.AccessControlEntry
-#* This table contains the information necessary for implementing Grooper’s node-level security architecture.
+#* This table contains the information necessary for implementing '''Grooper’s''' node-level security architecture.
+::: {| class="wikitable"
+! Column Name !! Data Type !! Size !! Description
+|-
+| Id || int || 10 || unique number referencing the table's row
+|-
+| NodeId || uniqueidentifier || 16 || guid associated with node from '''Grooper''' node tree
+|-
+| PrincipalId || uniqueidentifier || 16 ||
+|-
+| Permissions || in || 4 ||
+|}
 # dbo.CustomStats
+#* ''Definition incoming ...''
+::: {| class="wikitable"
+! Column Name !! Data Type !! Size !! Description
+|-
+| SessionId || int || 4 ||
+|-
+| Name || Varchar || 64 ||
+|-
+| Value || Float || 8 ||
+|}
 # dbo.FileStoreEntry
-#* This table contains the information necessary for associating filestore objects with Grooper nodes.  Note: a node with associated files will also contain those files’ filestore object locations, names, and mimetypes in the “files” column of TreeNode.
+#* This table contains the information necessary for associating filestore objects with '''Grooper''' nodes.  Note: a node with associated files will also contain those files’ filestore object locations, names, and mimetypes in the “files” column of TreeNode.
+::: {| class="wikitable"
+! Column Name !! Data Type !! Size !! Description
+|-
+| Id || int || 4 || sequential number assigned to object in row
+|-
+| FileId || uniqueidentifier || 16 || guid assigned as the name of the object (stored in the filestore with this number followed by .grp)
+|-
+| NodeId || uniqueidentifier || 16 || guid associated with node from grooper tree
+|-
+| FileStoreId || uniqueidentifier || 16 || guid given to the node referencing the file store
+|}
 # dbo.License
-#* This table contains information relating to Grooper licensing, including the licensing and tamper keys.
+#* This table contains information relating to '''Grooper''' licensing, including the licensing and tamper keys.
+::: {| class="wikitable"
+! Column Name !! Data Type !! Size !! Description
+|-
+| Id || int || 4 || unique number referencing the table’s row
+|-
+| SerialNumber || uniqueidentifier || 16 || guid of serial number
+|-
+| LicenseText || Varchar || 1024 ||
+|-
+| CustomerId || Varchar || 256 || number given to customer owning the license
+|-
+| CustomerName || varchar || 256 || string name given to customer owning the license
+|-
+| MachineId || varchar || 40 || unique identifier of the machine which the license was generated for
+|-
+| LicenseCode || varchar || 50 || shorthand for the type of license assigned
+|-
+| LicenseDesc || varchar || 256 || description of the type of license assigned
+|-
+| Quantity || int || 4 || amount of licenses given upon assignment
+|-
+| Expires || int ||4  ||
+|-
+| ExpirationDate || datetime || 8 || date and time the assigned license expires
+|-
+| ResetInterval || varchar || 12 || basis on which the assigned licenses resets
+|-
+| LastReset || datetime || 8 || date and time the license was last reset
+|-
+| RemainingCount || int || 4 || amount of licenses remaining since assigned
+|-
+| TamperKey || varchar || 256 ||
+|}
 # dbo.LicenseCheckout
 #* This table contains information relating to checked out licenses.
@@ Line 30: / Line 95: @@
 #* This table contains one row per locked node; prevents overlapping access to various resources.
 # dbo.Log
-#* This table contains the Grooper log.
+#* This table contains the '''Grooper''' log.
 # dbo.NodeReference
 #* This table contains a list of all referenced nodes, and what other nodes they are referenced by.  Used to protect referenced nodes from deletion, and in determining what nodes are necessary on exporting.
@@ Line 36: / Line 101: @@
 #* This table contains tasks submitted for activity processing as part of production batches.
 # dbo.ServiceInstance
-#* This table contains a list of installed services.  Note: If an installed service does not have a reference in this table, it will not show up in Grooper Config.
+#* This table contains a list of installed services.  Note: If an installed service does not have a reference in this table, it will not show up in '''Grooper''' Config.
 # dbo.SessionStats
-#* This table contains statistical information regarding batches in Grooper.
+#* This table contains statistical information regarding batches in '''Grooper'''.
 # dbo.Setting
 #* This table contains only the Database version, including build number.
 # dbo.TreeNode
-#* The main Grooper table, TreeNode contains one entry for every object in the Grooper node tree.  Composed of the following columns:
+#* The main '''Grooper''' table, TreeNode contains one entry for every object in the '''Grooper''' node tree.  Composed of the following columns:
-#** ID – The node’s unique ID.  A GUID autogenerated by Grooper.
+#** ID – The node’s unique ID.  A GUID autogenerated by '''Grooper'''.
 #** Row ID – Identity Column for the table.  Generated by SQL server.
 #** Row Version – a timestamp column, updated when the node is changed.  Used by certain processes to determine if the node has been updated.
 #** Name – The name of the node.
 #** TypeName – The name of the object type of the node.
-#** ParentID – ID of the node’s parent.  The Grooper Root node has a parent ID with all zeros.
+#** ParentID – ID of the node’s parent.  The '''Grooper''' Root node has a parent ID with all zeros.
 #** NodeIndex – Index of the node in it’s containing object (usually a folder)
 #** Attributes – A flags attribute indicating whether the node has the following attributes: read-only, fixed contents (no children may be added), sorted (children will always show up in alphabetical order), has ACL, is a system object (may not be changed).
@@ Line 56: / Line 121: @@
 #** Files – Files associated with this node.
-Using a SQL database as the core of Grooper allows for great efficiency. Every property of every object in Grooper, as a result of being a simple entry in a table, can be loaded into memory and accessed nearly instantly.  This would not be the case otherwise (if for example Grooper repositories were defined by something like a project file), as file i/o is one of the slowest aspects of modern computing. This also allows the discrete management of objects on an individual basis to allow multiple users to work in one environment and prevent work overlap by locking objects.
+Using a SQL database as the core of '''Grooper''' allows for great efficiency. Every property of every object in '''Grooper''', as a result of being a simple entry in a table, can be loaded into memory and accessed nearly instantly.  This would not be the case otherwise (if for example '''Grooper''' repositories were defined by something like a project file), as file i/o is one of the slowest aspects of modern computing. This also allows the discrete management of objects on an individual basis to allow multiple users to work in one environment and prevent work overlap by locking objects.
 ===The File Store===
-The '''[[File Store]]''' in Grooper is a file share in a Windows environment. It houses the files associated with objects in Grooper that have information that would otherwise be inefficient to store in (a cell in) a database table.
+The '''[[File Store]]''' in '''Grooper''' is a file share in a Windows environment. It houses the files associated with objects in '''Grooper''' that have information that would otherwise be inefficient to store in (a cell in) a database table.
+The '''Grooper Filestore''' exists at a user-specified location.  This may be a local or an network path, but if a filestore is given a local path, computers connecting to that repository remotely will not be able to access it.  If you want to set up a repository so that other computers can connect to it, make sure you reference the filestore using a UNC path!
+The filestore contains three levels of directories.  A typical filestore entry will exist on disk as, e.g. 00 > 00 > 00 > [guid].grp.  Each of the lowest-level folders in the filestore will have a maximum of 256 files, at which point a new folder at that level will be created.  If the lowest level contains 256 folders, a new folder will be created at the level above; this gives the '''Grooper''' filestore a limit of 256 ^ 4 = 4,294,967,296 files stored on disk.
+While the filestore entries are all given .grp extensions, the contents of the file are unaltered from their “actual” form.  If you navigate, for example, to the .grp file associated with an pdf imported using full import, you can open it and view it with a pdf viewer.  The files in the filestore are intentionally obfuscated to prevent users from interacting with them outside of '''Grooper''', as they are essentially '''Grooper-internal''' objects.
+Although the majority of files in the filestore relate to batch objects (split pages, imported documents, image processing undo objects, etc.), some files are the result of other in-grooper processes such as layout data, OCR character data, etc.
 ===The Application===
-This is the most visible aspect of Grooper as it is the software you interact with that displays the currently active repository.  It consists of several pieces:
+This is the most visible aspect of '''Grooper''' as it is the software you interact with that displays the currently active repository.  It consists of several pieces:
 # '''[[Grooper Design Studio]]'''
 # '''[[Grooper Config]]'''
 # '''[[Grooper Attended Client]]'''
 # '''[[Grooper Unattended Client]]'''
-Every object in the Grooper '''[[Node Tree]]''' is an object, or '''[https://en.wikipedia.org/wiki/Row_(database) row]''', in a specific table, the dbo.TreeNode table. The '''[https://en.wikipedia.org/wiki/Graphical_user_interface GUI]''' of Grooper is essentially wrapping information from the Grooper database, and associated files from the Grooper File Store, into a series of grids and windows that allow you to directly interact with that database, and its related Windows file share, without writing SQL queries.
+Every object in the '''Grooper''' '''[[Node Tree]]''' is an object, or '''[https://en.wikipedia.org/wiki/Row_(database) row]''', in a specific table, the dbo.TreeNode table. The '''[https://en.wikipedia.org/wiki/Graphical_user_interface GUI]''' of '''Grooper''' is essentially wrapping information from the '''Grooper''' database, and associated files from the Grooper File Store, into a series of grids and windows that allow you to directly interact with that database, and its related Windows file share, without writing SQL queries.