Main Page: Difference between revisions

From Grooper Wiki
← Older editNewer edit →
No edit summary
No edit summary
Line 1: Line 1:
{|cellpadding="15" cellspacing="10" width="100%"
Technically speaking, what is '''Grooper'''?  '''Grooper''' is a '''[https://en.wikipedia.org/wiki/Information_repository repository]''' of information made up by 13 tables in a '''[https://en.wikipedia.org/wiki/Microsoft_SQL_Server SQL Server]''' '''[https://en.wikipedia.org/wiki/Database Database]''' and associated files in a '''[https://en.wikipedia.org/wiki/Microsoft_Windows Windows]''' file share. This information is displayed to the user via an '''[https://en.wikipedia.org/wiki/Application_software application(s)]''' that displays a 1 to 1 relationship of what exists in that database and fileshare. Gaining an understanding of how to properly create using '''Grooper''' begins by first understanding the structure of its architecture, and why it is built the way it is. This sets a foundational groundwork for knowing how to think and interact with the application.
|-style="background-color:#f89420; color:white; font-size:16pt"
|colspan="4"|'''Getting Started'''
|-style="background-color:#fde6cb" valign="top"
|rowspan="3" style="width:75%"|
Grooper was built from the ground up by BIS, a company with 35 years of continuous experience developing and delivering new technology. Grooper is an intelligent document processing and digital data integration solution that empowers organizations to extract meaningful information from paper/electronic documents and other forms of unstructured data.


The platform combines patented and sophisticated image processing, capture technology, machine learning, natural language processing, and optical character recognition to enrich and embed human comprehension into data. By tackling tough challenges that other systems cannot resolve, Grooper has become the foundation for many industry-first solutions in healthcare, financial services, oil and gas, education, and government.
==Grooper infrastructure==
[[file:GrooperInfrastructure01.png]]


|[https://xchange.grooper.com/discussion/57/read-me-getting-started Getting Started]
==System Architecture and Requirements==
|-style="background-color:#fde6cb" valign="top"
===End User PC===
|[[Install and Setup]]
* The ''end user pc'' would include '''Grooper Thick Clients''' and installations on premisis.
|-style="background-color:#fde6cb" valign="top"
* Network latency should be (ideally) less than 50ms.
|[https://grooper.bisok.com/Documentation/2.90/Main/HTML5/index.htm#t=Start_Page.htm 2.90 Reference Documentation]
* Network throughput should be (ideally) greater than 50MB.
|}
 
===Grooper Processing Server===
* A '''Grooper''' processing environment can have as many servers as is necssary to meet business requirements.
* [https://en.wikipedia.org/wiki/Windows_10 Windows 10] and/or [https://en.wikipedia.org/wiki/Windows_Server_2012 Windows Server 2012] or later is required.
* Server RAM should be 16GB or more.
* The CPU should consist of 4 or more cores.
* The port for licensing is 13900 by default.
 
===Grooper SQL Server===
* [https://en.wikipedia.org/wiki/Microsoft_SQL_Server#SQL_Server_2012 SQL Server 2012] or later is required.
* The '''Grooper Database ''' should allow for a size of 20GB or more.
* The scaling of the '''Grooper Database''' should be set to percentage, ideally, not the default of a few MB. This will reduce frequent scaling that hinders performance.
* The SQL Port is 1433.
 
===Grooper FileStore Share===
* This can be local to the '''Grooper SQL Server''' or managed on a separate server.
* The ieal filesize would accomodate for roughly 150GB for small to medium installs.
* The authentication requires CIFS or SMB.


===Windows Prerequisites===
The following are included in the installation package of '''Grooper''' and are required for installation (listed are x64 packages.)


{|cellpadding="15" cellspacing="10"
* '''vc_redist_2012_update4_x64'''
|-style="background-color:#36b0a7; color:white; font-size:16pt"
*: Microsoft Visual C++ 2012 Redistributable (x64) - 11.0.61030
|style="width:50%"|'''Featured Articles'''||'''Did you know?'''
* '''vc_redist_2010_sp1_x64'''
|-style="background-color:#d8f3f1" valign="top"
*: Microsoft Visual C++ 2010  x64 Redistributable Setup
|
* '''vc_redist_2013_x64'''
[[File:microsoft_office_integration_000.png|right|thumb|Native text for Microsoft Office applications is a powerful data integration tool in Grooper.]]
*: Microsoft Visual C++ 2013 Redistributable (x64) - 12.0.30501
<blockquote style="font-size:14pt">
* '''vc_redist.x64'''
[[Microsoft Office Integration]]
*: Microsoft Visual C++ 2017 Redistributable (x64) - 14.10.25017
</blockquote>
: Visual C++ Redistributable Packages install runtime components of Visual C++ Libraries on a computer that does not have Visual C++ installed. The libraries are required to run applications that are developed by using the corresponding version of Visual C++.
'''[https://en.wikipedia.org/wiki/Microsoft_Office Microsoft Office]''' integration allows a '''Grooper''' user to leverage the native text of files generated in the Microsoft Office Suite such as '''[https://en.wikipedia.org/wiki/Microsoft_Word Microsoft Word]''' documents and '''[https://en.wikipedia.org/wiki/Microsoft_Excel Microsoft Excel]''' '''[https://en.wikipedia.org/wiki/Personal_Storage_Table spreadsheets]'''. This feature can pull the native text from and perform type-specific activities on these files.
* '''NDP462-DevPack-KB3151934-ENU'''
*: Microsoft .NET Framework 4.6.2 Developer Pack
: .NET is an integral part of many apps running on Windows and provides common functionality for those apps to run. For developers, the .NET Framework provides a comprehensive and consistent programming model for building apps that have visually stunning user experiences and seamless and secure communication.
* '''SQLSysClrTypes'''
: In LINQ to SQL, the data model of a relational database maps to an object model that is expressed in the programming language of your choice. When the application runs, LINQ to SQL translates the language-integrated queries in the object model into SQL and sends them to the database for execution. When the database returns the results, LINQ to SQL translates the results back to objects that you can work with in your own programming language.
: In order to translate data between the object model and the database, a type mapping must be defined. LINQ to SQL uses a type mapping to match each common language runtime (CLR) type with a particular SQL Server type. You can define type mappings and other mapping information, such as database structure and table relationships, inside the object model with attribute-based mapping. Alternatively, you can specify the mapping information outside the object model with an external mapping file.
* '''AccessDatabaseEngine_x64'''
:Microsoft Access Database Engine contains essential components that make it possible for Microsoft Office programs to interact with software services and applications that do not belong to Microsoft. For instance, if you are working in a program that gives you the possibility to export tables directly to Microsoft Access, then you need the database engine to make that happen.
:The Microsoft Access Database Engine is available in multiple editions, which mainly depend on which Microsoft Office product you’re using. The hint is in the year. As such, if you use Microsoft Office 2016, then you need Microsoft Access Database Engine 2016.


===Supported File Types===
* ReportViewer
* Microsoft Word documents (.doc and .docx)
: Used for viewing built in '''Grooper Reports''', which uses a Microsoft Reporting framework.
** For Word documents, you can generate a Grooper-usable document with the '''Execute''' activity, using the ''Word to PDF'' command for the '''Word Document''' object type.  The PDF will contain all the native text from the Word document, obtainable for further Grooper processing using the '''Recognize''' activity.
* Microsoft Excel spreadsheets (xls and xlsx)
** For Excel documents, you can generate a Grooper-usable document with the '''Execute''' activity, using the ''Excel to CSV'' command for the '''Excel Document''' object type.  CSV files are natively readable by Grooper in version 2.90.  The '''Recognize''' activity is ''not'' required.
|
The earliest examples of OCR (Optical Character Recognition) can be traced back to the  1870s.  Early OCR devices were actually invented to aid the blind.  This included "text-to-speech" devices that would scan black print and produce sounds a blind person could interpret, as well as "text-to-tactile" machines which would convert luminous sensations into tactile sensations.  Machines such as these would allow a blind person to read printed text not yet converted to Braille.


The first business to install an OCR reader was the magazine ''Reader's Digest'' in 1954.  The company used it to convert typewritten sales reports into machine readable punch cards.
[[file:SystemArchitecture.png]]


It would not be until 1974 that OCR starts to form as we imagine it now with Ray Kurzweil's development of the first "omni-font" OCR software, capable of reading text of virtually any font.
==The Four Layers of Grooper==
|}
'''Grooper''' consists of four main components that constantly interact with one another:
# Database
# '''[[File Store]]'''
# Licensing
# Application


===The Database===
This is the core component of '''Grooper''' as all aspects of the functionality and configuration of a '''Grooper''' repository are stored here as '''[https://en.wikipedia.org/wiki/Metadata metadata]''' in '''[https://en.wikipedia.org/wiki/Table_(information) tables]'''. There are 13 tables that make up the '''Grooper''' database:


{|cellpadding="15" cellspacing="10"
'''dbo.AccessControlEntry'''
|-style="background-color:#f89420; color:white; font-size:16pt"
* This table contains the information necessary for implementing '''Grooper’s''' node-level security architecture.
|style="width:50%"|'''New in 2.9'''||'''Featured Use Case'''
:: {| class="wikitable"
|-style="background-color:#fde6cb" valign="top"
! Column Name !! Data Type !! Size !! Description
|
|-
{|cellpadding=5 cellspacing=2 style="margin:auto"
| Id || int || 10 || unique number referencing the table's row
!colspan="4" style="padding: 25px" | <blockquote style="font-size:14pt">Welcome to '''Grooper 2.9'''!<br/>Below you will find helpful links to all the articles about the new/changed functionality in this version of '''Grooper'''.</blockquote>
|-
| NodeId || uniqueidentifier || 16 || guid associated with node from '''Grooper''' node tree
|-
| PrincipalId || uniqueidentifier || 16 ||
|-
| Permissions || in || 4 ||
|}
'''dbo.CustomStats'''
* ''Definition incoming ...''
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| SessionId || int || 4 ||
|-
| Name || Varchar || 64 ||
|-
| Value || Float || 8 ||
|}
'''dbo.FileStoreEntry'''
* This table contains the information necessary for associating filestore objects with '''Grooper''' nodes.  Note: a node with associated files will also contain those files’ filestore object locations, names, and mimetypes in the “files” column of TreeNode.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || sequential number assigned to object in row
|-
| FileId || uniqueidentifier || 16 || guid assigned as the name of the object (stored in the filestore with this number followed by .grp)
|-
| NodeId || uniqueidentifier || 16 || guid associated with node from '''Grooper''' tree
|-
| FileStoreId || uniqueidentifier || 16 || guid given to the node referencing the file store
|}
'''dbo.License'''
* This table contains information relating to '''Grooper''' licensing, including the licensing and tamper keys.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique number referencing the table’s row
|-
| SerialNumber || uniqueidentifier || 16 || guid of serial number
|-
| LicenseText || Varchar || 1024 ||
|-
| CustomerId || Varchar || 256 || number given to customer owning the license
|-
| CustomerName || varchar || 256 || string name given to customer owning the license
|-
| MachineId || varchar || 40 || unique identifier of the machine which the license was generated for
|-
| LicenseCode || varchar || 50 || shorthand for the type of license assigned
|-
| LicenseDesc || varchar || 256 || description of the type of license assigned
|-
| Quantity || int || 4 || amount of licenses given upon assignment
|-
| Expires || int ||4  ||
|-
| ExpirationDate || datetime || 8 || date and time the assigned license expires
|-
| ResetInterval || varchar || 12 || basis on which the assigned licenses resets
|-
| LastReset || datetime || 8 || date and time the license was last reset
|-
| RemainingCount || int || 4 || amount of licenses remaining since assigned
|-
| TamperKey || varchar || 256 ||
|}
'''dbo.LicenseCheckout'''
* This table contains information relating to checked out licenses.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique number referencing the table’s row
|-
| LicenseCode || varchar || 50 || string defining the license code
|-
| LicenseDesc || varchar || 256 || definition of the type of license in use
|-
| UserName || varchar || 256 || windows user to whom the license is checked to
|-
| MachineName || varchar || 256 || hostname of machine using the license
|-
| ModuleName || varchar || 256 || '''Grooper''' module occupying the license
|-
| CheckoutTime || datetime || 8 || date and time the license was checked out
|-
| CheckoutId || uniqueidentifier || 16 ||
|}
'''dbo.Lock'''
* This table contains one row per locked node; prevents overlapping access to various resources.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| NodeId || uniqueidentifier || 16 || guid of node from '''Grooper''' hierarchy tree
|-
| UserName || varchar || 64 || windows user that applied the unlock
|-
| MachineName || varchar || 64 || hostname of the machine where the unlock occurred
|-
| ModuleName || varchar || 64 || '''Grooper''' module that executed the unlock
|-
| ProcedureName || varchar || 64 || the procedure executed to apply the unlock
|-
| CreateDate || smalldatetime || 4 || date and time the unlock occurred
|}
'''dbo.Log'''
* This table contains the '''Grooper''' log, which is represented in the '''Event Viewer''' of '''Grooper Design Studio'''.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique row identifier
|-
| TimeStamp || datetime || 8 || date and time the event was logged
|-
| Type || int || 4 || type of event logged
|-
| ProcName || varchar(MAX) || -1 || name of procedure associated with the event
|-
| Message || varchar(MAX) || -1 || message displayed in and defining, in general, the event
|-
| UserAccount || varchar || 64 || windows user under which the event triggered
|-
| MachineName || varchar || 64 || hostname of the machine under which the event triggered
|-
| NodeId || uniqueidentifier || 16 || guid of parent node associated with ‘NodeName’
|-
| NodeName || varchar || 256 || Item Name, or name of node associated with the triggering of the event
|-
| NodeType || varchar || 64 || type of '''Grooper''' Node defining the ‘NodeName’
|}
'''dbo.NodeReference'''
* This table contains a list of all referenced nodes, and what other nodes they are referenced by.  Used to protect referenced nodes from deletion, and in determining what nodes are necessary on exporting.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| NodeId || uniqueidentifier || 16 || guid given to node in '''Grooper''' hierarchy tree
|-
| ReferencingNodeId || uniqueidentifier || 16 || guid of parent node associated with ‘NodeId’
|}
'''dbo.ProcessingTask'''
* This table contains tasks submitted for activity processing as part of production batches.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique row identifier
|-
| BatchId || uniqueidentifier || 16 || guid of batch related to running process
|-
| NodeId || uniqueidentifier || 16 || guid of node related to batch in running process
|-
| Settings || varchar(MAX) || -1 || json properties of running node
|-
| ModuleName || varchar || 256 || module running current task
|-
| Status || int || 4 || status of current task (i.e. running, error…)
|-
| UserName || varchar || 256 || windows user that submitted task
|-
| MachineName || varchar || 256 || hostname of machine running task
|-
| PageCount || int || 4 || number of pages in batch being processed
|-
| Submitted || datetime || 8 || date/time when processing batch was submitted
|-
| Started || datetime || 8 || date/time when processing batch started
|-
| Completed || datetime || 8 || date/time when batch process completed
|-
| Priority || int || 4 || numeric value (1-5) given to sort batch priority
|-
| Scope || int || 4 || the level of the batch that a step is processing
|-
| StepNo || int || 4 || current step of batch process
|-
| BatchName || varchar || 256 || name of batch in processing
|-
| BatchProcessName || varchar || 256 || name of process associated with batch in processing
|-
| StepName || varchar || 256 || name of step of process associated with batch in processing
|}
'''dbo.ServiceInstance'''
* This table contains a list of installed services.  Note: If an installed service does not have a reference in this table, it will not show up in '''Grooper''' Config.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique row identifier
|-
| RepositoryId || uniqueidentifier || 16 ||
|-
|-
|[[Image:Compile_stats_02.png|center|200px|link=Compile Stats]]
| MachineName || varchar || 64 || hostname of pc running '''Grooper''' service
|[[Image:Microsoft_office_integration_000.png|center|200px|link=Microsoft Office Integration]]
|[[Image:document_viewer_00.png|center|150px|link=Document Viewer]]
|[[Image:Separation_and_review_18.png|center|175px|link=Separation and Separation Review]]
|-
|-
|style="text-align:center"|'''[[Compile Stats]]'''
| ServiceName || varchar || 256 || name of '''Grooper''' service running
|style="text-align:center"|'''[[Microsoft Office Integration]]'''
|style="text-align:center"|'''[[Document Viewer]]'''
|style="text-align:center"|'''[[Separation and Separation Review]]'''
|-
|-
|[[Image:data_review_00.png|center|200px|link=Data Review]]
| TypeName || varchar || 256 || typer of '''Grooper''' service running
|[[Image:Weighted_rules_00.png|center|200px|link=Confidence Multiplier]]
|[[Image:Data_element_overrides_000.png|center|150px|link=Data Element Overrides]]
|[[Image:Database_export_002.png|center|200px|link=Database Export]]
|-
|-
|style="text-align:center"|'''[[Data Review]]'''
| PropertiesJson || varchar || 2048 || json information defining /Node Properties/Properties
|style="text-align:center"|'''[[Confidence Multiplier]]'''
|style="text-align:center"|'''[[Data Element Overrides]]'''
|style="text-align:center"|'''[[Database Export]]'''
|-
|-
|[[Image:Cmis_lookup_002.png|center|200px|link=CMIS Lookup]]
| UserName || varchar || 64 || windows user running '''Grooper''' service
|[[Image:Content_type_filter_000.png|center|100px|link=Content Type Filter]]
|[[Image:Output_extractor_key_000.png|center|200px|link=Output Extractor Key]]
|[[Image:box_cmis_binding_000.png|center|200px|link=Box (CMIS Binding)]]
|-
|-
|style="text-align:center"|'''[[CMIS Lookup]]'''
| Password || varchar || 64 || password of windows user running '''Grooper''' service
|style="text-align:center"|'''[[Content Type Filter]]'''
|style="text-align:center"|'''[[Output Extractor Key]]'''
|style="text-align:center"|'''[[Box (CMIS Binding)]]'''
|-
|-
|colspan="4"|[[Image:Linq_to_grooper_objects_001.png|center|200px|link=LINQ to Grooper Objects]]
| InstanceNo || int ||4  || number of a specific service on a machine
|-
|-
|colspan="4" style="text-align:center"|'''[[LINQ to Grooper Objects]]'''
| ThreadPirority || int || 4 || numeric value given to service thread to determine processor priority
|}
'''dbo.SessionStats'''
* This table contains statistical information regarding batches in '''Grooper'''.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || int || 4 || unique row identifier
|-
| BatchId || uniqueidentifier || 16 || guid for batch node
|-
| BatchName || varchar || 256 || name of batch
|-
| BatchProcessId || uniqueidentifier || 16 || guid of process node related to the batch
|-
| BatchProcessName || varchar || 256 || name given to process by end user
|-
| StepName || varchar || 256 || name of step in the process
|-
| ProcessingScope || in || 4 || the level of the batch that a step is processing
|-
| ModuleName || varchar || 256 || name of '''Grooper''' module
|-
| ModuleDisplayName || varchar || 256 || name of '''Grooper''' module given to batch process step
|-
| MachineName || varchar || 256 || hostname of pc running batch
|-
| UserName || varchar || 256 || windows user under which the batch is running
|-
| StartTime || datetime || 8 || time batch was initiated
|-
| EndTime || datetime || 8 || time batch ended or was terminated
|-
| TasksProcessed || int || 4 || number of tasks processed in batch
|}
'''dbo.Setting'''
* This table contains only the Database version, including build number.
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Name || varchar || 64 || DatabaseVersion
|-
| Value || varchar || 1024 || Current installed '''Grooper''' Database version
|}
'''dbo.TreeNode'''
* The main '''Grooper''' table, TreeNode contains one entry for every object in the '''Grooper''' node tree.  Composed of the following columns:
:: {| class="wikitable"
! Column Name !! Data Type !! Size !! Description
|-
| Id || uniqueidentifier || 16 || The node’s unique ID.  A GUID autogenerated by '''Grooper'''.
|-
| RowId || int || 4 || Identity Column for the table.  Generated by SQL server.
|-
| RowVersion || timestamp || 8 || A timestamp column, updated when the node is changed.  Used by certain processes to determine if the node has been updated.
|-
| Name || varchar || 256 || The name of the node.
|-
| TypeName || varchar || 64 || The name of the object type of the node
|-
| ParentId || uniqueidentifier || 16 || ID of the node’s parent.  The '''Grooper''' Root node has a parent ID with all zeros.
|-
| NodeIndex || int || 4 || Index of the node in its containing object (usually a folder)
|-
| Attributes || int || 4 || A flags attribute indicating whether the node has the following attributes: read-only, fixed contents (no children may be added), sorted (children will always show up in alphabetical order), has ACL, is a system object (may not be changed).
|-
| NumChildren || int || 4 || Number of children.
|-
| Properties || varchar(MAX) || -1 || Column containing the JSON properties for this node.
|-
| NodeValues || varchar(MAX) || -1 || metadata about the node that pertains to '''Grooper''' modules
|-
| Files || varchar(MAX) || -1 || objects in the '''Grooper''' file store associated with the node
|}
|}


|
Using a SQL database as the core of '''Grooper''' allows for great efficiency. Every property of every object in '''Grooper''', as a result of being a simple entry in a table, can be loaded into memory and accessed nearly instantly.  This would not be the case otherwise (if for example '''Grooper''' repositories were defined by something like a project file), as file i/o is one of the slowest aspects of modern computing. This also allows the discrete management of objects on an individual basis to allow multiple users to work in one environment and prevent work overlap by locking objects.
[[File:American-airlines-credit-union-financial-services-document-data-capture-integration-grooper.jpg|400px|right|link=https://www.bisok.com/case-studies/electronic-data-discovery-case-study/]]
 
<blockquote style="font-size:14pt">
'''They’re Saving Over 5,000 Hours Every Year in Data Discovery and Processing'''
</blockquote>


===The File Store===
The '''[[File Store]]''' in '''Grooper''' is a file share in a Windows environment. It houses the files associated with objects in '''Grooper''' that have information that would otherwise be inefficient to store in (a cell in) a database table.


American Airlines Credit Union has transformed their data workflows, quickly saving thousands of hours in electronic data discovery , resulting in much greater efficiency and improved member services.
The '''Grooper Filestore''' exists at a user-specified location.  This may be a local or an network path, but if a filestore is given a local path, computers connecting to that repository remotely will not be able to access it. If you want to set up a repository so that other computers can connect to it, make sure you reference the filestore using a UNC path!


Discover how they:
The filestore contains three levels of directories.  A typical filestore entry will exist on disk as, e.g. 00 > 00 > 00 > [guid].grp. Each of the lowest-level folders in the filestore will have a maximum of 256 files, at which point a new folder at that level will be created. If the lowest level contains 256 folders, a new folder will be created at the level above; this gives the '''Grooper''' filestore a limit of 256 ^ 4 = 4,294,967,296 files stored on disk.  
 
* Quickly found 40,000 specific files among one billion
* Easily integrated with data silos and content management systems when no other solution would
* Have cut their mortgage processing time in half (and they process mortgages for 47 branch offices!)
* Learn from the document and electronic data discovery experts at BIS!
 
[https://www.bisok.com/case-studies/electronic-data-discovery-case-study/ You can access the full case study clicking this link].
|}
==== <span style="color:white">Feedback</span> ====
 
{|cellpadding="15" cellspacing="10" width="100%"
|-style="background-color:#36b0a7; color:white; font-size:16pt"
|colspan=2|'''Feedback'''
|-style="background-color:#d8f3f1" valign="center"
|style="width:25%"|
[[File:Gartner pi.jpg|link=https://www.gartner.com/reviews/market/data-and-analytics-others/vendor/bis/product/grooper]]
|
<span style="font-size:14pt">We value your feedback!<br><br>Help us improve our product by leaving us a review on [https://www.gartner.com/reviews/market/data-and-analytics-others/vendor/bis/product/grooper Gartner.com].<br><br>Click the image to the left to submit a review.</span>
|}
 
 
{|cellpadding="15" cellspacing="10" width="100%"
|-style="background-color:#36b0a7; color:white; font-size:16pt"
|colspan="4"|'''Other Resources'''
|-style="background-color:#d8f3f1" valign="top"
|style="width:25%"|
* [https://xchange.grooper.com Grooper x Change]
* [https://xchange.grooper.com/categories/downloads-and-resources Grooper Downloads] - Including Grooper installer files
* [https://www.bisok.com/grooper-data-integration-platform/ Grooper.com]
|style="width:25%"|
* [http://grooper.bisok.com/Documentation/2.80/Main/HTML5/index.htm#t=Start_Page.htm 2.80 Reference Documentation]
* [http://grooper.bisok.com/Documentation/2.80/SDK/HTML5/index.htm#t=Developer_Reference.htm 2.80 SDK Documentation]
* [https://grooper.bisok.com/Documentation/2.90/Main/HTML5/index.htm#t=Start_Page.htm 2.90 Reference Documentation]
|style="width:25%"|
* [https://blog.bisok.com/webinars Webinars and Video]
* [https://www.bisok.com/white-papers/ BIS White Papers]
* [https://www.bisok.com/case-studies/ Case Studies]
* [https://go.bisok.com/learning-center BIS Blog]
|style="width:25%"|
* [[ACE Training Schedule]]
* [https://go.bisok.com/first-tuesday-grooper-technical-user-group First Tuesday User Group Signup]
|}


<!--
While the filestore entries are all given .grp extensions, the contents of the file are unaltered from their “actual” form.  If you navigate, for example, to the .grp file associated with an pdf imported using full import, you can open it and view it with a pdf viewer.  The files in the filestore are intentionally obfuscated to prevent users from interacting with them outside of '''Grooper''', as they are essentially '''Grooper-internal''' objects.  
== Getting started (MediaWiki) ==
<strong>MediaWiki has been installed.</strong>


Consult the [https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents User's Guide] for information on using the wiki software.
Although the majority of files in the filestore relate to batch objects (split pages, imported documents, image processing undo objects, etc.), some files are the result of other in-'''Grooper''' processes such as layout data, OCR character data, etc.
===The Application===
This is the most visible aspect of '''Grooper''' as it is the software you interact with that displays the currently active repository.  It consists of several pieces:
# '''[[Grooper Design Studio]]'''
# '''[[Grooper Config]]'''
# '''[[Grooper Dashboard]]'''
# '''[[Grooper Attended Client]]'''
# '''[[Grooper Unattended Client]]'''
Every object in the '''Grooper''' '''[[Node Tree]]''' is an object, or '''[https://en.wikipedia.org/wiki/Row_(database) row]''', in a specific table, the dbo.TreeNode table. The '''[https://en.wikipedia.org/wiki/Graphical_user_interface GUI]''' of '''Grooper''' is essentially wrapping information from the '''Grooper''' database, and associated files from the Grooper File Store, into a series of grids and windows that allow you to directly interact with that database, and its related Windows file share, without writing SQL queries.


* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
===Licensing===
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
A '''Grooper''' license is provided to you after confirmation of purchase. This license key is entered in '''Grooper Design Studio''' either on the '''Licensing''' tab of the '''Grooper Root Node''' or on a '''License Server''' object by pressing the '''Activate Online''' button. The licensing is ultimately stored on the '''License Server''' object, so either of these methods accomplishes the same end goal. This applies the licensing to the '''Grooper''' repository that was being represented by '''Grooper Design Studio''' upon activation. This license can, subsequently, be hosted from that repository to any other '''Grooper''' repository via an instance of a '''Grooper License Server''' service pointed at the repository that has an active license.
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Localisation#Translation_resources Localise MediaWiki for your language]
* [https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Combating_spam Learn how to combat spam on your wiki]
--!>

Revision as of 12:17, 12 April 2021

Technically speaking, what is Grooper? Grooper is a repository of information made up by 13 tables in a SQL Server Database and associated files in a Windows file share. This information is displayed to the user via an application(s) that displays a 1 to 1 relationship of what exists in that database and fileshare. Gaining an understanding of how to properly create using Grooper begins by first understanding the structure of its architecture, and why it is built the way it is. This sets a foundational groundwork for knowing how to think and interact with the application.

Grooper infrastructure

System Architecture and Requirements

End User PC

  • The end user pc would include Grooper Thick Clients and installations on premisis.
  • Network latency should be (ideally) less than 50ms.
  • Network throughput should be (ideally) greater than 50MB.

Grooper Processing Server

  • A Grooper processing environment can have as many servers as is necssary to meet business requirements.
  • Windows 10 and/or Windows Server 2012 or later is required.
  • Server RAM should be 16GB or more.
  • The CPU should consist of 4 or more cores.
  • The port for licensing is 13900 by default.

Grooper SQL Server

  • SQL Server 2012 or later is required.
  • The Grooper Database should allow for a size of 20GB or more.
  • The scaling of the Grooper Database should be set to percentage, ideally, not the default of a few MB. This will reduce frequent scaling that hinders performance.
  • The SQL Port is 1433.

Grooper FileStore Share

  • This can be local to the Grooper SQL Server or managed on a separate server.
  • The ieal filesize would accomodate for roughly 150GB for small to medium installs.
  • The authentication requires CIFS or SMB.

Windows Prerequisites

The following are included in the installation package of Grooper and are required for installation (listed are x64 packages.)

  • vc_redist_2012_update4_x64
    Microsoft Visual C++ 2012 Redistributable (x64) - 11.0.61030
  • vc_redist_2010_sp1_x64
    Microsoft Visual C++ 2010 x64 Redistributable Setup
  • vc_redist_2013_x64
    Microsoft Visual C++ 2013 Redistributable (x64) - 12.0.30501
  • vc_redist.x64
    Microsoft Visual C++ 2017 Redistributable (x64) - 14.10.25017
Visual C++ Redistributable Packages install runtime components of Visual C++ Libraries on a computer that does not have Visual C++ installed. The libraries are required to run applications that are developed by using the corresponding version of Visual C++.
  • NDP462-DevPack-KB3151934-ENU
    Microsoft .NET Framework 4.6.2 Developer Pack
.NET is an integral part of many apps running on Windows and provides common functionality for those apps to run. For developers, the .NET Framework provides a comprehensive and consistent programming model for building apps that have visually stunning user experiences and seamless and secure communication.
  • SQLSysClrTypes
In LINQ to SQL, the data model of a relational database maps to an object model that is expressed in the programming language of your choice. When the application runs, LINQ to SQL translates the language-integrated queries in the object model into SQL and sends them to the database for execution. When the database returns the results, LINQ to SQL translates the results back to objects that you can work with in your own programming language.
In order to translate data between the object model and the database, a type mapping must be defined. LINQ to SQL uses a type mapping to match each common language runtime (CLR) type with a particular SQL Server type. You can define type mappings and other mapping information, such as database structure and table relationships, inside the object model with attribute-based mapping. Alternatively, you can specify the mapping information outside the object model with an external mapping file.
  • AccessDatabaseEngine_x64
Microsoft Access Database Engine contains essential components that make it possible for Microsoft Office programs to interact with software services and applications that do not belong to Microsoft. For instance, if you are working in a program that gives you the possibility to export tables directly to Microsoft Access, then you need the database engine to make that happen.
The Microsoft Access Database Engine is available in multiple editions, which mainly depend on which Microsoft Office product you’re using. The hint is in the year. As such, if you use Microsoft Office 2016, then you need Microsoft Access Database Engine 2016.
  • ReportViewer
Used for viewing built in Grooper Reports, which uses a Microsoft Reporting framework.

The Four Layers of Grooper

Grooper consists of four main components that constantly interact with one another:

  1. Database
  2. File Store
  3. Licensing
  4. Application

The Database

This is the core component of Grooper as all aspects of the functionality and configuration of a Grooper repository are stored here as metadata in tables. There are 13 tables that make up the Grooper database:

dbo.AccessControlEntry

  • This table contains the information necessary for implementing Grooper’s node-level security architecture.
Column Name Data Type Size Description
Id int 10 unique number referencing the table's row
NodeId uniqueidentifier 16 guid associated with node from Grooper node tree
PrincipalId uniqueidentifier 16
Permissions in 4

dbo.CustomStats

  • Definition incoming ...
Column Name Data Type Size Description
SessionId int 4
Name Varchar 64
Value Float 8

dbo.FileStoreEntry

  • This table contains the information necessary for associating filestore objects with Grooper nodes. Note: a node with associated files will also contain those files’ filestore object locations, names, and mimetypes in the “files” column of TreeNode.
Column Name Data Type Size Description
Id int 4 sequential number assigned to object in row
FileId uniqueidentifier 16 guid assigned as the name of the object (stored in the filestore with this number followed by .grp)
NodeId uniqueidentifier 16 guid associated with node from Grooper tree
FileStoreId uniqueidentifier 16 guid given to the node referencing the file store

dbo.License

  • This table contains information relating to Grooper licensing, including the licensing and tamper keys.
Column Name Data Type Size Description
Id int 4 unique number referencing the table’s row
SerialNumber uniqueidentifier 16 guid of serial number
LicenseText Varchar 1024
CustomerId Varchar 256 number given to customer owning the license
CustomerName varchar 256 string name given to customer owning the license
MachineId varchar 40 unique identifier of the machine which the license was generated for
LicenseCode varchar 50 shorthand for the type of license assigned
LicenseDesc varchar 256 description of the type of license assigned
Quantity int 4 amount of licenses given upon assignment
Expires int 4
ExpirationDate datetime 8 date and time the assigned license expires
ResetInterval varchar 12 basis on which the assigned licenses resets
LastReset datetime 8 date and time the license was last reset
RemainingCount int 4 amount of licenses remaining since assigned
TamperKey varchar 256

dbo.LicenseCheckout

  • This table contains information relating to checked out licenses.
Column Name Data Type Size Description
Id int 4 unique number referencing the table’s row
LicenseCode varchar 50 string defining the license code
LicenseDesc varchar 256 definition of the type of license in use
UserName varchar 256 windows user to whom the license is checked to
MachineName varchar 256 hostname of machine using the license
ModuleName varchar 256 Grooper module occupying the license
CheckoutTime datetime 8 date and time the license was checked out
CheckoutId uniqueidentifier 16

dbo.Lock

  • This table contains one row per locked node; prevents overlapping access to various resources.
Column Name Data Type Size Description
NodeId uniqueidentifier 16 guid of node from Grooper hierarchy tree
UserName varchar 64 windows user that applied the unlock
MachineName varchar 64 hostname of the machine where the unlock occurred
ModuleName varchar 64 Grooper module that executed the unlock
ProcedureName varchar 64 the procedure executed to apply the unlock
CreateDate smalldatetime 4 date and time the unlock occurred

dbo.Log

  • This table contains the Grooper log, which is represented in the Event Viewer of Grooper Design Studio.
Column Name Data Type Size Description
Id int 4 unique row identifier
TimeStamp datetime 8 date and time the event was logged
Type int 4 type of event logged
ProcName varchar(MAX) -1 name of procedure associated with the event
Message varchar(MAX) -1 message displayed in and defining, in general, the event
UserAccount varchar 64 windows user under which the event triggered
MachineName varchar 64 hostname of the machine under which the event triggered
NodeId uniqueidentifier 16 guid of parent node associated with ‘NodeName’
NodeName varchar 256 Item Name, or name of node associated with the triggering of the event
NodeType varchar 64 type of Grooper Node defining the ‘NodeName’

dbo.NodeReference

  • This table contains a list of all referenced nodes, and what other nodes they are referenced by. Used to protect referenced nodes from deletion, and in determining what nodes are necessary on exporting.
Column Name Data Type Size Description
NodeId uniqueidentifier 16 guid given to node in Grooper hierarchy tree
ReferencingNodeId uniqueidentifier 16 guid of parent node associated with ‘NodeId’

dbo.ProcessingTask

  • This table contains tasks submitted for activity processing as part of production batches.
Column Name Data Type Size Description
Id int 4 unique row identifier
BatchId uniqueidentifier 16 guid of batch related to running process
NodeId uniqueidentifier 16 guid of node related to batch in running process
Settings varchar(MAX) -1 json properties of running node
ModuleName varchar 256 module running current task
Status int 4 status of current task (i.e. running, error…)
UserName varchar 256 windows user that submitted task
MachineName varchar 256 hostname of machine running task
PageCount int 4 number of pages in batch being processed
Submitted datetime 8 date/time when processing batch was submitted
Started datetime 8 date/time when processing batch started
Completed datetime 8 date/time when batch process completed
Priority int 4 numeric value (1-5) given to sort batch priority
Scope int 4 the level of the batch that a step is processing
StepNo int 4 current step of batch process
BatchName varchar 256 name of batch in processing
BatchProcessName varchar 256 name of process associated with batch in processing
StepName varchar 256 name of step of process associated with batch in processing

dbo.ServiceInstance

  • This table contains a list of installed services. Note: If an installed service does not have a reference in this table, it will not show up in Grooper Config.
Column Name Data Type Size Description
Id int 4 unique row identifier
RepositoryId uniqueidentifier 16
MachineName varchar 64 hostname of pc running Grooper service
ServiceName varchar 256 name of Grooper service running
TypeName varchar 256 typer of Grooper service running
PropertiesJson varchar 2048 json information defining /Node Properties/Properties
UserName varchar 64 windows user running Grooper service
Password varchar 64 password of windows user running Grooper service
InstanceNo int 4 number of a specific service on a machine
ThreadPirority int 4 numeric value given to service thread to determine processor priority

dbo.SessionStats

  • This table contains statistical information regarding batches in Grooper.
Column Name Data Type Size Description
Id int 4 unique row identifier
BatchId uniqueidentifier 16 guid for batch node
BatchName varchar 256 name of batch
BatchProcessId uniqueidentifier 16 guid of process node related to the batch
BatchProcessName varchar 256 name given to process by end user
StepName varchar 256 name of step in the process
ProcessingScope in 4 the level of the batch that a step is processing
ModuleName varchar 256 name of Grooper module
ModuleDisplayName varchar 256 name of Grooper module given to batch process step
MachineName varchar 256 hostname of pc running batch
UserName varchar 256 windows user under which the batch is running
StartTime datetime 8 time batch was initiated
EndTime datetime 8 time batch ended or was terminated
TasksProcessed int 4 number of tasks processed in batch

dbo.Setting

  • This table contains only the Database version, including build number.
Column Name Data Type Size Description
Name varchar 64 DatabaseVersion
Value varchar 1024 Current installed Grooper Database version

dbo.TreeNode

  • The main Grooper table, TreeNode contains one entry for every object in the Grooper node tree. Composed of the following columns:
Column Name Data Type Size Description
Id uniqueidentifier 16 The node’s unique ID. A GUID autogenerated by Grooper.
RowId int 4 Identity Column for the table. Generated by SQL server.
RowVersion timestamp 8 A timestamp column, updated when the node is changed. Used by certain processes to determine if the node has been updated.
Name varchar 256 The name of the node.
TypeName varchar 64 The name of the object type of the node
ParentId uniqueidentifier 16 ID of the node’s parent. The Grooper Root node has a parent ID with all zeros.
NodeIndex int 4 Index of the node in its containing object (usually a folder)
Attributes int 4 A flags attribute indicating whether the node has the following attributes: read-only, fixed contents (no children may be added), sorted (children will always show up in alphabetical order), has ACL, is a system object (may not be changed).
NumChildren int 4 Number of children.
Properties varchar(MAX) -1 Column containing the JSON properties for this node.
NodeValues varchar(MAX) -1 metadata about the node that pertains to Grooper modules
Files varchar(MAX) -1 objects in the Grooper file store associated with the node

Using a SQL database as the core of Grooper allows for great efficiency. Every property of every object in Grooper, as a result of being a simple entry in a table, can be loaded into memory and accessed nearly instantly. This would not be the case otherwise (if for example Grooper repositories were defined by something like a project file), as file i/o is one of the slowest aspects of modern computing. This also allows the discrete management of objects on an individual basis to allow multiple users to work in one environment and prevent work overlap by locking objects.

The File Store

The File Store in Grooper is a file share in a Windows environment. It houses the files associated with objects in Grooper that have information that would otherwise be inefficient to store in (a cell in) a database table.

The Grooper Filestore exists at a user-specified location. This may be a local or an network path, but if a filestore is given a local path, computers connecting to that repository remotely will not be able to access it. If you want to set up a repository so that other computers can connect to it, make sure you reference the filestore using a UNC path!

The filestore contains three levels of directories. A typical filestore entry will exist on disk as, e.g. 00 > 00 > 00 > [guid].grp. Each of the lowest-level folders in the filestore will have a maximum of 256 files, at which point a new folder at that level will be created. If the lowest level contains 256 folders, a new folder will be created at the level above; this gives the Grooper filestore a limit of 256 ^ 4 = 4,294,967,296 files stored on disk.

While the filestore entries are all given .grp extensions, the contents of the file are unaltered from their “actual” form. If you navigate, for example, to the .grp file associated with an pdf imported using full import, you can open it and view it with a pdf viewer. The files in the filestore are intentionally obfuscated to prevent users from interacting with them outside of Grooper, as they are essentially Grooper-internal objects.

Although the majority of files in the filestore relate to batch objects (split pages, imported documents, image processing undo objects, etc.), some files are the result of other in-Grooper processes such as layout data, OCR character data, etc.

The Application

This is the most visible aspect of Grooper as it is the software you interact with that displays the currently active repository. It consists of several pieces:

  1. Grooper Design Studio
  2. Grooper Config
  3. Grooper Dashboard
  4. Grooper Attended Client
  5. Grooper Unattended Client

Every object in the Grooper Node Tree is an object, or row, in a specific table, the dbo.TreeNode table. The GUI of Grooper is essentially wrapping information from the Grooper database, and associated files from the Grooper File Store, into a series of grids and windows that allow you to directly interact with that database, and its related Windows file share, without writing SQL queries.

Licensing

A Grooper license is provided to you after confirmation of purchase. This license key is entered in Grooper Design Studio either on the Licensing tab of the Grooper Root Node or on a License Server object by pressing the Activate Online button. The licensing is ultimately stored on the License Server object, so either of these methods accomplishes the same end goal. This applies the licensing to the Grooper repository that was being represented by Grooper Design Studio upon activation. This license can, subsequently, be hosted from that repository to any other Grooper repository via an instance of a Grooper License Server service pointed at the repository that has an active license.

Retrieved from "https://wiki.grooper.com/index.php?title=Main_Page&oldid=6578"