Export (Activity) - 2021

From Grooper Wiki
(Redirected from Export)
Jump to navigation Jump to search

The Export activity exports processed document content to an external storage platform.

Export is an Unattended Activity, typically added as of of the last steps (if not the last step) of a Batch Process. It allows Grooper users to deliver processed Batch content to an external system. Whether exporting Batch Folders as PDF files to a Windows folder, exporting extracted Data Model fields to a SQL database, exporting to a content management system, or some combination of multiple exports to multiple systems, the Export activity handles how document Batch Folders in a Batch ultimately leave Grooper after they have been classified and had their data extracted.

How documents are exported (what gets exported, where they go, and what format the exported content takes) is all controlled by Export Behaviors. This is a set of properties configured to control how Batch Folder content is exported based on its Document Type classification. Export Behaviors can be configured locally, configured as part of the Export activity's property configuration, or can be configured for a particular Content Type, by configuring the Behaviors property of a Content Model and/or its descendant Content Categories or Document Types.

About

Asset 22@4x.png

You may download and import the file below into your own Grooper environment (version 2021). This contains a Batch with the example document(s), a Content Model, and a Batch Process discussed in this article

So you've ingested some documents into a Batch. You've obtained their full text data with the Recognize activity, either through OCR or extracting their native embedded text. You've classified these documents, assigning the Batch Folders a Document Type from a Content Model during the Classify activity. You've collected the data you want from these documents during the Extract activity. Now what?

You need to get these documents and that data out of Grooper!

Enter the Export activity. Grooper is designed to be a document processing platform. It is a powerful tool to model document sets and their data (according to a Content Model) and put unprocessed pages or files through a step by step list of processing instructions (according to a Batch Process) to ultimately organize them and collect information from them. However, Grooper is not designed to be a content management system or a storage platform. Once your documents are organized and Grooper has extracted the data you want from them, you generally want to put those files and data in an external endpoint, such as a file system, a database, a content management system or some combination thereof.

The Export activity's job is to get document content out of Grooper, according to your specifications. Using one or more Export Behavior configurations, you can control how processed document content is exported, how its indexed in which storage location, what data goes where, what file format certain content should take, and more.

FYI How you export documents in Grooper underwent some serious changes in version 2021. In previous versions, there were two separate export activities: Document Export and Database Export.

To simplify things, we combined these two Activities into the singular Export activity. Whether you're exporting document files or data to a database, you use the Export activity and Export Behavior configurations in either case.

Just What Is "Document Content"?

We're going to talk a lot about "document content" throughout this article. Ultimately, the Export activity controls what content is exported and how it is exported. So, what do we mean by "document content"?

In terms of its content, you can break up a document processed by Grooper into (at least) three meaningful components:

  1. The document's image
  2. The document's full text
  3. The document's extracted data

Each of these different kinds of content is another layer that comprises a whole document (represented as a Batch Folder in a Batch, its child Batch Pages and/or files attached to the Batch Folder). Grooper's job is to take source material (scanned pages or imported files), derive the content you desire (such as extracting Data Elements from a Data Model), and using the Export activity recombine this content into derivable files or data to one or more storage endpoints.

Image Content

Export-about-1.png

The document's image is simply what the viewer physically sees when viewing the document. Whether scanned pages or a digital file, like a PDF, this content comprises the pixels on the screen you're looking at when reading a document. This content can be altered in a Batch Process by the Image Processing activity, which is a typical part of processing scanned documents to clean up the image before OCR. Upon Export, Grooper can build a new file from these images, or just export whatever image content was originally imported.

Full Text Content

Export-about-2.png

A good deal of document processing automation requires machine readable text to parse words, phrases and other text data. Grooper obtains a document's full text data through the Recognize activity, OCRing images or extracting embedded digital text. These results can then be embedded into the exported file as another part of its content during Export.

Extracted Data Content

Export-about-3.png

Last but not least, the Extract activity in a Batch Process will collect information from the document, according to its classified Document Type and Data Model. This may be simple indexing data, even just the Document Type assigned during the Classify activity. This may be every meaningful data point on the document, obtained from a Data Model with hundreds of extracted Data Elements. Regardless, this needs to be stored somewhere and somehow, such as in a SQL database, content management system, or as a separate data file, like an XML or CSV file.

How you merge this content into new files, define what storage platform it goes to, and how extracted data can drive indexing considerations is all controlled by the Export activity's Export Behavior configuration.

Export Behaviors

The Export activity exports documents according to an Export Behavior. This is a set of export property configurations based on the Content Type (i.e. Document Type of a Content Model) assigned to a document Batch Folder during document classification. Once a Batch Folder is assigned a Document Type, you have something you can point to that controls the flow of traffic out of Grooper.

For documents "A", build a PDF file and put them in folder "A" in a file system, for example. For documents "B", put them in folder "B" and export their data to a database while you're at it. For document "C", you might do something entirely different. Or, you might perform essentially the same export for all Document Types in a Content Model. Export Behavior configurations are how you tell Grooper what to do for one Document Type or another upon export.

Export Behaviors can be configured for any Content Type object. This includes a parent Content Model or any of its descendant Document Types or Content Categories.

Export-export-behaviors-content-type-1.png

Export-export-behaviors-content-type-2.png

Export-export-behaviors-content-type-3.png

This allows you to use the Content Model's hierarchy to determine how you want to export documents of a certain Document Type.

  • If you want to perform the same, generic export for all Document Types in a Content Model, you can configure a single Export Behavior solely for the Content Model applying to all its child Document Types.
  • If a group of Document Types under a single Content Category all should be exported in the same manner, you can configure an Export Behavior for the Content Category. Those settings will apply to any of its child Document Types.
  • If every Document Type or certain Document Types have their own specific export configuration, you can configure individual Export Behaviors for one or more Document Types (or all of them!).

Export Behaviors can be configured in one of two ways:

  1. Using the Behaviors property of a Content Type object
    • A Content Model
    • A Content Category
    • Or, a Document Type
  2. As part of the Export activity's property configuration

In either case, export settings are added as one or more Export Definitions of the Export Behavior. Once a document is classified and it is assigned a Document Type its Export Behavior's configured Export Definition(s) will define how the document content is exported. The main difference is how you get to the Export Behavior property.

Content Type Export Behaviors

An Export Behavior configuration can be added to any Content Type object (i.e. Content Models, Content Categories, and Document Types) using its Behaviors property. Doing so will control how a Document Type "behaves" upon export.

  1. For example, here we have a Content Model selected in the Node Tree.
  2. To add an Export Behavior, first select the Behaviors property.
  3. Then, press the ellipsis button at the end of the property.

Export-export-behaviors-1.png

  1. This will bring up the Behaviors collection editor window.
  2. Press the "Add" button.
  3. Select Export Behavior.
    • You can only configure one Export Behavior per Content Type object.
    • Children Content Type objects will inherit export settings from their parent Content Type's Export Behavior configuration.
    • However, multiple Export Behaviors may be added by configuring the Behaviors property of multiple Content Types. For example, if every Document Type needed a unique Export Behavior configuration, you could configure the Behaviors property for each one, adding one Export Behavior to the Behaviors list for each one.

Export-export-behaviors-2.png

  1. You will see the Export Behavior added to the Behaviors list.
  2. Selecting it, you can now add one or more Export Definitions with the Export Definitions property.


FYI When configured using the Behaviors property of a Content Type object, the Export activity will export Batch Folder content in a Batch according to the Export Definition settings configured for the Batch Folder's assigned Document Type
  • Or its parent Content Category or parent Content Model depending on which Content Type's Behavior property is configured in the Content Model's hierarchy.
  • Export-export-behaviors-3.png

    Export Activity Export Behaviors

    Export Behaviors can also be configured as part of the Export activity's configuration. These are called "local" Export Behaviors. They are local to the Export activity in the Batch Process.

    1. For example, here we have a working Batch Process selected in the Node Tree.
    2. And we have the Export step of the Batch Process selected.
    3. To add an Export Behavior, select the Export Behaviors property.
    4. Then, press the ellipsis button at the end of the property.

    Export-export-behaviors-4.png

    1. This will bring up the Export Behaviors collection editor window.
    2. Press the "Add" button to add a new Export Behavior
    3. An Export Behavior will be added to the list.
    4. With the Export Behavior selected you must define which Content Type the behavior applies to using the Content Type property.
      • Note in both cases, a Content Type is involved in configuring Export Behaviors. Whether local to the Export activity or as part of a Content Model's configuration, Grooper needs to know what to do upon export, given a certain Content Type (and its children Content Types if scoped to a Content Model or Content Category). Once Grooper knows what kind of document it's looking at, we can then inform it what to do in terms of exporting its document content.
    5. Using the dropdown menu, select which Content Type scope should utilize the Export Behavior by selecting either a top-level parent Content Model or one of its child Content Categories or Document Types.
      • Keep in mind you can only select a single Content Type here. You can only configure one Export Behavior per Content Type object.
      • Children Content Type objects will inherit export settings from their parent Content Type's Export Behavior configuration.
    6. However, multiple Export Behaviors may be added locally to the Export activity. For example, if every Document Type needed a unique Export Behavior configuration, you could add one Export Behavior to the list for each one.

    Export-export-behaviors-5.png

    1. Once a Content Type is selected, you can add one more more Export Definitions with the Export Definitions property.

    Export-export-behaviors-6.png

    Export Definitions

    Regardless of whether the Export Behavior is set up directly on the Content Type object or with the Export activity's local property grid, how document content is exported is defined using one or more Export Definitions.

    Export Definitions functionally determine three things:

    1. Location - Where the document content ends up upon export. In other words, the storage platform you're exporting to.
    2. Content - What document content is exported: image content, full text content, and/or extracted data content.
    3. Format - What format the exported content takes, such as a PDF file or XML data file.

    Your primary consideration is Location. Where do you want these files and/or data to end up? Are you exporting files to a Windows file system? Are you exporting data to a database? Are you exporting content to a content management system, like Box.com?

    When configuring an Export Definition the first thing you will add is an Export Type. This determines what export endpoint you're using to export document content. The Export activity will deliver document content to the storage platform determined by the Export Type.

    1. To add an Export Type, press the "Add" button in the Export Definitions collection editor.
    2. This can be one of the following options:
      • CMIS Export - To export content using a CMIS Connection
      • Data Export - To export data to a SQL database or ODBC compliant database
      • File Export - To export files to a Windows file system
      • FTP Export - To export files to an FTP server
      • IMAP Export - To export files to an IMAP email server
      • SFTP Export - To export files to an SFTP server

    Export-export-definitions-1.png

    Export Types

    Each Export Type defines connection to the endpoint storage location slightly differently.

    CMIS Export

    For CMIS Export, document content is exported over a CMIS Connection.

    1. The CMIS Repository property here establishes connection to a storage platform connected to Grooper via the CMIS Connection.
      • The CMIS Connection defines the connection settings for one of several storage platforms available as CMIS Bindings. CMIS Repositories are storage locations imported into Grooper as children of the CMIS Connection object.
      • Depending on the CMIS Binding the CMIS Repository will represent a different storage location. This could be a Windows file system folder for the NTFS binding. This could be a SharePoint site for the SharePoint binding. This could be an email inbox for the Exchange or IMAP bindings.

    For more information, please visit the CMIS Repository and CMIS Export articles.

    Export-export-definitions-2.png

    Data Export

    For Data Export, extracted data content is exported to a SQL database or ODBC compliant database, using a Data Connection object.

    1. The Connection property connects Grooper to the database connected to Grooper via the referenced Data Connection object.
      • The Data Conneciton object stores all the connection settings required for Grooper to transmit data to the database, including the database server's name and any required logon settings.
    2. The Table Mappings property defines what Data Model elements should be exported to which database table (or tables) in the database.


    FYI In previous versions of Grooper, collected document data were exported to a database using the Database Export activity. Files, on the other hand, were exported using the Document Export activity.

    In version 2021, the two export activities were combined into a single activity, the Export activity. You will now add and configure a Data Export Export Type to export data to a SQL or ODBC compliant database.

    Export-export-definitions-7.png

    File Export

    For File Export, document content is exported to a Windows file system folder.

    1. The Target Folder property defines what folder you want to export content to. It is always best practice to use a fully qualified UNC path, to disambiguate file and folder locations on one networked machine from another.


    The File Export Export Type is a carry over from older methods of exporting to a Windows file system in previous versions of Grooper. File Export exists mostly for backwards compatibility, but it can still be utilized for simple file system exports.

    In current versions, using the NTFS CMIS Binding and the CMIS Export Export Type is a preferable method to export document content to a Windows file system.

    Export-export-definitions-3.png

    IMAP Export

    For IMAP Export document content is exported to email servers using the IMAP protocol.

    1. The Mail Server property defines the host name (or IP address) of the email server you want to export to.
      • For example, the server used to connect to an Outlook 365 inbox is "outlook.office365.com"
    2. The User Name and Password properties define the logon information to connect to the mailbox.
    3. The Target Folder property defines what email folder you want to export content to.


    The IMAP Export Export Type is a carry over from older methods of exporting across the IMAP protocol in previous versions of Grooper. IMAP Export exists mostly for backwards compatibility, but it can still be utilized for simple exports to email boxes.

    In current versions, using the IMAP CMIS Binding and the CMIS Export Export Type is a preferable method to export document content to an IMAP server.

    Furthermore, when connecting to a Microsoft Outlook inbox, the Exchange CMIS Binding is preferable to the IMAP binding. The Exchange binding has increased functionality specifically designed for the Outlook messaging system.

    Export-export-definitions-4.png

    FTP Export

    For FTP Export, document content is exported to an FTP site using the FTP protocol.

    1. The FTP Server URL property defines what site you want to export to.
    2. The User Name and Password properties define the logon information to connect to the FTP site.


    The FTP Export Export Type is a carry over from older methods of exporting to FTP sites in pervious versions of Grooper. FTP Export exists mostly for backwards compatibility, but it can still be utilized for simple exports to FTP folders.

    In current versions, using the FTP CMIS Binding and the CMIS Export Export Type is a preferable method to export document content to an FTP site.

    Export-export-definitions-5.png

    SFTP Export

    For SFTP Export, document content is exported to an SFTP site using the SFTP protocol.

    1. The Host Name property defines what site you want to export to.
    2. The User Name and Password properties define the logon information to connect to the SFTP site.


    The SFTP Export Export Type is a carry over from older methods of exporting to SFTP sites in previous versions of Grooper. SFTP Export exists mostly for backwards compatibility, but it can still be utilized for simple exports to SFTP folders.

    In current versions, using the SFTP CMIS Binding and the CMIS Export Export Type is a preferable method to export document content to an SFTP site.

    Export-export-definitions-6.png

    When choosing an Export Type, you should be asking yourself "Do I want to export files, data, or both?". What content you want to export will inform which location (or locations) you export to. Your answer to this question will impact which Export Type you choose and how you configure it to export document Batch Folder content.

    Data Only

    If you're purely exporting document data content (values collected from the Extract activity) and nothing else, you're likely looking to export data to a database.

    • Use Data Export to do this.

    Files Only

    If you're looking to export files, such as PDFs, TIFs, and text files, you have more options depending on the storage location you want your document files to wind up in.

    Use any of the following Export Formats, depending on where you want to export.

    • CMIS Export
    • File Export
    • FTP Export
    • SFTP Export
    • IMAP Export

    All of these Export Types have a configurable Export Format property, which will allow you to build an export file of a given format out of Batch Folder content.

    • For example, the PDF Format can be configured to build a PDF file from the Batch Folder's image content (using the images of its child Batch Pages or its attached PDF) and the full text data content obtained from OCR.

    Both Data and Files

    When exporting document content, there are a variety of ways to export both data and files.

    1. Commonly, if you want to export both data and files, you will simply add multiple Export Type definitions.
      • A Data Export to export data to a database and one of the other Export Types to export files to a storage repository of your choice.
    2. You can also export data as a file itself, such as an XML, JSON, or text data file.
      • These are Export Format options available when configuring CMIS Export, File Export, FTP Export, SFTP Export, or IMAP Export.
      • We will detail all Export Formats below.
    3. If you're using CMIS Export and connecting to a content management system that supports custom metadata fields, you may be able to export both files and data using CMIS Export only.
      • If you can do this will be highly dependent on the capabilities of the storage platform and the CMIS Connection type.
      • For more information, please visit the CMIS Export article.
    4. Grooper's PDF Data Mapping Behavior more fully leverages the capabilities of the PDF format when exporting PDF files. With this functionality, you can store classification and extraction data via bookmarking and PDF metadata.
      • This allows you to store most, if not all, document content in the PDF file itself.
      • For more information, please visit the PDF Data Mapping article.

    Export Formats

    When exporting content to an export location you must determine what format that content takes. There's all different types of files out there. Some of them are better suited to house different types of content than others. XML files are great for storing data, but not so much for image content. TIF files are great for image content, but not so much for full text data.

    Export Formats can be configured for any of the Export Types that export files:

    • CMIS Export
    • File Export
    • FTP Export
    • SFTP Export
    • IMAP Export
    1. When adding an Export Format, look for the Export Formats property of these Export Types.
    2. Press the ellipsis button at the end of the property.

    Export-export-definitions-8.png

    1. This will bring up the Export Formats collection editor.
    2. To add an Export Format, press the "Add" button.
    3. The, select the file format you wish to export from the list.

    Export-export-definitions-9.png

    Below we will briefly describe each Export Format to give you a better idea of what content you can export with each format.

    PDF Format

    1. The PDF Format will output a PDF file from the Batch Folder content, either from an imported PDF file attached to the Batch Folder upon import, or from the child Batch Pages of the Batch Folder.
    2. As well as the page image content, this format can also embed the full text OCR data obtained via the Recognize activity, adding text-behind to the image-based PDF pages.
      • Under the Build Options turn Searchable to True.
    3. Another important property to note is the Always Build property. This property will force the PDF file to be generated, even if there is a native PDF file attached to the Batch Folder.
      • Enabling this property is important when using the Split Pages activity to process individual pages as Batch Pages. Turning Always Build to True will stitch together a new PDF file from the processed pages, rather than re-exporting whatever originally imported PDF is attached to the Batch Folder.
      • Enabling this property is also important when using the PDF Data Mapping Behavior to ensure the data mapped file is built and exported, rather than a PDF file that was imported originally.

    Export-export-formats-01.png

    This will export a PDF file, according to the PDF Format's property grid settings.

    1. You can see here, we have text-behind generated from the Searchable property. We can select this text with our cursor.

    Export-export-formats-02.png

    XML Format

    1. The XML Metadata format will output extracted Data Model values to an XML file.

    Export-export-formats-03.png

    1. Document level information (including the Batch Folder's classified Document Type) are found in the <Document> tag.
    2. Data Field values and information are found in the <Field> tags.
    3. Extracted table values are found in <TableCell> tags as children of <TableRow> and <Table> parent tags.
    FYI XML data can be reformatted using XSLT style sheets using the XML Transform activity.

    Export-export-formats-04.png

    JSON Metadata

    1. The JSON Metadata format will output extracted Data Model values to a JSON file.

    Export-export-formats-05.png

    1. This will export the "Grooper.DocumentData.json" file generated by the Export activity for each Batch Folder.

    Export-export-formats-06.png

    Simple Metadata

    1. The Simple Metadata format will output extracted Data Field values to a text file.
    2. This file formats Data Fields and their values as simple "key-value pairs", where the Data Field's name (the key) delineates the extracted value, according to the character entered for the Delimiter property.

    Export-export-formats-08.png

    As you can see here, Data Fields are exported to a text file as a simple list of key-value pairs.

    1. Data Field names are on the left.
    2. The Delimiter character (an equals sign by default) is in the middle.
    3. Extracted values are on the right.

    Export-export-formats-09.png

    Delimited Metadata

    1. The Delimited Metadata format also outputs extracted Data Field values to a text file.
    2. This formats Data Field values as a delimiter-separated value array, delimited by the character entered for the Delimiter property (a comma by default).
    3. You must enter a value for the Delimiter Escape property.
      • This property will replace a delimiter character occurring in a Data Field's extracted value.
      • For example, we've entered a semicolon character. So, a comma (our Delimiter character) would be swapped for a semicolon.
    4. Using a header row is useful for this kind of format. The header row is populated with the Data Field's names. Turn Include Header to True to include a header row in the text file.

    Export-export-formats-11.png

    1. Extracted 'Data Field values are exported as a comma-delimited list of values in a text file.
    2. Note since we enabled Include Header, we have a header row of our Data Field' names output as well.

    Export-export-formats-12.png

    TIF Format

    1. The TIF Format will output image content only as a TIF (Tagged Image Format) file.

    Export-export-formats-14.png

    TIF (or TIFF) is a format used to store high quality raster graphics for graphic design or publishing. However, keep in mind this is an image only format. If you want text-behind embedded in your files, you must use the PDF Format.

    Export-export-formats-15.png

    Text Format

    1. The Text Format is will output full text content only, generated from OCR data, as a text file.

    Export-export-formats-16.png

    Upon export, this will generate a text file from the Batch Folder's raw OCR text data, generated from the Recognize activity.

    Export-export-formats-17.png

    Attachment

    For document files imported from a digital source, the Attachment format will output the Batch Folder's attachment file.

    Export-export-formats-18.png

    1. This option can also output any file attached to a Batch Folder by referencing a filename with the Filename property.
      • This is how Grooper exports custom generated files from activities such as XML Transform or custom scripted activities.

    Export-export-formats-19.png

    FYI If the Batch Folder has no attachment, this option will generate an image version of the document from all child Batch Pages in the folder.

    Export-export-formats-20.png

    Shared Behavior Modes

    Remember, Export Behaviors can be configured in one of two ways:

    1. Using the Behaviors property of a Content Type object
    2. As part of the Export activity's property configuration

    In general, most users will choose to do one or the other. This may just be as simple as what your preference is. Personally, I prefer to set up Export Behaviors on the Content Model and/or its child Document Types and Content Categories. You may prefer to configure one or more Export Behaviors local to the Export activity's property panel. Either way, the Export Behavior (or set of Export Behaviors) will export documents as you configure it.

    However, what happens if you have both? For example, Export Behaviors configured on one or more Content Type objects as well as local to the Export activity.

    Grooper needs to understand which one should take priority preference, or if both should execute in one way or another. This can accommodate more complex exports, but there are different ways you can define how 'Export Behaviors are shared between Content Types and local Export activity configurations.

    This is what the Shared Behavior Mode property of the Export activity is for. It defines how "local" and "shared" Export Behaviors are executed when the Export activity exports Batch Folders in a Batch.

    Local Behaviors

    Shared Behaviors

    "Local behaviors" are Export Behaviors configured in an Export activity's local property grid.

    "Shared behaviors" are Export Behaviors configured for a Content Type object (Content Models, Content Categories, and Document Types), using its Behaviors property.

    Export-shared-behaviors-01.png

    Export-shared-behaviors-02.png

    The Shared Behavior Mode property is configured in the Export activity's property grid.

    It can be set to one of the following values:

    • None - Only the Export step's locally configured Export Behaviors will execute.
      • NO shared Export Behaviors configured on Content Types will execute.
    • LocalOrShared - The Export activity will execute all local Export Behaviors first.
      • Shared Export Behaviors only execute if none are specified for that Content Type in the local Export Behaviors.
    • SharedOrLocal - The Export activity will execute all shared Export Behaviors first.
      • Local Export Behaviors only execute if none are specified for a Content Type's shared Export Behaviors.
    • LocalAndShared - Both sets of Export Behaviors will execute, but local Export Behaviors are executed first.
    • SharedAndLocal - Both sets of Export Behaviors will execute, but shared Export Behaviors are executed first.

    Export-shared-behaviors-03.png

    Imagine you have both "shared" and "local" Export Behaviors for two Document Types: "Orange" and "Red"

    Shared Behaviors (Configured with the "Orange" and "Red" Document Type's set of Behaviors properties)

    • The "Orange" Document Type's Export Behavior will export a PDF file to a folder named "Folder A".
    • The "Red" Document Type's Export Behavior will export a PDF file to a folder named "Folder A".

    Export-shared-behaviors-04.png

    Local Behaviors (Configured with the Export activity's local property grid)

    • The "Orange" Document Type's Export Behavior will export an XML file to a folder named "Folder B"
    • There is no Export Behavior configured for the "Red" Document Type.

    Export-shared-behaviors-07.png

    Depending on how you configure the Shared Behavior Mode property, you're going to end up with different results for your export.

    With the Export activity left unconfigured, no local Export Behaviors are applied. Only the Document Types' behaviors will execute.

    • The "Orange" documents would export as a PDF file and be placed in "Folder A".
    • The "Red" documents would export as a PDF file and be placed in "Folder B".

    Export-shared-behaviors-05.png

    Export-shared-behaviors-08.png

    With local Export Behaviors and the Shared Behavior Mode set to None, only the local Export Behaviors will execute.

    • The "Orange" documents would export as an XML file and be placed in "Folder B".
    • No files would export for the "Red" documents. There is no local Export Behavior for this Document Type.

    Export-shared-behaviors-06.png

    Export-shared-behaviors-09.png

    With the Shared Behavior Mode set to SharedOrLocal, shared behaviors will execute first. Local behaviors will only execute if no shared behavior is present.

    Only our shared Export Behaviors execute in this instance. Grooper doesn't even bother to look at the local Export Behavior for the "Orange" and "Red" Document Types because shared Export Behaviors are present for these Document Types.

    • The "Orange" documents would export as a PDF file and be placed in "Folder A" (and not an XML file in "Folder B".
    • The "Red" documents would export as a PDF file and be placed in "Folder A".

    Export-shared-behaviors-10.png

    Export-shared-behaviors-08.png

    With the Shared Behavior Mode set to LocalOrShared, local behaviors will execute first. Shared behaviors will only execute if no local behavior is present.

    In our case, no local Export Behavior is present for "Red" documents, but there is a shared Export Behavior configured on the "Red" Document Type.

    • So, the "Orange" documents export an XML file to "Folder B", using the local behavior.
    • However, the "Red" documents export a PDF file to "Folder A", using the shared behavior since no local behavior was found.

    Export-shared-behaviors-11.png

    Export-shared-behaviors-12.png

    For the And modes (LocalAndShared and SharedAndLocal) both local and shared Export Behaviors execute. The only difference is which one executes first.

    • For LocalAndShared local behaviors execute first.
    • For SharedAndLocal shared behaviors execute first.


    In our case, we end up with the same result either way.

    • The "Orange" documents export a PDF file to "Folder A", using the shared behavior, and they export an XML file to "Folder B", using the local behavior.
    • The "Red" documents export a PDF file to "Folder A", using the local behavior.

    Export-shared-behaviors-13.png

    Export-shared-behaviors-14.png

    Export-shared-behaviors-15.png

    Now, you should be asking yourself "If the result was the same for both LocalAndShared and SharedAndLocal, why even have two different options?"

    That's not always going to be the case. And this is where things can get tricky with Shared Behavior Modes.

    Issues can occur if you are exporting the same file type, with the same name, to the same folder with both shared and local Export Behaviors. If both Export Behaviors are configured to export a PDF Format file, for example, but with different File Format configurations, you could end up with a situation where one behavior overwrites the other's export. This may be what you want to do. It may not. Just be aware since both Export Behaviors execute, either the local or shared behavior can potentially overwrite whichever one exported a file first.

    Imagine our shared and local behavior configurations were a little bit different.

    Shared Behaviors

    • The "Orange" Document Type's Export Behavior will export a PDF file to a folder named "Folder A".
    • The "Red" Document Type's Export Behavior will export a PDF file to a folder named "Folder A".

    Export-shared-behaviors-04.png

    Local Behaviors

    • The "Orange" Document Type's Export Behavior will export an XML file to a folder named "Folder B"
    • The "Red" Document Type's Export Behavior will also export a PDF file to "Folder A".
      • However, it has a different Export Format configuration, indicated by "v2" in the diagram.

    Export-shared-behaviors-16.png

    With these configurations and Export's Shared Behavior Mode set to SharedAndLocal, we would end up overwriting a file. First, the shared behaviors would execute, then the local behaviors would execute. For the "Red" documents, the shared behavior would export its version of a PDF, then the local behavior would export its version of a PDF. If the two files, both the same PDF file type, share the same name, the default configurations will overwrite existing files in a folder location.

    • So, the "Orange" documents export a PDF file to "Folder A", using the shared behavior, and they export an XML file to "Folder B", using the local behavior.
      • This is the same result as demonstrated previously.
    • The difference is for the "Red" documents.
      • The shared behaviors would execute first, exporting a PDF file to "Folder A". Then, the local behaviors would execute, exporting the local Export Behavior's configuration of the PDF (version "2"), overwriting the first export, leaving you with the local behavior's PDF format.

    Export-shared-behaviors-14.png

    Export-shared-behaviors-17.png

    Be aware of this possibility when configuring SharedAndLocal or LocalAndShared exports. If the file names, types and export folder locations are the same, you may end up overwriting a file. If this is your intention, great! If not, you will need to ensure the file names for the files generated by shared and local behaviors are unique to avoid one file being overwritten.

    Thread Pool Guidance

    When automating Export steps in a Batch Process, you may need to execute the activity single threaded.

    Unattended Activities in a Batch Process can be automated using an Activity Processing Grooper service. The Activity Processing service will act like a Windows service and automatically start tasks in a Batch, as processing threads in your system's resources become available. This is one of the ways Grooper leverages your system resources for parallel processing.

    Imagine you're running Grooper on a machine with eight (8) processing threads. If you have a Batch with five (5) Batch Folders, and each one is on the Recognize step of the Batch Process, there's no need for your system to process each Batch Folder sequentially (with each Batch Folder waiting to be processed until the one before it is finished).

    • You have 8 threads and 5 Batch Folders in this scenario.
    • Each one of those threads can process one Batch Folder as a single task.
    • With 8 available threads, all 5 Batch Folders could be processed concurrently by 5 individual threads.
    • This is multi-threaded Activity processing.

    However, depending on which external storage system you're exporting to, you may run into errors if you attempt to run the Export activity multi-threaded. Particularly when it comes to cloud-based systems, like SharePoint online or Box.com, their file transfer protocol expects users to upload files one at a time. If you have 5 threads all attempting to upload 5 different Batch Folders from the same machine, 4 of those Batch Folders are going to kick back to Grooper in an error state.

    Instead, you must run the activity single-threaded, ensuring only one Batch Folder is processed at a time. As well as automating Batch Processing activities, Activity Processing services allow you to control thread resources by assigning activities a Thread Pool and limiting the number of maximum threads available for that Thread Pool.

    Next, we will show you how to create a single threaded Thread Pool for an Export activity, and set up an Activity Processing service that utilizes it. This will effectively throttle your export, so Batch Folders are indeed only exported one at a time, avoiding any issues with external platforms that cannot handle multi-threaded exports.

    How To: Assign a Thread Pool to an Activity Processing Service to Run Export Single-Threaded

    Add a Thread Pool

    The first thing you'll need to do is add a Thread Pool object. A Thread Pool defines the "bucket" of threads available to one step or another in a Batch Process. In our case, this will allow us to limit the number of threads the Export step uses to a single thread.

    To add a Thread Pool:

    1. Expand the Infrastructure folder in the Node Tree.
    2. Right-click the Thread Pools folder.
    3. Select "Add" then "Thread Pool..."
    4. This will bring up a new window to name the Thread Pool. Enter a name and press "OK".
      • We named ours "Export Throttle"

    Export-thread-pools-01.png

    1. This will add a new Thread Pool object to the Node Tree.
    2. FYI: No further object configuration is technically required at this point.
      • However, if you want the safest implementation of a single-threaded Thread Pool, totally ensuring only a single Export task is processed per repository environment, you can change the Concurrency Mode property from Multiple to Single. With the Single mode, only a single task will run per Grooper repository.

    Export-thread-pools-02.png

    Assign the Thread Pool

    Next, we need to tell our Batch Process which step should use our new Thread Pool.

    1. By default, all Batch Process steps use the "Default" Thread Pool.
    2. In the Batch Step property grid, Thread Pools are assigned with the Thread Pool property.

    Export-thread-pools-03.png

    We want to tell the Export step of this Batch Process to use a different Thread Pool, the new one we just created.

    1. Select the Export step in the Batch Process.
    2. Select the Thread Pool property.
    3. Using the dropdown menu, select the Thread Pool you wish to use.
      • In our case, the "Export Throttle" Thread Pool.

    Export-thread-pools-04.png

    Configure an Activity Processing Service

    On to Grooper Config! Grooper services are installed and edited using Grooper Config. Open Grooper Config to install a new Activity Processing service.

    Grooper Config must be run as an administrator to install and edit services.
    1. To install a new service, press the "Edit Services..." button in Grooper Config.


    FYI If you have multiple Grooper Repository connections, as is the case for this environment, you will need to ensure one thing first.

    Make sure the Default Repository selected is the Grooper Repository with the newly added Thread Pool.

    Export-thread-pools-05.png

    1. This will bring up the "Service Manager" window.
    2. Press the "Install..." button.

    Export-thread-pools-06.png

    1. In the "Install New Service" window, select Activity Processing.
    2. Press the "OK" button.

    Export-thread-pools-07.png

    1. This will bring up a window to edit the service.
    2. Select the Thread Pool property.
    3. Using the dropdown menu, select the Thread Pool you wish to use for the Activity Processing service.

    Export-thread-pools-08.png

    1. The Number Of Threads property allows you to define how many of your system's processing threads should be utilized by the Activity Processing service.
      • In our case, we're using this Activity Processing Service to throttle our Export activity, forcing it to run single-threaded. So, we will keep this set to 1.
    2. You must also provide user credentials to run the service. This must be a user who has the following access rights:
      • Have the "Logon as Service" permission on the machine which the Grooper service is running.
      • Have access to the Grooper Repository's database.
      • Have access to the Grooper Repository's file store location.
    3. Press the "Execute" button when done.

    Export-thread-pools-09.png

    1. This will add a Grooper Activity Processing service instance to the services list.
    2. Newly installed services, must be started. Notice its Status is listed as Stopped.
    3. Select the service, and press the "Start" button to start it.

    Export-thread-pools-10.png

    1. The Status will change to Running
    2. The icon next to the service name will also change from red to green.
    3. Your service is now installed and running. Press the "Close" button to exit the Service Manager window.

    Export-thread-pools-11.png

    1. Upon processing a Batch the Export step in our Batch Process will be executed by the Activity Processing Service.
      • This automates the step's execution using the "Export Throttle" Thread Pool and limits the number of threads processing Batch objects (Batch Folders in this case) to a single thread, preventing export errors to systems that cannot handle multi-threaded exports.
    FYI You can have multiple Activity Processing services running, each with their own Thread Pool.

    You can automate the rest of the Unattended Activities in a Batch Process by added a second Activity Processing service, assigning it the "Default" Thread Pool, and assign the number of threads you want to utilize with the Number of Threads property.

    Export-thread-pools-12.png