2023.1:PDF Data Mapping (Behavior): Difference between revisions
Dgreenwood (talk | contribs) |
Dgreenwood (talk | contribs) No edit summary |
||
| Line 282: | Line 282: | ||
{|cellpadding=10 cellspacing=5 | {|cellpadding=10 cellspacing=5 | ||
|style="width:40%" valign=top| | |style="width:40%" valign=top| | ||
Annotations are one of the configuration options for the ''PDF Generate Behavior''. | Annotations are one of the configuration options for the ''PDF Generate Behavior''. A '''Content Type''' '''''Behavior''''' can tell an activity (specifically the '''Export''' activity, in the case of the ''PDF Generate Behavior'') how to use the '''Content Type''' to do something (how to use the '''Content Model's''' collected '''Data Fields''' to insert additional content when generating a PDF upon export, in this case). | ||
# All '''''Behaviors''''' are added to a '''Content Type''' object. | # All '''''Behaviors''''' are added to a '''Content Type''' object. | ||
| Line 672: | Line 672: | ||
<tab name="About" style="margin:20px"> | <tab name="About" style="margin:20px"> | ||
=== About === | === About === | ||
Bookmarks in PDFs aid readers when navigating through multipage documents. The ''PDF Generate Behavior'' can insert bookmarks into the generated PDF to take advantage of this functionality. This can be done in one of two ways (or both): | Bookmarks in PDFs aid readers when navigating through multipage documents. The ''PDF Generate Behavior'' can insert bookmarks into the generated PDF to take advantage of this functionality. This can be done in one of two ways (or both): | ||
| Line 773: | Line 772: | ||
|} | |} | ||
</tab> | </tab> | ||
<tab name="Add the Behavior and Configure | <tab name="Add the Behavior and Configure It for Bookmarking" style="margin:20px"> | ||
=== Add the Behavior === | === Add the Behavior and Configure It for Bookmarking=== | ||
{|cellpadding=10 cellspacing=5 | {|cellpadding=10 cellspacing=5 | ||
|style="width:40%" valign=top| | |style="width:40%" valign=top| | ||
Bookmarking is one of the configuration options for the ''PDF Generate Behavior''. A '''Content Type''' '''''Behavior''''' can tell an activity (specifically the '''Export''' activity, in the case of the ''PDF Generate Behavior'') how to use the '''Content Type''' to do something (in this case, how to use the '''Content Model's''' '''Document Types''' to insert bookmarks into the PDF upon export). | |||
# All '''''Behaviors''''' are added to a '''Content Type''' object. | # All '''''Behaviors''''' are added to a '''Content Type''' object. | ||
| Line 810: | Line 809: | ||
=== Configure PDF Generation for Metadata === | === Configure PDF Generation for Metadata === | ||
<tabs style="margin:20px"> | |||
<tab name="About" style="margin:20px"> | |||
=== About === | |||
{|cellpadding=10 cellspacing=5 | |||
|style="width:50%" valign=top| | |||
The ''PDF Generate Behavior'' has the ability to create and insert additional metadata into the generated PDF as well, using information collected during Grooper's document processing. The metadata you are able to create falls into one of three categories: | |||
# Editing the PDF's default metadata fields. This includes the following metadata fields that are standard to every PDF file: | |||
#* Title | |||
#* Author | |||
#* Subject | |||
#* Creation Date | |||
#* Modification Date | |||
#* Application (Used to establish the "creator" application which created the original file. This can be useful if the original file was created in a different application, like Microsoft Word, and converted to a PDF format with the '''PDF Generate Behavior''.) | |||
# Creating custom metadata fields | |||
#* This is done using extracted '''Data Field''' values collected during the '''Extract''' activity. | |||
# Adding "Keywords" to the PDF metadata | |||
#* This can be done using expression based or extraction based methods. | |||
{|cellpadding="10" cellspacing="5" | |||
|-style="background-color:#f89420; color:white" | |||
|style="font-size:22pt"|'''⚠''' | |||
| | |||
Notice what's not included in this list is the exported document's ''filename'' (i.e. "Im_a_file.pdf"). Filename mappings are always configured using an ''Export Behavior''. | |||
|} | |||
| | |||
[[File:Pdf-generate-howto-43.png]] | |||
|} | |||
</tab> | |||
<tab name="Prereqs - Data Extraction" style="margin:20px"> | |||
=== Prereqs - Data Extraction === | |||
</tab> | |||
<tab name="Add the Behavior and Enable Metadata" style="margin:20px"> | |||
=== Add the Behavior and Enable Metadata === | |||
{|cellpadding=10 cellspacing=5 | |||
|style="width:40%" valign=top| | |||
Metadata is one of the configuration options for the ''PDF Generate Behavior''. A '''Content Type''' '''''Behavior''''' can tell an activity (specifically the '''Export''' activity, in the case of the ''PDF Generate Behavior'') how to use the '''Content Type''' to do something (how to use the '''Content Model's''' collected '''Data Fields''' and other information to edit the generated PDF's metadata, in this case). | |||
# All '''''Behaviors''''' are added to a '''Content Type''' object. | |||
#* We will add the ''PDF Generate Behavior'' to this '''Content Model''' named "PDF Generate - UNESCO Packet". | |||
# All '''''Behaviors''''' are added using the '''''Behaviors''''' property. Select the '''''Behaviors''''' property and press the ellipsis button at the end to add the ''PDF Generate Behavior''. | |||
# This will bring up the '''''Behaviors''''' editor window. | |||
# Press the "Add" button to add a '''''Behavior'''''. | |||
# Choose "PDF Generate Behavior" from the list. | |||
| | |||
[[File:Pdf-generate-howto-06.png]] | |||
|- | |||
|valign=top| | |||
# Once added, you will see the ''PDF Generate Behavior'' added to the list on the left. Select it. | |||
# To enable the bookmarking functionality, in the right panel, select the '''''Metadata''''' property. | |||
# Change it from ''Disabled'' to ''Enabled''. | |||
|} | |||
</tab> | |||
</tabs> | |||
=== Export the Generated PDF === | |||
== Version Differences == | == Version Differences == | ||
'''''Behaviors''''' are a new functionality in '''Grooper 2021'''. Much of the ''PDF Generate Behavior'' functionality was not available in previous versions. Prior to version '''2021''', only annotation creation was possible using the '''[[Generate PDF]]''' activity. In version '''2021''', this activity has been replaced by the ''PDF Generate Behavior'', expanding its capabilities to generate bookmarks and document metadata as well. | '''''Behaviors''''' are a new functionality in '''Grooper 2021'''. Much of the ''PDF Generate Behavior'' functionality was not available in previous versions. Prior to version '''2021''', only annotation creation was possible using the '''[[Generate PDF]]''' activity. In version '''2021''', this activity has been replaced by the ''PDF Generate Behavior'', expanding its capabilities to generate bookmarks and document metadata as well. | ||
Revision as of 15:36, 4 March 2021
|
2021 |
This article is in development for the upcoming version of Grooper, Grooper 2021. PDF Generate is a new Content Type Behavior option in 2021. This information is incomplete and/or may change by the time of release. |
The PDF Generate Behavior is a Content Type Behavior designed to create an exportable PDF file with additional native PDF elements, using the classification and extraction content of a Batch Folder. This includes capabilities to export extracted data as PDF metadata, inserting bookmarks, and creating PDF annotations, such as highlighting, checkbox and signature widgets.
About
The PDF Generate Behavior (or PDF Generate for short) allows Grooper users to more fully leverage the capabilities of the PDF file type. The standard PDF Export Format in Grooper will use the page image files and their text data to create a multipage PDF file for each document folder upon Export. However, this is just the "display information" required to open and read the document. There's a lot more to what a PDF can be than just a multipage document with page images and machine readable text. PDF content can also include metadata, keywords, bookmarks, annotations, and more!
The PDF Generate Behavior creates an exportable PDF file that includes some of this additional content available to the PDF format. This is part of Grooper's evolving "Smart PDF Architecture". This is a design philosophy striving to more fully utilize the capabilities of the PDF file type and merge them with Grooper's own document processing capabilities.
The expanded PDF Generate Behavior functionality can be divided into three categories:
- Annotations
- Bookmarks
- Metadata
Annotations
|
Annotations are additional objects you can add to PDF documents. Grooper uses information from Data Elements in a Data Model collected during the Extract activity to add these annotations (also called "widgets"). These annotations can increase the readability and add components for the reader to interact with the document, such as checkboxes and signature boxes. The kinds of annotations you can add are:
Grooper uses the data instance information from extracted Data Fields to insert these annotations. For example, here we set up a Content Model with a Data Field named "Last Name". After the document's data was collected during the Extract activity, Grooper has a data instance it can associate with the "Last Name" Data Field, including its size and location coordinates on the document. We then used the Highlight Annotation to highlight the extracted last name on the document in yellow. The size of all these annotations can also be adjusted using a Padding property if the size of the extracted data instance is too small for your needs. |
Bookmarks
|
Bookmarks allow easy navigation for multipage PDF documents. When exporting a single PDF comprised of multiple child sub-documents, you can create bookmarks for each child document. This way, you can keep all the documents together in a single PDF file, easily navigating from one section of the document to another. For example, this document is an application packet for a study abroad program. Each document in the packed was separated and classified as a child document folder of one Document Type or another. The PDF Generate Behavior was used to export the packet as a single PDF and a bookmark was inserted for each sub-document and named after its Document Type. Grooper can create bookmarks from extracted Data Fields in the document as well. |
Metadata
|
Metadata refers to a PDF file's content beyond the information required to display the document (the page images and encoded text data). Prior to implementing the PDF Generate Behavior functionality, Grooper only had access to edit minimal PDF metadata, notably the file's name upon export. The PDF Generate Behavior allows Grooper to alter and store additional collected metadata as well, including Data Field values collected during the Extract activity. This means Grooper can now create a viewable document with all the extracted data associated with the document itself, independent of that data being stored elsewhere (such as a database table or content management system). This metadata can be accessed by opening a PDF in a PDF viewer application, such as Adobe Acrobat, and opening the "Document Properties" window from the File menu. |
|||
|
There are several pieces of metadata Grooper has access to.
|
|||
|
|
As a Behavior, PDF Generate is configured on a Content Type object, commonly a Content Model or a Document Type.
|
|||
|
|||
|
Along with the PDF Generate Behavior, you will also need an Export Behavior configured to export a PDF formatted file. The PDF Generate Behavior does the job of configuring all the extra content (metadata, bookmarks and/or annotations) you want to add to the exported PDF. The Export Behavior does the job of actually creating the PDF (with the content configuration information supplied by the PDF Generate Behavior) and sending it off to an external storage platform. Export Behaviors can be added to Content Types, such as the Content Model here.
|
|||
|
Once the Export Behavior is added, you will need to add an Export Definition. This will control how the file is exported, most notably where the file is exported. Whether exporting to a Windows file system, or an IMAP email mailbox, or a CMIS content management system, Grooper needs to know where to put the file. An Export Definition is how Grooper knows where the file goes. Importantly for the PDF Generate Behavior, you will also use an Export Definition to define what type(s) of file you want to export. For whichever Export Definition you choose, you will need to ensure you've configured an Export Format for a PDF formatted file in order to export the generated PDF.
We will review some specifics of the PDF Format option's configuration later. For now, just be aware adding a PDF Export Format is a necessary step to export the PDF file generated by the PDF Generate Behavior. |
How To
The following tutorials use a mock UNESCO Laura W. Bush Traveling Fellowship application to detail more specific set up for a PDF Generate Behavior. This is a packet of documents from a single applicant containing five different kinds of documents.
Application
|
This document consists of two pages. The first is a coversheet for the whole application packet. The second is the application form itself. Primarily, this document will allow us to demonstrate the different kinds of annotations available when using a PDF Generate Behavior to generate a PDF file (using its Annotations property configuration). We will see how to set up one example of each of the following annotation types available in Grooper:
Importantly for any annotation type, a Data Field must be extracted in order to place the annotation. How does Grooper know what you want to highlight? It uses the extraction result of a Data Field, which includes information about where that value is located on the page. Even if the extraction result is just a blank zone without returning any actual information, Grooper needs some kind of coordinates to know where to place the annotation. Since we're going to end up extracting some data in order to place these annotations, this will also give us the opportunity to see some of the collected data inserted as PDF metadata as well. |
Essay
|
This application also includes an essay from the student. This document will demonstrate how to add keywords to the PDF's metadata. We will use an extractor to count the number of words in the essay and configure the PDF Generate Behavior's Metadata properties to insert a keyword of "long essay", "medium essay", or "short essay" depending on the essay's length. |
Other Documents
|
This packet contains three other kinds of documents as well:
These documents (as well as the rest) will allow us to see how to insert bookmarks into the generated PDF, using the PDF Generate Behavior's Bookmarking property configuration. |
|||
|
The original document, imported as a single multipage PDF file, has been processed a bit to facilitate this.
This means we can process the full imported application packet document, and export a single file with easily navigable bookmarks for its component documents. There's no need to export individual documents for each component document and figure out a way to index them, or put them in their own folder, or any other method you may come up with to relate them to each other in their final storage location. With the PDF Generate Behavior's bookmarking capabilities, you can export just one file with each child Document Type bookmarked. |
|||
Configure PDF Generation for Annotations
About
|
The PDF Generate Behavior has the capability of inserting various annotations and native pdf widgets into the generated PDF. This increases the document's readability and adds functionality for the reader to interact with the document through widgets such as radio group buttons, checkboxes and signature fields. We will demonstrate how to configure one example for each of the Annotation Types.
|
|
|
We will also use the Textbox Widget to insert editable text boxes into the document's coversheet. These text boxes will also be populated with some corresponding information from the rest of the document.
|
Prereqs - Data Fields & Extracted Data
Before a PDF annotation can be generated, a document's data must be extracted. Put another way, the Extract activity must run before the Export activity (when the PDF Generate Behavior ultimately builds the PDF and exports it).
Each of the Annotation Types point to a Data Field in a Data Model as part of their configuration. If the Data Field does not collect data during the Extract activity, the PDF Generate Behavior won't know where to place the annotation.
|
Add the Behavior
|
Annotations are one of the configuration options for the PDF Generate Behavior. A Content Type Behavior can tell an activity (specifically the Export activity, in the case of the PDF Generate Behavior) how to use the Content Type to do something (how to use the Content Model's collected Data Fields to insert additional content when generating a PDF upon export, in this case).
|
|
We will detail collection and configuration of the various Annotation Types in the next tabs of this tutorial. |
Highlight Annotation
|
In this example, we will use the Highlight Annotation to highlight the extracted "Last Name", "First Name" and "Middle Initial" fields from the application form. |
|
|
|
|
|
|
Optionally, you can control how the highlight looks. Its color, size, opacity and whether or not there's a stroke around the highlighted rectangle.
|
Radio Group Widget
|
For example, we will create a Radio Group Widget annotation from the "US Citizen" Data Field's result. We have two choices, either "Yes" or "No". Only one or the other can be chosen. So, this is well suited for a radio button group. |
|
|
|
|
|
|
Let's briefly look at this "US Citizen" Data Field and see what's happening behind the scenes when the PDF Generate Behavior creates the radio buttons.
|
| FYI |
|
Checkbox Widget
| WIP | The Checkbox Widget documentation needs to be finalized after getting some guidance from dev. If it seems incomplete or images don't match up with text, that is why. |
|
For example, we will create a Checkbox Widget annotation for the checkboxes in the "Checklist" section of this document, the "Application", "Proposal Summary", "Essay", "Resume" and "Recommendation Letter" Data Fields. These are Boolean OMR checkboxes, returning "true" if the box next to the corresponding label is checked, and "false" if unchecked. In either case, checked or not, the Checkbox Widget will insert an editable checkbox element into the generated PDF. |
|
|
|
|
Signature Widget
|
For example, we will create a Signature Widget annotation for the signature line on the application form, using the "Signature" Data Field of our Data Model. The Checkbox Widget will insert an interactable signature element into the generated PDF. |
|
|
|
|
Just like any Annotation Type, the extraction result from the Data Field is critical for placing the signature annotation on the generated PDF. Let's look at the "Signature" Data Field's result to understand a little better how these results are used to create the signature widget.
In our case, we're using the Detect Signature extractor type to supply these results. The Detect Signature extractor is perfectly suited for the Signature Widget Annotation Type.
- It actually combines both Zonal and OMR based extraction techniques to determine if a signature is present in the zone. It sets the boundaries of where you expect to find a signature using Zonal based methods and detects if the signature is present by counting the percentage of filled pixels in the zone, which is the basis of OMR based extraction methods. You can then output different values if the zone is filled above or below a certain percentage. In this case, the extractor returns "Not Signed" because there aren't enough pixels present in the extraction zone to count as filled. If there were a signature present, there'd be more pixels present, accounting for a higher filled percentage.
This is great for our purposes because it gives us the exact information we need for the Signature Widget, which is an extraction zone. Grooper needs a data instance indicating the size and location for the generated signature widget.
- But wait there's more! We also get some bonus information about whether or not there's a signature present. Does the Signature Widget Annotation Type need to know if there's a signature present? No. It does not. It will place the widget no matter what the result is. But might that information be otherwise useful to you? Probably.
|
Textbox Widget
|
The Textbox Widget Annotation Type will insert editable text boxes into the generated PDF. One simple way to use this functionality is to use the Highlight Zone extractor type to place a blank zone where you want to place an empty text box on the PDF. However, any extractor type can be used to define the textbox's location. Furthermore, if the Data Field used to create the annotation collects a valued during the Extract activity, not only will a textbox be inserted into the generated PDF, but it will be prefilled with the Data Field's extracted value upon export. For example, we will use the Textbox Widget functionality to fill out the blank coversheet on the first page of our application packet. We will end up using a Highlight Zone extractor to define the size and location of the text box. However, we're going to go one step further and populate the Data Field's used with some information from other Data Field's in our Data Model. By the end of it, the PDF Generate Behavior will not only insert editable textboxes into the generated PDF, but fill them in with text. By the end of it, we end up with this blank coversheet automatically populated with some information collected during the Extract activity. |
|
|
|||
|
|||
|
The Textbox Widget annotation has some additional configuration options as well.
|
|||
|
As far as looking behind the scenes, there's at least two things going on with how we've set up these Data Fields' extraction, ultimately supplying the result used to insert the Textbox Widget annotation. First, we used the Highlight Zone extractor type to draw the textbox, defining the size and location of the annotation upon generating the PDF.
Second, we used an expression to return a value, using the results of other Data Fields in our Data Model.
|
|||
|
Configure PDF Generation for Bookmarks
About
Bookmarks in PDFs aid readers when navigating through multipage documents. The PDF Generate Behavior can insert bookmarks into the generated PDF to take advantage of this functionality. This can be done in one of two ways (or both):
- Using a Batch Folder's child document folders.
- Using the document's extracted Data Fields.
|
We will focus on the bookmarking method (as it is more common). Often it is the case you will import a file into Grooper that has multiple documents inside you want to separate and classify, but otherwise all belong together in one way or another. Such is the case with our study abroad application packet. The application packet as a whole consists of five separate and distinguishable documents.
|
|
|
Our goal is to create a bookmark in the generated PDF file for each of these component documents (or child documents as we will come to call them). Rather than exporting five separate PDF files for each component document, we will export a single PDF for the whole packet with navigable bookmarks corresponding to each component document.
|
Prereqs - Split Pages, Separation and Classification
|
In order to accomplish this goal, we're going to have to do some things to this application packet before we configure the PDF Generate Behavior. By the end of it, we're looking for a Batch whose documents have a structure like this. The documents in this batch consist of two Batch Folder levels.
This is what we want to end up with. How did we get there? Long story short, we have some document separation and classification requirements before we can insert bookmarks in the generated PDF. The bookmarks are inserted for each child document folder and named after their classified Document Type's name. In order to do that, we need to split out the pages of the imported document, separate them into child document folders, and classify them first. |
|||
|
The full application document came into Grooper like this. A 7 page PDF file with each of these 5 component documents was imported into a new Batch. This is now the parent document folder at Folder Level 1. But there's documents in them there document! How do we get them out? |
|||
|
First, we need to use the Split Pages activity to create child Batch Page objects. This will split out the pages of the imported PDF file, creating one child Batch Batch for each page in PDF on the parent document folder. Now we have page objects we can manipulate in our Batch. |
|||
|
Now that we have Batch Page objects in our Batch, we can use the Separate activity to insert the second folder level. This is the first step in organizing these pages into child documents. We need to distinguish between one collection of pages as a document and another collection of pages as a document. Creating a folders is the first part of that equation. Now, we have child document folders for this parent document folder, but they are just blank folders. There is nothing to distinguish one folder from the next.
|
|||
|
And, that's the second part of the organization equation, classification. Next, these folders will be assigned a Document Type from our Content Model using the Classify activity.
Now, we have everything we need to configure the bookmarking functionality of the PDF Generate Behavior. Bookmarks will be created every time a new child document is encountered and named after the Document Type assigned to that folder. When the full PDF is generated, a bookmark named "Application" will be inserted at the first page of the PDF. That child document is two pages long. The third page of the full PDF will be the proposal summary. So a bookmark named "Proposal Summery" will be inserted at page three. A "Resume" bookmark will be inserted at page four. And so on. |
| FYI |
There are many ways to separate and classify documents, including ESP Auto Separation which both separates and classifies documents time with a single activity (just Separate). But this is the general idea to get us where we need to go. One way or another, create classified child document folders from a parent document folder. That way when we generate the PDF for the parent document folder upon export, bookmarks will be created for the classified child document folders. |
Add the Behavior and Configure It for Bookmarking
|
Bookmarking is one of the configuration options for the PDF Generate Behavior. A Content Type Behavior can tell an activity (specifically the Export activity, in the case of the PDF Generate Behavior) how to use the Content Type to do something (in this case, how to use the Content Model's Document Types to insert bookmarks into the PDF upon export).
|
|
|
|
|
For our purposes, this is all we need to configure at this point. However, be aware of the Bookmarking configuration options.
|
Configure PDF Generation for Metadata
About
|
The PDF Generate Behavior has the ability to create and insert additional metadata into the generated PDF as well, using information collected during Grooper's document processing. The metadata you are able to create falls into one of three categories:
|
Prereqs - Data Extraction
Add the Behavior and Enable Metadata
|
Metadata is one of the configuration options for the PDF Generate Behavior. A Content Type Behavior can tell an activity (specifically the Export activity, in the case of the PDF Generate Behavior) how to use the Content Type to do something (how to use the Content Model's collected Data Fields and other information to edit the generated PDF's metadata, in this case).
|
|
|
Export the Generated PDF
Version Differences
Behaviors are a new functionality in Grooper 2021. Much of the PDF Generate Behavior functionality was not available in previous versions. Prior to version 2021, only annotation creation was possible using the Generate PDF activity. In version 2021, this activity has been replaced by the PDF Generate Behavior, expanding its capabilities to generate bookmarks and document metadata as well.























































