Dgreenwood: Dgreenwood moved page 2024:Import Mode and Document Linking (Concept) to 2024:Import Mode (Property)

2025-05-27T19:10:49Z

Dgreenwood moved page 2024:Import Mode and Document Linking (Concept) to 2024:Import Mode (Property)

← Older revision	Revision as of 14:10, 27 May 2025
(No difference)

Dgreenwood: Dgreenwood moved page Import Mode and Document Linking (Concept) to 2024:Import Mode and Document Linking (Concept)

2025-03-05T14:45:49Z

Dgreenwood moved page Import Mode and Document Linking (Concept) to 2024:Import Mode and Document Linking (Concept)

← Older revision	Revision as of 09:45, 5 March 2025
(No difference)

Randallkinard: // via Wikitext Extension for VSCode

2024-12-11T15:38:47Z

// via Wikitext Extension for VSCode

← Older revision		Revision as of 10:38, 11 December 2024
Line 1:		Line 1:
	{{~~Migrated~~}}		{{AutoVersion}}
	{{~~2023~~:~~{{PAGENAME}~~}}}
			<blockquote>{{#lst:Glossary\|Import Mode and Document Linking}}</blockquote>

			{\|class="fyi-box"
			\|
			'''FYI'''
			\|
			"Sparse Import" redirects here. How you configure an '''''Import Provider's''''' '''''Import Mode''''' determines if documents are imported sparsely.
			* Set '''''Import Mode''''' to ''Sparse'' to perform a sparse import.
			\|}

			== About ==
			=== A Side Note on Importing in General ===
			Forget about '''''Import Modes''''' for a second. How do you even import documents into Grooper ''at all''? You import documents into a Grooper environment using an [[Import Provider]].

			The simplest way to import in Grooper is to "Submit a new import job" from the "Imports" page.

			The '''Import Providers''' highlighted in turquoise below are considered "Legacy" '''Import Providers''' and should only be used for older configurations that have been kept through upgrading, or in very specific circumstances where '''CMIS Connections''' do not provide the desired connectivity. The properties for connecting to systems using these '''Import Providers''' are set on the "Import Job" rather than on a specific object in '''Grooper''', therefore their settings are not re-usable.

			It is considered best practice to use the '''Import Providers''' highlighted in red below. These leverage '''CMIS Repository''' objects for their connection configuraiton, which give them the most functionality and are the most developed means of importing in '''Grooper'''.
			Of the two shown, ''Import Query Results'' should be your first choice as it is even more fully featured than the ''Import Descendants'' option. However, ''Import Query Results'' can only be leveraged by "query-able", indexed content systems. ''Import Descendants'' should only be used in cases where the content system connected to by your '''CMIS Connection''' is not "query-able".

			There are a myriad of articles related to "CMIS" here on the Grooper Wiki, but the ones most related to this topic would be:
			* [[CMIS Connection]]
			* [[CMIS Import]]


			# Start by going to the Imports page.
			# Click the "Add" button (indicated by a + icon) in the top right of the screen to submit an Import job.
			# The "Submit Import Job" window will open.
			# A description of the Import job must be entered into the Description property.
			# Click the drop-down for the Provider property and select an Import Provider.
			#* The following are considered "legacy" Import Providers and should only be used in very specific circumstances:
			#** File System Import
			#** FTP Import
			#** Mail Import
			#** OPEX Import
			#** SFTP Import
			#** Test Batch
			#* The following are considered "best practice" Import Providers as they leverage CMIS Repository objects which provide the most functionality and support:
			#** Import Descendants
			#** Import Query Results

			[[File:2023_Import-Mode-and-Document-Linking_01_About_01.png]]

			==== Ad-Hoc Import Jobs vs Automated Import Jobs ====
			Starting an "Import Job" from the "Imports" page is considered ad-hoc, so an "Import Job" that only happens once. While you can manually repeat a "Import Job" that you have not cleared from the list of completed "Import Jobs" from the "Imports Page", it is necessarily not an automated procedure.

			If you want an "Import Job" to repeat indefinitely based on some kind of schedule, you would need to use an '''Import Watcher''' service established in '''Grooper Command Console'''. The configuration of an "Import Job" on an '''Import Watcher''' service is identical to what is pictured above in an ad-hoc import, but the '''Import Watcher''' service itself has built in configuration for scheduling that allows for the automation of starting "Import Jobs".

			=== What is an Import Mode? ===

			When importing documents (i.e. files in an external storage platform), they contain two important sets of information:

			*Content - The file itself, such as a PDF.
			*Properties - Metadata associated with the file originating from the source storage platform. This can be as basic as the file's name or something more custom like fields in a Box.com metadata template.

			There are three '''''Import Modes''''' in Grooper:

			# ''Full'' - This mode fully imports each file as a '''Batch Folder''' in the '''Batch'''. Both their content and their properties are loaded into the Grooper Filestore upon import.
			#* Because the files are fully copied from the source into a Grooper environment, this is the slowest of the three '''''Import Modes'''''.
			#* Import speed can be further impacted by network traffic required to copy the files associated with each document from their original source to the Grooper Filestore.
			# ''Sparse'' - The ''Sparse'' '''''Import Mode''''' loads a file's properties as it does in ''Full'' mode. However, instead of fully importing the document's content, a link between Grooper and their content at the import source is created.
			#* When Grooper needs to access the document's content, it travels the link attached to the '''Batch Folder''' to retrieve the attached file from the import source.
			#* Because import operations must run single threaded in grooper, when importing large document sets, this can greatly reduce the time it takes to import documents.
			#* If needed, the content can also be loaded into the Grooper Filestore in parallel using the '''[[Execute (Activity)\|Execute]]''' activity.
			# ''LinkOnly'' - This mode only creates an appropriate object in '''Grooper''' and only links to both the content of the document and its properties.
			#* This is by far the fastest of the three '''Import Modes'''. Only a '''Batch Folder''' and a link to the source document are created for each imported file.
			#* Like ''Sparse'' imports, the content of ''as well as'' the properties of the document can be loaded in parallel using the '''Execute''' activity.
			#** Please note, these properties must map to Grooper '''Data Fields''' in a '''Data Model'''. As such, the documents must be classified first before their properties can be loaded.

			=== How Does Sparse Import Save Time? ===
			Consider what is occurring when importing a document into '''Grooper''' using the ''Full'' '''''Import Mode''''':
			# An object is made in '''Grooper''' representing a document (a '''Batch Folder''' in a '''Batch''').
			#* This takes little to no time to accomplish as all that is occurring is a row is being added in SQL to the TreeNode table of the '''Grooper Database''' representing the document object.
			# The properties of the document are loaded onto the created document object.
			#* This is only slightly more time consuming than creating an object in '''Grooper''' as ''some'' data is being copied through the network.
			# The content associated with the document is copied from its source location to the '''Grooper Filestore''' and associated with the created object.
			#* This is by far the most time consuming portion of this process as the content of a document can vary wildly in storage size. If a document is fully electronic in nature it will be ''relatively'' small, but this of course depends on the size of the document itself. However, documents that contain images to represent their pages can range from small to gargantuan depending on the number of pages of the document, the color depth of the images, the resolution, the number of pages, etc.

			A ''Sparse'' import only does the first two tasks listed above and simply links to the document's contents where they exist in their system of origin. This vastly speeds up the import process.
			The main thing to consider about a ''Sparse'' import is that while the document is being processed in '''Grooper''', the original document '''''should not''''' be moved from its original source or the link between the document object in '''Grooper''' and the original file will be broken. Upon completion of processing in '''Grooper''', however, the document could later be moved.
			However, at the time of import, using the ''Sparse'' '''''Import Mode''''', you can simultaneous import and move the document via configuration of the '''''Import Provider'''''. The object made in '''Grooper''' will point to the new location of the document while still only existing as a link.

Randallkinard at 16:07, 28 August 2024

2024-08-28T16:07:31Z

← Older revision		Revision as of 11:07, 28 August 2024
Line 1:		Line 1:
	{{~~Migrated2023~~}}		{{Migrated}}
	{{2023:{{PAGENAME}}}}		{{2023:{{PAGENAME}}}}

Randallkinard: Randallkinard moved page Import Mode and Document Linking (Property) to Import Mode and Document Linking (Concept) without leaving a redirect: new naming convention

2024-04-17T19:16:05Z

Randallkinard moved page Import Mode and Document Linking (Property) to Import Mode and Document Linking (Concept) without leaving a redirect: new naming convention

← Older revision	Revision as of 14:16, 17 April 2024
(No difference)

Randallkinard: Randallkinard moved page Import Mode and Document Linking to Import Mode and Document Linking (Property): new naming convention

2024-04-17T19:12:20Z

Randallkinard moved page Import Mode and Document Linking to Import Mode and Document Linking (Property): new naming convention

← Older revision	Revision as of 14:12, 17 April 2024
(No difference)

Dgreenwood: Removed redirect to Import Mode and Document Linking - 2023

2023-12-29T23:03:41Z

Removed redirect to Import Mode and Document Linking - 2023

← Older revision		Revision as of 18:03, 29 December 2023
Line 1:		Line 1:
	~~#REDIRECT [[Import Mode and Document Linking -~~ 2023]]		{{Migrated2023}}
			{{2023:{{PAGENAME}}}}

Randallkinard: Changed redirect target from Import Mode and Document Linking - 2.80 to Import Mode and Document Linking - 2023

2023-11-09T22:35:00Z

Changed redirect target from Import Mode and Document Linking - 2.80 to Import Mode and Document Linking - 2023

← Older revision		Revision as of 17:35, 9 November 2023
Line 1:		Line 1:
	#REDIRECT [[Import Mode and Document Linking - ~~2.80~~]]		#REDIRECT [[Import Mode and Document Linking - 2023]]

Randallkinard: Randallkinard moved page Import Mode and Document Linking to Import Mode and Document Linking - 2.80

2023-11-09T20:56:53Z

Randallkinard moved page Import Mode and Document Linking to Import Mode and Document Linking - 2.80

New page

#REDIRECT [[Import Mode and Document Linking - 2.80]]

2024:Import Mode (Property) - Revision history