2023:CMISQL Query

This article is about an older version of Grooper. Information may be out of date and UI elements may have changed.
2025	2023	2021

A CMISQL Query (aka CMIS Query) is Grooper's way of searching for documents in CMIS Repositories. Commonly, CMISQL Queries are used by Import Query Results to import documents from a CMIS Repository. CMISQL Queries are also used by CMIS Lookup to lookup data from a CMIS Repository. CMISQL Queries are based on a subset of the SQL-92 syntax for querying databases, with some specialized extensions added to support querying CMIS sources.

• CMISQL Queries are configured using the "CMIS Query" property found in "Import Query Results" and "CMIS Lookup".

The querying language CMISQL Queries use is based on a subset of the SQL-92 grammar. Where SQL is a querying language to search and select data in a database, "CMISQL" is a querying language to search and select documents (and their metadata properties) in a storage location, represented by a CMIS Repository in Grooper.

Because the property used to configure a CMISQL Query is always called "CMIS Query", we may refer to CMISQL Queries as CMIS Queries as well.

About

CMIS Queries use clauses to retrieve content from a CMIS Repositories based on certain metadata property values, much like a SQL query uses clauses to retrieve data from a database based on column values. In both cases, a SELECT clause is used to select content based on certain conditions determined by a WHERE clause

In general the CMISQL statement takes the following form:

SELECT * FROM <Content Type> WHERE <Conditions> ORDER BY <Sort Criteria>

For example, if you wanted to retrieve all email messages in your Inbox folder using an "Exchange" CMIS Repository from a particular sender and order results by the date you received the email (oldest first), you could use the following query:

SELECT * FROM Message WHERE (Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox')) ORDER BY DateTimeReceived ASC

Next, let's break down this query further and look at what each clause is doing.

Clause Information

SELECT

SELECT * FROM Message WHERE (Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox')) ORDER BY DateTimeReceived ASC

The SELECT clause defines a set of property values to be returned with each query result. "Property values" refers to the available metadata properties for the object type selected for the From statement. For example, when querying email messages using in an Exchange CMIS Repository, there are metadata properties for the sender, the subject line, the date the email was sent, and more. All these metadata properties are part and parcel to the email message file itself.

From a practical standpoint, your CMIS Query is ultimately returning documents. Files in a digital storage location. You aren't retrieving individual metadata values, rather the whole file with all its various metadata included. To that end you will use the * character to select all property values, when crafting a CMISQL query.

You will always start your CMIS Query with SELECT * when using a CMIS Query to import documents.

FYI

When would you use something besides SELECT *? When would you construct a CMISQL query, such as SELECT Property_A, Property_B, Property_C and so on? The answer is CMIS Lookups. A CMIS Lookup operates much like a database lookup. Instead of querying a database table's columns for data, a CMIS Query queries the metadata properties of documents in a storage location. Technically, a CMIS Query is used in that case as well. However, CMIS Queries are much more commonly used to retrieve documents from a storage location upon import, which is the focus of this article. For more information on CMIS Lookups please visit the CMIS Lookup article.

FROM

SELECT * FROM Message WHERE (Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox')) ORDER BY DateTimeReceived ASC

The FROM clause determines what type of content you're searching for, such as a digital file or a folder and their metadata properties.

• The type of content you select determines what metadata properties you have access to when building the conditional logic in your query's WHERE clause.

Depending on the CMIS Repository, you will have different options to select from. Some have simple "Document" and "Folder" content types, representing documents and folders respectively. Some have a multitude of content types, each with their own set of properties. Furthermore, some of these properties are queryable and some are not.

All of this is determined by the CMIS Repostiory's CMIS Binding (aka the CMIS Connection's connection type).

• Example: The Exchange binding's "Message" content type has property values relating to email messages, such as Subject, Sender and DateTimeSent.
• Example: The SharePoint binding's content types can be Document Libraries, giving you access to custom metadata properties you've created on the SharePoint site by adding custom columns to the Document Library.

In our example, we wanted to query email messages in an Exchange repository. So, we selected the "Message" type by adding FROM Message to our query.

WHERE

SELECT * FROM Message WHERE (Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox')) ORDER BY DateTimeReceived ASC

The WHERE clause defines what search conditions must be met for a document to be retrieved. Each condition is defined by a "predicate". You can use any metadata available to the content type (as determined by the FROM clause) in your query conditions.

• Multiple conditions/predicates can be joined with the AND or OR operators.
• You can manipulate the logical order of operations by using nested parenthesis.
• You can logically negate predicates using the NOT operator.

Where Predicates

"Where Predicate" is the base class in Grooper for search conditions that can be applied to the WHERE clause of a CMISQL Query. Each predicate represents a logical condition that must be satisfied for an item to be included in the query results.

Predicate	Description	Example
Comparison Predicate	Specifies search criteria for an individual property using a comparison operator. Such as "equals to" = or "less than" < or "greater than" >. The LIKE and IS operators are also available for certain properties. Note: Not every property type may be able to utilize every operator. Example: The Exchange binding's "Subject" property cannot use the = operator. The LIKE predicate must be used instead.	DateTimeCreated < '12/31/2007'
In Predicate	Specifies search criteria for an individual property, using a list of allowed values.	FileExtension IN ('.pdf', '.docx', '.xlsx')
Contains Predicate	Specifies a full-text query. You can use AND, OR and NOT operators when crafting search parameters. Note: Only platforms that support full text searching are compatible with with the Contains Predicate.	CONTAINS('mortgage AND payment AND NOT vehicle')
Scope Predicate	Restricts the search scope to the direct children (IN_FOLDER) or all descendants (IN_TREE) of a specific folder. Note: The NOT operator cannot be used with the Scope Predicate.	IN_FOLDER('/Documents/Grooper')
Predicate List	Defines a collection of predicates to be evaluated as a group using a logical operator (AND or OR). Allows multiple search conditions in a single CMISQL query.	(Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox/'))

• Note: Not every CMIS Binding type may be able to utilize every predicate equally.
  • Example: The SharePoint binding does not support the Scope Predicate's IN_FOLDER option. IN_TREE must be used instead.
  • Example: The NTFS binding will not support the CONTAINS() predicate (for full text searching) unless the Windows file system has been indexed by the Windows Search service.

In our example, we used a Predicate List that collected two predicates: a Comparison Predicate and a Scope Predicate. This queried email messages from certain sender (Sender LIKE '%user@example.com%') and in our inbox folder (IN_FOLDER('/Inbox/')) Since we wanted to filter content by multiple criteria, the two conditions were joined with the AND operator.

ORDER BY

SELECT * FROM Message WHERE (Sender LIKE '%user@example.com%' AND IN_FOLDER('/Inbox')) ORDER BY DateTimeReceived ASC

The ORDER BY clause is an optional clause which allows you to specify the order in which results are returned.  You can sort by multiple properties using a comma separated list.  Each property name may be followed by ASC or DESC to indicate ascending or descending sort direction.

• The default sort direction is ascending. However, it is still considered best practice to include the ASC tag to properties you wish to sort in ascending order.

In our example, we wanted to sort our incoming messages by oldest received first.

CMISQL Query Editor

If you are not familiar with the CMISQL syntax (or even SQL), you may find it easier to use the CMIS Query Editor, instead of writing the full query yourself. This allows you to configure the CMISQL query using a series of property grids and menu selections instead of writing out the query from scratch.

The CMIS Query Editor is accessible in two locations:

1. When configuring the Import Query Results Import Provider's "CMIS Query" property.
2. Using a CMIS Repository's "Search Repository" tab.

⚠

Only certain external storage platforms are currently queryable with a CMIS Query. The following CMIS Binding sources cannot be queried currently. FTP SFTP NTFS (If the folder path is not indexed by the Windows Search service and/or Windows Search is not running on the storage server) As such, these bindings are not suitable for the Import Query Results provider (You should instead use Import Descendants instead). Furthermore, you will not have access to the "Search Repository" tab for these bindings when selecting a CMIS Repository for these types. Again, the exception to this is NTFS. If the folder path IS indexed in Windows and the Windows Search service IS running, then you CAN use the "Search Repository" tab and CMIS Queries.

How to Access the CMIS Query Editor

Option 1: Import Query Results - CMIS QueryOption 2: The Search Repository Tab

Option 1: Import Query Results > CMIS Query

Most commonly, you will use a CMISQL query to import documents using the Import Query Results provider.

FYI	The Import Query Results provider is one of two Import Providers that use a CMIS Connection to bring document content into Grooper. If you need more information on Import Query Results, please visit the CMIS Import article.

When configuring the import settings, you will first select which CMIS Repository you want to query by selecting one using the Repository property. Then, select the CMIS Query property. Note: Some CMIS Bindings are not CMISQL queryable (For example, the FTP and SFTP CMIS Bindings). If the CMIS Repository is not queryable, you will not see this CMIS Query property. Press the ellipsis button at the end of the property to bring up the CMIS Query Editor.

This is the CMIS Query Editor. Using this interface you can craft a CMISQL query in one of two ways: You can use this property grid to configure the various clauses of the query statement. You can free-type the query in this text editor. We will demonstrate how to use this query editor in the next section

Option 2: The Search Repository Tab

You can also use the CMIS Query Editor when searching for documents in a CMIS Repository using the "Search Repository" tab.

Select the CMIS Repository you want to query in the Node Tree. Navigate to the "Search Repository" tab. Note: Some CMIS Bindings are not CMISQL queryable (For example, the FTP and SFTP CMIS Bindings). If the CMIS Repository is not queryable, you will not see this "Search Repository" tab. You can use this property grid to configure the various clauses of the query statement. You can free-type the query in this text editor.

We will demonstrate how to use this query editor in the next section

How to Use the CMIS Query Editor

Next, we're going to illustrate how to use the CMIS Query Editor in order to filter email messages based on certain criteria. The CMIS Query Editor's interface allows you to configure the CMISQL query based on available metadata to the CMIS Binding.

We will use the Exchange binding, which has a selection of queryable metadata for email messages, such as the email's subject and date the message was received. Our query will import email messages based on the following conditions:

1. Only messages are to be imported (as opposed to other Exchange content, such as appointments or tasks).
2. Only messages in a certain folder will be imported.
3. Only messages containing specific text search terms will be imported.
4. Only messages from a certain sender will be imported.
5. Only messages that have not been read yet will be imported.

This will get us through the basics of building a CMISQL query using the CMIS Query Editor.

The SELECT and FROM ClausesThe WHERE ClauseAdding Search PredicatesThe ORDER BY Clause (and Testing the Query)

The SELECT and FROM Clauses The property grid is divided into the various clauses that form the CMISQL query. Adjusting these properties will adjust the CMISQL query. The Select property will adjust the SELECT clause. By default this property is set to *. This forms the start of the query SELECT * The From property will adjust the FROM clause. Each CMIS Binding will default to a different content type. Generally, the default content type corresponds to a "document" in the storage location. The Exchange binding's default is Item. However, we actually want to change this. An "item" is too generic. It refers to message files, appointment files, task files, and note files in an Exchange inbox. We want to import email messages, and define our query conditions to filter out emails using metadata specifically accessible to those types of files. We need to use the Message content type to do this properly.

Changing Content Types in the FROM Clause We will use the CMIS Query Editor's property panel to edit our query going forward. Since we want to adjust the content type in our FROM clause, we will select the From property. Using the dropdown menu, we've selected Message. This will query specifically message files over our Exchange connection, and give use access to their metadata for our querying conditions. This automatically updates the full text of our CMISQL query. SELECT * FROM Item changed to SELECT * FROM Message As we use the property grid to further configure our query, the query's text will update automatically. This gives you a method of editing the CMISQL query without free-typing the query. However, you certainly can type out the full query text if you prefer not to use the property grid to enter the CMISQL query. You can also go back and forth between the property grid and the text editor.

The WHERE Clause Next, we will use the Where property to configure the various filter conditions we've set for ourselves. Select the Where property. Using the dropdown list, select the search predicate you wish to use. If you are filtering based of multiple criteria. you will ALWAYS choose Predicate Collection. In our example, we need to filter based on the values of multiple metadata properties and search a specific folder. We need to use both a Comparison Predicate and a Scope Predicate.

Using Predicate Collection for Multiple Search Conditions Expand the Where sub-properties to add your query criteria. The default Logical Operator is AND. This is appropriate for our situation, as we want to use the AND operator. ALL conditions must be met in our case. But be aware you also have access to the OR operator. Using this operator will look for any number of the defined predicates. So if at least one of the predicates is present in the document, it will be returned as a result. Next, we will add our various query conditions using the Expressions property. Press the ellipsis button at the end to bring up the Where Predicate Expressions collection editor.

Adding Search Predicates Now that we're in the Where Predicate Expressions collection editor, we can start adding and configuring our search conditions. Press the Add button to add a predicate to the list. We will start by adding a Scope Predicate to restrict the search scope to a specific folder.
The Scope Predicate The Scope Predicate allows you to choose a folder from which you want to retrieve documents. Select the Search Scope property and press the ellipsis button at the end. This will bring up a folder browser, representing your connected CMIS Repository's hierarchical folder structure. We're going to restrict our scope to this subfolder named "Wiki" in the connected email account. Press OK to finalize your selection. FYI If Include Subfolders is False, the IN_FOLDER() predicate will be used. • This will restrict the query's search to the single selected folder in the folder hierarchy. If Include Subfolders is True, the IN_TREE() predicate will be used. • This will restrict the query's search to the selected folder and search any of its subfolders in the folder hierarchy.
The Contains Predicate If the storage location your CMIS Repository is connected to supports full text searching, you can use the Contains Predicate to query documents based on specific text search terms. For example, we want to retrieve email messages pertaining to "Wiki Vitals", a specific kind of report generated nightly about the Grooper wiki and sent to my email. To add a Contains Predicate: Press the Add button and select Contains Predicate This adds a new Contains Predicate to our list of predicates Using the Search String property, enter the text you would like to search for. In our case, only messages with the text Wiki Vitals somewhere in the subject line or email body or otherwise in the message's metadata will be retrieved. Notice Grooper automatically generates the property syntax for the query, CONTAINS('Wiki AND Vitals'). ⚠ Please note only external platforms that support full text searching are compatible with with the Contains Predicate.
The NOT Operator All predicates can be logically negated using the NOT operator. For example, most of these "Wiki Vitals" reports are daily reports. These are the emails I want to process. However, once a month, there is a monthly report. I DO NOT want to process those documents. The only way I know it's a monthly report is the term "monthly report" is listed in the body of the email. I can use the Contains Predicate' and the NOT operator to exempt these monthly reports from the query. To logically negate a predicate: Add the predicate using the Add button. We've added another Contains Predicate. Configure the predicate's search parameters. What do we not want to retrieve in this case? Messages with the text Monthly Report somewhere in their body. So, we've configured the Search String property to search for that term. Change the Not property to True. This will negate the predicate's logic. If the email contains the search term, it will not be retrieved.
The Comparison Predicate The Comparison Predicate specifies search criteria based on a document's metadata property values, using comparison operators such as >, < or =. For example, we will use a Comparison Predicate to only import emails that have not been read yet, whose "IsRead" property value equals "False". To add a Comparison Predicate: Press the Add button and select Comparison Predicate This adds a new Comparison Predicate to our list of predicates Using the Property Name property, select a property from a list of the documents' available metadata properties. What metadata properties are available will be determined by the CMIS Repository's binding/connection type and the content type selected in the FROM clause. We've selected the IsRead property. Using the Comparison Operator select which operator you want to use for comparison. Depending on the value's type (string, decimal, Boolean, etc.) different comparison operators may be available. Here, we are checking for equivalency using the = operator. Using the Search Value property, enter the control variable you want to use for comparison. In our case, we want to check if these emails have not been read, indicated by the email message's "IsRead" property being False.
The LIKE Operator The LIKE operator is used to match substrings of string values. In many cases, it must be used instead of the = when you're comparing string variables. For example, if we want to use a Comparison Predicate to filter messages based on the "Sender" (a string value), we will need to use the LIKE variable to do so. Furthermore, we will also need to encase the search term in the % wildcard symbol. Here, we've added a new Comparison Predicate to only retrieve messages sent by a specific email address. We've selected Sender for the Property Name This property must use the LIKE Comparison Operator. For the Search Value property we've entered %cdearner@bisok.com%. Effectively, we just enter the email address we want to search for (cdearner@bisok.com) with percent sign wildcards at the front and end.
FYI The LIKE operator specifies that a property value must contain a substring. It takes the general form PropertyName LIKE 'Expression', where PropertyName is the query name of a string property, and Expression is a string literal using the % and _ characters as wildcards. The LIKE operator is not case sensitive. • The % wildcard substitutes for zero or more characters. • The _ wildcard substitutes for exactly one character. • A LIKE expression with no wildcards is equivalent to using the = operator. • For example, the expression 'grooper%' will match cases where the property value begins with "grooper", while '%grooper' will match cases where the property value ends with "grooper". The expression 'grooper%review' will match cases where the property value begins with "grooper" and ends with "review". Some CMIS Bindings have limited support for LIKE expressions, and will raise errors when unsupported features are used. Examples of such issues are as follows: • Some bindings only support the % wildcard at the beginning and end of the expression (i.e. '%grooper%'). • Some bindings do not support the "ends with" form of LIKE (i.e. '%grooper'). • Some bindings do not support the _ wildcard at all.
Finish Adding Predicates When you have finished adding predicates to the collection list, press the OK button.
When finished, the search predicates will form the conditional parameters of the query's WHERE clause.

We could be done at this point. We have the three basic parts of a CMIS Query.

1. The query starts with SELECT *
2. The FROM clause determines which content (documents/folders and their metadata properties) we're querying from the CMIS Repository/storage location.
3. The WHERE clause defines the query's search conditions.

Optionally, you may want to order the documents Grooper retrieves. We will do that next, with the ORDER BY clause.

The ORDER BY Clause

The ORDER BY clause is an optional CMISQL querying clause used to change the sort order of the documents Grooper retrieves. You can sort by any metadata property available.

For example, we may want to sort by the date the email message was sent. And specifically we want the most recent email first in the list and the last email last in the list. We're going to need an ORDER BY clause to do this.

To add an ORDER BY clause: Select the Order By property, and press the ellipsis button at the end.
This brings up the Order By collection editor. Press Add to add a new sort element. Using the Property Name' property, select which document metadata property you want to sort by. In our case, we've chosen the email messages DateTimeSent property, storing what date and time the email was sent. Using the Sort Direction property, select which direction you want to sort by. Choose ASC to sort in ascending order. Choose DESC to sort in descending order.
Testing the Query With this ORDER BY clause added, our query is now complete. This message indicates the query is valid. There is nothing wrong with our syntax. You can test the query by pressing the Execute Query button. The query's results will appear in the Query Results panel If you double-click on an item in the list, it will appear in the Document Preview panel.

Unsupported query configurations

NTFS

The NTFS connection binding will not support the CONTAINS() predicate (for full text searching) unless the Windows file system has been indexed by the Windows Search service.

SharePoint

The SharePoint connection binding does not support the IN_FOLDER predicate.

• You must use the IN_TREE predicate instead when selecting a folder location to query.
• Be aware the IN_TREE search is recursive where IN_FOLDER is not. This means any subfolders will be queried as well as the targeted folder.

The SharePoint connection binding has a hard limitation when it comes to querying larger repositories, with many thousands of folders and files.

• There is currently a threshold limit of 5000 rows.
• This means if a query has to scan more than 5000 items to return an item, SharePoint will return the following error:
  • "Internal Server Error (500): The attempted operation is prohibited because it exceeds the list view threshold."

Box

Search values are required for Box CMISQL Queries. You must include a WHERE clause with at least one valid search condition.

The Box API has some built in requirements to perform search queries. This is not something Grooper can get around. In the past, this has caused Grooper users a lot of confusion when configuring WHERE clauses in CMISQL search queries.

To avoid confusion the following properties are not queryable at this time:

• Created by (cmis:createdBy)
• Modified by (cmis:lastModifiedBy)

The Box binding does not support does not support the IN_FOLDER predicate.

• You must use the IN_TREE predicate instead when selecting a folder location to query.
• Be aware the IN_TREE search is recursive where IN_FOLDER is not. This means any subfolders will be queried as well as the targeted folder.

Box has some documented limitations in its CMIS implementation in general. In most cases, Grooper is able to circumvent these issues, but not in all cases. Grooper is only able to return what Box gives us when executing a CMISQL query.

• Documents are not immediately queryable after their metadata templates have been edited in Box. If you have just added/edited fields in a Box document's metadata template, there is some lag time between when Grooper can query that data. You may need to wait several hours before a CMISQL query will return the document.
• We have encountered sporadic issues when using aliases (i.e. SELECT propertyName AS 'AliasName') in a Box CMIS Lookup query. As such, CMIS Lookups used to populate Grooper fields may be unreliable. However, please continue to report this issue to the support team at support@bisok.com. Additional reports will help the QA team diagnose this issue and help the Development team implement a solution, if possible.

Exchange

When configuring a WHERE clause (using a Comparison Predicate), the "Subject", "Sender", "To Recipients", "Cc Recipients", and "Bcc Recipients" cannot use the = operator.

• Use the LIKE operator instead.
  • Incorrect syntax: Sender = '%user@example.com%'
  • Correct syntax: Sender LIKE '%user@example.com%'
• BE AWARE: EWS only supports substring matching for these properties. You must use the % wildcard on either side of the search term when using the LIKE operator.
  • Incorrect syntax: Sender LIKE 'user@example.com'
  • Correct syntax: Sender LIKE '%user@example.com%'

IMAP

Gmail and other email clients does not support the IN_FOLDER or IN_TREE predicates because these are not part of the standard IMAP protocol.

• Instead Gmail uses "labels" which are not searchable by Grooper's querying mechanism.
• Some email clients do support these search predicates. We do not have a complete list of providers that do and do not support IN_FOLDER and IN_TREE at this time.

Object Model info

Type name: Grooper.CMIS.CmisQuery

Inheritance

Grooper Object (Grooper.GrooperObject)

Connected Object (Grooper.ConnectedObject)

Embedded Object (Grooper.EmbeddedObject)

CMISQL Element (Grooper.CMIS.QueryElement)

CMISQL Query (Grooper.CMIS.CmisQuery)

Derived Types

CMISQL Query is the Grooper object that selects documents or folders from a CMIS Repository.

• CMISQL Query has no derived types.
• However, several sibling types are key to its composition and use in Grooper.

CMISQL Element (Grooper.CMIS.QueryElement)

CMISQL Query (Grooper.CMIS.CmisQuery)

Join Clause (Grooper.CMIS.JoinClause) - Used to create the JOIN clause of a CMISQL Query

ORDER BY Element (Grooper.CMIS.OrderByElement) - Used to create the ORDER BY clause of a CMISQL Query

Select Element (Grooper.CMIS.SelectElement) - Used to create the SELECT clause of a CMISQL Query

Where Predicate (Grooper.CMIS.WherePredicate) - Used to create the WHERE clause of a CMISQL Query. Its derived types expose search conditions that can be added to the WHERE clause.

AT_LEVEL Predicate (Grooper.CMIS.AtLevelPredicate) - Matches documents/folders at a specific level below the base folder.

Comparison Predicate (Grooper.CMIS.ComparisonPredicate) - Specifies search criteria for an individual property using a comparison operator (=, <, >, LIKE etc)

CONTAINS Predicate (Grooper.CMIS.ContainsPredicate) - Specifies full-text search criteria

IN Predicate (Grooper.CMIS.InPredicate) - Specifies search criteria for an individual property, using a list of allowed values.

MATCHES Predicate (Grooper.CMIS.MatchesPredicate) - Specifies filtering criteria for a string property, where the property value must match a regular expression.

Predicate List (Grooper.CMIS.PredicateCollection) - Defines a collection of search conditions separated by logical AND/OR operators.

Scope Predicate (Grooper.CMIS.ScopePredicate) - Restricts the search scope to the direct children (IN_FOLDER) or all descendants (IN_TREE) of a folder.