Web Service Lookup (Lookup Specification)

From Grooper Wiki

This article is about the current version of Grooper.

Note that some content may still need to be updated.

2025 2023

Web Service Lookup is a Lookup Specification that looks up external data at an API endpoint by calling a web service.

You may download and import the file below into your own Grooper environment (version 2025). This contains a Project with the Content Models used as examples throughout this article.

About

Web Service Lookup allows Grooper to collect and validate data by calling a web service. The Web Service Lookup issues an HTTP GET or POST request and parses one or more records from the response. For example, you could use the Web Service Lookup to validate mailing addresses using the USPS Web Tools API. The Web Service Lookup would pass Data Field values Grooper collects in the request parameters and receive various values from the API from the web call. If you passed a Grooper extracted zip code in the web request, you could return the city and state from the API.

  • For GET requests, parameters may be passed in the URL using "@variables" to replace a portion of the URL with a Data Field's value.
  • For POST requests, parameters may be passed using "@variables" in the request body.

The Web Service Lookup supports both JSON and XML message formats.

How To

Below is a brief guide on how to access a Web Service Lookup in Grooper, as well as a quick glance at its components.

Adding a Web Service Lookup

Getting started

For this overview, we'll be going over how to access the Web Service Lookup. This particular lookup will will be accessing an API through GET/POST requests and to retrieve (or submit) a zip code and its related information (city and state). Further details will be expanded upon later in this article.

  1. Select the Data Model.
  2. Click the ellipses to the right of "Lookups" property.
  3. This will bring up the viewing window. Click the add button.
  4. Select Web Service Lookup.
    • From there, you must configure both the Request and Response sections.
  • There are a number of considerations when configuring these properties, which will be determined by how the API endpoint expects to receive a request and return a response. These configurations can change depending on the request type you want to use, GET or POST.
  • In the next section, we will give a generic overview of the Request and Response property groups.
  • In the #GET vs POST Requests section of this article, we will explain some of the differences between GET and POST requests.

Request

The "Request" is the first half of the Web Service Lookup. These properties determine how the API call is initiated. Essentially, it's what does the fetching of the information we want to pass into Grooper, and is broken down into the following pieces:

URL:

  • The URL is required to connect to the API. Simply put, the URL defines the web address Grooper needs to go to.
  • You can use @variables to reference extracted results from a Data Field when editing the URL address. This will merge extracted values into the URL address.
    • For example, if you had a Data Field named "Item Number" you could merge the extracted value by entering @Item_Number.
      • If this is what you enter, https://www.example.com/@Item_Number
      • And Grooper extracts "12345" for the "Item Number" Data Field.
      • Then, this is the URL Grooper passes "https://www.example.com/12345"

Message Format:

  • Specifies the format of the message being passed from the web service into Grooper.
  • This can either be JSON or XML. What you choose depends on the requirements of the API endpoint you're calling. Be sure to read through that API's documentation to understand which one is appropriate.

Verb:

  • A vital aspect of this lookup. Verb specifies which HTTP verb (GET or POST) to use when performing your lookup.
  • Whether you choose GET or POST depends on the requirements of the API endpoint you're calling. Be sure to read through that API's documentation to understand which one is appropriate.
  • GET and POST are discussed further at length below.

Headers:

  • HTTP Headers that you can choose to include with your request. Headers are sometimes necessary to pass the API additional information. What information (if any) is required in an HTTP header should be found in your API endpoint's documentation.
  • You will enter headers as a list of key value pairs using the following syntax:
key1=value1
key2=value2
key3=value3
  • Most commonly, you'll use a header to pass an API key.

Authentication:

  • If the API requires a user and password logon, you can enter security credentials in the Authentication property.

Response

The is is the second half of the Web Service Lookup. The "Response" properties determine what information is passed into Grooper and where in the Data Model that data goes. It is divided into the following:

  • Record Selector:
    • Used to specify which JSON or XML entities represent records in the result set. Selectors are required when you're trying to access sub elements of a JSON or XML response.
    • Our GET request example in this article has a JPATH selector called "places". This selects elements in an array in the JSON file the API returns to Grooper.
  • Value Selectors:
    • Where one or more values extracted from each record are mapped to fields in Grooper.
    • These are further divided into "Path" and "Field".
      • The Path is a JSON or XML expression that pulls a value from the response.
      • The Field is the target Data Field where the value will be stored in Grooper.
  • Timeout:
    • The amount of time, measured in seconds, in which Grooper will wait for a response from the web service before declaring a timeout error.

Configuring a Web Service Lookup

With an understanding of the components, we can now set a request. Below, we'll explain how to set up both GET and POST requests.

GET vs POST Requests

In the example below, we will use a Web Service Lookup to lookup a city and state for a given zip code. It will use a GET verb for the web request. For more information on GET and POST requests, please click the following link: W3Schools

⚠

Be aware:

  • Whether you configure a Web Service Lookup with a GET or POST is going to be determined by the parameters set by your API endpoint.
    • Some API's will expect a GET request. Some will expect a POST request. It's entirely dependent on how the API is set up.
  • Each individual API will have their own requirements as far as constructing either a GET or POST request. An improperly formatted request called to an API endpoint may not yield a valid response or the API will return an error.


Be sure to go through the API endpoint's documentation to understand:

  1. Whether a GET or POST request is appropriate.
  2. How the request must be formatted.

The below example issues a GET request to an API. GET requests retrieve data from an API.

  • For GET requests, the entire web call is made using the URL. The call itself is altered by simply changing part of the URL. Data Field values are merged with the URL by using @variables.
  • Furthermore, the message received in this example is JSON formatted.
    • This example will use the Record Selector property to return values of an array in the JSON message.
    • However, be aware the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.
  • Be aware! GET requests are less secure than POST requests. GET requests should NEVER be used when dealing with sensitive data.

POST requests send data to an API.

  • Be aware! Even though a POST is sending data, the goal in doing so is to get some other data back from the API.
    • So, as far as the Web Service Lookup goes, both the GET and POST requests do similar things. We send the API some data from Data Fields in our Data Model and receive additional data from the API endpoint.
    • The only difference is how that data is sent. For POST requests, the data is sent in the "request body".
  • For POST requests, the URL starts the call the web service, but additional information in the request body is required to complete the request and return a response. You will need to use the Request Body property to define Data Field values used in the request.
    • Be aware! the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.

GET Request Example

Please note: This is just one example. You will need to know specifics about the API endpoint you're calling, such as if the request requires any HTML headers, to successfully configure any Web Service Lookup. You will need to refer to the the endpoint API's documentation for this information.

GET requests fetch information from an API, and merge data from the API into a Grooper Data Field through @variables.

  1. Select the Data Model under the GET Example Content Model.
  2. Click the ellipses to the right of the Lookups property.
  3. The default of the Message Format property, "JSON", will be used for this example.
  4. The default of the Verb property, "GET", will be used for this example.
  5. Click the ellipsis button to the right of the URL property to open its editor.
  6. The following URL will be used for this example. 
    https://api.zippopotam.us/us/@ZIP_Code
    • Note the @variable at the end. This is a key part of the URL. This is the information (zip codes in our case) that is passed into the API and fed back to Grooper.
    • Since we're wanting to retrieve city and state information related to zip codes, we'll be using Zippopotamus.
  7. Now, for the Record Selector.
    • Since we're passing multiple pieces of information back to Grooper (city and state name relevant to our zip code), the JSON format of our GET request parses said information into an array called "places." See here for visual representation: Zippopotamus API Endpoint
    • Set the Record Selector to the string "places".
  8. To set up the Value Selectors, click the ellipses to the right of the property.
  9. Click the add button.
  10. Path is an expression which selects a scalar value from the record; our Record selector "places" in this case.
    • Thinking back to the array, "['place name']" is associated with city, and "['state']" with State respectively. Thus, these will be the paths we set up.
  11. Set the Path property to ['place name'].
  12. Click the drop-down for the Target Field property, and select the appropriate Data Element. For our example, select "City" from the drop-down menu.
    • Notice that these fields are the same as the Data Fields in our Data Model. The Target Field is the final destination for the data retrieved through the GET request, and where it will be displayed.
  13. Add another Value Selector.
  14. Set the Path property to ['state'], then set the Target Field property to "State".
  15. Back on the Data Model, enter a valid zip code into the Data Model Preview and you'll get a successful GET request that returns city and state information based on the zip code.

JSON vs XML

Grooper offers the option to choose between JSON or XML formatting when creating your lookup. The chosen Message Format depends the type of data you're working with and what format the destination API uses. The differences between the two are highlighted briefly below.

For more information on JSON and XML, please click the following link: AppMaster

JSON

JSON (Java Script Object Notation) is used for interchanging data. It is language independent and is ideal for sharing information to APIs due to its way of serializing complex information into a string. It is quicker and easier to write due to its lack of need for end tags and use of arrays, and can be parsed into a Java Script object.

For more information on how JSON differs from XML, click here: W3Schools

XML

XML (Extensible Markup Language) is standard for storing and exchanging data. In XML, data is a collection of elements and attributes which can be nested within one another and are contained within opening and closing tags, instead of arrays. Said tags are used to define specific pieces of data for ease of coding. XML is designed to carry data. It is more secure than JSON and supports comments.

For more information on how XML differs from JSON and the benefits it may provide, click here: GeeksforGeeks

Remaining Properties overview

Below is a list of Web Service Lookup properties that weren't covered (the Request and Response properties) earlier in this article.

Lookup Options

  • Trigger Mode
    • Controls when the lookup is executed.
    • Determines the conditions under which a lookup operation is performed in Grooper. This setting allows you to control whether lookups are executed automatically, only under certain conditions, manually, or based on a custom expression. Choosing the appropriate trigger mode is essential for balancing automation, user control, and data integrity in your solution.
  • Miss Disposition
    • Specifies what happens if the lookup returns no results.
    • Determines how Grooper responds when a lookup operation does not return any matching records from the external data source. This setting controls whether errors are flagged, target fields are cleared, or no action is taken.
  • Conflict Disposition
    • Specifies what happens if the lookup returns multiple results.
    • Controls how Grooper handles situations where a lookup operation returns more than one matching record from the external data source. This setting determines whether errors are flagged, data is cleared, the first result is accepted, or no action is taken.
  • Field Population
    • Specifies how target fields are populated with the lookup results.
    • Controls whether and how the results of a lookup operation are written to the target fields in Grooper. This setting determines if lookup results will overwrite existing values, supplement only empty fields, or leave all fields unchanged.

Lookup Info

  • Lookup Fields
    • The list of fields and variables used as lookup criteria.
    • This property displays a comma-separated list of all Data Fields and Variable Definitions that are used as input for the lookup query. These are the values that will be sent to the external data source to perform the lookup.
  • Target Fields
    • The list of fields that will be populated by the lookup operation.
    • This property displays a comma-separated list of all Data Fields that will be set or validated by the results of the lookup. These are the target fields that will receive values from the external data source.
Retrieved from "https://wiki.grooper.com/index.php?title=Web_Service_Lookup_(Lookup_Specification)&oldid=31747"