2023:Web Service Lookup (Lookup Specification)

WIP	This article is a work-in-progress or created as a placeholder for testing purposes. This article is subject to change and/or expansion. It may be incomplete, inaccurate, or stop abruptly. This tag will be removed upon draft completion.

Web Service Lookup is a type of Lookup that can be performed in Grooper. It looks up external data at an API (Application Programing Interface) endpoint by calling a web service.

About

You may download and import the files below into your own Grooper environment (version 2023). This contains a Project with the Content Models used as examples throughout this article. Media:2023 Project - Wiki - Web Service Lookup.zip

Web Service Lookup is a new data lookup method in Grooper 2023. It allows Grooper to collect and validate data by calling a web service. The Web Service Lookup issues an HTTP GET or POST request and parses one or more records from the response. For example, you could use the Web Service Lookup to validate mailing addresses using the USPS Web Tools API. The Web Service Lookup would pass Data Field values Grooper collects in the request parameters and receive various values from the API from the web call. If you passed a Grooper extracted zip code in the web request, you could return the city and state from the API.

• For GET requests, parameters may be passed in the URL using "@variables" to replace a portion of the URL with a Data Field's value.
• For POST requests, parameters may be passed using "@variables" in the request body.

The Web Service Lookup supports both JSON and XML message formats.

How To

Below is a brief guide on how to access a Web Service Lookup in Grooper, as well as a quick glance at its components.

Adding a Web Service Lookup

Getting StartedRequestResponse

Getting Started

For this overview, we'll be going over how to access the Web Service Lookup. This particular lookup will will be accessing an API through GET/POST requests and to retrieve (or submit) a zip code and its related information (city and state). Further details will be expanded upon later in this article.

First, select the Data Model. We'll be looking at two Content Models within our Project—GET Example and POST Example. This will help you understand the different types of Verbs available within Grooper and how to set up the Lookup using one of these Verbs. Information on GET vs POST requests will be expounded upon later in this article. Select the ellipses to the right of Lookups under Behavior.

This will bring up the viewing window. Click the add button. Select Web Service Lookup. From there, you must configure both the Request and Response sections. There are a number of considerations when configuring these properties. These configurations can change depending on the request type you want to use, GET or POST. In the next section we will explain some of the differences between GET and POST requests and give an example for each.

Request

The "Request" is the first half of the Web Service Lookup. Essentially, it's what does the fetching of the information we want to pass into Grooper, and is broken down into the following pieces:

URL: Simply put, the URL defines a template using @variables to merge field data into the URL. Message Format: Specifies the format of the message being passed from the web service into Grooper. This can either be JSON or XML. Verb: A vital aspect of this lookup. Verb specifies which HTTP verb (GET or POST) to use when performing your lookup. GET and POST are discussed further at length below. Headers: HTTP Headers that you can choose to include with your request. Authentication: Set up through an API Key, authentication can be passed through automatically with the current user's credentials, or through Basic or NTLM Authentication

Response

The latter half of the Web Service Lookup; this is where the information is passed into Grooper. It is divided into the following:

Record Selector: Used to specify which JSON or XML entities represent records in the result set. For example, our GET Example has a record selector called "places". This acts as an array and selects elements from the API to send back to Grooper. More detail is provided in the next section. Value Selectors: Where one or more values are extracted from each record and mapped to fields in Grooper. These are further divided into Paths and Fields. The former is a JSON or XML expression that pulls a value from the record, while the latter is the target field into which the value will be displayed in Grooper. Timeout: The amount of time, measured in seconds, in which Grooper will wait for a response from the web service before declaring a timeout error.

Click here to return to the top of the tab

Configuring a Web Service Lookup

With an understanding of the components, we can now set a request. Below, we'll explain how to set up both GET and POST requests.

GET vs POST Requests

In the two examples below, we will use a Web Service Lookup to lookup a city and state for a given zip code. One will use a GET verb for the web request, the other will use the POST verb. For more information on GET and POST requests, please click the following link: W3Schools

⚠

The following two examples are to give very generic guidance on using the GET and POST requests. Be aware: Whether you configure a Web Service Lookup with a GET or POST is going to be determined by the parameters set by your API endpoint. Some API's will expect a GET request. Some will expect a POST request. It's entirely dependent on how the API is set up. Each individual API will have their own requirements as far as constructing either a GET or POST request. An improperly formatted request called to an API endpoint may not yield a valid response or the API will return an error. Be sure to go through the API endpoint's documentation to understand: Whether a GET or POST request is appropriate. How the request must be formatted.

The first example issues a GET request to an API. GET requests retrieve data from an API.

• For GET requests, the entire web call is made using the URL. The call itself is altered by simply changing part of the URL. Data Field values are merged with the URL by using @variables.
• Furthermore, the message received in this example is JSON formatted.
  • This example will use the Record Selector property to return values of an array in the JSON message.
  • However, be aware the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.
• Be aware! GET requests are less secure than POST requests. GET requests should NEVER be used when dealing with sensitive data.

The second example issues a POST request to an API. POST requests send data to an API.

• Be aware! Even though we're sending data. Our goal in doing so is to get some other data back from the API.
  • So, as far as the Web Service Lookup goes, both the GET and POST requests do similar things. We send the API some data from Data Fields in our Data Model and receive additional data from the API endpoint.
  • The only difference is how that data is sent. For POST requests, the data is sent in the "request body".
• For POST requests, the URL starts the call the web service, but additional information in the request body is required to complete the request and return a response. We will need to use the Request Body property to define Data Field values used in the request.
  • This example will use an XPath expression in the Record Selector property to return the values we want in the XML.
  • Be aware! the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.

GET Request

Add LookupRequestResponseResults

GET requests fetch information from an API, and merge data from the API into a Grooper Data Field through @variables.

Select the Data Model under the GET Example Content Model. Select the ellipses to the right of Lookups under Behavior.
Select the Add Button Click Web Service Lookup

Notice that GET is automatically selected. First, set up the URL.
Enter the URL that will start the call to your API. Note the @variable at the end. This is a key part of the URL. This is the information (zip codes in our case) that is passed into the API and fed back to Grooper. Since we're wanting to retrieve city state information related to zip codes, we'll be using Zippopotamus. [1] When finished, click OK.

Now, for the Record Selector. Since we're passing multiple pieces of information back to Grooper (city and state name relevant to our zip code), the JSON format of our GET request parses said information into an array called "places." See here for visual representation: ZIP Hence, we'll call our Record Selector "places."
To set up the Value Selectors, click the ellipses to the right of the property.
Click the add button. Path is an expression which selects a scalar value from the record; our Record selector "places" in this case. Thinking back to the array, "place name" is associated with city, and "state" with State respectively. Thus, these will be the paths we set up.
Select the Target Field that you would like the data to be mapped to. Notice that these fields are the same as the Data Fields in our Data Model. The Target Field is the final destination for the data retrieved through the GET request, and where it will be displayed.

With everything configured, click OK.
Click Save Enter a zip code, and viola! A successful GET request that returns city and state information based on the zip.

POST Request

POST requests send, or post, data to a requested server. The data is included in the URL path and uploaded for storage, making a change on that server. POST requests are not cached and do not remain in browser history, making them ideal for more sensitive data as opposed to GET requests.

Add LookupRequestResponseResults

Select the Data Model under the Content Model, POST Example. Click the ellipses to the right of Lookups under Behavior.
Click the Add button. Select Web Service Lookup.

Select Verb and Message Format, changing them to POST and XML respectively.
Next, add your URL. For the POST Example, we're using shippingapis.com.
Next, set up the Request Body. This is the part of your lookup that tells the API exactly where we plan to go to get our desired information.
The Request Body is set up in a format similar to HTML. We first tell the API that we're doing a city/state lookup, that we're specifically looking for a zip code, then we pass the @variable into the API.

Once we have the data "in hand" so to speak, it's communicated back to Grooper through the help of the Record and Value Selectors in the Response Section.
Unlike the GET request, we're not passing our data back to Grooper through an array. Instead, we're appending a URL. Now that the Record selector is set up, click the ellipses to the right of the Value Selector property to configure said selectors.
Click the Add button. Similar to the GET request, we need to set up the Path and Target Field. Unlike the GET request, since we're not passing data through an array, we can give the Paths and Target Fields the same name. Once the Path has been named, select the matching Target Field to which the data will be sent (i.e. the matching Data Field in the Data Model where the data is displayed).
With everything finished, select OK to close to accept the changes and close the window.

Save. Enter a zip code, and ta da! The relevant city and state are displayed along with the zip.

JSON vs XML

Grooper offers the option to choose between JSON or XML formatting when creating your lookup. The chosen Message Format depends on your preferences as well as the type of data you're working with. The differences between the two are highlighted briefly below. For more information on JSON and XML, please click the following link: AppMaster

JSON

JSON (Java Script Object Notation) is used for interchanging data. It is language independent and is ideal for sharing information to APIs due to its way of serializing complex information into a string. It is quicker and easier to write due to its lack of need for end tags and use of arrays, and can be parsed into a Java Script object. For more information on how JSON differs from XML, click here: W3Schools

XML

XML (Extensible Markup Language) is standard for storing and exchanging data. In XML, data is a collection of elements and attributes which can be nested within one another and are contained within opening and closing tags, instead of arrays. Said tags are used to define specific pieces of data for ease of coding. XML is designed to carry data. It is more secure than JSON and supports comments. For more information on how XML differs from JSON and the benefits it may provide, click here: GeeksforGeeks