2023:Web Service Lookup (Lookup Specification)

WIP	This article is a work-in-progress or created as a placeholder for testing purposes. This article is subject to change and/or expansion. It may be incomplete, inaccurate, or stop abruptly. This tag will be removed upon draft completion.

Web Service Lookup is a type of Lookup that can be performed in Grooper. It looks up external data at an API (Application Programing Interface) endpoint by calling a web service.

About

Web Service Lookup is a new data lookup method in Grooper 2023. It allows Grooper to collect and validate data by calling a web service. The Web Service Lookup issues an HTTP GET or POST request and parses one or more records from the response. For example, you could use the Web Service Lookup to validate mailing addresses using the USPS Web Tools API. The Web Service Lookup would pass Data Field values Grooper collects in the request parameters and receive various values from the API from the web call. If you passed a Grooper extracted zip code in the web request, you could return the city and state from the API.

• For GET requests, parameters may be passed in the URL using "@variables" to replace a portion of the URL with a Data Field's value.
• For POST requests, parameters may be passed using "@variables" in the request body.

The Web Service Lookup supports both JSON and XML message formats.

Overview

Below is a brief guide on how to access a Web Service Lookup in Grooper, as well as a quick glance at its components.

Getting StartedRequestResponse

For this overview, we'll be going over how to access the Web Service Lookup. This particular lookup will will be accessing an API through GET/POST requests and to retrieve (or submit) a zip code and its related information (city and state). Further details will be expanded upon later in this article.

First, select the Project. We'll be looking at two Content Models within our Project—GET Example and POST Example. This will help you understand the different types of Verbs available within Grooper and how to set up the Lookup using one of these Verbs. Information on GET vs POST requests will be expounded upon later in this article. Select Data Model in your Content Model, followed by the ellipses to the right of Lookups under Behavior.

This will bring up the viewing window. Click the add button. Select Web Service Lookup. From there, you can configure the properties in the Request and Response sections.

The "Request" is the first half of the Web Service Lookup. Essentially, it's what does the fetching of the information we want to pass into Grooper, and is broken down into the following pieces:

URL: Simply put, the URL defines a template using @variables to merge field data into the URL. Message Format: Specifies the format of the message being passed from the web service into Grooper. This can either be JSON or XML. Verb: A vital aspect of this lookup. Verb specifies which HTTP verb (GET or POST) to use when performing your lookup. GET and POST are discussed further at length below. Headers: HTTP Headers that you can choose to include with your request. Authentication: Set up through an API Key, authentication can be passed through automatically with the current user's credentials, or through Basic or NTLM Authentication

The latter half of the Web Service Lookup; this is where the information is passed into Grooper. It is divided into the following:

Record Selector: Used to specify which JSON or XML entities represent records in the result set. For example, our GET Example has a record selector called "places". This acts as an array and selects elements from the API to send back to Grooper. More detail is provided in the next section. Value Selectors: Where one or more values are extracted from each record and mapped to fields in Grooper. These are further divided into Paths and Fields. The former is a JSON or XML expression that pulls aa value from the record, while the latter is the target field into which the value will be displayed in Grooper. Timeout: The amount of time, measured in seconds, in which Grooper will wait for a response from the web service before declaring a timeout error.

Click here to return to the top of the tab

Now that we understand where in Grooper to access the Web Service Lookup as well as its composite parts, we're ready to dive further in and explore how the Request and Response section work though the use of Verb and Message Format.

GET vs POST Requests

In the two examples below, we will use a Web Service Lookup to lookup a city and state for a given zip code. One will use a GET verb for the web request, the other will use the POST verb. For more information on GET and POST requests, please click the following link: W3Schools

⚠

The following two examples are to give very generic guidance on using the GET and POST requests. Be aware: Each individual API will have their own requirements as far as constructing either a GET or POST request. An improperly formatted request called to an API endpoint may not yield a valid response or the API will return an error. Be sure to go through the API endpoint's documentation to understand how the request must be formatted.

The first example issues a GET request to an API. GET requests retrieve data from an API.

• For GET requests, the entire web call is made using the URL. The call itself is altered by simply changing part of the URL. Data Field values are merged with the URL by using @variables.
• Furthermore, the message received in this example is JSON formatted.
  • This example will use the Record Selector property to return values of an array in the JSON message.
  • However, be aware the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.
• Also, it is important to be aware that GET requests should NEVER be used when dealing with sensitive data.

The second example issues a POST request to an API. POST requests send data to an API.

• In the case of our Web Service Lookup, we send the API some data from Data Fields in our Data Model and receive additional data from the API endpoint.
  • For POST requests, the URL starts the call the web service, but additional information in the request body is required to complete the request and return a response. We will need to use the Request Body property to define Data Field values used in the request.
• Thus, the message received in this example is XML formatted.
  • This example will use an XPath expression in the Record Selector property to return the values we want in the XML.
  • However, be aware the message format depends on each individual API. Be sure to read the API's documentation to know if the API supports XML or JSON requests.

GET RequestPOST Request

A GET requests, as their name states, get data from an API, then pass it into Grooper; usually in a JSON format. Once the data has been received from the source, it is given back to Grooper in a neat little bundle—which can be made neater still with Record and Value Selectors, which are detailed below.

For GET requests, we'll take a look at the GET Example Content Model. To access the GET request, select the Data Model, then click the ellipses to the right of Lookup under Behavior.
Here, you will choose which type of request (referred to as a Verb) you wish to use. As you can see, we have chosen GET. Verbs are vital. Much like how you cannot have a sentence without a verb, you cannot have a Web Service Lookup without a request/Verb. While HTTP requests are varied, Grooper only concerns itself with GET and POST Now, we'll move ono Record and Value Selectors. Record Selectors are optional, as leaving the section blank will give you the option to simply select the type of record at the root of either the JSON or XML document. However, for this particular case, since we're going to be displaying a place's location based on its ZIP Code, we will need a Record Selector to pass the pieces of information we want into the appropriate fields in Grooper. This is because we have multiple pieces of data that we need to feed into our Data Model. Thus, the Record Selector, "Places" acts as an array through which the information associated with whatever zip code our URL retrieves will pass. Said passing is then refined by the Value Selectors we have set up. These act as signs along the path which our retrieved information will travel, pointing out the field in which the specific piece of information needs to be placed.
As evidenced, we have two Value Selectors, "state" and "[place name]"; "state" receives the name of the state associated with the zip code pulled by the API and "[place name]" is the city name relative to that zip code. Once this information is obtained by the API, it is then passed back to Grooper and filtered first through the array that is our record selector, then additionally through our value selectors. This is done by setting the "Path", i.e. a scalar value from the record as well as a Target Field—the Data Field where the information will be displayed. For example, when entered into the URL, the zip code 57701 will return the following: {"post code": "57701", "country": "United States", "country abbreviation": "US", "places": [{"place name": "Rapid City", "longitude": "-103.2052", "state": "South Dakota", "state abbreviation": "SD", "latitude": "44.1415"}]} As you can see, the place name, and state are the paths where we pick up the desired information. Once they're sent back to our Data Model, "place name" is deposited into the Target Field of City, and "state" into State respectively.

Unlike GET requests, POST requests send, or post, data to a requested server. The data is included in the URL path and uploaded for storage, making a change on that server. POST requests are not cached and do not remain in browser history, making them ideal for more sensitive data as opposed to GET requests.

To view the POST request, select the Data Model under the POST Examples Content Model, and click the ellipses to the right of the Lookup property, under Behavior.
The main difference between GET and POST requests is the Request Body. As stated earlier, a GET request merely retrieves information from a URL and passes it into Grooper. A POST request on the other hand, sends data to a requested server. The Request Body defines the XML or JSON data that is sent with the request.
First, we tell Grooper which path we're using, JSON or XML. As you can see, we've chosen XML for this request. Just like our GET example, we want to look up zip codes, and pass that information, along with the city and state related to said zip code, into our Data Model. For POST requests, the API expects to know where the request is coming from, thus you will need to input your computer's/business' user id. Thus, we outline where we wish to go: CityStateLookup -> ZipCode and then end our html statements accordingly. Note the @variable, @Zip_Code. This is the information we want to retrieve and pass into Grooper.
As before, we have set up our Record and Value Selectors. Since we're using a POST request, instead of an array through which information will be fed and sorted, we enter what looks to be the latter portion of a URL path. Specifically, we're telling the API and Grooper that once we reach CityStateLookup, we're utilizing the zip code to obtain it and its related information.
Similar to the GET request, once we have the zip code data and its related information in Grooper, we feed it through the value selectors and sort each piece of information accordingly. City and State in this case display the city and state names respectively.
And viola!

Click here to return to the top of the tab

JSON vs XML

Grooper offers the option to choose between JSON or XML formatting when creating your lookup. The chosen Message Format can depend on your preferences as well as the type of data you're working with. The differences between the two are highlighted briefly below. For more information on JSON and XML, please click the following link: AppMaster

JSON

JSON (Java Script Object Notation) is used for interchanging data. It is language independent and is ideal for sharing information to APIs due to its way is serializing complex information into a string. It is quicker and easier to write due to its lack of need for end tags and use of arrays, and can be parsed into a Java Script object. For more information on how JSON differs from XML, click here: W3Schools

XML

XML (Extensible Markup Language) is standard for storing and using data exchange. In XML, data is a collection of elements and attributes which can be nested within one another and are contained within opening and closing tags, instead of arrays. Said tags are used to define specific pieces of data for ease of coding. XML is designed to carry data. It is more secure than JSON and supports comments. For more information on how XML differs from JSON and the benefits it may provide, click here: GeeksforGeeks