Making an AI Assistant from HTTP Import (Simple Functionality)

This article is about the current version of Grooper.

Note that some content may still need to be updated.

2025

You may download the ZIP file below for use in your own Grooper environment (version 2025). This is a Project ZIP file.

2025-Project_Making-an-AI-Assistant-from-HTTP-Import.zip

Introduction

This article demonstrates how to build a fully functional, domain-specific AI Assistant in Grooper by ingesting web content, indexing it, and connecting it as a knowledge resource for conversational use. This article showcases how Grooper can transform publicly available website content into a searchable knowledge base that powers intelligent, context-aware responses.

The intention of this article is to provide a clear, end-to-end example of how several key Grooper features—HTTP Import, Batch Processing, AI Search, and AI Assistants—work together to create a practical AI solution. Instead of relying on pre-existing datasets, this workflow dynamically pulls content from a live website, processes it as documents within a Batch, and prepares it for retrieval through a search index.

The guide walks through configuring the required services and repository options, setting up a search index with vector-based retrieval, importing and processing web pages, and finally connecting that indexed content to an AI Assistant. The result is an assistant capable of answering questions using the imported material as its knowledge source.

By the end of this article, readers will understand how to ingest external web content into Grooper, structure it for search and retrieval, and leverage it within an AI Assistant—demonstrating a powerful pattern for creating custom, knowledge-driven conversational experiences.

Initial setup

This portion of the article ensures the environment is properly configured before any ingestion or AI functionality is used. It begins by verifying that required services—Activity Processing and Import Watcher—are installed and running, which can be confirmed in the Machines folder of the Node Tree on the Design page. These services are essential for executing Batch Processes and handling automated imports.

It also includes configuring Repository Options at the Root node level by adding and setting up both an LLM Connector (for AI-driven functionality) and an AI Search option (for indexing and retrieval). This establishes the foundational connections Grooper needs to interact with external AI models and search infrastructure.

Select the Machines node in the Node Tree. Verify an Activity Processing and Import Watcher service are added and running.
Select the Root node, then click the ellipsis button for the Options property to open the "Options" editor.
Make sure an LLM Connector is added in the "Options" editor, then verify it is property configured in the Service Providers editor.
Verify an "AI Search" option is added and property configured.

Search Index setup

This section focuses on preparing the Content Model for indexing and enabling search capabilities. It involves configuring the Indexing Behavior, including selecting an embeddings model for vector-based search and understanding how document content will be stored and retrieved.

Once configured, the search index is created and associated with the Content Model. This step ensures that any documents processed later can be added to the index, making them searchable and available as a knowledge resource for the AI Assistant.

Expand the Node Tree and select the "Constitution Index" Content Model from the provided Project, then click the ellipsis button for the Behaviors property to open the "Behaviors" editor.
In the "Behaviors" editor, verify the Name, Included Elements, and Built in Fields properties are properly configured. Vector Search should be enabled. Choose an Embeddings Model you prefer, and set the Chunking Method to "Fixed Chunker".
Right-click the "Constitution Index" Content Model, then select the "Search", "Create Search Index" object command from the pop-out menu.
Click the "Execute" button in the subsequent dialogue window.
To verify our newly created indices, select the Root node in the Node Tree, then click the "Indexes" tab.
You will see two indices are listed for our desired content. These two indices are linked to one another via values in their content.

Import setup

This portion walks through configuring and submitting an HTTP Import job to ingest web-based content into Grooper. Users define a target website, specify how links are discovered and filtered, and ultimately execute the import.

The goal is to bring external web pages into Grooper as documents in a Batch, where they can be processed just like any other document set. This demonstrates how Grooper can acquire knowledge from online sources and prepare it for downstream processing and indexing.

Click the "Imports Page" button to go to the Imports Page, then click the "Add Import Job" button in the top-right of the UI.
Type something for the Description property, then set the Provider property to "HTTP Import".
In the sub-properties of the Provider property set to "HTTP Import", click the ellipsis button for the Sources property to open the "Sources" editor.
Add a "Source", then set the URL property to: "https://ballotpedia.org". This will use the main page as the entry point.
Click the ellipsis button for the Relative Page URLs property to open the "Relative Page URLs" editor.
Add Oklahoma_Constitution in the "Relative Page Urls" editor. This will restrict the import to only look at this sub-page of the site, and not the entire site. Multiple sub-sites could be added here if desired.
Click the ellipsis button for the Link Selectors property to open the "Link Selectors" editor.
Add a "Link Selector", then set the Selector property to "a". This is referring to the "a" "h-ref" attribute used in HTML for hyperlinks.
"Enable" the Included Url Property and set the Pattern property to Oklahoma_Constitution$. The dollar sign refers to the end of string of the URL, so it would be the base URL that ends with "Oklahoma_Constitution".
Provide something for the Description property.
Leave the Batch Creation property set to "(no process)", then click "Submit" to start the import. Not linking a Batch Process will put our Batch in the "Imported Batches" folder of the "Test" branch of the Batches node.
Wait for the job to complete with 41 files, then click the "Design Page" button to go to the Design Page.
Expand the Node Tree and select the newly created Batch, then click the "Viewer" tab.
You should see the contents of your newly imported Batch.

Batch Process testing

This section focuses on validating the HTTP Ingestion Batch Process by examining and testing each of its steps. Each stage of the process—such as loading content, processing documents, and preparing them for indexing—is reviewed to ensure it is configured correctly and functioning as expected.

By testing individual steps, users gain insight into how imported web content is transformed within Grooper. This includes confirming that documents are properly created, text is accessible, and indexing preparation steps are executed successfully. This approach helps identify and resolve issues before relying on the process in a fully automated workflow.

AI Assistant setup

This final portion explains how to configure an AI Assistant to use the indexed documents as its knowledge resource. It includes selecting an LLM model, assigning the previously created search index as a resource, and optionally refining the assistant’s instructions and behavior.

Once configured, the AI Assistant can be used from the Chat page to ask questions about the imported content. This step demonstrates how Grooper brings everything together—turning ingested and indexed web data into a conversational interface capable of retrieving and presenting relevant information on demand.

Example Questions for the AI Assistant

What are the different branches of government in Oklahoma, and how do they function?

What rights do Oklahoma citizens have that go beyond the U.S. Constitution?

How can the Oklahoma State Constitution be amended?

How has the Oklahoma State Constitution changed over time?

How does the Oklahoma Constitution address public education and school funding?

What provisions exist in the Constitution regarding taxation and economic development?

What are the rules for elections and voter rights in Oklahoma?

What powers does the state government have over cities and counties?

For more information