Retrieve Data from any Site

The First Retriever for any Website! The RAG.pro retriever is an agent that can filter your query by specified domains. We handle the RAG pipeline for you so you don't have to set up your own retriever!

Getting Started

Step 1: Get Your API Key

You can find your API key on the Account Page. Keep it secure and include it in your requests as shown below.

Step 2: Base URL

All requests should be made to the following endpoint:

https://2q5vp3y0d0.execute-api.us-east-1.amazonaws.com/dev/retrieve

Authentication

Include your API key in the request headers as x-api-key.

Example Header:

x-api-key: YOUR_API_KEY

Making a Request

Send a POST request with a JSON payload containing the following parameters.

Request Parameters

Headers:

  • x-api-key (string, required): Your API key.

Body:

  • queryStringParameters (object, required):
    • query (string, required): The query string you want to send.
    • model (string, required): The size of the model to use. Options are "small", "medium", or "large".
    • sources (string, required): Whether to return sources. Options are "true" or "false".
    • in-text-sources (string, optional): Whether to include sources inline in the response text instead of using citation indicators. Only works when sources is "true". Options are "true" or "false".
  • multiValueQueryStringParameters (object, optional):
    • urls (array of strings, optional): A list of URLs to filter search domains.

Request Example

{
  "headers": {
    "x-api-key": "YOUR_API_KEY"
  },
  "queryStringParameters": {
    "query": "What is RAG?",
    "model": "small",
    "sources": "true",
    "in-text-sources": "false"
  },
  "multiValueQueryStringParameters": {
    "urls": [
      "https://docs.example.com/rag"
    ]
  }
}

Response

The API responds with a JSON array containing the response content and metadata.

Response Format

With in-text-sources: "false" (default):

[
  {
    "page_content": "Your answer here [1] with citation indicators [2]",
    "metadata": {
      "source": [
        "https://www.relevant-site.com",
        "https://www.another-site.com"
      ]
    }
  }
]

With in-text-sources: "true":

[
  {
    "page_content": "Your answer here (from https://www.relevant-site.com) with inline sources (from https://www.another-site.com)",
    "metadata": {
      "source": [
        "https://www.relevant-site.com",
        "https://www.another-site.com"
      ]
    }
  }
]

Error Handling

The API uses standard HTTP status codes to indicate success or failure.

  • 200 OK: The request was successful.
  • 400 Bad Request: The request was invalid.
  • 401 Unauthorized: API key is missing or invalid.
  • 500 Internal Server Error: An error occurred on the server.

Notes

Use the urls parameter to focus the agent's knowledge on specific websites.

Examples

URLs
curl -X POST \
'https://2q5vp3y0d0.execute-api.us-east-1.amazonaws.com/dev/retrieve' \
-H 'Content-Type: application/json' \
-H 'x-api-key: YOUR_API_KEY' \
-d '{
  "queryStringParameters": {
    "query": "What is RAG?",
    "model": "small",
    "sources": "true",
    "in-text-sources": "false"
  },
  "multiValueQueryStringParameters": {
    "urls": [
      "https://docs.aws.amazon.com/"
    ]
  }
}'