How to request paginated LightDB Stream data

Golioth Cloud Help
1

Description

I’m trying to query LightDB Stream data via the API and I’m trying to figure out how get all the data within a certain date range using pagination.

Here’s the query I’m using:

curl -X 'POST' \
  'https://api.golioth.io/v1/projects/water-level-sensor-b159cd/devices/6849be5632c3e874e7ec9c5a/stream' \
  -H 'accept: application/json' \
  -H 'x-api-key: <redacted>' \
  -H 'Content-Type: application/json' \
  -d '{
  "start": "2025-07-15T22:39:22.212Z",
  "end": "2025-07-17T22:39:22.212Z",
  "query": {
    "fields": [
      {
        "path": "time"
      },
      {
        "path": "deviceId"
      },
      {
        "path": "sensor"
      }
    ]
  },
  "page": 0,
  "perPage": 1
}'

This is the response I’m getting:

{
  "list": [
    {
      "deviceId": "6849be5632c3e874e7ec9c5a",
      "sensor": {
        "accel": {
          "x": 0.083334,
          "y": 0.151962,
          "z": -9.571155
        },
        "tilt": {
          "pitch": -0.9095788157055332,
          "roll": -0.4987866350428875
        },
        "water_level": {
          "float_height": -0.5714812202120836,
          "float_length": 36,
          "float_offset": 0
        }
      },
      "time": "2025-07-17T06:48:58.919656+00:00"
    }
  ],
  "page": 0,
  "perPage": 1,
  "total": 0
}

Should the total field report the total number of stream records within the date/time range I provided in the original query?

I guess I’m not totally clear on exactly how I should be using the pagination to request all the records within this date/time range. I would assume that I should be calculating the number of pages to request (and therefore the number of page requests to make via the API) based on the total number of records reported.

What’s the correct way to get all the records for a time/date range if the total number of records exceeds the max 100 records per request?

Environment

API v1

2

FYI, my end goal was to try to use the Golioth API as a data source for the Grafana Infinity plugin (JSON).

This plugin supposedly supports pagination: Pagination of API responses · grafana/grafana-infinity-datasource · Discussion #601 · GitHub although I haven’t been able to figure out how to get this working with the Golioth API.

3

I also tried omitting the pagination fields to see if that gives me a correct total number of records:

curl -X 'POST' \
  'https://api.golioth.io/v1/projects/water-level-sensor-b159cd/devices/6849be5632c3e874e7ec9c5a/stream' \
  -H 'accept: application/json' \
  -H 'x-api-key: <redacted>' \
  -H 'Content-Type: application/json' \
  -d '{
  "start": "2025-07-15T22:39:22.212Z",
  "end": "2025-07-17T22:39:22.212Z",
  "query": {
    "fields": [
      {
        "path": "time"
      },
      {
        "path": "deviceId"
      },
      {
        "path": "sensor"
      }
    ]
  }
}'

This returns 100 records (the max), but the total is still 0:

{
  "list": [
    {
      "deviceId": "6849be5632c3e874e7ec9c5a",
      "sensor": {
        "accel": {
          "x": 0.083334,
          "y": 0.151962,
          "z": -9.571155
        },
        "tilt": {
          "pitch": -0.9095788157055332,
          "roll": -0.4987866350428875
        },
        "water_level": {
          "float_height": -0.5714812202120836,
          "float_length": 36,
          "float_offset": 0
        }
      },
      "time": "2025-07-17T06:48:58.919656+00:00"
    },
    ... (other 98 records removed for brevity)
    {
      "deviceId": "6849be5632c3e874e7ec9c5a",
      "sensor": {
        "accel": {
          "x": 0.058824,
          "y": 0.176472,
          "z": -9.590763
        },
        "tilt": {
          "pitch": -1.0541152798922866,
          "roll": -0.35135414044570484
        },
        "water_level": {
          "float_height": -0.6622828009906346,
          "float_length": 36,
          "float_offset": 0
        }
      },
      "time": "2025-07-17T06:40:43.32002+00:00"
    }
  ],
  "page": 0,
  "perPage": 100,
  "total": 0
}
4

Hey @cdwilson,

The total value is a legacy field, and to preserve the existing public API, we continue returning it with the value always set to zero. That said, page and perPage need to be included in the request body to handle pagination. In your case, you can set perPage to 100 and keep incrementing the page value until the response returns fewer than 100 items.

1 Like
6

Just to close the loop on this for anybody running across this in the future: I ended up abandoning the Grafana Infinity plugin because it can’t handle the pagination required by the Golioth API.

Instead, I’m just using Golioth pipelines to route the data into an InfluxDB database, and visualizing it in Grafana using the InfluxDB data source. This post on the blog describes how to set it all up.

1 Like