Sending JSon sensor data to lightdb stream using pipeline but not displaying on dashboard

Golioth Device Help
1

Description

Hi all, I’ve been debugging LightDB Stream all day and I’m completely stuck.
Wi-Fi connects and logs show device is online
golioth_lightdb_set_async() works — values show up in LightDB State
But nothing ever shows up in LightDB Stream, even after hours of testing

GOLIOTH_LOG() works — I can see device logs via goliothctl logs listen
Stream publishing returns GOLIOTH_OK — device says rows are being sent

Steps to Reproduce

This is the code i am currently using in the golioth_manager.cpp file:
#include “golioth_manager.h”
#include “golioth/client.h”
#include “golioth/lightdb_state.h”
#include “golioth/golioth_status.h”
#include “golioth/golioth_sys.h”
#include “golioth/stream.h”
static struct golioth_client* s_client = nullptr;

static void on_golioth_event(struct golioth_client* /client/,
enum golioth_client_event event,
void* /arg/)
{
switch (event) {
case GOLIOTH_CLIENT_EVENT_CONNECTED:
ESP_LOGI(“GOLIOTH_MGR”, “connected to Golioth”);

break;

case GOLIOTH_CLIENT_EVENT_DISCONNECTED:
ESP_LOGW(“GOLIOTH_MGR”, “lost Golioth connection”);
break;

default:
ESP_LOGI(“GOLIOTH_MGR”, “event %d”, (int)event);
break;
}
}

/* ------------------------------------------------------------------ /
/ Helper: build a compile‑time config struct /
/ ------------------------------------------------------------------ /
static void build_golioth_config(struct golioth_client_config cfg)
{
memset(cfg, 0, sizeof(*cfg));

cfg->credentials.auth_type = GOLIOTH_TLS_AUTH_TYPE_PSK;

cfg->credentials.psk.psk_id     = GOLIOTH_PSK_ID;
cfg->credentials.psk.psk_id_len = strlen(GOLIOTH_PSK_ID);

cfg->credentials.psk.psk        = GOLIOTH_PSK;
cfg->credentials.psk.psk_len    = strlen(GOLIOTH_PSK);

}

void golioth_manager_init(void)
{
if (s_client) {
ESP_LOGW(TAG, “Golioth already initialised”);
return;
}

struct golioth_client_config cfg;
build_golioth_config(&cfg);

s_client = golioth_client_create(&cfg);
if (!s_client) {
    ESP_LOGE(TAG, "Failed to create Golioth client (NULL returned)");
    return;                            // avoids NULL‑dereference crash
}

golioth_client_register_event_callback(s_client, on_golioth_event, nullptr); 

ESP_LOGI(TAG, "Golioth client created");
golioth_client_start(s_client);


ESP_LOGI(TAG, "Golioth client started");

}

bool golioth_is_connected(void)
{
return s_client &&

(golioth_client_is_connected(s_client) == GOLIOTH_OK);

}

void golioth_send_sensor_data(void)
{
if (!golioth_is_connected()) return;

sens_reading_t s;
sensors_get_latest(&s);

uint64_t ts_ms = esp_timer_get_time() / 1000ULL;

//static uint32_t loop = 0; 
char json[300];
int n = snprintf(json, sizeof(json),
"{"
"\"temp\":%.2f,\"humidity\":%.1f,"
"\"red\":%d,\"green\":%d,\"blue\":%d,\"ir\":%d,"
"\"moist1\":%d,\"moist2\":%d,"
"\"percent\":%d,"
"\"voltage\":%.3f,"
"\"connected\":true,"
"\"timestamp\":%" PRIu64
"}",
s.temp, s.humi,
s.red, s.green, s.blue, s.ir,
s.moist1, s.moist2, s.percent,
s.voltage,
ts_ms);
if (n < 0 || n >= sizeof(json)) {
    ESP_LOGW("GOLIOTH_TX", "JSON buffer too small");
    return;
}

enum golioth_status rc = golioth_stream_set_async(
    s_client,          
    "sensor_test",                // root path
    GOLIOTH_CONTENT_TYPE_JSON,      
    (const uint8_t*)json,
    strlen(json),
    nullptr, nullptr);               

ESP_LOGI("GOLIOTH_TX", "stream row → %s", golioth_status_to_str(rc));
ESP_LOGI("GOLIOTH_PAYLOAD", "sending JSON: %s", json);

}

Expected Behavior

The data sent from the device should appear in the LightDB Stream tab

Actual Behavior

Nothing shows up in LightDB Stream, ever.
Even after multiple successful logs and stream sends (GOLIOTH_OK).
Pipeline is running and assigned to the device, but data seems silently dropped.

Impact

This completely blocks our telemetry pipeline.
We can’t visualize or query our device’s historical sensor data — a critical feature for our product.

Environment

Device: ESP32-C3

ESP-IDF: v5.1.1

Golioth SDK: golioth-firmware-sdk (latest as of May 2025)

Firmware: Uses golioth_stream_set_async(…) with JSON payload + millisecond timestamps

Logs and Console Output

I (7891) LIGHTDB_TASK: loop 2
I (7891) GOLIOTH_TX: stream row → GOLIOTH_OK
I (7891) GOLIOTH_PAYLOAD: sending JSON: {“temp”:35.50,“humidity”:60.8,“red”:187,“green”:338,“blue”:192,“ir”:120,“moist1”:3817,“moist2”:3818,“percent”:0,“voltage”:0.000,“connected”:true,“timestamp”:7902}

Attempts to Resolve

Confirmed pipeline filter/path matches (sensor_test)

Added “timestamp” field (int64, UNIX ms)

Tried manually sending stream via goliothctl (couldn’t verify due to stream error)

Added logging of payload JSON to confirm structure

Confirmed device is online + assigned to pipeline

3

I have been able to resolve this.

After realizing that my initial payload was too large, and since this was my first time testing LightDB Stream; I decided to start with a minimal test to better understand how it works with pipelines.

I sent a simple “Hello, World” message through the pipeline and successfully saw it appear on the dashboard. I also followed the documentation on pipelines, which helped me get a better grasp of the setup.

From there, I created a minimal send_temperature_only() helper function and gradually started building on top of that.

2 Likes