Despite the 14-day log retention in most plans, it appears that only the first 250 log messages are available thorough the web management interface.
Is there a way to access the older logs?
Hey Eric - thanks for reporting this! Sounds like it might be a pagination issue, we’re looking into this and will respond back once we have more information.
Hi @EricNRS! Currently the console will return the first 250 logs within the provided time window (and with the provided filters if viewing logs at the project level). Logs within the time window that exceed the first 250 can be accessed by shifting the time window and / or constraining it more narrowly.
We are looking at improving this functionality and welcome your feedback!
Hi @hasheddan
Logs within the time window that exceed the first 250 can be accessed by shifting the time window and / or constraining it more narrowly.
This is a tedious work-around requiring 6 clicks to manually shift the time and mentally remembering the last bisection point when trying to find out the time of a particular event. In addition, since the time selector only has a resolution of minutes, this approach will fail for the case where there are more than 250 logs in a minute.
Instead of doing the equivalent of select * from logs order by timestamp limit 250 and then splitting those across the local UI logs-per-page setting, you may want to consider retrieving the exact log count for one page and then create links for the additional pages (+/- 5 pages from the current location would be a good option). This is just the equivalent of select * from logs order by timestamp limit LOGS_PER_PAGE OFFSET PAGE_NUMBER*LOGS_PER_PAGE query.
The exact query obviously needs to be tailored to the query language of your backend data store, but limit/offset type queries are normally universal across most type of databases and datastores.
If you don’t have the ability to change the pagination, then you could add next and previous links to the time selector, but instead of a fixed 250 log pages, you need to increase that to the maximum log entries per second to ensure you don’t drop anything.
@EricNRS thanks for the feedback! We are working on improving the functionality for querying logs and will make sure to post updates when available.
Thanks @hasheddan. To summarize, here is the use case where the current UI falls down IMHO:
- Normal logging on the device
- A single failure is triggered
- A high rate of repetitive log messages caused by the failure occurs
Right now, the user has to manually shift the time as you suggested to find the failure in step 2 since there are many pages of the repetitive log messages.
1 Like