SaladCloud Container Logs just got much better. The team has been hard at work making numerous improvements up and down the stack to deliver a host of new logging features with rock-solid reliability. Let’s dig in.

Improved Reliability

A recent logging degradation event sent us down quite a rabbit-hole, deep into our logging stack. Like many cloud-based and containerized solutions, SaladCloud’s logging is powered by fluentbit, an excellent open-source log processor and forwarder. We were experiencing silent, intermittent log dropping, with no easily discernable pattern. After several late nights and many dead ends and red herrings, the team traced the issue to an interaction between specific versions of the json and stdin plugins for fluentbit. It has since been resolved, with the fix fully deployed across the network, and logs are flowing unobstructed at a rate of tens of thousands of log messages per second.

Log Query API

The new Query Log Entries Endpoint introduces a simple but expressive query language for advanced log filtering. It includes first-class support for JSON structured logs, so you can execute efficient application-specific log queries. It’s not just application logs from your containers, either. You can also query system events, including container reallocations, replica count adjustments, and more. Perhaps most exciting, though, is the ability to query logs and events from across your entire organization with a single request. Many of our customers deploy hundreds or even thousands of container groups, which has historically presented some observability challenges with Salad’s on-platform logging tools. Now, no matter how many projects or container groups you run, you can query across exactly the subset you need. This is a huge quality of life improvement.

To start making the most of this new capability, check out our Logging Best Practices.