Skip to main content

Road to 1.0

· 4 min read
David Delassus
David Delassus
Co-creator of FlowG

🎉 FlowG v0.54.0 has been released, finalizing the MVP milestone of our roadmap.

Minimum Viable Product​

For the MVP, we wanted to have at least the following features:

  • ability to ingest and store logs
  • interoperability with many third party services, done via Forwarders
  • basis of clustering with the most common cluster formation strategies

After months of work, this is now done. Special thanks to our main contributors, without who we would not be here today:

FlowG has all you need to make sense of the logs of your production environments, and we're quite active on the bug tracker, so don't hesitate to ask any question, or report eventual problems.

There is still a lot of work to be done, and it's all going towards the 1.0, first "stable" version.

Road to 1.0​

What we call "stable" here means that the API will be locked down, no breaking change guaranteed.

We have 2 big projects for this, and probably a few more down the line that are not planned yet, or not even fleshed out yet:

Frontend Redesign​

Historically, the frontend was made with Templ, HTMX, and Tailwind. Due to the difficulties of handling the amount of client-state we had, and keeping the API and UI features in sync, we decided to migrate to a React SPA with React MUI.

Unfortunately, lots of Tailwind code remained, and the result is a mix of 2 conflicting design systems, leading to some visual glitches and inconsistencies that are hard to debug and/or fix.

A proper redesign is waranted:

  • removing Tailwind and fully adopting MUI
  • reorganizing the code to be more consistent and following best practices
  • documenting the architecture to facilitate onboarding new contributors

Over the next few months, this work is gonna be done by @coyote2190, our most recent contributor. Special thanks to him as well!

Replication​

We currently have an experimental implementation of the replication layer, allowing FlowG to be a highly available service.

This implementation is buggy, costly in terms of performance, and wrong on many levels.

A complete redesign is also planned for the next few months. We'll keep using the SWIM Protocol, via Hashicorp Memberlist Go package, for the automatic cluster formation. But instead of relying on the costly "TCP Push/Pull" mechanism to synchronize storages, we'll implement a proper Operation Log, and synchronize changes via a custom workflow, separate from the SWIM Protocol.

As for the replication model, we still want eventual consistency, that hasn't changed. Only the implementation will change, and be correct.

Correctness will be proven with an exhaustive test suite, trying to ensure that most edge cases are covered.

Testing the chaos​

We want to prove that FlowG can perform well in production environments, where many things can go wrong. To do so, we will continuously improve the test suite and set up test environments, following the principles of Chaos Engineering.

Traces and Metrics with OpenTelemetry?​

At the moment, FlowG only supports logs. The OpenTelemetry integration also only supports those.

But OpenTelemetry also has "metrics" and "traces", which are essential parts of any observability solutions. It is not yet clear if we want that in FlowG or not, there is an ongoing discussion about it on Github, and would appreciate any feedback on this.

Nothing has been fleshed out yet for this. It might not be part of the 1.0 release, which is fine because it does not look like that introducing this feature would be a breaking change.

Conclusion​

The list of contributors keeps growing, and you have all our thanks for that!

Any feedback is welcomed, feel free to join our community either on the Github or on Discord.

If you are using FlowG in production, give us a ping, we'll add you to our README.

To infinity and beyond!