Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(grafana): add grafana support #137

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

edobry
Copy link
Contributor

@edobry edobry commented Jan 16, 2025

Description

This PR builds on the previously-added Prometheus support to add Grafana support by:

  • adding a grafana_params section to the top-level observability parameter section
  • deploying a Grafana server
  • implementing dashboard provisioning

The grafana module from the ethereum-package package was used as inspiration, but modified to simplify devX by removing support for inline dashboards and improving remote dashboard source support.

Additionally, this PR implements API provisioning using the official grizzly tool, over the existing file-based provisioning approach to simplify the process of keeping Kurtosis Grafana in-sync with hosted Grafana. To this end, a new grafana-dashboards repository has been created, with the intention of tracking extant dashboards in hosted Grafana. This repository is currently set to internal visibility, and must be approved for open-sourcing prior to this PR being merged.

This PR has been tested and successfully deploys a Grafana server with including all dashboards present on our hosted Grafana instance, organized into the same folder structure:
image

Not all dashboards are yet at full parity, but a fair number of them do show data:
image

If you want to try this out locally, add the following snippet to your params file:

optimism_package:
  observability:
    grafana_params:
      dashboard_sources:
        - github.com/edobry/grafana-dashboards/resources@1a027431c2c1ba98a17963710896fcf5c40b346d

Note that dashboard_sources is currently pointed at https://github.com/edobry/grafana-dashboards as a temporary measure until I get admin approval to publicize the ethereum-optimism repo.

Remaining work:

  • continue converging metrics to enable additional dashboards
  • implement promtail/loki support to enable log-based dashboard panels
  • deploy any services required for certain dashboards (ie replica-healthcheck) (?)
  • support a subset of existing dashboards using tags/folders
  • automatic updates of the grafana-dashboards repository

Copy link

@axelKingsley axelKingsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't leave an Approval here, since I am very inexperienced with Kurtosis &co, but looking over the premise, this seems to be what we want, a grafana deployment that comes up just after prometheus, and uses some existing dashboard definition.

Left minor comments, but generally excited for this one!

README.md Show resolved Hide resolved
src/util.star Outdated Show resolved Hide resolved
@parithosh
Copy link
Member

is the link https://github.com/ethereum-optimism/grafana-dashboards valid? it returns 404 for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants