Observability Events Issue #1

“What people have the capacity to choose, they have the ability to change.” ― Madeleine Albright

Oct 29, 2023

Hello folks!
This is the first issue for Observability Events monthly digest. Please take a look, share it with your friends, and post your feedback on Discord channel 😏

Featured Observability events

Observer Cluster - Monitoring Kubernetes
https://www.meetup.com/open-operations-meetup/events/296552704
Monday, October 30, 2023 at 4:00 PM to Monday, October 30, 2023 at 5:00 PM EET
🌏Virtual conference
ObservabilityCon by Grafana Labs
https://grafana.com/about/events/observabilitycon/2023/
Tuesday , November 14-15, 2023 | London, UK
📍In person conference
Elasticon EU:
https://www.elastic.co/events/elasticon/frankfurt
Frankfurt, November 7, 2023
📍In person conference
https://www.elastic.co/events/elasticon/amsterdam
Amsterdam, November 21, 2023
📍In person conference
https://www.elastic.co/events/elasticon/tel-aviv
Tel Aviv, November 27, 2023
📍In person conference
https://www.elastic.co/events/elasticon/london
London, February 13, 2024
📍In person conference

Elasticon USA:
https://www.elastic.co/events/elasticon/toronto
Toronto, March 6, 2024
📍In person conference

https://www.elastic.co/events/elasticon/chicago
Chicago, March 12, 2024
📍In person conference

https://www.elastic.co/events/elasticon/new-york-city
New York City, March 26, 2024
📍In person conference

Elasticon Asia-Pacific:
https://www.elastic.co/events/elasticon/bengaluru
Bengaluru, November 22, 2023
📍In person conference

https://www.elastic.co/events/elasticon/sydney
Sydney, February 20, 2024
📍In person conference

https://www.elastic.co/events/elasticon/singapore
Singapur, March 5, 2024
📍In person conference
Open conference
https://www.open-conf.gr/
Friday, November 10-11, 2023 | Athens, GR
📍In person conference
Open Source Monitoring Conference
https://osmc.de/
Tuesday, November 7-9, 2023 | NUREMBERG, GERMANY
📍In person conference
KubeCon NA
https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america
Monday, November 6-9, 2023 | CHICAGO, ILLINOIS
📍In person conference
Istio Day
https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/co-located-events/istio-day/
Monday, November 6, 2023 | CHICAGO, ILLINOIS
📍In person conference
Open Source Summit Japan 2023
https://linuxfoundation.smapply.io/prog/oss_japan_2023
DECEMBER 5-6, 2023 | TOKYO, JAPAN
The premier event for open source developers, technologists and community leaders to collaborate, share information, solve problems and gain knowledge, furthering open source innovation and ensuring a sustainable open source ecosystem. It is the gathering place for open source code and community contributors.
📍In person conference
Voxxed Days CERN 2024
https://cern.voxxeddays.com/
Monday, January 22-23, 2023 | Meyrin, Switzerland
Voxxed Days CERN returns for a third edition on 22nd & 23rd January, 2024. This 2 day conference offers you the chance to visit one of the world’s most respected centres of scientific research, where you can meet fellow developers, share knowledge and listen to fascinating ideas from some of the most respected speakers of the sector.
📍In person conference

Reading corner:

“Don’t sleep on the basics. Someone probably solved your problem in the 80s”
Great post about using AWK and R(programming language) to parse 25 TB of log data when popular services like AWS Athena or Parquet didn't help to achieve this job.
https://livefreeordichotomize.com/posts/2019-06-04-using-awk-and-r-to-parse-25tb/index.html
You're overpaying for OpenTelemetry's verbosity by at least 30%!
https://coroot.com/blog/you-are-overpaying-for-opentelemetry-verbosity
Railway shared their journey of adopting Clickhouse as storage for 100 billion logs. They moved from a local filesystem and Google Cloud storage to add support for filtering logs, aggregate metrics queries (count, group by) and solve the problem to scale horizontally. After the first proof-of-concept, Vector, a log ingestion tool was configured to write to Clickhouse, and the UI queries were updated to match the new SQL format. Interesting read, especially if you plan to build your Observability storage backend. More insights in this Twitter/X thread.
An overview of cost monitoring in k8s using VictoriaMetrics and OpenCost
Measuring Git performance with OpenTelemetry
https://github.blog/2023-10-16-measuring-git-performance-with-opentelemetry
Getting Started with 🐝eBPF: Monitoring TCP Retransmissions Using eBPF, Go and Prometheus
https://www.israelo.io/blog/ebpf-net-viz
Julio Merion from Snowflake describes the hard work of profiling their new “build farm service” based on Build Barn.
Monitoring vs Observability: What Engineers Need to Know
v1.0 of the Observability whitepaper, issued by the CloudNativeFoundation's technical advisory group for O11y.
Monitor Amazon EKS Control Plane metrics using AWS Open Source monitoring services

Available for iOS and Android

Interesting projects:

Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe performance counters and other system stats, use ML models to estimate workload energy consumption based on these stats, and exports them as Prometheus metrics.
https://github.com/sustainable-computing-io/kepler
🧑‍🚒 Benchmark compares VictoriaLogs with ELK stack and Grafana Loki/
Project “promcolor” that colorize piped Prometheus metrics in the terminal
there is also a browser extension which makes plain Prometheus/OpenMetrics endpoints easier to read.
Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors.
https://github.com/hyperdxio/hyperdx
Grafana Plugin for VictoriaMetrics and enhancements for Grafana dashboards provided at PromCon EU 2023 by Roman Khavronenko.
Autometrics is an observability micro-framework built for developers. It makes it easy to instrument any function with the most useful metrics: request rate, error rate, and latency. Autometrics uses instrumented function names to generate Prometheus queries so you don’t need to hand-write complicated PromQL.
Effortless, Low-Overhead, 🐝 eBPF-based Kubernetes MonitoringAlaz is an open-source Ddosify eBPF agent that can inspect and collect Kubernetes (K8s) service traffic without the need for code instrumentation, sidecars, or service restarts. This is possible due to its use of eBPF technology. Alaz can create a Service Map that helps identify golden signals and problems like high latencies, 5xx errors, zombie services, SQL queries. Additionally, it can gather system information and resources via the Prometheus Node Exporter, which is readily available on the agent.
https://github.com/ddosify/alaz
New Perses updates from PromCon, an OSS dashboard builder project with a GitOps-first approach, a CNCF alternative for Grafana.
https://github.com/perses/perses
Distributed tracing without code changes. 🚀 Instantly monitor any application using OpenTelemetry and 🐝 eBPF.
https://github.com/keyval-dev/odigos

Interesting discussions:

“After using OpenTelemetry, I understand why DataDog can bill like this”

https://twitter.com/JustJake/status/1715395828620407068

https://twitter.com/lorenc_dan/status/1714984999739433074

“lol there's a CVE in one of the pre-1.0 otel go libraries now, RIP to everyone's go.mod's. CVE-2023-45142”

”Frankly if Otel had a single go package instead of like 10-15 different ones, it would have been much more manageable.”

To watch:

PromCon EU 2023
Day 1 Live - https://www.youtube.com/watch?v=pKYhMTJgJUU
Day 2 Live - https://www.youtube.com/watch?v=ymR57Q0qqg4
ClickHouse Academy - How to sessions, YouTube playlist
https://www.youtube.com/playlist?list=PL0Z2YDlm0b3gtIdcZI3B_8bMJclDOvY8s
Let's Profile OpenTelemetry Collector
https://youtu.be/vkMQRjiNTHM

Thanks for reading Observability digest! Subscribe for free to receive new posts and support my work.

Observability digest