<#import "/templates/guide.adoc" as tmpl>
<#import "/templates/kc.adoc" as kc>
<#import "/templates/options.adoc" as opts>
<#import "/templates/links.adoc" as links>
<#import "/templates/profile.adoc" as profile>
<#import "/templates/features.adoc" as features>

<@tmpl.guide
title="Centralize your observability stack with OpenTelemetry"
summary="Learn about OpenTelemetry integration for centralized observability and telemetry data.">

https://opentelemetry.io/docs/what-is-opentelemetry/[OpenTelemetry] (OTel) is an open-source observability framework designed to provide consistent and reliable ways to collect telemetry data across distributed systems.
It is a project under the https://www.cncf.io/projects/opentelemetry/[Cloud Native Computing Foundation (CNCF)] as an incubating project.
Its goal is to provide a unified standard and toolset for instrumenting, generating, and exporting telemetry data from modern applications, making it easier to monitor, debug, and understand complex services.

The goal of {project_name} is to support OpenTelemetry Logs, Metrics, and Traces to create a centralized observability stack without any additional deployment overhead.

== General options
{project_name} provides support for global settings of telemetry data export.
It means that you can configure connection parameters to a single https://opentelemetry.io/docs/collector/[OpenTelemetry collector] that will be used for all supported components.

In order to have OpenTelemetry enabled, at least one component needs to be turned on, for example <<traces, Traces>>.

NOTE: The OpenTelemetry feature is _enabled_ by default. Disable the feature in order to disable OpenTelemetry components.

=== Endpoint
You can configure the OpenTelemetry collector's global endpoint that is shared across all components by using the `telemetry-endpoint` option.
The default value is `+http://localhost:4317+`.

You can change the endpoint via CLI as follows:

<@kc.start parameters="--telemetry-endpoint=http://my-otel-endpoint:4317"/>

=== Service name
You can configure the global OpenTelemetry service name used for identifying the exporter of the telemetry data.
The default service name is `keycloak`, specified via the `telemetry-service-name` property, which takes precedence over `service.name` defined in the `telemetry-resource-attributes` property.

You can change the service name via CLI as follows:

<@kc.start parameters="--telemetry-service-name=my-keycloak-iam"/>

=== Protocol
You can configure the global OpenTelemetry transport protocol used as a communication channel between {project_name} and the OpenTelemetry collector.
The default value is `grpc`.

{project_name} supports these communication protocols:

<@opts.expectedValues option="telemetry-protocol"/>

You can change the protocol via CLI as follows:

<@kc.start parameters="--telemetry-protocol=http/protobuf"/>

== Traces

See the https://www.keycloak.org/observability/tracing[Root cause analysis with tracing guide].

You can override the global OpenTelemetry settings via the Tracing options if you want to export traces to a different OpenTelemetry collector.

== Logs

<@features.techpreview feature="opentelemetry-logs"/>

NOTE: The OpenTelemetry feature (`opentelemetry`) needs to be turned on (by default).

It is possible to export {project_name}'s logs to the OpenTelemetry collector and be managed by various logging backends that support OpenTelemetry.

=== Enable Logs
You can enable OpenTelemetry Logs via CLI as follows:

<@kc.start parameters="--features=opentelemetry-logs --telemetry-logs-enabled=true"/>

For more information on how to set up logging, see the https://www.keycloak.org/server/logging[Configuring Logging guide].

=== Log level
By default, all log levels are exported by {project_name}.
You can optionally configure the most verbose log level to be exported via the `telemetry-logs-level` option.

For example, if you want only to export `WARN` and `ERROR` logs, you can change the log level via the CLI as follows:

<@kc.start parameters="--telemetry-logs-level=WARN"/>

<@profile.ifCommunity>

== Development setup

For development purposes, you can use the https://github.com/grafana/docker-otel-lgtm[Grafana OTel-LGTM service], containing OpenTelemetry Collector and backends for logs (Loki), metrics (Prometheus), and traces (Tempo).

.Service architecture
image::observability/grafana-otel-lgtm.png[]

You can start the service by using Docker/Podman as follows:

[source,shell]
----
docker run -p 3000:3000 -p 4317:4317 -p 4318:4318 --rm -ti grafana/otel-lgtm
----

Then, you can navigate to Grafana UI by accessing `+localhost:3000+` and then you can explore all the data sent to OpenTelemetry Collector.

</@profile.ifCommunity>

<@opts.printRelevantOptions includedOptions="telemetry-*" excludedOptions="telemetry-logs-*">

=== Traces
<@opts.includeOptions includedOptions="tracing-enabled tracing-endpoint tracing-protocol tracing-service-name tracing-resource-attributes"/>

=== Logs
<@opts.includeOptions includedOptions="telemetry-logs-*"/>

</@opts.printRelevantOptions>

</@tmpl.guide>
