[[log-correlation]]
== Log Correlation

The Go agent provides integrations for popular logging frameworks, which take care of
injecting trace ID fields into your application's log records. You can find a list of
the supported logging frameworks under <<supported-tech-logging, supported technologies>>.

If your favorite logging framework is not already supported, there are two other options:

* Open a feature request, or contribute code, for additional support, as described in <<contributing>>.
* Manually inject trace IDs into log records, as described below in <<log-correlation-manual>>.

Regardless of how you integrate APM with logging, you can use {filebeat-ref}[Filebeat] to
send your logs to Elasticsearch, in order to correlate your traces and logs and link from
APM to the {apm-app-ref}/xpack-logs.html[Logs app].

[float]
[[log-correlation-manual]]
=== Manual log correlation

If the agent-provided logging integrations are not suitable or not available for your
application, then you can use the agent's <<api, API>> to inject trace IDs manually.
There are two main approaches you can take, depending on whether you are using structured
or unstructured logging.

[float]
[[log-correlation-manual-structured]]
==== Manual log correlation (structured)

For correlating structured logs with traces, the following fields should be added to your logs:

 - `trace.id`
 - `transaction.id`
 - `span.id` (if logging in the context of a span)

Given a transaction object, you can obtain its trace ID and transaction ID using
the <<transaction-tracecontext, apm.Transaction.TraceContext>> method. Similarly,
given a span object, you can obtain its span ID using <<span-tracecontext, apm.Span.TraceContext>>.

If you use the context APIs to start transactions and spans, then you can obtain
the context's current transaction using <<apm-transaction-from-context, apm.TransactionFromContext>>,
and current span using <<apm-span-from-context, apm.SpanFromContext>>. Note that if
a transaction is not sampled, `apm.TransactionFromContext` will return `nil`.
Similarly, spans may be dropped by the agent, so `apm.SpanFromContext` may also return `nil`.

[source,go]
----
labels := make(map[string]string)
tx := apm.TransactionFromContext(ctx)
if tx != nil {
	traceContext := tx.TraceContext()
	labels["trace.id"] = traceContext.Trace.String()
	labels["transaction.id"] = traceContext.Span.String()
	if span := apm.SpanFromContext(ctx); span != nil {
		labels["span.id"] = span.TraceContext().Span
	}
}
----

You can follow this article in order to ingest JSON-encoded logs with Filebeat:
{blog-ref}how-to-instrument-your-go-app-with-the-elastic-apm-go-agent#logs[How to instrument your Go app with the Elastic APM Go agent].

[float]
[[log-correlation-manual-unstructured]]
==== Manual log correlation (unstructured)

For correlating unstructured logs (e.g. basic printf-style logging, like the standard library's
`log` package), then you will need to need to include the trace IDs in your log message, and then
extract them using Filebeat.

If you already have a transaction and/or span object, then you can use the
<<transaction-tracecontext, Transaction.TraceContext>> and <<span-tracecontext, Span.TraceContext>>
methods. The trace, transaction, and span ID types all provide `String` methods that yield
their canonical hex-encoded string representation.

[source,go]
----
traceContext := tx.TraceContext()
spanID := span.TraceContext().Span
log.Printf("ERROR [trace.id=%s transaction.id=%s span.id=%s] an error occurred", traceContext.Trace, traceContext.Span, spanID)
----


If instead you are dealing with context objects, you may prefer to use the
<<apm-traceformatter, TraceFormatter>> function. For example, you could supply it as an argument
to `log.Printf` as follows:

[source,go]
----
log.Printf("ERROR [%+v] an error occurred", apm.TraceFormatter(ctx))
----

This would print a log message along the lines of:

    2019/09/17 14:48:02 ERROR [trace.id=cd04f33b9c0c35ae8abe77e799f126b7 transaction.id=cd04f33b9c0c35ae span.id=960834f4538880a4] an error occurred

For log correlation to work, the trace IDs must be extracted from the log message and
stored in separate fields in the Elasticsearch document. This can be achieved by
{filebeat-ref}/configuring-ingest-node.html[parsing the data by using ingest node], in particular
by using {ref}/grok-processor.html[the grok processor].

[source,json]
----
{
  "description": "...",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": ["%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{TIME} %{LOGLEVEL:log.level} \\[trace.id=%{TRACE_ID:trace.id}(?: transaction.id=%{SPAN_ID:transaction.id})?(?: span.id=%{SPAN_ID:span.id})?\\] %{GREEDYDATA:message}"],
        "pattern_definitions": {
          "TRACE_ID": "[0-9A-Fa-f]{32}",
          "SPAN_ID": "[0-9A-Fa-f]{16}"
        }
      }
    }
  ]
}
----
