Loki and Elasticsearch

Loki and Elasticsearch can both store and query logs, but they make different tradeoffs. Elasticsearch indexes the contents of each document, enabling fast and flexible full-text search. Loki primarily indexes labels and leaves the log contents compressed in chunks, reducing indexing cost while encouraging queries that first select a small set of log streams.

This article explains Loki, introduces LogQL, and compares Loki with Elasticsearch as a log platform.

Where Loki Fits #

An observability system commonly works with three kinds of telemetry:

• Metrics describe aggregate numeric behavior. Prometheus stores and queries them.
• Logs record discrete events. Loki or Elasticsearch can store and query them.
• Traces follow requests across services. Tempo, Jaeger, and similar systems store and query them.

Grafana can display all three. A dashboard might use Prometheus for an error-rate graph, Loki for the corresponding error messages, and Tempo for traces of the failed requests.

Loki is often described as “Prometheus for logs.” This does not mean it stores logs as Prometheus metrics. It means Loki organizes logs into labeled streams and uses a query language whose selection and aggregation model resembles PromQL.

How Loki Organizes Logs #

A Loki log stream is identified by a set of labels:

{cluster="production", namespace="shop", app="checkout"}

Every log line carrying exactly that label set belongs to the same stream:

2026-07-17T05:00:01Z request completed status=200 duration=42ms
2026-07-17T05:00:02Z payment provider timed out status=503

Loki indexes the labels and timestamps needed to locate the stream’s chunks. It does not normally build a full-text index over every word in every line. During a query, Loki selects streams through their labels and then scans the matching chunks for text or parsed fields.

This design has two important consequences:

1. Storage and indexing can be considerably cheaper than a full-text search engine for large log volumes.
2. Queries work best when selective labels narrow the search before log content is inspected.

Loki Components #

A production Loki deployment can separate several responsibilities:

• Distributor: Receives log entries, validates them, and sends them to ingesters.
• Ingester: Builds compressed chunks from incoming streams and persists them.
• Query frontend and querier: Split, schedule, cache, and execute queries.
• Object storage: Stores durable log chunks and index data.
• Compactor: Maintains index files and applies retention or deletion rules.

Loki can also run as a single process for local development or small installations. Larger installations split these responsibilities so ingestion and querying can scale independently.

Loki itself does not discover and read application log files. Logs must be shipped to it by an agent. Common choices include Grafana Alloy, Fluent Bit, Vector, and the OpenTelemetry Collector.

Choosing Labels #

Good Loki labels describe bounded infrastructure dimensions:

cluster
region
namespace
application
environment
container
log_level

Avoid labels with numerous or unbounded values:

request_id
trace_id
user_id
email
timestamp
raw_url

Every unique label set creates another stream. High-cardinality labels therefore produce many small streams, enlarge the index, increase memory use, and make ingestion and queries less efficient.

Values such as request IDs and user IDs can remain in the log body or be attached as structured metadata. They can then be filtered after the query has selected streams using stable labels.

The distinction is similar to Prometheus: labels should identify useful, bounded dimensions, not individual events.

Querying Logs with LogQL #

LogQL queries begin with a stream selector:

{cluster="production", app="checkout"}

Label matchers use the same operators as PromQL:

{app="checkout"}                # Equal
{app!="checkout"}               # Not equal
{app=~"checkout|payments"}      # Regular expression match
{namespace!~"test-.*"}          # Negative regular expression match

After selecting streams, add a line filter:

{app="checkout"} |= "timeout"

The available text filters include:

{app="checkout"} |= "timeout"   # Contains text
{app="checkout"} != "health"    # Does not contain text
{app="checkout"} |~ "5[0-9]{2}" # Matches a regular expression
{app="checkout"} !~ "2[0-9]{2}" # Does not match a regular expression

Put inexpensive, selective filters early in the pipeline so later stages process fewer lines:

{app="checkout"} |= "payment" != "health"

Parsing Structured Logs #

Applications should preferably emit structured logs. LogQL can parse JSON:

{app="checkout"}
  | json
  | status >= 500

Given this line:

{"level":"error","route":"/checkout","status":503,"duration_ms":1200}

the json stage extracts fields that later pipeline stages can filter or format:

{app="checkout"}
  | json
  | status >= 500
  | line_format "{{.status}} {{.route}} took {{.duration_ms}} ms"

For logfmt data, use the logfmt parser:

{app="checkout"}
  | logfmt
  | level="error"

For unstructured text, a pattern expression is often clearer and faster to maintain than a regular expression:

{app="nginx"}
  | pattern `<ip> - - [<_>] "<method> <uri> <_>" <status> <size>`
  | status >= 500

Parsed fields are query-time values. They do not become permanent indexed labels merely because a query extracts them.

Deriving Metrics from Logs #

LogQL can turn matching log lines into metric values. Count error lines per second:

sum by (app) (
  rate({namespace="shop"} |= "error" [5m])
)

Count the number of errors during a period:

sum by (app) (
  count_over_time({namespace="shop"} |= "error" [1h])
)

Structured numeric fields can also become samples. Calculate the 95th percentile of request duration from JSON logs:

quantile_over_time(
  0.95,
  {app="checkout"}
    | json
    | unwrap duration_ms
    | __error__="" [5m]
)

The unwrap stage turns an extracted field into numeric samples. The __error__="" filter excludes lines that could not be parsed or converted.

Log-based metrics are useful when an application does not expose an equivalent metric. However, frequently used service-level indicators should normally be instrumented directly as Prometheus metrics. Metrics are cheaper to query and do not depend on parsing every matching log line.

Connecting Logs, Metrics, and Traces #

Consistent metadata makes telemetry systems work together. Prometheus metrics, Loki logs, and traces should use compatible names for fields such as service, cluster, namespace, and environment.

Grafana can then provide links between data sources:

• A Prometheus graph can link to Loki logs for the same service and time range.
• A log containing a trace ID can link to the corresponding trace.
• A trace can link back to logs for the participating service.

Elasticsearch Data Model #

Elasticsearch stores JSON documents in indices. An index is divided into primary shards, and each primary can have replica shards. A document is routed to one primary shard; replicas provide redundant copies and can serve searches. Primary and replica counts affect capacity and recovery, so they should be selected from expected data volume and availability requirements, rather than from the number of nodes currently available.

Each field has a mapping. A string field often needs two representations:

{
  "message": {
    "type": "text",
    "fields": {
      "keyword": { "type": "keyword", "ignore_above": 256 }
    }
  }
}

The text field is analyzed into terms for full-text search. Its keyword multi-field preserves one value for exact filtering, sorting, and aggregations. Analysis applies both when indexing and when a query targets that text field; use an explicitly selected analyzer when language-specific tokenization is required.

Mappings should be defined before production ingestion. Dynamic mapping can create incompatible field types when one producer sends a number and another sends a string. Index templates provide a repeatable mapping, settings, and index pattern for time-based data streams.

Elasticsearch Queries and Operations #

Use the bool query to separate scoring clauses from filters:

{
  "query": {
    "bool": {
      "filter": [
        { "term": { "service.keyword": "checkout" } },
        { "range": { "@timestamp": { "gte": "now-15m" } } }
      ],
      "must": [{ "match": { "message": "payment timeout" } }]
    }
  }
}

filter clauses do not calculate relevance scores and are cacheable in appropriate contexts. match analyzes full-text input, whereas term matches the supplied term exactly and therefore belongs with keyword, numeric, boolean, date, and similarly exact fields.

For log retention, use data streams with Index Lifecycle Management. A policy can roll over a backing index by size, age, or document count, move it through data tiers, and delete it at the retention boundary. Snapshots to a registered repository are the supported backup mechanism; replicas do not protect against accidental deletion or a cluster-wide fault.

Elasticsearch is the better fit when search requires arbitrary text retrieval, complex aggregations, or document-centric analysis. Loki is the better fit when log streams have stable, selective labels and operational queries can narrow them before scanning log content.

Each backend remains optimized for its data model while Grafana can provide a common interface.

Loki Compared with Elasticsearch #

Neither design is universally better. Loki spends less work indexing log content and may scan more data during a content query. Elasticsearch spends more resources while ingesting and indexing documents so later searches can use those indexes.

When to Choose Loki #

Loki is a strong choice when:

• Logs are primarily used for application and infrastructure observability.
• Operators usually begin with a service, cluster, namespace, and time range.
• Prometheus and Grafana are already central to the monitoring stack.
• Large log volumes make full-text indexing unnecessarily expensive.
• Object storage is preferred for durable, scalable retention.
• Correlation among metrics, logs, and traces is more important than unrestricted document search.

Loki is less attractive when most investigations begin with an arbitrary term across all historical logs and no useful labels are known. Such queries may require scanning a large amount of log data.

When to Choose Elasticsearch #

Elasticsearch is a strong choice when:

• Fast, flexible full-text search is a primary requirement.
• Users need rich searches across many indexed fields.
• Logs support security analytics, compliance investigations, or other document-oriented workloads.
• Existing applications already rely on Elasticsearch and Kibana.
• The organization can operate and tune index mappings, shards, lifecycle policies, and data tiers.

Its flexibility has a cost. Indexing consumes CPU, memory, and storage, and poorly controlled dynamic fields can create mapping and performance problems. Shard sizing and lifecycle management also require deliberate operational planning.

They Can Coexist #

Adopting Loki does not require replacing Elasticsearch. An organization might:

• Send routine application logs to Loki for economical retention and Grafana correlation.
• Keep security or audit events in Elasticsearch for indexed investigation.
• Route a short-retention copy to Loki and a curated subset to Elasticsearch.
• Retain an existing Elasticsearch deployment while using Loki for new Kubernetes workloads.

The right boundary depends on how logs are queried, not merely how many logs are generated.

Practical Design Guidelines #

1. Emit structured JSON or logfmt instead of inventing an unstructured format.
2. Use stable, bounded Loki labels such as service, environment, and namespace.
3. Keep request IDs, trace IDs, user IDs, and raw URLs out of labels.
4. Preserve the original event timestamp and use synchronized clocks.
5. Avoid logging secrets, credentials, or unnecessary personal data.
6. Set retention from operational and compliance needs rather than keeping all logs indefinitely.
7. Instrument important rates and latency distributions as Prometheus metrics instead of repeatedly deriving them from logs.
8. Test representative queries before choosing a backend. Ingestion cost alone does not reveal whether the system will satisfy investigation workflows.

Loki treats logs as compressed streams selected by labels, whereas Elasticsearch treats logs as searchable documents. Choose Loki when observability-oriented stream selection and economical storage fit the workload. Choose Elasticsearch when indexed document and full-text search justify the additional ingestion and operational cost.

References #

• Grafana Loki Documentation
• LogQL Reference
• Loki Storage
• Elasticsearch Mapping
• Elasticsearch Data Streams
• Index Lifecycle Management