Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 37 ยท Data Sources

The Loki Data Source and LogQL Basics

Connect Grafana to Loki and learn LogQL: stream selectors, line filters, parsers, label filters, metric queries like rate and count_over_time, and the Logs panel.

Loki stores log lines cheaply by indexing only their labels, not their text. You search by first choosing streams with labels, then filtering the text. Its query language, LogQL, deliberately resembles PromQL, so it feels familiar once you know the Prometheus data source.

Loki does not run on this site. LogQL appears in read-only blocks. The runnable examples are stand-ins that filter and count sample log lines in JavaScript so you can see what each step does.

Connecting

Add Loki under Connections. The URL is typically http://loki:3100. Important settings:

  • Maximum lines: how many lines a Logs panel may fetch (default 1000).
  • Derived fields: regular expressions that turn a piece of the line (such as trace_id=abc) into a link to another data source (Tempo) or URL.
  • Headers: X-Scope-OrgID when Loki runs multi-tenant.

Log queries

Every LogQL query starts with a stream selector in braces:

TEXT
{app="shop"}
{app="shop", env="prod"}
{namespace=~"prod-.+", level!="debug"}

Then add a pipeline of stages, left to right:

TEXT
{app="shop"} |= "error"             # line contains "error"
{app="shop"} != "healthcheck"       # line does not contain it
{app="shop"} |~ "timeout|refused"   # regex match
{app="shop"} !~ "GET /static"       # regex not match

Parsers

Parsers extract labels from the text so you can filter and aggregate on them:

TEXT
{app="shop"} | json                                   # for JSON log lines
{app="shop"} | logfmt                                 # key=value lines
{app="shop"} | pattern "<ip> - - <_> \"<method> <uri> <_>\" <status> <size>"
{app="shop"} | regexp "(?P<status>\\d{3})"

After parsing, filter on the extracted labels:

TEXT
{app="shop"} | json | status >= 500 | duration > 2s
{app="shop"} | logfmt | level="error" | line_format "{{.method}} {{.path}} {{.status}}"

Metric queries from logs

Wrapping a log query in a range function gives numbers you can graph or alert on:

TEXT
count_over_time({app="shop"} |= "error" [5m])
rate({app="shop"} |= "error" [5m])
sum by (status) (rate({app="shop"} | json [5m]))
sum(rate({app="shop"} | json | status >= 500 [5m]))
  / sum(rate({app="shop"} | json [5m]))

Use [$__auto] or [$__interval] in Grafana to match the panel's resolution.

Stand-in: stream selector and line filters

The stand-in holds sample lines with labels and applies the same steps: select stream, filter text, parse key=value.

JS Stand-in: a mini log pipeline

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Stand-in: count_over_time

This computes a count per time bucket, the heart of count_over_time(... [5m]).

JS Stand-in: count log lines per window

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The Logs panel and Explore

In Explore, Loki shows a Logs volume histogram, log lines with level colors, a Live tail button, and Fields detected by parsers. Click a line to expand labels and the Log context (lines before and after). A Logs panel on a dashboard displays the same lines; options include Time, Unique labels, Wrap lines, Prettify JSON, Order and Deduplication.

Performance tips

  • Narrow labels first. {app="shop", env="prod"} is much faster than {env="prod"} |= "shop".
  • Put cheap filters (|=) before parsers (| json).
  • Use a short time range while exploring.
  • Do not use user IDs, URLs or other unbounded values as labels (see the ecosystem lesson).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does Loki index?

    Loki indexes labels; the line text is stored compressed and filtered at query time.
  2. 2. Which operator keeps lines that match a regular expression?

    |~ matches a regex, |= a plain substring; != and !~ exclude.
  3. 3. Which parser reads JSON log lines into labels?

    | json extracts fields; logfmt does the same for key=value lines.
  4. 4. What turns a log query into a number you can graph?

    Wrap the log query in a range aggregation to get a metric.

Summary

  • Loki queries start with a label selector, then a pipeline of filters and parsers.
  • |=, !=, |~, !~ filter text; | json and | logfmt extract fields.
  • rate and count_over_time turn logs into metrics.
  • Derived fields link log lines to traces.
  • Keep labels low-cardinality and selectors narrow.

Next: SQL Data Sources: MySQL and PostgreSQL