FuseQL Best Practices

The fastest FuseQL queries on Kloudfuse touch the least data. Every log line has three layers of structure — indexed stream labels, ingest-time facets, and the raw log body — and the right operator for each layer is different. This page explains the layers, how the query engine evaluates them, and the habits that keep queries fast at scale.

Know the three layers of a log line

Layer Examples How to filter Cost

Indexed stream labels

source, level, kube_namespace, kube_service, pod_name, cluster_name, host, availability_zone, project, docker_image, kube_container_name, instance_id, instance_type

Stream selector or pre-pipe condition: source="nginx" and level="error"

Cheapest — resolved against the label index before any data is read

Ingest-time facets

Fields Kloudfuse extracted when the log arrived: statusCode, endpoint, duration, userId, and any other key your pipeline parses

Facet filter with the @ prefix: @statusCode=200

Cheap — resolved from stored facet columns; no log-body scan

Raw log body

The original message text

Term search: 'error'; grep: "err*"; regex: =~"pattern"

Most expensive — requires scanning the log_line column; term search is index-backed and much faster than grep or regex

Structure every query in that order: select streams by label, narrow by facet, and only then search the remaining text.

Start with the most selective stream selector

The stream selector is evaluated first and determines how much data the rest of the pipeline reads. Name the source and add every indexed label you know:

source="grafana" and level="error"

The full set of indexed labels — resolved as direct column lookups before any row is fetched — is:

source, level, kube_namespace, kube_service, pod_name, cluster_name, host, availability_zone, project, docker_image, kube_container_name, instance_id, instance_type

Two habits keep selectors fast:

  • Prefer = to =~. An equality matcher is a direct columnar lookup; a regex matcher runs RE2 against every label value in the result set.

  • Never search for what a label already states. source="app" and "ERROR" scans every line; source="app" and level="error" reads only the error streams and catches all capitalizations because Kloudfuse normalizes level at ingest.

Avoid match-everything patterns such as source=~".+" — they make the stream selector a no-op and turn the entire time range into the query’s working set.

Filter on ingest-time facets, not on strings

Kloudfuse extracts facets from structured content at ingest and stores them alongside the line. Filter on them directly with the @ prefix — no parser stage is needed:

source="grafana" and @statusCode=200

Numeric comparisons work the same way:

source="grafana" and @statusCode>=500

To compare a duration facet, convert it to milliseconds with toduration() and compare the result. Duration facets are stored as text with a unit, such as 95.046µs, so a direct comparison such as @duration>100ms compares the text, not the time value, and returns wrong results:

source="grafana" and @duration:string
| toduration(@duration) as duration_ms
| where duration_ms > 100

Compare this with the string-search version, source="grafana" and "statusCode=200", which scans every line, breaks if the field order or spacing changes, and also matches lines that merely mention that text.

Keep facet conditions before the first pipe, and always write the @ prefix:

source="grafana" and @statusCode=503 and @endpoint="/api/query"

Two mistakes look valid but return the wrong result:

  • A facet condition after a pipe, such as source="grafana" | where duration > 100ms, does not filter on the facet and returns every line. where after a pipe works on fields that an earlier parse stage extracted; see Reach for a parser only when the facet does not exist.

  • Grouping by a facet also needs the prefix: count by (@statusCode) returns one row per status code, but count by (statusCode) returns a single null group.

When you need to search the raw log body, prefer term search over grep or regex. Term search resolves through the Lucene inverted index that Kloudfuse builds from every log line at ingest — it is an index lookup, not a scan:

'OutOfMemoryError'

Term search is case-insensitive by design and matches whole tokens only. Combine with stream and facet conditions so the index result set is already narrowed:

source="payment" and level="error" and @endpoint="checkout" and 'OutOfMemoryError'

Grep ("pattern") supports wildcards and is a good choice when you need partial-word matching or a prefix:

"conn*"         # matches connection, connect, connected
"*refused"      # matches suffix

Internally, grep with a literal prefix pre-filters candidates through the index before doing per-row work — significantly faster than a pure regex scan. Reserve =~ and !~ for cases where neither term search nor grep expresses the intent.

Reach for a parser only when the facet does not exist

Parsers (parse, parse regex, json, and split) re-extract fields from the raw log body on every line, on every query. That work is wasted when the field is already an ingest-time facet. Use the facet directly:

source="nginx" and @statusCode=503

Use a parser when a field is genuinely not extracted at ingest — a one-off pattern inside the message body, or a format your pipeline does not parse:

source="nginx" | parse regex "(?P<method>[A-Z]+) (?P<path>/[^ ]*)" | where method="POST"

Fields extracted by a parser are filtered with where after the parse stage, as above. Keep parser stages as simple as possible.

Keep metric queries pushdown-friendly

FuseQL pushes metric computation down into the Pinot storage layer when the query shape allows it. Pushed-down queries are dramatically faster than those computed in the engine. Label and facet filters before the first pipe, followed by metric aggregations — count, sum, avg, min, max, and the percentile operators (p50, p95, p99, …) — combined with a timeslice grouping are pushdown-friendly:

source="grafana" | timeslice 5m | count by (_timeslice, level)

For the stages that prevent pushdown, see Pushdown constraints.

Use compose carefully

The compose operator runs a subquery and uses its results as a filter. It has a hard result limit of 10,000 entries (default: 2,500). Exceeding the limit truncates the subquery result silently and may cause the outer query to miss matching rows:

# Safe — subquery returns a small, bounded set
source="grafana" and level="error"
  | count by kube_namespace
  | compose kube_namespace maxresults=500

Keep compose subqueries as selective as possible. If you need to match a large set, consider writing the condition directly as a facet filter instead of using compose.

Bound the scope and the time range

Query cost scales with the data volume selected. The time range is a multiplier on everything above. While iterating on a query, work against a short window — the last 5 or 15 minutes — and widen it only when the query is final.

For dashboards, size the timeslice to the panel’s step rather than defaulting to a fine-grained slice: timeslice 5m on a panel with a 5-minute step returns one point per step, while timeslice 1m returns five times as many points for no visible gain.

Add limit N to log-retrieval queries to cap the result set. The maximum is 10,000 rows; queries that omit a limit use the default and may return more data than needed.

  • FuseQL Limitations — hard row limits, series caps, timeout constraints, and pushdown restrictions.

  • Text search operators — the full term search and grep reference: tokens, wildcards, field-scoped search.

  • Predicate operators — label and facet filter reference: string, numeric, duration, and regex comparisons.

  • LogQL Best Practices — LogQL-specific guidance for queries using the Loki-compatible syntax.