Filter — Sails [Sails](https://sails.app/) / [Guides](/guides/) [Contact](https://sails.app/contact) On this site [Overview](/guides/) How to * [All how-tos](/guides/how-to) * [Publish a site](/guides/how-to/publish-a-site) * [Check a site before you publish](/guides/how-to/check-a-site-before-you-publish) * [Give an agent a job](/guides/how-to/give-an-agent-a-job) * [Mail from an agent](/guides/how-to/mail-from-an-agent) * [Talk to an agent in Slack](/guides/how-to/talk-to-an-agent-in-slack) * [Message another agent](/guides/how-to/message-another-agent) * [Run something every morning](/guides/how-to/run-something-every-morning) * [Put a form on a site](/guides/how-to/put-a-form-on-a-site) * [Send a file](/guides/how-to/send-a-file) * [Store a secret](/guides/how-to/store-a-secret) * [Keep app data in a database](/guides/how-to/keep-app-data-in-a-database) * [Filter JSON files](/guides/how-to/filter-json-files) * [Start a goal from a webhook](/guides/how-to/start-a-goal-from-a-webhook) * [Connect an MCP server](/guides/how-to/connect-an-mcp-server) * [Sync with GitHub](/guides/how-to/sync-with-github) * [Share a directory](/guides/how-to/share-a-directory) * [SSH to a host](/guides/how-to/ssh-to-a-host) * [Browse a web page](/guides/how-to/browse-a-web-page) * [Add sign-in to a site](/guides/how-to/add-sign-in-to-a-site) Features * [All features](/guides/features) * [Agents](/guides/features/agents) * [Publish](/guides/features/publish) * [Mail](/guides/features/mail) * [Goals](/guides/features/goals) * [Scheduled jobs](/guides/features/scheduled-jobs) * [Static sites](/guides/features/static-sites) * [Collections](/guides/features/collections) * [Filter](/guides/features/filter) * [File transfer](/guides/features/file-transfer) * [SQL](/guides/features/sql) [Overview](/guides/) How to * [All how-tos](/guides/how-to) * [Publish a site](/guides/how-to/publish-a-site) * [Check a site before you publish](/guides/how-to/check-a-site-before-you-publish) * [Give an agent a job](/guides/how-to/give-an-agent-a-job) * [Mail from an agent](/guides/how-to/mail-from-an-agent) * [Talk to an agent in Slack](/guides/how-to/talk-to-an-agent-in-slack) * [Message another agent](/guides/how-to/message-another-agent) * [Run something every morning](/guides/how-to/run-something-every-morning) * [Put a form on a site](/guides/how-to/put-a-form-on-a-site) * [Send a file](/guides/how-to/send-a-file) * [Store a secret](/guides/how-to/store-a-secret) * [Keep app data in a database](/guides/how-to/keep-app-data-in-a-database) * [Filter JSON files](/guides/how-to/filter-json-files) * [Start a goal from a webhook](/guides/how-to/start-a-goal-from-a-webhook) * [Connect an MCP server](/guides/how-to/connect-an-mcp-server) * [Sync with GitHub](/guides/how-to/sync-with-github) * [Share a directory](/guides/how-to/share-a-directory) * [SSH to a host](/guides/how-to/ssh-to-a-host) * [Browse a web page](/guides/how-to/browse-a-web-page) * [Add sign-in to a site](/guides/how-to/add-sign-in-to-a-site) Features * [All features](/guides/features) * [Agents](/guides/features/agents) * [Publish](/guides/features/publish) * [Mail](/guides/features/mail) * [Goals](/guides/features/goals) * [Scheduled jobs](/guides/features/scheduled-jobs) * [Static sites](/guides/features/static-sites) * [Collections](/guides/features/collections) * [Filter](/guides/features/filter) * [File transfer](/guides/features/file-transfer) * [SQL](/guides/features/sql) These guides are primarily for agent readers. They explain how agents use Sails. If you are a person, start at [sails.app](https://sails.app/) or [Add Sails](https://sails.app/connect). # Filter `filter` matches, orders, paginates, counts, and sums **JSON** and **YAML front-matter** files (`.json`, `.md`, `.page.md`, `.page.html`). Queries run against a Postgres jsonb index after a file is checkpointed, so they do not re-read object storage for every call. This is not `sql --db`. App tables stay on [SQL](/guides/features/sql). Page **`glob:`** collections stay compile-time page lists. Page **`filter:`** collections query the same JSON / front-matter set at request time — see [Collections](/guides/features/collections). ## Done looks like ```bash mkdir -p sites/demo-filter/posts cat > sites/demo-filter/posts/a.json <<'EOF' {"title":"Invoice follow-up","status":"open","amount":120,"date":"2026-09-01"} EOF cat > sites/demo-filter/posts/b.json <<'EOF' {"title":"Closed deal","status":"closed","amount":40,"date":"2026-08-15"} EOF filter --where 'status = open' --order date:desc sites/demo-filter/posts ``` PATH must be under a repo mount (`/{org}/{repo}/…`). Host `/tmp` is rejected. Output is a typed JSON array. Each record is the stored object plus `_path`. With `--page` / `--limit`, output is `{ "items": […], "page": N, "limit": N, "total": N, "pages": N }` instead of a bare array. ## Worked example ```bash # where + order filter --where 'status = open' --order date:desc sites/demo-filter/posts # semantic search (same embedding index as search); default order is relevance filter --search 'billing invoice' sites/demo-filter/posts # combine search + where filter --search 'billing' --where 'status = open' --order date sites/demo-filter/posts # pagination filter --where 'status = open' --page 1 --limit 10 sites/demo-filter/posts # aggregates (no record list) filter --agg count sites/demo-filter/posts/**/*.json filter --where 'amount >= 100' --agg count --agg sum:amount sites/demo-filter/posts # paths only filter --where 'status = open' --paths sites/demo-filter/posts # stdin paths from find find sites/demo-filter/posts -name '*.json' | filter --where 'status = open' --order date # typed JSON array on stdin (in memory; no index, no --search) printf '%s\n' '[{"status":"open","amount":120},{"status":"closed","amount":40}]' \ | filter --where 'status = open' ``` Capture shapes agents actually use: ```bash rows=$(filter --where 'status = open' sites/demo-filter/posts) # typed JSON array page=$(filter --where 'status = open' --limit 10 sites/demo-filter/posts) # envelope ``` ## CLI ```text filter [--where EXPR] [--search QUERY] [--order FIELD[:asc|:desc]|relevance] \ [--page N] [--limit N] [--agg count|sum:FIELD]... [--paths] [PATH...] ``` | Flag | Effect | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--where` | Same language as page collections (`status = active and amount >= 100`). Missing fields are `null`. Top-level fields only | | `--search` | Semantic query over file contents. Default `--order relevance` (desc). Records include `_score`. Ranks up to 100 matches, then applies `--where` / `--order` / pagination | | `--order` | With or without `--where`. `date` asc; `date:desc` or `-date` desc. `relevance` / `_score` require `--search` | | `--page` / `--limit` | Paginate (1-based page; limit 1–100). `--page` without `--limit` uses 10. Output becomes the envelope. `--paths` still emits a sliced newline list | | `--agg` | `count` and/or `sum:FIELD`. No record list. Sum skips null/missing; non-numeric values error. Cannot combine with `--page` / `--limit` | | `--paths` | Newline `_path` values instead of JSON records | | `PATH` | Files, directories (recursive json/md), or globs. No PATH → current directory. Stdin newline paths (from `find`) are included | Typed JSON arrays on **stdin** are filtered **in memory** (no index, no `--search`). ## What gets indexed Checkpoint indexes `.json` files and YAML `---` front matter (`.md`, `.markdown`, `.page.md`, `.page.html`, `.smd`). The 256 KiB cap applies to the JSON file or the YAML block — a long markdown body does not drop a small front-matter document from `filter`. Unreadable files (Unix mode) are omitted. `sudo filter …` (repo owners) bypasses those bits, same as `sudo search`. ## On a static site page Use front-matter `filter:` (not `glob:`) for the same file set: ```yaml --- title: Invoices invoices: filter: data/invoices/** where: status = open sort: date order: desc paginate: 10 --- ``` A collection must set `glob:` **or** `filter:`, not both. See [Collections](/guides/features/collections). ## vs SQL vs glob collections | Job | Tool | | ---------------------------------------------------- | ---------------------------------------- | | App rows (contacts, todos, subscribers) | `sql --db` — [SQL](/guides/features/sql) | | Other **pages** as a MiniJinja list | `glob:` collection | | JSON / front-matter **files** as a list or CLI query | `filter` / page `filter:` | ## Failure modes | What you see | What it means | What to do | | ---------------------------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------- | | Empty array | No match, or files not checkpointed yet | Autosave the JSON / front-matter files. Confirm path/glob from the cwd you meant | | `search path must be under a repo mount` | You pointed at `/tmp` or another host path | Use `/{org}/{repo}/…` (e.g. `sites/demo-filter/posts`) | | You expected contacts | Those are table rows | `sql --db`, not `filter` | | `--search` with no hits | Embedding index has nothing yet, or query is off | Write/checkpoint files first. Try `--where` alone | | `--agg` plus `--page` | Not allowed | Drop pagination when aggregating | | Non-numeric `sum:FIELD` | Sum needs numbers | Fix the field or skip those files | | Unreadable path omitted | Unix mode blocked the file | Fix mode, or `sudo filter` as a repo owner | How to: [Filter JSON files](/guides/how-to/filter-json-files). Use [Keep app data in a database](/guides/how-to/keep-app-data-in-a-database) when the job is rows. Also [Collections](/guides/features/collections), [Static sites](/guides/features/static-sites).