Data¶

Everything under data.sources: is one map of named entries. An entry is either a source, which fetches from outside (an HTTP API, a command, a file, a WebSocket, inline data), or a pipeline operator, which reads other entries and reshapes them. Components and wrangl use entries by name, and don't care which kind they are.

Every field is in Reference → Sources and Reference → Pipelines. For the model underneath, see How the data layer works.

Sources¶

data:
  sources:
    pods:
      type: http
      url: http://localhost:8001/api/v1/pods
      root: items          # the list lives under "items"
      refresh: 5s          # poll

    branches:
      type: exec
      command: [gh, api, repos/jsdrews/tui-builder/branches]

    logs:
      type: exec
      follow: true         # stream lines as they arrive
      command: [kubectl, logs, -f, deploy/web]

How often a source delivers follows from its fields: follow: true (or websocket) streams, refresh: polls, and otherwise it's fetched once and again on r.

root: picks the list out of the response. format: text keeps the response as a string, for logs.

Pipelines¶

Operators take from: (or several inputs) and produce a new entry:

data:
  sources:
    pods: {type: http, url: http://localhost:8001/api/v1/pods, root: items, refresh: 5s}

    failing:
      type: filter
      from: pods
      where: "status.phase != 'Running'"

    slim:
      type: project
      from: failing
      keep:
        name: metadata.name
        phase: status.phase
        restarts: "status.containerStatuses[0].restartCount"

When the steps in between aren't worth naming, chain them inline with pipe::

failing_slim:
  type: http
  url: http://localhost:8001/api/v1/pods
  root: items
  pipe:
    - {type: filter, where: "status.phase != 'Running'"}
    - {type: project, keep: {name: metadata.name, phase: status.phase}}

Expressions are expr-lang; see Templating → Expressions.

Parameters¶

A source can declare typed inputs and reference them as ${params.NAME}. That makes one definition serve many uses: a drill-down screen, a detail pane that follows a cursor, a join, or the command line.

pod:
  type: http
  parameters:
    namespace: {required: true}
    name: {required: true}
  url: http://localhost:8001/api/v1/namespaces/${params.namespace}/pods/${params.name}

wrangl ops.yaml pod --param namespace=default --param name=web-0

In the TUI, a push binding's bind: or a component's on_cursor.bind supplies them from the focused row.

Big remote sets¶

For an API with more rows than you'd want in memory, there are two options:

• paginate: walks every page up front into one list. The table then filters and sorts locally. Fine up to a few thousand rows.
• window: fetches only the rows on screen and sends the filter and sort to the server, so author:tolkien searches the whole set. Binding a table to a windowed source switches it into this mode.

books:
  type: http
  url: https://openlibrary.org/search.json
  root: docs
  window:
    offset_param: offset
    limit_param: limit
    total_path: numFound
    search_param: q
    filters: {Author: author}

window: works with exec too, with the request templated into the argv. See Reference → window.

Walking from an edge¶

Some APIs can't jump to row N at all: Elasticsearch past 10,000 hits (search_after), logs filtered by "before / after this timestamp", and anything that pages with an ?after=<token>. Declare cursor: in the window and the source is Anchored: each request walks from the oldest or newest item held, using that item's cursor.

logs:
  type: http
  method: POST
  url: ${env.ES_URL}/app-logs/_search
  root: hits.hits
  body: '{"query": {"match_all": {}}, "size": ${window.limit}}'
  window:
    cursor: sort
    older: {sort: [{"@timestamp": desc}], search_after: "${window.cursor}"}
    newer: {sort: [{"@timestamp": asc}],  search_after: "${window.cursor}"}

older: and newer: are what the request changes for each direction, merged into the body. An eventlog or a table can bind an Anchored source. See Reference → Anchored windows.

Still growing¶

A running job keeps adding events. window.growing: says so, either always (true) or while another source says the job is running:

window:
  growing: {source: job, while: 'status in ["pending", "running"]'}
  follow_every: 2s

While it grows, a bound eventlog follows the newest item and polls; scroll away and new items are counted instead of moving you.