Documentation
Learn Weave
Pipelines

Pipelines

Pipeline operators are the heart of Weave. They replace most loops and let you express data transformations as readable, left-to-right chains.

The Three Operators

Operator Name What it does
|> Pipe Pass a value into a function
*> Map Transform each item in a container
&> Reduce Accumulate values into a single result

Pipe |>

Passes data as the first argument to a function:

fn display(data) { puts(data) }

fn process(d) { d *> ^(x) { x * 2 } }

data = [1, 2, 3]
data |> display    # same as display(data)

# Chain multiple calls
data |> process |> display

Map *>

Transforms each item in a container by applying a function:

fn double(x) { x * 2 }

[1, 2, 3] *> double              # [2, 4, 6]
[1, 2, 3] *> ^(x) { x + 10 }    # [11, 12, 13]

# Extract fields from structured data
people = [
    [name: "Alice", age: 30],
    [name: "Bob", age: 25]
]
people *> ^(p) { p[:name] }      # ["Alice", "Bob"]

Reduce &>

Accumulates values into a single result. The function must have a parameter with a default value that serves as the accumulator’s initial value:

fn sum(val, acc: 0) { acc + val }
fn product(val, acc: 1) { acc * val }

[1, 2, 3, 4] &> sum        # 10
[1, 2, 3, 4] &> product    # 24

# Building strings
fn join(s, acc: "") {
    if acc.len > 0 { acc + ", " + s } else { s }
}
["a", "b", "c"] &> join    # "a, b, c"

Chaining Operators

Weave’s real power comes from combining operators:

fn is_even(x) { x % 2 == 0 }

# Squares of even numbers using reduce with array accumulator
[1, 2, 3, 4, 5, 6]
    &> ^(x, acc: []) { if is_even(x) { acc + [x * x] } else { acc } }
    |> print    # [4, 16, 36]

A Real-World Pipeline

Sum the “total” column across all CSV files in a directory:

# Define our building blocks
csvs = ^() { split(`ls *.csv`[:output], "\n") &> ^(f, acc: []) { if f != "" { acc + [f] } else { acc } } }
totals_in = ^(filename) { read(filename, :csv) *> ^(x) { x[:total] } }
sum = ^(val, acc: 0) { acc + val }
fn sum_lists(vals) { vals &> sum }

# Chain them together
total = csvs()
    *> totals_in     # Map each filename to a list of totals
    *> sum_lists     # Sum each list
    &> sum           # Sum the final list
    |> ^(total) { puts("Total: {total}") }

Each step’s output flows directly into the next. The pipeline reads like a description of what you want: get the CSV files, extract totals from each, sum the sublists, sum the results, and print.

Inline Lambdas

Lambdas work naturally with pipeline operators for quick one-off transformations:

read("data.csv", :csv)
    *> ^(row) { row[:amount] }
    &> ^(v, acc: 0) { acc + v }
    |> ^(total) { puts("Total: {total}") }

Pipelines with Named Functions

For more complex transformations, define named functions and compose them:

fn extract_ip(line) { split(line, " ")[0] }

fn count_occurrences(items) {
    counts = []
    items *> ^(item) {
        current = if !counts[item] { 0 } else { counts[item] }
        counts[item] = current + 1
    }
    counts
}

# Find top IPs from access logs
split(`cat /var/log/nginx/access.log`[:output], "\n")
    *> extract_ip
    |> count_occurrences
    |> ^(c) { c.sort(:desc) }
    |> ^(counts) {
        i = 0
        while i < 5 {
            ip = counts.keys[i]
            puts(ip + ": " + counts[ip])
            i += 1
        }
    }
ContainersFile I/O