Tag: workflows

Posted on

RESERVE v1.2.2: Opening the Official Workflow Catalog

RESERVE v1.2.2 opens the first public window into the official RESERVE workflow catalog.

The previous release, v1.2.1, established an important architectural boundary: the official reserve-workflows collection would live separately from local workflows and examples, and the CLI would access that collection through the RESERVE workflow API rather than depending directly on GitHub, object storage, or another distribution mechanism.

v1.2.2 puts that boundary to work

For the first time, RESERVE can browse the official workflow catalog and retrieve versioned workflow definitions through the permanent API at:

https://api.reservecli.dev/v1

This is intentionally a read-only preview.

RESERVE does not install remote workflows in v1.2.2. It does not publish them, update them, authenticate private repositories, or execute downloaded pipeline content.

The goal of this release is narrower: make the official catalog visible through the CLI, prove the API boundary, and establish a foundation that later releases can build on without weakening the local-first workflow model.

Browsing the Official Catalog

The most visible addition is:

reserve workflow browse

This command lets users inspect the official workflow catalog directly from RESERVE.

The complete catalog can be browsed from the CLI, or the results can be narrowed to a particular collection.

Catalog output follows RESERVE’s existing output conventions. The normal command-line experience uses a readable table, while JSON output is available for tools, agents, scripts, and other programmatic consumers.

This is the first point at which the reserve-workflows catalog becomes something an installed RESERVE client can discover rather than merely an architectural concept.

In v1.2.1, the official repository had a name and an API boundary.

In v1.2.2, that boundary has something useful on the other side.

Retrieving an Official Workflow

Browsing tells you what exists. The second new command lets you inspect the workflow itself:

reserve workflow get [version]

RESERVE can retrieve an official workflow definition from the API, including a specific version when requested.

The default output is the raw workflow YAML.

That choice is deliberate. A workflow is ultimately a portable document, and when a user asks RESERVE to retrieve one, the most useful default is the document itself rather than an additional presentation layer around it.

For programmatic consumers, JSON envelopes are also supported.

Together, browse and get provide the first complete read path from the RESERVE CLI into the official workflow catalog:

reserve workflow browse

Find a workflow.

reserve workflow get ...

Retrieve its definition and then inspect it. Nothing is installed. Opening the door to this future ecosystem is a big thing and v1.2.2 deliberately takes a small step.

The preview does not include:

  • remote workflow installation or updates;
  • workflow publishing;
  • repository management;
  • private repository authentication;
  • ratings or search;
  • remote execution;
  • user enrollment.

The catalog is available for inspection, but the CLI does not turn retrieved content into executable behavior. This keeps the new network boundary small enough to reason about. A remote workflow definition can be useful without automatically being trusted. RESERVE v1.2.2 treats those as separate concerns.

Retrieval Is Not Execution

That distinction is especially important for pipeline workflows.

RESERVE v1.2.2 never executes workflow pipeline content. 

workflow render

does not invoke Bash or another shell. Rendered pipeline stages remain descriptive output intended for operator or agent review.

They should not be interpreted as a promise that arbitrary pipeline content is portable, deterministic, or safe to execute across platforms. From a security perspective, this is critical. Repository-style references are also constrained so that they cannot traverse or escape the configured workflow root, including through symlinks. These restrictions are not temporary obstacles to work around. They reflect the security model RESERVE is trying to preserve as workflows become increasingly shareable.

Execution may come later, but only after its semantics are defined well enough to make it secure, deterministic, and cross-platform. Until then, remote workflow discovery and remote workflow execution remain intentionally separate capabilities.

# Browse the official workflow catalog
reserve workflow browse

# Browse one collection
reserve workflow browse inflation

# Retrieve the latest CPI workflow
reserve workflow get inflation cpi-history

# Retrieve a specific immutable version
reserve workflow get inflation cpi-history 1.0.0

# Retrieve workflow metadata as JSON
reserve workflow get inflation cpi-history 1.0.0 --format json

The API Is the Contract

One of the most important parts of v1.2.2 may be something users never have to think about.

Official workflow reads go through:

https://api.reservecli.dev/v1

The CLI does not expose GitHub, R2, or another storage provider as the repository contract.

That follows directly from the architecture introduced in v1.2.1.

The official workflow collection may be authored in one place, indexed somewhere else, and distributed through infrastructure that changes over time. None of those choices should require users to learn a new workflow model or force the CLI to encode a particular hosting provider’s repository layout.

The RESERVE API is the stable boundary.

v1.2.2 includes typed internal registry client support for catalog, collection, and artifact reads so that this separation exists in the implementation as well as in the documentation.

That gives future releases room to evolve the service without making storage infrastructure part of RESERVE’s public CLI contract.

A Preview of a Larger Service Boundary

The API in v1.2.2 is deliberately public and read-only.

It is not yet RESERVE’s authenticated service model.

In particular, the preview does not use a FRED API key as RESERVE authentication.

That distinction matters because a FRED API key identifies access to FRED. It should not quietly become a general-purpose credential for RESERVE services.

The next stage of the service boundary is planned around explicit RESERVE identity and authentication: verified-email enrollment, one-time FRED-key validation, RESERVE token issuance, and authenticated API enforcement.

Those capabilities remain v1.2.3 work.

Keeping them out of v1.2.2 lets this release answer a simpler question first:

Can RESERVE expose a stable, useful, provider-independent read path to the official workflow catalog?

Now it can.

Local Workflows Still Stay Local

The addition of an official catalog does not change the role of RESERVE’s existing local workflow system.

The distinction established in v1.2.1 remains:

examples/workflows/

Educational examples shipped with the source tree.

~/.reserve/workflows/personal/

User-created and locally managed workflows.

reserve-workflows/

The curated official workflow collection.

RESERVE workflow API

The distribution boundary through which the CLI accesses official workflow content.

v1.2.2 connects the final two pieces for read operations.

It does not collapse them into the local workflow model.

Users can discover and retrieve official workflows while local authoring remains its own concern. That separation gives RESERVE room to introduce installation and repository management deliberately rather than making every remote artifact implicitly local.

# List locally available workflows
reserve workflow list

# Show a normalized workflow document
reserve workflow show examples/workflows/simple-observation.yaml

# Inspect required runtime inputs
reserve workflow contract examples/workflows/parameterized-workflow.yaml

# Validate a workflow
reserve workflow validate examples/workflows/parameterized-workflow.yaml

# Render a copy-ready pipeline without executing it
reserve workflow render examples/workflows/parameterized-workflow.yaml 2015-01-01 2024-12-31

Designed for People and Tools

The new commands also continue a pattern that is becoming increasingly important for RESERVE: CLI output should work for both human operators and programmatic consumers.

Catalog browsing supports the familiar table view for interactive use and JSON for structured consumption.

Workflow retrieval defaults to raw YAML because that is the natural representation for a workflow definition, while JSON envelopes are available when metadata and structured integration matter.

This makes the preview useful beyond an interactive terminal.

An operator can browse the catalog.

A script can consume the catalog as JSON.

An agent can retrieve a workflow definition for inspection.

And in every case, retrieval remains retrieval. RESERVE does not silently turn inspection into execution.

Verified Before Release

The v1.2.2 release has been tested across the CLI and internal packages.

make test passes, as does:

go test ./cmd ./internal/… ./tests

Registry client tests cover catalog reads, collection reads, versioned artifact retrieval, and bearer-header behavior.

go vet ./… also passes.

The release distribution build produces the expected platform archives and SHA256SUMS, with release documentation and manifests updated for v1.2.2.

From Boundary to Interface

v1.2.1 was about deciding where things belong.

v1.2.2 is about making one of those boundaries useful.

The official reserve-workflows collection is no longer only a future repository described by the architecture. It is now discoverable through the RESERVE CLI.

But the scope remains intentionally constrained.

  • Browse.
  • Retrieve.
  • Inspect.
  • Do not execute.
  • Do not assume trust.
  • Do not couple the CLI to the infrastructure behind the API.
  • Those constraints give RESERVE a much stronger foundation for what comes next.

Future releases can add identity, authenticated access, installation, private repositories, publishing, and eventually carefully defined execution semantics on top of an API boundary that already exists and a catalog model that is already usable.

RESERVE v1.2.2 is the first public connection between the CLI and the official workflow ecosystem.

It opens the catalog without opening the door to everything at once.

Posted on

Why JSONL Matters for FRED Data

When most people think about working with FRED data, they think in terms of CSV files or API JSON responses. CSV files make immediate sense to the desktop economist: download the series, open it in Excel, chart it, regress it, annotate it, and move on. Plain JSON serves a different audience. It is the natural format for applications, APIs, and software systems that need structured data with fields, arrays, and metadata. Both formats are familiar, both are widely supported, and both are perfectly adequate for small-scale analysis. But once you begin building downstream pipelines around economic observations — particularly pipelines that transform, stream, enrich, or continuously ingest data — the limitations of those formats begin to show.

This is where JSONL becomes interesting.

What is JSONL?

JSONL stands for JSON Lines.

Instead of storing all records inside a single JSON array, JSONL stores one JSON object per line.

For example:

{"date":"2024-01-01","value":"100.2"}
{"date":"2024-02-01","value":"101.1"}
{"date":"2024-03-01","value":"101.8"}

Each line is independently valid JSON.

That small design difference has major implications for data engineering workflows.

JSON vs JSONL

Traditional JSON usually looks like this:

{
  "observations": [
    {"date":"2024-01-01","value":"100.2"},
    {"date":"2024-02-01","value":"101.1"},
    {"date":"2024-03-01","value":"101.8"}
  ]
}

Traditional JSON encourages you to think in terms of documents. A FRED response becomes one large object containing metadata, arrays, nested structures, and observation collections. Before a downstream process can begin its work, the entire payload typically needs to be materialized in memory and parsed as a complete unit.

JSONL encourages a different mental model. Instead of one document containing observations, the observations themselves become the primary unit of computation. Each line can be read independently, transformed independently, validated independently, and streamed independently. The data stops behaving like a static export and starts behaving like a sequence of economic events flowing through a pipeline.

That distinction matters enormously for time-series systems.

Economic observations are naturally append-oriented. New CPI values arrive. New unemployment figures arrive. GDP revisions arrive. Rarely do we need to reinterpret the entire historical universe every time a new observation appears. More commonly, we want to process the newest data incrementally while preserving provenance and transformation history downstream.

CSV works extremely well right up until the moment your workflow becomes more complicated than a spreadsheet. A single economic series with a date column and a value column is simple. But once you start combining multiple series, revision histories, release metadata, transformations, or provenance tracking, the format starts showing its age. Important context often ends up scattered across filenames, sidecar documents, README notes, or pipeline assumptions rather than traveling with the observations themselves.

Traditional JSON solves some of those problems, but introduces another. The data becomes hierarchical and application-centric rather than stream-centric. Appending a single new observation often means rewriting an entire array. Distributed processing becomes less elegant. Line-oriented tooling becomes harder to leverage.

JSONL occupies a remarkably useful middle ground. It preserves the self-describing nature of JSON while remaining operationally compatible with streaming systems, append-only workflows, Unix pipelines, and distributed analytical tooling.

That alignment becomes especially important inside RESERVE.

Within RESERVE, FRED observations are not treated merely as downloadable datasets. They are treated as composable records that can move through transformation pipelines. Once observations are emitted as JSONL, the output becomes naturally compatible with downstream operators:

reserve fred observations CPIAUCSL --format jsonl

At that point, transforms stop feeling like post-processing steps attached to static files and begin behaving more like streaming operators acting on a flow of economic observations.

A rolling average transform no longer needs to conceptualize itself as “opening a file.” It can instead consume observations incrementally as they arrive:

reserve fred observations FEDFUNDS --format jsonl \
  | reserve transform moving-average --window 12

This is ultimately why JSONL matters for FRED data. The format aligns with the actual shape of economic time-series workflows. Observations arrive sequentially. Pipelines process sequentially. Transformations operate sequentially. JSONL simply removes unnecessary structural friction between the data and the systems operating on it.

Consumers as First Class Citizens

RESERVE was designed and developed to democratize access to FRED data. One key design goal was to treat humans, other programs, and LLM-based agents as first-class consumers.

RESERVE’s tabular formats and charting functions target human consumers. CSV targets desktop tools and relational data stores. JSON targets API-driven applications and LLMs, while JSONL caters to data pipelines and streaming-oriented applications.

A naysayer might ask, “So what? Any program or script can reformat data!”

There is an important nuance.

Accessing FRED data requires consumers to adhere to the usage requirements of the underlying sources. A FRED source generally falls into one of three categories:

  • Public Domain: Citation Requested
  • Copyrighted: Citation Required
  • Copyrighted: Contact Source (Permission is Required)

As a software program, RESERVE checks source-usage requirements during retrieval and carries the appropriate disclosure information throughout its processing, including into its various output formats.

In the example below, GDP is a public-domain data series for which citation is requested:

reserve obs latest gdp --format jsonl
{"series_id":"GDP","date":"2026-04-01","value":32486.066,"value_raw":"32486.066","citation_text":"Source: Bureau of Economic Analysis via FRED"}

Compliance with FRED’s source-usage requirements involves more than using a standard library to flip data from one output format to another. The disclosure must remain semantically attached to the data as it moves through RESERVE’s processing pipeline and into the format requested by the consumer.

This behavior was deliberately architected into RESERVE. Its validation is also part of the automated test suite and is revalidated with every release.

Summary

Modern software can generally convert data back and forth between formats. But not having the right format at the point of consumption creates inefficiencies, unnecessary processing, and performance headwinds. This is particularly relevant when working with large datasets, such as those potentially delivered by the FRED API.

RESERVE treats all consumers—and the formats they consume—as first-class citizens. Humans can work with tables and charts. Desktop tools and databases can consume CSV. Applications and LLMs can consume JSON. Data pipelines can consume JSONL.

Moreover, RESERVE recognizes that the format is only part of the contract. Source disclosure and usage requirements must travel with the data. By preserving that information across formats, RESERVE delivers data that is not only convenient to consume, but also carries the context necessary to use it responsibly.

Posted on

RESERVE v1.2.1: Putting the Workflow Boundaries in Place

RESERVE v1.2.1 is a deliberately focused release.

After the substantial workflow foundation introduced in v1.2.0, this release takes the next practical step: establishing where workflows belong, clarifying the boundaries between different kinds of workflow repositories, and adding a small set of examples that make the existing workflow system easier to understand.

It is a plumbing and documentation release.

There are no new remote workflow commands in v1.2.1. No catalog browsing, installation, publishing, private repository authentication, or remote execution has been added. Instead, the release prepares the project for those capabilities without prematurely building them.

Examples Are Examples

RESERVE now includes four curated workflow examples under:

examples/workflows/
├── simple-observation.yaml
├── transform-pipeline.yaml
├── multi-series-analysis.yaml
├── parameterized-workflow.yaml
└── README.md

Together, these examples demonstrate several parts of the workflow model:

  • a simple single-series observation;
  • a transformation pipeline;
  • multi-series analysis;
  • runtime parameters and workflow contracts.

They are intentionally readable and modest. Their purpose is to show workflow syntax and CLI capabilities, not to serve as production-grade economic research.

The examples also establish an important rule:

examples/workflows/ is educational source-tree content. It is not the official RESERVE workflow catalog.

That distinction matters as the project grows. Examples may demonstrate simplified data, unusual features, or authoring patterns that do not belong in a curated public collection.

A Clearer Workflow Ecosystem

RESERVE workflows now have clearly defined homes:

examples/workflows/
    Educational examples shipped with the source tree.

~/.reserve/workflows/personal/
    User-created and locally installed workflows.

reserve-workflows/
    The future canonical, curated public workflow collection.

Private repositories/
    Future user- or organization-controlled collections.

RESERVE workflow API/
    The future distribution boundary between the CLI and repositories.

The official repository is named reserve-workflows.

That name now exists as a first-class internal repository descriptor rather than as a collection of hard-coded GitHub URLs or storage-specific assumptions. The CLI’s future repository operations can depend on the concept of an official repository while remaining independent of how that repository is authored, published, indexed, or stored.

GitHub may be the source of truth for the curated workflow collection. A cloud-based API and supporting distribution services may eventually provide access to that collection. Those implementation details should remain behind the RESERVE workflow API boundary.

The Official Repository Boundary

The internal official repository descriptor identifies:

id:       official
name:     reserve-workflows
type:     official
endpoint: https://api.reservecli.dev/v1

This does not yet enable remote workflow operations. It simply establishes the abstraction that future releases can build upon.

The initial API boundary is intentionally small:

GET /workflows
GET /workflows/{id}
GET /workflows/{id}/{version}

The exact service implementation can evolve later without requiring the CLI to understand GitHub repository layouts, static object paths, R2 buckets, or other infrastructure details.

What Users Will Notice

For most installed users, v1.2.1 is intentionally quiet.

The existing workflow commands continue to work as before:

reserve workflow create
reserve workflow list
reserve workflow show
reserve workflow edit
reserve workflow remove
reserve workflow validate
reserve workflow render
reserve workflow contract

The release does not add commands such as:

reserve workflow browse
reserve workflow install
reserve workflow update
reserve workflow publish

The new examples are shipped in the RESERVE source repository for developers and contributors. They are not automatically copied into a user’s ~/.reserve/workflows/ directory, and they are not presented as the official catalog.

That keeps the local-first behavior of RESERVE unchanged while giving workflow authors clearer reference material.

Go 1.27.1

RESERVE v1.2.1 also updates the supported release toolchain note to Go 1.27.1.

This is a small implementation detail, but documenting the toolchain used for the release improves reproducibility and gives users and maintainers a clearer baseline for future builds.

A Small Release With a Long-Term Purpose

The goal of v1.2.1 is not to make the workflow system larger. It is to make its boundaries harder to misunderstand.

Examples should remain examples.

Personal workflows should remain local.

The official catalog should have its own repository.

Private repositories should arrive only when their authentication and authorization model is ready.

And the CLI should communicate with repositories through a stable RESERVE abstraction rather than through infrastructure-specific knowledge.

That gives future releases room to add official workflow browsing and installation without turning the local workflow foundation into a tangle of special cases.

RESERVE v1.2.1 establishes the physical and architectural boundaries of that ecosystem, while keeping the user-facing CLI focused, local, and predictable.