Roadmap
0-roadmap.RmdVision
blockr.assistant is a blockr.dock
extension that puts a chat interface next to a blockr
board. The chat is driven by ellmer and
equipped with a set of tools — R functions exposed to the LLM —
that allow a model to inspect and manipulate board state. Tool
invocations are translated into the same update(...)
reactive payloads that interactive user actions use, so the assistant
is, from the board’s perspective, just another caller of the public
extension API.
The intended user experience is conversational pipeline construction:
“Load the
irisdataset, then drop the species column, then plot sepal length vs. width.”
…producing the corresponding blocks, links and stack on the board.
Scope
In scope:
- A
dock_extensionproviding a chat panel. - Provider-neutral chat construction by consuming the board’s
new_llm_model_option(). - LLM tools for CRUD operations on blocks, links and stacks.
- Block-parameter editing via
blockr.core’s external-control mechanism (external_ctrl = TRUEblocks). - Streamed responses and tool-call telemetry via
shinychat.
Explicitly out of scope (for now):
- Building our own LLM provider abstractions —
ellmerhandles that. - Application-level persistence wiring. The extension’s constructor
and
stateshape make system prompt and chat history round-trip throughblockr.dock’s standard ser/des (see Phase 1), but the integration that triggers cross-session save/restore (e.g. viablockr.session) is out of the initial roadmap. - Mutation safeguards (undo, dry-run, snapshot-before-mutation) — the
initial implementation applies tool calls directly to the board.
Snapshotting will likely be revisited once
blockr.sessionexposes the right hooks. - Multi-board / multi-session reasoning — assistant context is scoped to the single board it is mounted on.
Architecture sketch
┌──────────────────────────────────────────────────────────────┐
│ dock_board │
│ │
│ ┌────────────────────┐ ┌────────────────────────────┐ │
│ │ board state │ ←── │ assistant_extension │ │
│ │ (blocks, links, │ │ │ │
│ │ stacks) │ │ ┌──────────────────────┐ │ │
│ │ │ │ │ chat panel (UI) │ │ │
│ │ update reactive │ ←── │ │ shinychat module │ │ │
│ └────────────────────┘ │ └──────────┬───────────┘ │ │
│ ↑ │ │ │ │
│ │ │ ┌──────────┴───────────┐ │ │
│ │ │ │ ellmer::Chat with │ │ │
│ │ │ │ registered tools │ │ │
│ │ │ │ ┌──────────────┐ │ │ │
│ └──────────────────┼──┤ │ board tools │ │ │ │
│ │ │ │ (CRUD) │ │ │ │
│ │ │ └──────────────┘ │ │ │
│ │ └──────────────────────┘ │ │
│ └────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
Concretely:
-
new_assistant_extension()returns adock_extensionwhose UI is ashinychat::chat_mod_ui()panel and whose server constructs anellmer::Chatfrom the board’sllm_modeloption, registers the tool layer over closures of(board, update, session), and hands the chat toshinychat::chat_mod_server(). - Each tool is an R closure that captures the live
boardandupdatereferences. Inspection tools read fromboard$board. Mutation tools emitupdate(list(blocks = …, links = …, stacks = …))payloads, or write to externally-controlled block reactives. - The chat’s system prompt is composed from a fixed assistant persona plus a dynamically generated board summary (block IDs, types, links, stacks, available block constructors).
Architectural decisions
Two decisions shape every phase below and deserve to be stated up front:
Provider neutrality via
new_llm_model_option(). We never construct an
ellmer chat client directly. The board exposes an
llm_model board option whose value is a chat-constructor
function of signature
function(system_prompt = NULL, params = NULL) returning an
ellmer::Chat. The extension reads that option and calls it.
Providers, model names and keys are configured globally via
blockr_option("chat_function", …) — out of scope for this
package. The option always resolves to a usable function (falling back
to blockr.core’s default_chat), so the
extension never needs to handle a missing chat backend.
Narrow tools, auto-batched dispatch. The LLM sees
many small, focused tools (add_block,
add_link, …) rather than one monolithic
modify_board. Each tool call does not
dispatch an update(...) immediately; it stages its
change into a pending payload held inside the extension server.
Validation runs against the current board merged with the pending
payload, so per-tool feedback is meaningful. When the assistant’s turn
completes, the accumulated payload is flushed as a single
update(...) call.
This combination gives us:
- LLM-friendly tool ergonomics (small, focused calls, easy to describe).
- A single atomic dispatch per user turn (one validation pass, one
re-render, one logical snapshot point for future
blockr.sessionintegration). - The “add link before block” ordering trap disappears within a turn — ordering of stage operations is irrelevant as long as the final payload is valid.
The staging layer is the central architectural deliverable of the package; every mutation tool is a thin façade over it.
Decomposition into phases
Each phase corresponds to its own design doc under
vignettes/design/, named
<n>-<topic>.Rmd. These files are deliberately
excluded from the package tarball via .Rbuildignore — they
are not installed vignettes (R CMD check rejects digit-prefixed vignette
filenames) but pkgdown walks vignettes/
recursively and picks them up as articles served at
articles/design/<n>-<topic>.html. Phases are
ordered so that the package compiles and demos something at the end of
each one.
Phase 1 — Extension shell with ellmer chat
File: 1-shell.Rmd
new_assistant_extension() constructor, dock UI/server, a
shinychat::chat_mod_ui / chat_mod_server
panel, and an ellmer::Chat constructed from the board’s
llm_model option. Includes from day one:
- Streaming responses, history, basic token telemetry.
- A visible “stop” affordance — provided for free by
shinychat::chat_mod_server(), which constructs anellmer::stream_controller()internally and wires its own cancel input. No work for us beyond verifying it end-to-end. - A default, overridable assistant persona shipped with the package —
configurable via a
system_promptargument onnew_assistant_extension(). The default persona is short and scoped to “you are a blockr board assistant”; subsequent phases add dynamic board context on top. - A
stateshape that round-trips constructor arguments (system_prompt,messages) viablockr.dock’s ser/des. Constructor signature ↔︎ state keys is a structural contract (see1-shell.Rmdfor the implications). Phase 1 also adds amessages = NULLconstructor argument so that a saved conversation can be seeded back into the chat on restore. Model parameters (temperature, max tokens, …) are deliberately not in the extension’s surface — they belong to the chat constructor, supplied viablockr_option("chat_function", …)or the board’sllm_modeloption.
No tools yet — the assistant can hold a conversation but is board-blind.
Phase 2 — Read-only tool layer
File: 2-read-tools.Rmd
Establish the “tool kit” abstraction: closures over
(board, update, session), JSON-schema argument definitions
via ellmer::type_*, and result shaping (strings, structured
lists, errors). Implement the read-only inspection tools:
-
list_blocks,describe_block -
list_links,list_stacks list_available_block_typesget_block_result_preview
This phase locks in the patterns later mutation phases follow.
Phase 3 — Staging and dispatch model
File: 3-staging.Rmd
The architectural keystone. No new LLM-facing tools, but a clean foundation for Phase 4.
- A pending-update
reactiveValprivate to the extension server. - A “merge current board + pending payload” helper for tools to validate against.
- A turn-completion hook that flushes the pending payload via
update(...)and resets it. - Error formatting for tool results (so the model can self-correct).
- Behaviour on flush failure (partial rollback, error to user, error to model).
Phase 4 — Mutation tools
File: 4-mutation-tools.Rmd
All thin façades over the Phase 3 staging layer:
- Blocks:
add_block(type, args),remove_block(id),modify_block(id, args)(viaexternal_ctrlreactives). - Links:
add_link(from, to, input),remove_link(id). - Stacks:
add_stack(blocks, name),remove_stack(id),modify_stack(id, blocks, name).
Design decisions to nail down here: ID generation policy, validation
of constructor args before staging, behaviour when external control is
not available on a modify_block target, link validity
checks (referenced blocks exist, target input exists, no cycles).
Phase 5 — Context, examples and polish
File: 5-polish.Rmd
- Dynamic board-summary in the system prompt (refreshed each turn).
- Model-facing error formatting and recovery hints.
- Example apps in
inst/examples/. - A top-level tutorial vignette
intro.Rmd. - README,
pkgdownsite setup.
No new features beyond polish on the context layer.
Splitting Phase 4 if it grows
If Phase 4 turns out larger than expected, the natural split is block mutation vs. link + stack mutation. Blocks carry the bulk of the complexity (external control, registry lookup, ID generation); links and stacks are mechanical once the block patterns are in place. This is cheap to do retroactively — the design doc cleaves at that boundary.
Future work (out of the initial roadmap)
These are flagged so we don’t lose them, but they are explicitly not phases of the initial roadmap:
-
Snapshot / undo — integrate with
blockr.sessionto snapshot board state before each mutation and expose an undo tool to the LLM. Requires hooks inblockr.sessionthat don’t yet exist. -
Conversation persistence wiring — Phase 1 already
ships the state shape and constructor surface needed for chat history to
round-trip via ser/des. What’s missing is the integration that actually
triggers save/restore — either through
blockr.sessionor viashinychat::chat_mod_server(bookmark_on_*). -
Per-block-type tool registration — allow block
packages to contribute tools specific to their blocks (e.g. a
blockr.dplyrblock exposing aset_filter_expressiontool). - Multi-agent / multi-tool-call orchestration — explicit support for long-running plans, tool-call retries, structured planning.
- UI affordances for tool calls — inline “approve this change” UI in the chat, surfacing tool diffs to the user before they apply.