Teach: from recordings to deterministic workflows
Answer me with artifact · 2026-10-25 11:26

New version 0.5.7

You have 0.5.5. Found when this page was made.

What's new:

Run in a terminal:

npx skills update answer-me-with-html -y

Or tell your agent:

Update answer-me-with-html
See what's new

Teach: from recordings to deterministic workflows

Short answers to your questions, then the plan. Today Teach records browser work only. It turns a recording into a model-written skill, and Replay asks a model to redo it. For businesses that is too slow and too loose. The goal is a recording that compiles into a script, which runs without a model and calls one only when something breaks.

A

Your questions

Question Answer
Should "every placeholder becomes an input" be the default? Yes, and stricter: every value that changes is a declared, typed input. A step may not use an undeclared value. Saving is blocked until each {{name}} is declared and each input is used.
Placeholders or a hard block? Both. Placeholders are the only way a run can vary. Everything else is blocked: no free text the model fills in at run time, no invented values.
Is Teach browser-only today? Yes. It records clicks and typing only inside Allternit's browser. Other apps show up only as app switches and screenshots, and Replay can only drive a browser.
Can skills be structured by category? Yes: a step kind per surface (browser, desktop app, API, files, shell), each with its own locator and its own check.
Files and folder paths? Typed inputs (path, folder) checked against allowed roots, plus a per-business profile for fixed values (base URL, shared drive root, account).
B

What a compiled skill looks like

code
skill: invoice-export          # version 3, owner: acme-opsinputs:  month:   {type: date-month, example: 2026-09}  out_dir: {type: folder, root: profile.reports_root}profile: [reports_root, erp_url]          # set once per businesssteps:  - browser.goto:  "{{profile.erp_url}}/invoices"  - browser.fill:  {target: {label: "Month", css: "#m"}, value: "{{month}}"}  - browser.click: {target: {label: "Export CSV", role: button}}    expect: {download: "*.csv"}  - file.move:     {from: "~/Downloads/invoices*.csv", to: "{{out_dir}}/{{month}}.csv"}  - desktop.act:   {app: "Excel", path: ["File", "Open"], file: "{{out_dir}}/{{month}}.csv"}    expect: {window_title: "{{month}}.csv"}volatile:                       # known to change, allowed  - {step: 2, field: css, why: "id changes per deploy"}

Each step names its target several ways (label, role, CSS or accessibility path). It also says what must be true afterwards, so the runner can check its own work.

C

How it runs

try primary locatorpassedfailedmodel, only nowRun skillStepCheck expected resultNext stepTry fallback locatorsRepair
  • Fast path: no model. Steps run as a script, through the Allternit Driver for apps and DOM actions for pages.
  • Repair path: a model looks at the screen only when a check fails. It fixes the run, and saves a proposed new skill version for a person to approve.
  • Drift: every run records which locator worked. A step whose primary locator keeps failing is flagged as stale before it breaks.
D

Recording desktop apps too

  • Capture clicks and typing in any app through the macOS accessibility API: which app, which window, which control (role + label path).
    • The Allternit Driver already reads that tree (read_ui, act, verify).
    • This needs Accessibility permission. Windows uses UI Automation next.
  • Capture file actions in chosen folders (created, moved, renamed) as file.* steps, with paths turned into inputs.
  • Keep screenshots on the device as evidence; they are not the instructions.
E

Build order

  1. Strict inputs + validation: typed inputs, the save block, and the business profile. Small; UI + API.
  2. Compiler + runner for browser steps: turn today's recordings into the step format above and run them without a model. The browser-skills WorkflowRunner already runs workflow.json with parameters, so extend it.
  3. Checks, fallbacks, repair, and versioned skill fixes, plus the volatile-field allowlist.
  4. Desktop recording + desktop steps through the Allternit Driver (Mac first).
  5. File steps and folder inputs.
  6. Business flow: record, dry-run with approval at each step, certify, publish, schedule. Per-skill run history and drift report.
Which part should I start with?