EXPERTA

1.1 Goal

The tool makes two-level fractional factorial designs from a request in plain language.

• A large language model (LLM) translates the request into a typed specification ("spec").
• The R package FrF2 makes the design. FrF2 runs in the browser through WebR (R compiled to WebAssembly).
• The tool shows the design table, the resolution, the word length pattern (WLP), the minimum aberration (MA) status, the number of clear two-factor interactions (2FIs) and the factors that have all 2FIs clear (if any).
• The LLM never makes the design. The LLM only translates.

1.2 Users

• Quality engineering practitioners.
• Design of experiments (DoE) researchers.
• The tool is public. Users do not sign in.

1.3 Two roles of the tool

1. A public tool for practitioners.
2. A research artifact for a paper. The paper measures the accuracy of the translation step.

1.4 Success criteria

1. Construction is exact. Each design is the design that FrF2 makes for the confirmed spec. Tests prove this (Section 11).
2. Properties are correct. The tool computes the resolution and the WLP from the design matrix. Three independent sources must agree (Section 7.6).
3. The tool does not claim more than FrF2 guarantees. The guarantee statement comes from a reviewed table (Section 7.7).
4. Translation accuracy is measured. A benchmark with a frozen test set measures it (Section 12).
5. The user confirms the spec before R runs. No design appears without this confirmation.
6. Data stay on the device. Only the request text leaves the device during design work. In Phase 2, chart data leave the device only after the user opts in.
7. The tool is accessible. It meets WCAG 2.1 AA (Section 9.10).

1.5 Phases

1.6 Scope rule

• If a request does not fit what a two-level fractional factorial can do, the tool says that the request is outside its scope.
• The tool does not recommend a different approach.
• If a request is a two-level fractional factorial feature that this version does not support, the tool says "This version does not support {feature}."
• For required estimable interactions, the message adds: "Step 3 lists the clear two-factor interactions (2FIs). To change which factors have them, change the factor order in step 2." (R22). This request is in scope, so the first rule does not apply to it.
• Both messages come from fixed templates. The LLM does not write these messages.

1.7 Supported browser

• The tool is tested in desktop Google Chrome. The end-to-end tests run in Chromium.
• Other browsers get a short notice. The notice does not block the tool.
• Reason: ChatISA found a CodeMirror problem with NVDA in Firefox (issue #10, open, cause upstream). Phase 1 has no code editor. The R code view is a highlighted <pre> block, so this problem does not apply to Phase 1.

1.8 Principles of two-level fractional factorial designs

Two-level fractional factorial designs add three principles to the classical principles of experimentation (randomization, blocking and replication). The text comes from Alan R. Vazquez (review decision R20). The text of effect heredity follows his answer of 2026-10-19 (decision F8).

• Effect hierarchy. Low-order effects are more likely to be active than high-order effects. Main effects and 2FIs are usually more important than interactions of three or more factors. Thus, the evaluation of a two-level fractional factorial design usually does not look at interactions of three or more factors.
• Effect sparsity. Only a few effects (main effects or 2FIs) are active. An active effect has a larger absolute size than the noise.
• Effect heredity. Strong heredity: an interaction is more likely to be active if the main effects of all its factors are active. Weak heredity: an interaction is more likely to be active if the main effect of one of its factors is active.

Wu and Hamada (2021, Section 4.6) state the three principles. Box and Meyer (1986) is the source of effect sparsity, and Hamada and Wu (1992) of effect heredity. Published studies show that the principles hold in practice (Li, Sudarsanam and Frey, 2006; Ockuly et al., 2017). Alan R. Vazquez confirms this list (decision P4 of the Milestone 3B plan).

How the tool uses the principles:

• Phase 1 follows the hierarchy principle, as FrF2 does. The report shows the aliasing among main effects and 2FIs, the number of clear 2FIs and the factors with all 2FIs clear (Section 7.5). The WLP still counts words of every length (Section 7.6), because minimum aberration is defined on the full WLP.
• Phase 2 uses sparsity and heredity in the analysis. Lenth's method and the half-normal plot assume effect sparsity. The alias-aware linear model can use heredity to select among aliased 2FIs. The Phase 2 plan sets the rules.
• The glossary (Section 9.5) has entries for effect hierarchy, effect sparsity, effect heredity and clear 2FI. Claude drafted them in Milestone 3B. Alan R. Vazquez reviewed them on 2026-10-19 (Section 9.5).

1.9 Planned extension: other design families (Phase 4)

On 2026-10-19 Fadel M. Megahed asked to plan more design families through DoE.wrapper. Ulrike Grömping, the author of FrF2, also maintains DoE.base and DoE.wrapper. These packages return the same R class design, with the same design.info record. They share most argument names: factor.names, default.levels, randomize, seed, ncenter and blocks. So one translation pipeline and one report helper can grow one family at a time.

Facts (measured on 2026-10-19). The WebR package index for R 4.6 (repo.r-wasm.org) has DoE.wrapper 0.13 and all its dependencies. Seven packages are new to the app; the others are already in the runtime.

The new packages add about 4.5 MB to the 25.3 MB of R packages that the app loads now. RcmdrPlugin.DoE and mixexp are not in the WebR index.

Families. Each family gets its own spec schema, R template, property report, post-checks, claim statement and benchmark slice, as the Phase 3 families do.

Mixture designs and adaptive designs stay out of scope. DoE.wrapper::ccd.augment (a star portion added to a saved two-level design) joins "Augment this design" of Phase 3: screening first, then a response surface.

Routing rule (recommended). The family must not weaken the chain of Phase 1: the LLM extracts, rules decide, the user confirms, R computes and checks verify.

1. A request comes in one of two forms. A practitioner names the design ("central composite", "Latin hypercube"), or describes the problem and its limits. The model-facing schema gets one field, family, for a named design.
2. For a described problem, the voters extract the inputs of a decision table: the number of levels of each factor, quantitative or qualitative factors, the run budget against the size of the full factorial, the goal (screening, a curvature check, a quadratic model, a computer simulation) and the constraints. The table maps the inputs to a family. Example: two-level factors and a run budget below the full factorial give a regular fractional factorial (Phase 1). Alan R. Vazquez reviews the table. The LLM does not choose a family by its own judgment.
3. A request with no named family and no trigger gets a clarification question. The question lists the families as fixed choices, with one fixed sentence each.
4. The family is a field of the card, with its source and its agreement flag. The user confirms it in step 2, as every other field.
5. The scope rule (Section 1.6) stays: the tool does not recommend a different approach. A family that the version does not have gets the fixed message "This version does not support {feature}."

The alternative is to let the LLM grade which family fits the request best. That is design advice from the model. Its errors are hard to see on the card, and the evaluation cannot give it a gold label without a rule. So this specification does not plan it.

Order and gate. Before a Phase 4 plan, a feasibility spike runs each family function in WebR 0.6.0 and in native R with the same seed. It compares the designs (the differential tests of Section 11). The order follows the risk: general factorials and orthogonal arrays first (DoE.base is in the runtime now), then central composite and Box-Behnken designs, then Latin hypercube designs, then D-optimal designs. The app loads the packages of a family only when the confirmed family needs them, so the start of a two-level design does not change. Section 7.9 measures the start again before a release.

Effect on the evaluation (Section 12). The Phase 1 benchmark (S1 to S9) stays as a frozen version, so its results stay comparable. Phase 4 adds slices:

• A routing slice: requests of each family, with boundary pairs. "Check for curvature" stays a two-level design with center points. "Fit a quadratic model" goes to the response surface families.
• One slice for each family: spec fields, design equivalence by the rules of the family, and the claim statement.
• New metrics: the family confusion matrix, the silent error rate of each family (still the main safety metric) and the rate of a wrong family without a flag.
• Gold families come from the team and the decision table, reviewed by Alan R. Vazquez. Jev's questions cover two-level designs only. Its scope category can be an independent vote on the family; the spike measures if it helps.

Comparison with RcmdrPlugin.DoE. The same author offers RcmdrPlugin.DoE, an R Commander plugin with menus for DoE.base, FrF2 and DoE.wrapper.

• It needs a local R installation, R Commander and the packages in a given order. The author's page (read on 2026-10-19) calls it "still in beta shape" and advises users of the GUI to be "particularly cautious". Linux users are "partly on your own".
• It cannot run in the browser: it is not in the WebR index, and R Commander needs Tcl/Tk.
• EXPERTA needs no installation: the same packages run in WebR in the browser. A request in plain language replaces the menu choices. The card shows the sources and the agreement flags, and the user confirms each field.
• The report computes the properties, and the checks fail closed. The exported R script runs in any R, including R Commander, so the two tools work together.
• RcmdrPlugin.DoE has more options today (blocking, split-plot designs and all wrapper functions), runs at native speed and has analysis menus. It needs no network and no LLM cost for the design step.

2.1 Process

This specification came from one session in Claude Code, an AI agent that works in a terminal. It reads and writes files and runs commands with the user's approval. The model was Claude Opus 5.5 at the "xhigh" effort level. The agent loaded the brainstorming skill from the superpowers plugin, version 6.4.1. A skill is a packaged set of instructions for one kind of task. This skill sets the order of work: read the project, ask one question at a time, compare two or three approaches, present the design in sections, write the spec, review it, and get approval before any code. First, three research subagents worked in parallel. A subagent is a separate agent for one task, and it returns a written report. They read the two papers in refs/, the FrF2 2.3-5 manual (with probes in native R 4.6.1) and the ChatISA code and issues. Then the agent asked nine multiple-choice questions and discussed the answers in chat. This gave twelve decisions (D1 to D12, Section 2.4). The agent wrote version 1 of this spec, checked it for gaps and contradictions, and published it as a private web page. Fadel M. Megahed reviewed the page in sixteen points. Two more subagents read the TypeSafe Jev documentation and an earlier repository that solved equation rendering. The agent drew ten mockup screens in the visual companion of the brainstorming skill, with real FrF2 output. The authors chose among the options. This gave seventeen review decisions (R1 to R17), which version 2 applies. The transcripts, the mockups, the skills and the decision log are in replication_materials/ (Section 15).

2.2 Process figure

• "By date" lists the events. Each event names its stage and its actors.
• "By stage" groups the same events by stage (columns) and actor (rows). A link goes from each event to its cell, and from each cell back to its events.
• The paper uses the "By stage" view as a static figure.
• People write the text of each event. The sync script (Section 15) fills the counts and the times from the transcripts. A unit test checks that each cell of "By stage" matches its events.

2.3 Terms for readers new to agentic tools

2.4 Decisions

Decisions from the brainstorming session (version 1):

Review decisions (version 2). The decision log gives the full text and the evidence for each.

Review decisions (version 3), from the review by Alan R. Vazquez on 2026-10-16. The decision log gives the full text and the evidence for each.

Later plans record their decisions in the plan. Milestone 3B has P1 to P20 and Milestone 3C has C1 to C18. The alpha release has A1 to A18 in its design (2026-10-17-alpha-release-design.md) and AP1 to AP14 in its plan (2026-10-17-alpha-release-plan.md). Sections 13, 14.4, 16 and 17.1 give the result.

3.1 Evidence from the collaborators' papers

• Vazquez, Rother and Charles-Gonzalez (Quality Engineering, 2026) asked LLMs to write design tables directly.
  • Both models gave the optimal design reliably only up to about 8 factors.
  • Example: GPT-5.1 gave the MA design in 0 of 10 runs for 16 runs and 12 factors.
  • Some outputs had resolution 1 or 2. These outputs were not regular designs.
• Ying, Vazquez and Xu (arXiv 2607.05537, 2026) showed that the order and the wording of prompt parts change LLM success a lot.
  • Example: "n runs and m factors" gave lower success than "m factors and n runs".
• Conclusion for this tool: do not let the LLM make the design. Let FrF2 make it. Measure the translation.

3.2 Facts about FrF2 2.3-5 (measured with native R 4.6.1)

• FrF2() ignores resolution when you also give nruns. It gives only a warning.
• When no catalogue design meets the requested resolution, FrF2 makes a full factorial. For very large factor counts this can fail with a memory error.
• FrF2 accepts duplicate generators without an error. The result has aliased factors.
• Catalogue names with a running number (for example 7-3.1) are ranked by MA. Names without a running number (for example 40-29) are "good" designs and are not necessarily MA. Some sizes hold both: in 2048 runs with 28 factors, 28-17.1 is MA and 28-17 is an older design that is not MA.
• design.info(d)$catlg.entry holds the catalogue entry. Its stored WLP gives $(A_1, \ldots, A_L)$, where $L$ is between 4 and 13 and differs between entries.
• randomize = TRUE is the default. With seed, the run order is reproducible.
• seed must be an integer of at most 2,147,483,647 (.Machine$integer.max). A larger value is converted to NA with a warning.
• DoE.base::generators(d) works only when an object named catlg is on the search path. Without library(FrF2) it stops with "alias information can only be provided, if catlg is available".
• generators() is exported by DoE.base, not by FrF2. aliases() is exported by FrF2.
• All FrF2 dependencies exist as WebAssembly binaries for R 4.6 at repo.r-wasm.org (about 24.5 MB in total). WebR 0.6.0 contains R 4.6.0.
• design.info(d)$nruns includes the center points; ncube holds the cube runs.
• With center points, desnum holds real values and design.info(d)$coding holds the coding formulas.
• Full factorials have no aliased and no catlg.entry; without center points their desnum columns are named A1, B1, ....
• 19 catalogue entries share rank 1 with a letter suffix (41-34.1a).
• 12 rank-1 names in 256 runs lie outside the documented MA region.
• The catalogue has 89 entries with a wrong stored WLP, 129 with NA values in the stored WLP, 5 broken entries (fewer generators than added factors), and 2 entries with more generators than added factors (26-17.1 and 27-18.1). Each of these 2 entries stores 19 generators. FrF2 builds the design from the first 17 or 18, but design.info(d)$aliased names factors that the design does not have. The tool refuses the 5 broken entries and these 2 entries. See replication_materials/findings/.
• The probe scripts are in replication_materials/mockups/2026-10-14-ui/source/.

3.3 Facts about TypeSafe Jev (from the documentation, read on 2026-10-14)

• Jev is the first "System One" model of TypeSafe. It returns typed judgments and probabilities. It does not write text.
• Question types: Choice (one of up to 255 options), Noul (the probability of "yes") and Score (a position on 2 to 10 ordered levels).
• Most requests complete in about 100 ms. Input costs USD 0.042 per million tokens. Output is free.
• Versions can be pinned (jev-1.13.0). The alias jev-latest moves when a new version ships.
• Jev is not deterministic. With byte-identical requests, no label changed in 22 questions × 6 runs, but the probabilities moved by up to 0.16 (measured on 2026-10-15). TypeSafe's own test, in which the label changed on 2 of 8 Choice questions, put a random value in each request.
• The documentation says that Jev is not a calculator and that it leans toward the first Choice option.
• Measured: the option order moved the probabilities of borderline outcomes by up to 0.28, but not toward the first option. A lean toward the first option appeared only when no option fitted.
• Measured on 2026-10-16: Jev judged a 7-factor request with level ranges ("temperature 150 to 200 C, pressure 1 to 3 bar, ...") as out of scope. The three LLMs gave a design request, and Jev's own number answers read 16 runs and 7 factors. Since then, Jev's outcome is a flag only (Section 5.6).
• Inputs are not used for training. Retention for abuse monitoring is allowed. Zero retention is for enterprise customers only. The service is hosted in the United States. The SDK 0.6.0 was released on 2026-09-15. jev-1.14.0 was released on 2026-10-14 as jev-preview; on 2026-10-15, jev-latest still pointed to jev-1.13.0. The Master Customer Agreement (Sections 4.1 and 4.3) gives TypeSafe a perpetual right to derive "Telemetry", including "learnings", from requests and to process it without restriction.

3.4 ChatISA

• ChatISA is an in-house generative AI tool for information systems education at Miami University. Its source is on GitHub. This tool reuses its stack, its WebR worker pattern, its tan and ink design tokens and its accessibility fixes. Review decision R19 replaced the accent colour and the chart palette (Section 9.8).
• Paper: Megahed et al. (2024), arXiv:2407.15010 (Section 19).
• The current version on GitHub (main branch, 6.10.0) is much more advanced than the version that the latest arXiv version (v2, May 2025) describes. The latest GitHub release tag is v5.0.3 (December 2025).

3.5 Facts about WebR 0.6.0 (measured on 2026-10-15)

• FrF2 and jsonlite need 29 packages from repo.r-wasm.org (25.3 MB). The repository has .tgz files only.
• The worker installs each package with utils::untar(..., tar = "internal"). The default method calls system(), which WebR does not have.
• FrF2 orders its alias chains with the collation of the R session. WebR uses C; native R on Windows uses English_United States.utf8. So the report helper sorts the chains by factor position (Section 7.5).
• webR.interrupt() stops R, but WebR 0.6.0 then does not answer. So Cancel stops the worker and starts a new one (Section 7.9).
• In Node on Windows, WebR 0.6.0 loads R.js with a C:\ path, which Node refuses. The copy of WebR that the app serves (public/runtimes/webr/) has a one-line patch (scripts/lib/runtime-manifest.mjs) and a package.json that marks it as CommonJS, because the project is an ES module package. Browsers do not use either.
• The report JSON of WebR and of native R 4.6.1 is byte-identical, except for the R version (replication_materials/tests/perf/m2-differential.csv). All 879 cases of the full differential set agree.
• IDBFS is not supported under the SharedArrayBuffer channel of WebR 0.6.0 (measured in Chromium). The worker uses the PostMessage channel, which cannot interrupt R. Cancel does not need an interrupt (Section 7.9).
• WebR does not answer a syncfs request when IndexedDB fails inside its callback. So the worker first opens a small test database (ffd-probe). If that fails, the R library stays in memory for the session.

3.6 Facts about the LLM providers (measured on 2026-10-15 and 2026-10-16)

• gpt-6.1-sol, claude-opus-5-5 and gemini-4-argon answer with the model-facing schema of Section 5.2 as written, through the Vercel AI SDK 7 (generateText with Output.object). Each response reports the model id that answered.
• All three providers accept the schema's unions of string and number, its number enum and its nullable object.
• None of the three models accepts a temperature: OpenAI and Anthropic refuse it with HTTP 400, and Google ignores it.
• The lowest reasoning settings: OpenAI reasoningEffort: "low", Anthropic effort: "low" (thinking cannot be turned off), Google thinkingLevel: "low".
• Latency at the lowest setting, 100 calls for each LLM on 2026-10-16 (median, p95, p99): OpenAI 4.5, 7.9 and 8.4 s; Anthropic 4.1, 10.3 and 13.2 s; Google 8.1, 38.5 and 73.8 s. Jev (20 calls): 0.12, 0.22 and 0.24 s. The data are in replication_materials/tests/perf/.
• Google's delay does not depend on the length of the answer (correlation 0.08). A quarter of its calls took more than 20 s, and 5 of 100 took more than 40 s.
• Cost per translation (token means of the 100 calls): OpenAI about USD 0.005, Anthropic about USD 0.018, Google about USD 0.007, Jev about USD 0.0001.
• gemini-4-argon is not on the free tier: a free-tier key gets HTTP 429 with a quota of 0. The project's key works after billing was enabled on 2026-10-15.
• The probe scripts are in replication_materials/plans/m3a-probes/.

4.1 Overview

4.2 Rules

1. LLM output is data, not code. The LLM fills a typed spec. R code comes only from the codegen templates.
2. Only the request text leaves the device during design work. The text is the request and the clarification answers. In single mode it goes to one provider. In ensemble mode it goes to up to four providers (Section 5.6). The notice names the providers of the current mode.
3. Chart data leave the device only after an opt-in (Phase 2). A dialog shows the exact data before the first request.
4. The LLM is optional. The user can open an empty spec card and fill it. This path also works when the API is not available or a spend cap is reached.
5. Checks before R. Feasibility pre-checks use a catalogue summary. They stop FrF2's silent behaviours before R runs.
6. Checks after R. ffd_report() computes the properties from the design. If a check fails, the tool does not show the design ("fail closed").
7. One translator module. The API route and the evaluation harness use the same module. The paper measures the production code path.
8. No R on the server. R runs only in the browser (and in native R for tests).
9. Projects stay on the device. IndexedDB holds the projects. There are no accounts and no server storage.

4.3 Request lifecycle

1. The user writes a request in step 1 (Describe) and selects "Interpret request".
2. The client sends the request text and any clarification answers to POST /api/translate.
3. The server calls the voters of the current mode in parallel, validates each result and computes the agreement flags (Section 5.6).
4. The server returns one of four outcomes (Section 5.1) with the flags.
5. For design_request, the client validates the spec again, runs the evidence check and the pre-checks, and shows the spec card in step 2 (Confirm).
6. The user edits the card if necessary, resolves each flagged field, and selects "Generate design".
7. Codegen makes the R code. The worker runs it, then runs ffd_report().
8. Step 3 (Design) shows the results only if all post-checks pass. The project is saved on the device.
9. If the user edits the card again, steps 6 to 8 run again. There is no LLM call.

5.1 Outcomes

Both fixed messages have a ? button ("What this tool does"). The messages do not contain alternatives.

5.2 Model-facing schema

OpenAI strict structured output needs all fields present. So the model-facing schema uses nullable fields. All three providers accept its unions of string and number (Section 3.6). The server then applies a strict zod discriminated union. This is the ChatISA pattern. All three LLM providers get the same schema through the Vercel AI SDK.

TranslationResult = {
  outcome: "design_request" | "needs_clarification" | "out_of_scope" | "not_supported_in_version",
  spec: CoreSpec | null,                  // only for design_request
  clarification_question: string | null,  // one question
  scope_category: ScopeCategory | null,   // fixed enum
  evidence: { field: string, quote: string }[]
}

CoreSpec = {
  design_family: "regular",
  factors: { name: string, low: string | number, high: string | number,
             unit: string | null, quantitative: boolean }[],
  nruns: 4 | 8 | 16 | 32 | 64 | 128 | 256 | 512 | 1024 | 2048 | 4096 | null,
  min_resolution: number | null,   // integer >= 3; pre-checks test feasibility
  replications: number,     // integer >= 1, default 1
  repeat_only: boolean,     // default false
  center_points: number,    // integer >= 0, default 0
  randomize: boolean,       // default true
  seed: number | null       // null: the client sets the local date as yyyymmdd (Section 7.3)
}

Constraints that the strict zod schema applies:

• At least one of nruns and min_resolution is not null.
• factors.length >= 2. The number of factors is factors.length.
• Factor names are unique and valid as R names after a fixed sanitizing rule. The rule is in lib/spec/names.ts. Accents are removed, and other letters and digits outside ASCII become their code points in hexadecimal (for example "温度" becomes X6E29_5EA6), so names in any script give distinct R names.
• low and high are different.
• If center_points > 0, all factors are quantitative.
• seed, if given, is an integer from 1 to 2,147,483,647.

5.3 Scope categories (fixed enums)

• out_of_scope: more_than_two_levels, mixed_levels, mixture, response_surface_modeling, optimal_design, computer_experiment, adaptive_design, not_a_design_request.
  • adaptive_design means designs that change while the results arrive, for example sequential optimization. Foldover is not in this category.
• not_supported_in_version: blocking, split_plot, hard_to_change, estimable_interactions (Later, R21), user_generators, catalogue_choice, fraction_choice, alternative_criterion (for example MaxC2; minimum aberration stays the selection rule, R21), plackett_burman, foldover (Phase 3), analysis (until Phase 2).
• Boundary rule: "check for curvature" is in scope (center points). "Model curvature" is out of scope.

5.4 Rules for the translation

1. Do not invent a size. If the request has no run count and no resolution need, the outcome is needs_clarification.
2. Fixed phrase rules for resolution. The prompt contains the rules. Examples:
   • "main effects not aliased with each other" -> III.
   • "main effects clear of two-factor interactions" -> IV.
   • "estimate all two-factor interactions" -> V.
3. Defaults. If the request does not give a value, the spec uses the default. The card shows a "default" label.
4. Default factor names. If the request gives a count and no names, the names are A, B, C, and so on (FrF2's labels: no I and no i; above 50 factors, F1, F2, ...). Default levels are -1 and +1.
5. Any input language. The user can write in any language (for example Spanish). The interface is in English in Phase 1.

5.5 Evidence check (deterministic)

• Each value that is not a default must have an evidence item.
• The quote must occur in a user turn of the thread. The check normalizes case, white space and Unicode before it compares.
• For a number field (nruns, replications, center_points, seed, factor levels), the quote must contain the number as digits or as a number word. A small number-word table covers English and Spanish.
• For min_resolution, the quote must occur in the text. It does not need a number, because the phrase rules (Section 5.4) map words to a resolution.
• For a factor name, the quote must contain the name.
• For default factor names (A, B, C, ...), the quote must contain the number of factors.
• If a value fails the check, the voter gets one repair retry. Section 5.7 gives the result when the retry also fails.
• The card shows each quote in a "From your text" column. If the user edits a value, the column shows "Edited by you".
• Units and the quantitative flag need no evidence item. They follow from the levels.

5.6 Parallel translation and agreement flags

The translation step can call several independent translators ("voters") in parallel. Agreement among them is a signal for the user. It is not a vote that replaces the user.

Voters

• Jev cannot write factor names or levels. These fields come from the LLMs only.
• Code finds the candidate numbers (digits and number words). Each candidate has a label with its position and its context, so the same number can occur twice.
• Jev needs only the request text, so it runs at the same time as the LLMs.

Agreement rule

• The server normalizes each voter's spec (names, units, number formats) and compares the fields.
• If all voters agree on a field, the card shows "4 of 4 agree".
• If voters disagree on a field, the card marks the field and shows each value with its count, for example "16 runs: 3; 32 runs: 1". The user must select a value before "Generate design" is available. This is the same mechanism as "Check this value" (Section 5.7).
• If all voters agree on the outcome, the tool continues with that outcome.
• If the LLM voters disagree on the outcome, the tool asks one clarification question. It uses a question that an LLM voter wrote. If no voter wrote one, it uses a fixed template.
• Jev's outcome is a flag only. If it differs from the outcome of the LLM voters, the tool continues with the LLM outcome, and the agreement flags show Jev's outcome with its count. Jev's numbers count whenever an LLM voter gave a design. Jev's category is compared only when an LLM voter also gave a category (decision of the authors, 2026-10-16).

Deadline and missing voters

• The step has one deadline: the largest p95 of the LLM voters, rounded up to a whole 10 s. On 2026-10-16 it was 40 s, from 100 calls for each LLM voter (Section 3.6; decision of the authors). config/translate-budgets.json holds the value, and TRANSLATE_DEADLINE_MS overrides it.
• A voter that misses the deadline is left out at once: the server does not wait for its answer and does not read a late answer.
• A voter that misses the deadline or fails is left out. The card says "3 of 4 checked".
• The card names each voter (OpenAI, Anthropic, Google, Jev) with its state: answered, missed the deadline, failed, or left out before the call (spend cap reached or no key) (request of the authors, 2026-10-16; Milestone 3B).
• If fewer than two voters return a valid result, the tool shows the single result with the label "Not cross-checked".

Modes

• TRANSLATE_MODE=single calls one LLM. TRANSLATE_MODE=ensemble calls all voters.
• The evaluation compares the modes (Section 12.3). A rule written before the test-set run selects the production default (Section 12.4).
• The privacy notice names the providers of the current mode (Section 13.3).

5.7 Server pipeline

1. Apply input limits: at most 4,000 characters per turn and 3 clarification rounds.
2. Apply the rate limit and the spend caps.
3. Call the voters of the current mode in parallel, with the versioned prompt and the model-facing schema.
4. Validate each LLM result with the strict zod schema and the evidence check.
5. If an LLM result fails validation, make one repair retry for that voter. Send the validation errors to the model.
6. If the retry fails the schema validation, or a deadline or a provider error stops it, leave that voter out. If the first answer passed the schema validation, keep the first answer, with its failed fields marked (item 7).
7. If the retry passes the schema validation but fails the evidence check, keep the result. The card marks each failed field "Check this value". The user must edit or confirm each marked field.
8. Compute the agreement flags (Section 5.6).
9. If no voter returns a valid result, return "Could not interpret the request. Fill in the form yourself."
10. Return the result with prompt_version, schema_version, jev_questions_version, the mode, the voters of the mode that were left out before the call (left_out) and the model id that each provider returned.

5.8 Prompts and models

• The model ids come from .env (Section 13.2).
• gpt-6.1-sol has no dated snapshot. The server logs the model id that each response returns. Live smoke tests find changes.
• Jev is pinned to jev-1.13.0.
• None of the three models accepts a temperature (Section 3.6). Each call uses the lowest reasoning setting of its provider, and OpenAI calls set store: false. The development set decides the final values.
• The prompt is a versioned file. Few-shot examples come from the development set only.
• Prompt part order is an evaluation factor, because the order changes LLM success (Ying, Vazquez and Xu, 2026).

6.1 Catalogue summary

A build-time R script (scripts/build-catalogue.R) reads FrF2's catlg in native R. It writes data/catalogue-summary.json. For each number of factors, the file holds:

• the run sizes that the catalogue covers;
• the highest resolution in each run size, and the entry name;
• the smallest run size for each resolution.

6.2 Rules

Each rule has a native R test that reproduces the FrF2 behaviour that the rule prevents.

• The messages state facts. They do not tell the user what to do.
• The performance tests set the supported limits (Section 7.9).
• Codegen never passes nruns and resolution together to FrF2.
• The rules are in lib/spec/precheck.ts. replication_materials/tests/fixtures/precheck-cases.json has one input for each rule: a unit test checks the messages, and a native R test checks the FrF2 behaviour (Milestone 3B).

7.1 R code

All R code, in the app and in the exports, uses the pkg::fun() style. It contains no library() call. Example for the coating study used in the mockups:

d <- FrF2::FrF2(nruns = 16, nfactors = 7,
                factor.names = list(Temp = c(150, 200), Pressure = c(1, 3),
                                    Speed = c("slow", "fast"), Time = c(10, 20),
                                    Feed = c(0.10, 0.20), Depth = c(1, 2),
                                    Coolant = c("off", "on")),
                randomize = TRUE, seed = 20261014)

• If the spec has nruns, the call uses nruns and nfactors. It does not use resolution.
• If the spec has only min_resolution, the call uses nfactors and resolution.
• The factor order in the spec sets the factor-to-column assignment.
• The R code view shows the exact string that the worker runs. A unit test asserts this. The label is "R code".
• The exported .R file starts with a comment block: app version, R, WebR, FrF2 and DoE.base versions, and the date.

7.2 Namespaced calls and the search path

• Some FrF2 and DoE.base functions look up objects on the search path. DoE.base::generators(d) needs catlg there (Section 3.2). Under the pkg::fun() style, nothing is attached, so such calls fail.
• Rule: the report helper does not call functions that need the search path. It reads the generators from design.info(d)$catlg.entry[[1]]$gen (column numbers in Yates order) and maps them to the factor names.
• Test: every R snippet that the app runs or exports runs in a fresh R session with only the base packages attached, in native R and in WebR. The test fails on any error or warning.

7.3 Randomization and the seed

• By default, the run order is random (randomize = TRUE).
• The default seed is the local date of the user's device as yyyymmdd, for example 20261014. The card shows it with the label "default (today's date)".
• The user can type another seed: an integer from 1 to 2,147,483,647. A seed from the request text needs evidence (Section 5.5).
• The LLM never selects a seed.
• The seed is in the R code, the exports and the project file. The project file also records the full creation time and the time zone.
• Designs with the same number of runs made on the same day get the same run order. This is acceptable: each experiment is still randomized, and the user can change the seed.
• If the user turns randomization off, the design uses the standard order and the R code has no seed.
• Test: for a fixed set of seeds and designs, the run order in WebR (R 4.6.0) equals the run order in native R 4.6.1.

7.4 Report helper

• r/ffd_report.R is the one source file. Native R tests source it. The build copies it to public/r/ for WebR.
• ffd_report(d, spec) reads the design. It does not change the design.
• A "How we compute this" link in the help opens the file.

7.5 Report content

In the report, the Request section shows the request text once. The turns after the first follow under "Clarification", as "Question from EXPERTA" and "Answer" (decision R2 of the report print redesign).

7.6 WLP computation

For a two-level design with $N$ runs and $n$ factors, code each level as $-1$ or $+1$. Let $d_{ab}$ be the Hamming distance between runs $a$ and $b$: the number of factors at which the two runs differ. The word length pattern $(A_1, \ldots, A_n)$ is (Xu and Wu, 2001):

where $K_j$ is the Krawtchouk polynomial

The resolution is the length of the shortest word:

• A regular design is a coset of a linear code. A coset is distance-invariant: the distances from each run to all runs have the same distribution. So the tool uses the distances from run 1 only:

• This costs $O(Nn)$ operations instead of $O(N^2 n)$. The general double sum stays for non-regular designs (later phases) and for tests.
• The computation uses one replicate and the cube points only. Center points are not used.
• For a regular design, each $A_j$ must be an integer. The helper checks this within a small tolerance. If the check fails, the report fails closed.
• Tests compare three sources for each catalogue entry within the supported limits: this computation, DoE.base::GWLP(), and the stored catalogue WLP. The comparison with the stored WLP uses the stored values that are not NA and accepts the verified errata (Section 7.8, check 6).

7.7 Guarantee table

• scripts/build-catalogue.R puts each catlg entry into one class:
  • MA: the entry is first in an MA ranking that the FrF2 documentation calls complete.
  • catalogue_not_MA: FrF2 calls the entry "good, not necessarily MA".
  • full_factorial.
• The script quotes the FrF2 documentation text for each rule.
• Entries that need their own rule have more classes: rank1_tie (entries that share rank 1), rank1_outside_documented_region, ranked_not_first, stored_wlp_error and broken_entry.
• The interface shows only the statement for the claim of the design.

The claim. The report has two strings, catalogue.claim and catalogue.claim_basis (report 1.3.0, Section 8.1). The claim comes from the class of the entry, the size of the design and the reviewed range of its region (decisions F1 to F5 of 2026-10-19-alpha-review-fixes-plan.md). The statement comes from the claim.

The rules apply in this order. The first rule that fits sets the claim and its basis.

1. A full factorial design gets full_factorial (basis full_factorial).
2. An entry of class MA or rank1_tie, with a number of factors inside the reviewed range of its region, gets MA:
   • For 128 runs with 25 to 40 factors, the reference check applies (below). If the computed WLP equals the reference WLP, the basis is region_and_reference_wlp. If it differs, or the report has no WLP, the claim is none (basis reference_wlp_differs).
   • For a tie entry, the basis is region_and_equal_wlp_tie. For the other entries, it is region.
3. The entries 21-16.1 and 22-17.1 (class stored_wlp_error) get MA (basis errata_verified).
4. An entry of class catalogue_not_MA or rank1_outside_documented_region gets good (basis frf2_good).
5. All other designs get none (basis none).

The reviewed ranges. Each region has a range of factors in which the tool makes the MA claim (reviewed_min and reviewed_max in data/guarantee-table.json). For 12 regions, this range is the range that Alan R. Vazquez confirmed on 2026-10-19. For the region of 128 runs with 25 to 64 factors, he confirmed 40 to 64 factors (#61). For 25 to 39 factors, he answered "Not sure". The claim for these sizes rests on decision F2 (the evidence of Xu, 2009, and the check at generation time). This question is asked again. A later review can narrow one range. The source for each region (the FrF2 manual):

The reference check. For 128 runs with 25 to 40 factors, the report compares the WLP of the design with the WLP of the MA design of Xu (2009, Section 4; Table 9 of the preprint of 2007-09-28). r/ffd_report.R builds that design from the columns that Xu prints (FFD_XU2009_128; the function ffd_reference_wlp). It computes its WLP with the same function as the report (Section 7.6). The report passes the WLP that it computed from the cube points of the design. The two WLPs must be equal at every length. The check proves that the design has the aberration of Xu's design. It does not prove that Xu's design is MA: that rests on his complete enumeration of the odd 128-run designs of resolution IV, which exist only up to 40 factors. If the check fails, the claim is none, and the statement names only the entry (fail closed).

The other entries. Decisions F3 to F5 set the claims of the entries that the documented regions do not cover. Alan R. Vazquez is asked to confirm them (Section 14.4).

• The 19 entries that share rank 1 (128 runs with 41, 42, 43, 44 and 50 factors) get MA. They are the only catalogue entries of their sizes, and their WLPs are equal at every length (measured from the design matrix).
• The 223 entries without a running number (for example 29-19) and the 12 rank-1 names outside the documented region (69-61.1 to 80-72.1, 256 runs) get good.
• 21-16.1 and 22-17.1 get MA. Their stored WLP is wrong only because one number is split in two. By the computed WLP, each has less aberration than every other entry of its size.
• The other 87 entries with a verified stored-WLP error, the 2,443 ranked entries after the first, and the 7 refused entries get none. In all, the 3,099 catalogue entries have 327 MA, 235 good and 2,537 none.
• The 129 entries with NA values in the stored WLP are compared on the known values only (Section 7.8).
• The tool refuses the 5 broken entries and the 2 entries with more generators than added factors (26-17.1 and 27-18.1, the FrF2 defaults for 512 runs with 26 and 27 factors). To accept these 2 entries, the tool must recompute their aliases and clear 2FIs, and the review must check them first.

What the review found. Reviewer: Alan R. Vazquez, on 2026-10-19 (GitHub issues #7 to #65, commit a3790d3). The review block of the table has the status partial.

1. He agreed to the sentences of the classes and to 12 of the 13 regions. He kept the bounds of the regions 1024, 2048 and 4096 (answers #57 to #60). The claim now applies in these regions. For the 13th region (128 runs, 25 to 64 factors), he confirmed 40 to 64 factors.
2. He agreed that the claim holds with center points, replications and a request that gives only a resolution (answer #49). The claim of the selected entry stays in these cases.
3. He was not sure about the 128-run region for 25 to 39 factors (#61), the entries without an MA claim (#63) and the glossary entry for the factors with all 2FIs clear (#46). Four questions are asked again (Section 14.4): the 128-run region, the entries without an MA claim, and the glossary entries for the factors with all 2FIs clear and for the guarantee statement.
4. The tool still refuses 26-17.1 and 27-18.1. He is revising the FrF2 findings (#64). The question stays as it is and is not asked again. The two entries stay refused until his revision.

A software test also checks that FrF2 selects the .1 entry whenever one exists for the requested size.

7.8 Post-checks (fail closed)

The report checks the following before the tool shows the design:

1. The number of rows equals the expected count. The formula for the expected count (runs, replications, repeat_only, center points) comes from measured FrF2 behaviour. A native R test covers each combination of these options.
2. Number of factors = number of factors in the spec.
3. Each factor column is balanced in the cube points.
4. All coded values are -1 or +1 (or 0 for center points).
5. The computed resolution is at least min_resolution, if the spec gives one.
6. If the design has a catalogue entry, the computed WLP equals the stored catalogue WLP at each stored length that is not NA, or the entry is in the verified errata list (data/catalogue-errata.json; decision of 2026-10-15).
7. Each $A_j$ of a regular design is an integer.
8. If the design has a catalogue entry, the entry has one generator for each added factor, and each added factor column equals the product of the base factor columns that its generator names.
9. Each factor in the alias chains of design.info(d)$aliased is a factor of the design.
10. If the spec gives them, repeat_only, randomize and seed equal the settings that the design stores. The worker always gives them.

A check that cannot be computed fails. Examples: a WLP that is not computed because the design has NA values, and a value of length 0 or NA.

If a check fails, the tool shows "Report check failed. The design is not shown." and the name of the failed check. It does not show a table.

7.9 Runtime

• Pinned versions: WebR 0.6.0 (R 4.6.0), FrF2 2.3-5, DoE.base 1.2-5. A self-hosted package mirror holds the binaries. A manifest records the SHA-256 of each file.
• scripts/setup-runtimes.mjs copies WebR from npm and mirrors the packages from repo.r-wasm.org/bin/emscripten/contrib/4.6. The replication materials keep a copy of the 29 package files (replication_materials/runtime-archive/, decision of 2026-10-15). The setup takes each file from that copy and downloads only a file that the copy does not have.
• One module worker. The page starts the worker when it opens ("prewarm"). The worker loads WebR, checks the size and the SHA-256 of each package file, installs the packages, loads the FrF2 namespace, and sources the report helper. Before WebR starts, the worker checks that webr-worker.js, R.js and R.wasm can be loaded, so a missing file gives a stated reason.
• IndexedDB cache. The R library is kept in IndexedDB (IDBFS). After the first visit, and after a restart of the worker, it loads from there with no download. If IndexedDB cannot be opened, the library stays in memory for the session.
• Status. The footer shows "R engine: starting R" before the first package message, then "R engine: loading (n%)", then "R engine: preparing the R packages" after the last package byte, then "R engine: ready" (decision C3). A polite live region announces the changes.
• Headers. COOP same-origin and COEP require-corp on the pages, the worker path and the runtime path. These headers make the pages cross-origin isolated. The worker uses the PostMessage channel of WebR, because IDBFS is not supported under the SharedArrayBuffer channel (Section 3.5). Cancel stops the worker and starts a new one, because WebR 0.6.0 does not answer after an interrupt.
• Time budgets:
  • Engine cold start: no hard limit. A progress bar shows the bytes of the R packages. WebR's own start (R.wasm, 12.3 MB compressed) sends no progress; the footer says "R engine: starting R" (decision C3, measured). After the last package byte, the footer says "R engine: preparing the R packages" and the stall timer stops, because the save to IndexedDB and the load of FrF2 send no progress. "Retry" appears only if progress stops for a set time, counted from the first progress message.
  • Design run: from the start of the run, the footer shows "R engine: computing the design (m:ss)" and an activity bar. At the soft limit, it shows "R engine: still running (m:ss)" and a Cancel button. A hard limit (default 10 minutes, set in config) stops a worker that does not respond. Step 3 shows the same indicator and Cancel (Milestone 3B).
  • FrF2 search limits (Phase 3, for example max.time) must be less than the soft limit.
  • The soft limit is at least three times the observed p99 warm time from the performance tests. The measured values are in config/runtime-budgets.json, and the supported limits in data/supported-limits.json.

8.1 Project model

A project is the unit of work. It holds:

• the request text and the clarification turns (optional in exports: Step 6, "Request text", checked by default; decision C1);
• the confirmed spec, the seed and the factor order;
• the record of the interpretation: the model ids, the mode, the prompt, schema and Jev-question versions, and the time (older projects have none);
• the R code that ran, and the versions;
• the run sheet and its SHA-256;
• the report values (report version 1.3.0 since 2026-10-19: the field catalogue.claim and the claim statements of Section 7.7 are new; a stored report of version 1.2.0 is made again by the rule of Section 8.2);
• in Phase 2: the responses and the analysis settings (not images; charts are made again).

8.2 Storage on the device

• IndexedDB holds the projects. A write is complete only when its transaction completes, and the connection closes when another tab opens a newer version of the database.
• A stored record that matches no schema is listed in the sidebar as "cannot be opened", with Export and Delete. It is never hidden (decision P19 of the Milestone 3B plan).
• A report with another report version is made again from the stored spec and seed in step 3. It replaces the stored report only if the new run sheet has the stored SHA-256; otherwise the stored project stays unchanged and step 3 names the versions that differ (decision P1).
• localStorage holds two small values: a pointer to the last page opened, and the "Show explanations" switch.
• sessionStorage keeps the unsent text of step 1 (key ffd-draft:{project id or "new"}) until the interpretation is stored (decision C15). The text comes back after a plain link and after the browser's Back button. The draft is also removed when the user fills in the form by hand ("Fill in the form yourself"), when its project is deleted, and by "Delete all".
• All storage access is in try/catch. The tool works when storage is not available (for example in private browsing). Then projects last only for the session, and a notice says so.
• The sidebar shows the storage use and has "Export all (.zip)", "Import project" and "Clear". "Clear" asks for confirmation in the page.
• "Export all (.zip)" gives experta-projects-{yyyy-mm-dd}.zip. It holds one {stem}.project.json for each readable project. Names that repeat get -2, -3 (decision C8).
• "Import project" accepts several .json and .zip files. For a .zip file, it imports every *.project.json in the archive (decision C8).
• Each project has a disclosure button "Project actions: {name}" (aria-expanded) with four buttons: Rename, Duplicate, "Export project (.json)" and Delete. It is not an ARIA menu. Rename and Duplicate are not available for a project whose report waits for a rebuild. Delete asks for confirmation in the page (decision C10).
• After a save, the workspace replaces the saved project in its list. It does not read all projects again (decision C5).

8.3 Exports

• Downloads in context give single files where the user works. Step 6 (Report) gives the report and the archive of all files. The project menu renames, duplicates, exports and deletes a project. The sidebar imports projects.
• A test checks that each file is the same from each place where it is offered.
• Every download is announced once, politely: "Download started: {file name}."
• The project file is { "format": "experta-project", "format_version": 1, "app_version": "...", "project": <stored record> } as UTF-8 JSON with two-space indentation and a final newline. It has no export time, so equal projects give equal files. A project whose report waits for a rebuild is exported from its stored record. Import reads the record with the store's readRecord, so a record with the current or an older report version is read (decision C7).
• "Export project" in the project menu always gives the whole project. Without the request text (Step 6, "Request text" not checked), the report and the project file in "All files (.zip)" have no request and no clarification turns, and the project file also drops the quotes of the card: values "From your text" become interpretation values (decision C1). The project name is not part of the request text: while "Request text" is not checked, step 6 shows the note "The project name stays in the file names and in the report. To change it, rename the project."
• The project file in "All files (.zip)" is made from the stored record, as "Export project" makes it. So the two files are equal for the same project.
• The .csv file follows RFC 4180: quoted fields where needed, CRLF line ends, UTF-8 without a byte order mark, "NA" for a missing value, and a final line end. The columns are Run, Std, the real levels (headers with units), then the coded levels (headers "{name} (coded)") (decision C12). Values are written unchanged. A text that starts with "=", "+", "-" or "@" is not changed. The .xlsx file is the file for spreadsheet programs.
• The .xlsx file is written as OOXML with fflate; its hidden sheet experta holds the project id, the run-sheet SHA-256, the app version, the report version and the format version (decision C6).
• The report file is one HTML document: inline CSS, no script, one link (to the app), no external resource, system fonts (Georgia for headings, Arial for text), the accent #625d96, and a print style sheet. The page-break rule of decision C11 of the first plan (each section on a new page) is replaced by the rules below.
• The report has these sections, in this order. Each one has a check box in step 6. A title block with the logo, the project name, the design notation and the date opens the report.
  1. Request: the request text once, then "Clarification" with the later turns.
  2. Specification: the confirmed fields.
  3. How the specification was made: the model ids of the voters and the mode. It also shows the prompt, schema and Jev-question versions and the date of the interpretation, as the card stores them. A project without these values shows "Not recorded". A specification filled in by hand says so. The card stores how it was made (translation, hand or imported), and the section decides by that value. It names the factors that the user removed or added in step 2, and says when the user changed their order.
  4. Design properties and aliasing: the word length pattern in tables of at most 10 word lengths, and the alias groups in two parts.
  5. Run sheet.
  6. R code.
  7. Versions and citation: a table of versions, the SHA-256 of the run sheet and the citation in APA and BibTeX with the link https://expertadoe.app. A toggle that uses only CSS shows one form on screen; the print shows both. The Acknowledgment closes every report, outside the numbered sections: "The API and hosting costs of EXPERTA are funded by the Raymond E. Glos Professorship in Business at Miami University, held by Fadel M. Megahed."
• Print rules of the report (decisions R1 and R4 to R8 of the report print redesign). No script runs in the report, so these rules are in CSS only.
  • Sections flow with no forced page break. A heading stays with the text after it. A small table, a code block and an alias group do not split across pages.
  • The run sheet shows in a scroll box on screen, at most 75% of the window high. The header row and the Run column stay in view, and the rows are in groups of 256. The content-visibility property is not used. CSS containment does not apply to table row groups. The three view-time runs measured content-visibility first each time, so the run order and the effect cannot be told apart (replication_materials/tests/perf/print-layout.csv).
  • In print, the run sheet uses landscape pages. A measured width model (config/print-layout.json, packFactorBlocks) packs the factor columns into blocks that fit one page width. Each block repeats Run and Std order, and the header row repeats on each page. The last block has the blank columns Response and Notes. A page holds at least 25 rows, measured in Chromium for Letter and A4.
  • The number of rows in print is a choice in step 6 (Section 9.3). A limited print shows a note under the table. The HTML file has every row.
  • A running header and footer come from @page margin boxes with fixed app texts. The top has "EXPERTA · Design report" and the date. The bottom has "expertadoe.app · version {APP_VERSION}" and "Page {n} of {N}". Chromium and Edge print them. Other browsers print without them.
  • During print(), the app sets the page title to "{project name}: EXPERTA design report" and then restores the old title. So the PDF file name has the project name.
  • The design notation prints in the accent colour. Numbers use tabular digits. The report is the same file in the frame, in "Report (.html)" and in the archive.
  • Four reference prints are in replication_materials/findings/2026-10-19-report-print-after-*.pdf: 7 factors in 16 runs, 32 factors in 64 runs, a resolution III design, and 4,096 runs.
• Step 6 shows the report file in an <iframe> with srcdoc. The preview frame has sandbox="allow-same-origin allow-modals allow-popups allow-popups-to-escape-sandbox". Chromium ignores print() in a sandboxed frame without allow-modals. The link to the app has target="_blank", so it opens a new tab and the frame keeps the report. No script runs in the frame, because allow-scripts is not given. "Print / PDF" prints that frame.
• The SHA-256 of a run sheet is the SHA-256 of the UTF-8 text JSON.stringify({ labels, run, std, levels, coded }) of the report (lib/report/sha.ts, decision P12).

8.4 Round trip with responses (Phase 2)

1. The user downloads the run sheet as Excel and runs the experiment.
2. The user types each result into the response column. More response columns are allowed.
3. Step 4 (Responses) reads the file. The factor columns must equal the stored design, run by run. A sorted or edited sheet fails this check, and the tool names the first row that differs.
4. R adds the responses to the design object with DoE.base::add.response().

8.5 Import

• The tool validates the project file with its schema.
• The tool makes the design again from the spec and the seed, and compares the SHA-256 with the stored run sheet.
• If they are equal, the tool shows "Project imported. The rebuilt design matches the stored run sheet (SHA-256)."
• If they differ (for example after an FrF2 update), the tool keeps the stored run sheet, because that is the design that was run. It shows the versions that differ. When no version differs, the list has one item: "None. The stored R, webR, FrF2 and DoE.base versions equal the current versions."
• The check runs in step 3 of the imported project, in the session of the import (decision C9). Step 3 checks only an imported project that has a stored report, or that waits for a rebuild. A project with a spec and no report makes its first design as usual.
• The check never changes the stored record. A project that waits for a rebuild follows the rebuild rule of decision P1, with "Project imported. " before its text.
• A project with a spec and no card gets a card from its spec. The origin of its values is imported, and the label is "From the project file" (decision C9).
• An imported project whose id is already on the device gets a new id and the name "{name} (imported)". A file never overwrites a stored record. The ids that are already taken are the listed projects, the records that cannot be opened, and the files read earlier in the same selection (decision C2).
• Each import has one announcement: polite for the count only, and assertive when a file failed.
• The file input ("Import project", several files, .json,.zip) is cleared after each read.

9.1 Workspace layout

• Header: the two logos, the name and the current project. There is no Help item in the header.
• Sidebar: "New design", the projects on this device (name, design, size, date) and the storage controls. At narrow widths the sidebar becomes a menu.
• Step rail: 1 Describe, 2 Confirm, 3 Design, 4 Responses, 5 Analyze, 6 Report. Steps 4 and 5 are visible but marked "Phase 2" until Phase 2 ships.
• Footer: Home, Glossary, Architecture, Paper, the version and the R engine status.
• Phase 3 adds options to step 2 (blocking, split-plot, generators) and "Augment this design" (foldover) to step 3. The layout does not change.

9.2 Landing page and later visits

• The first visit to / shows the landing page: a short description, the introduction video, "Start a design", and three blocks titled "Maintained by", "Privacy" and "Citation". The landing page replaces a separate About page.
• On later visits to /, a small script in the page head reads the pointer in localStorage and opens the last page before the page is drawn.
• The logo and "Home" go to /?home. This address always shows the landing page.
• Without storage, every visit shows the landing page.

9.3 Steps

9.4 Design step layout

• The headline: $2^{k-p}$, the number of runs and the resolution, then the guarantee statement.
• Left: the run sheet. Controls: run order or standard order; real levels or coded levels; Print; Download.
• Right: design properties: resolution, WLP, generators, the number of clear 2FIs and the list of clear 2FIs, the factors that have all 2FIs clear (if any), run order and seed, FrF2 messages, and the aliased 2FIs with the real factor names. "Clear" does not take interactions of three or more factors into account (hierarchy principle, Section 1.8).
• Below the run sheet: "R code" (collapsed), with Copy.
• A note says: "To change which effects are aliased, change the factor order in step 2." If some factors, but not all, have all 2FIs clear, the note also names them. Example for 9 factors in 32 runs: "Only the factor in position 9 (now Feed) has all 2FIs clear. To give this property to a different factor, move that factor to position 9 in step 2."
• Below about 1100 px wide, the two columns stack.

9.5 Help at the ? button

• Where: each spec card field, each design property, each step and the two fixed scope messages.
• How: the ? button opens a panel in place, under the item. The panel pushes the content down. It covers nothing. A real <button> with aria-expanded controls it.
• Layers in each panel:
  1. What it means: the reviewed glossary text, with the reviewer and the date.
  2. In your design: sentences computed from the R output of this design. Example: "The 21 2FIs fall into 7 groups of 3. Example: Temp:Pressure = Speed:Feed = Time:Depth."
  3. Watch this part: a link to the matching chapter of the introduction video.
  4. Phase 2, charts only: "Explain this chart with AI", off by default (Section 9.5.1).
• Show explanations: a switch on the step rail opens all panels at once. The device remembers the switch.
• Sources: content/glossary.json (fields key, term, text, reference, reviewed_by, reviewed_on) and lib/report/explain.ts (fixed sentence templates). No LLM writes help text. scripts/build-glossary.mjs builds the MathML into data/glossary.json (decision P16). An entry without a reviewer shows "Draft, not yet reviewed" (decision P3). Until the video exists, a panel has layers 1 and 2 and a link to the glossary entry (decision P13).
• Review status (2026-10-19). Alan R. Vazquez reviewed 33 of the 35 entries. These 33 entries have reviewed_by and reviewed_on (decision F8). The texts of generators, clear_2fis and effect_heredity use his words. The entries all_2fis_clear and guarantee have new texts (decision F9). They show "Draft, not yet reviewed" until he answers again (Section 14.4). The references now include Mee (2009), Ryan and Bulutoglu (2010) and Xu (2009) (decision F10).

9.6 Factor order (isomorphism, Phase 1)

• The card shows the factor order. The user changes it with "Move up" and "Move down" buttons.
• The page announces each move. Example: "Temp moved to position 2 of 7."
• A ? panel explains: the order does not change the WLP, the resolution or the number of clear 2FIs. The order changes which named effects are aliased and which named factors have all 2FIs clear. Example: in the FrF2 design for 9 factors in 32 runs, only position 9 has all 2FIs clear.
• The design properties show the alias chains with the real factor names, so the user can see the effect of the order.

9.7 Wording

• Labels use plain academic wording: "Maintained by", "Privacy", "Citation", "R code", "All files (.zip)", "Report contents".
• No marketing language, no second-person slogans, no exclamation marks.
• Messages state facts. Error messages say what failed and what the user can change.

9.8 Branding

• Name: EXPERTA, with the subtitle "Fractional Factorial Design Translator" (R18). Constants APP_NAME, APP_SUBTITLE and APP_FULL_NAME in lib/site.ts. The browser title and the citation use the full name "EXPERTA: Fractional Factorial Design Translator".
• Interface tokens: accent #625d96 and accent dark #4c4977 (R19: the Dark2 purple #7570B3, darkened, because #7570B3 has only 4.44:1 on white); from ChatISA: light tan #edece2, medium tan #ccc9b8, dark tan #70685c, ink #000000, warm white #faf9f7. The accent has 5.96:1 on white, 5.66:1 on warm white and 5.02:1 on light tan; a unit test checks the WCAG AA values (brand-colours.test.ts).
• Fonts: Source Serif 4 (display) and Arial (body). Self-hosted. No font CDN in the app.
• Chart palette (ColorBrewer Dark2, measured, R19): series purple #7570B3, orange #D95F02, teal #1B9E77, gray #666666, in this order, at most four colours. Dark2 green (2.98:1) and mustard (2.06:1) fail the 3:1 contrast that chart marks need on white. In ColorBrewer's order, teal and magenta are almost the same colour for colour-blind readers (OKLab distance 1.7), and so are orange and brown (1.8). The smallest OKLab distance of the chosen order, for normal vision and simulated protanopia, deuteranopia and tritanopia, is 21.4 for 2 series, 10.7 for 3 and 7.2 for 4 (replication_materials/findings/palette-check.mjs). From 3 series up, direct labels or distinct shapes are required; above 4 series, shapes and labels carry the difference. Black is for text and axes only. FrF2's own plot functions get the palette through their colour arguments.
• Logos: both logos side by side with equal weight, each in its own colours on a white header.
  • Tecnológico de Monterrey: public/brand/tecnologico-de-monterrey-blue.svg (PNG fallback in the same folder).
  • Miami University: the horizontal lockup from ChatISA (assets/logo-horizontal-stacked.png).
• Favicon: an SVG cube with four of its eight corners filled in the accent: the $2^{3-1}$ half fraction.
• Phase 1 has a light theme only. The colours are tokens, so a dark theme can come later.

9.9 Mathematical notation

Lessons from hf_arxiv_control_charts (R Shiny with MathJax):

• Text that the app computes ($2^{7-3}$, $I = ABCE = \cdots$, resolution IV, $A_4 = 7$) is built as HTML elements (<sup>, <sub>). It is not parsed as LaTeX.
• The glossary math is converted to MathML at build time (KaTeX with output: "mathml", pinned). Chrome draws MathML natively, so the app loads no math fonts and no math style sheet, and screen readers read the formulas. This specification page uses the same method (pandoc with MathML output).
• Any LLM text (Phase 2 explanations) uses only \(...\) and \[...\]. The prompt forbids $ as a delimiter. A normalizer runs before rendering: it converts stray $ delimiters, protects currency, repairs JSON escapes that remove a backslash (for example \b in \beta), and removes unmatched delimiters.
• Sanitizing happens before KaTeX, not after it. KaTeX runs with trust: false.
• Display math scrolls horizontally inside its own box.
• A render test checks that the output contains no .katex-error element.

9.10 Accessibility

• Target: WCAG 2.1 AA. Also meet the WCAG 2.2 additions for target size and focus not obscured.
• Skip link. One global announcer (ChatISA pattern): the markup of the app has one polite and one assertive live region (components/a11y/Announcer.tsx). The route announcer of Next.js, in a shadow root, reads only the page title after a navigation.
• Focus moves to the step heading after each step change.
• Each step sets the page title "Step {n}: {title} | EXPERTA". The workspace keeps this title with an observer of the document head, because Next.js puts its metadata title back after each render (decision C13).
• Required fields (factor name, low level and high level) have aria-required="true", and their "*" marker is aria-hidden (decision C13).
• The accessibility sweep is a guard in npm run check on every page. It runs the five custom checks of Section 11 as Playwright assertions (decision C13).
• A select of step 2 whose value is not in its list shows that value as "{value} (not available)". The error of the field stays, and "Generate design" stays unavailable (decision C16).
• Focus rules (decision C14). No focus drops to the page body (WCAG 2.4.3).
  • "Delete all" moves focus to "New design". "Keep projects" returns it to "Clear".
  • "Delete record" moves focus to the "Your projects" heading. "Keep record" returns it to the "Delete" of that record.
  • After a failed "Delete project", focus returns to the "Project actions: {name}" button of that project. After a failed "Delete all", focus returns to "Clear". The store error is shown and announced.
  • Cancel of a translation returns focus to the request field, or to the Answer field.
  • A clarification question moves focus to the Answer field. This applies to the first one, which makes the project, and to each later one. A refusal of the first request moves focus to the request field.
  • The footer Retry and the footer Cancel of an R run move focus to the step heading.
  • "Previous rows" and "Next rows" use aria-disabled at the ends. "Interpret request" and "Send answer" use aria-disabled while busy.
• Status is always text. Colour is never the only signal.
• Required fields have a visible marker and a note that explains the marker.
• The run sheet is a real <table> with a caption and scope headers. Large tables use pagination, not virtual scrolling.
• Single column at high zoom.
• Reduced motion is respected.
• Phase 2 plots get text alternatives and a data table.
• The introduction video has captions and a transcript (Section 10).

10.1 Content

• One video of about 2 minutes 15 seconds for each minor or major release.
• Chapters follow the steps: introduction, 1 Describe, 2 Confirm, 3 Design, 6 Report and files, closing card. Phase 2 adds chapters for steps 4 and 5.
• The narration uses plain academic wording. The draft script has nine beats (mockup 06).
• A Spanish version (narration and captions) ships with version 1.0. A Spanish speaker reviews the translation.

10.2 Production

• Tool: the narrated-demo-video skill (Playwright footage, narrated beats, title cards, ffmpeg assembly).
• Narration: ElevenLabs v4. The word timings from the timestamp endpoint set the captions.
• Music: quiet CC-BY music under the narration, lowered while the narrator speaks, and credited on the closing card.
• Footage: Playwright films the real app. The translation step uses the recorded responses of the end-to-end tests, so filming makes no live API call. FrF2 runs in WebR.
• Output: self-hosted MP4 (1920 x 1080, 30 fps) with WebVTT captions in both languages and a transcript page. No third-party video player.
• Chapter times come from the measured beat lengths. The ? panels link to them ("Watch this part").

10.3 Title cards

• Warm-white field #faf9f7, an accent rule (#625d96, R19) at the top, both logos in their own colours with equal weight, and black titles.
• Measured contrast for normal vision and for simulated protanopia, deuteranopia, tritanopia and achromatopsia (version 2, with the Miami Red rule): black titles 19.9:1 or more, Tec blue 8.3:1 or more, Miami Red 4.8:1 or more (card_contrast.py). The accent rule is measured again before the video release (Section 17.1).

10.4 Keeping the video current

• A stale check compares screenshots of the filmed screens with the current app.
• A change flags the video for the next minor or major release. Patch releases do not need a new video.
• The landing page shows the version that the video records.

12.1 Benchmark slices (Phase 1)

• About 300 prompts. Each prompt runs 10 times.
• Gold specs are written by the team and reviewed by Alan R. Vazquez. Gold designs come from native R.
• Claude drafts the S2 paraphrase candidates. A person reviews each candidate. Claude does not set gold labels.

12.2 Metrics

1. Outcome accuracy ($4 \times 4$ confusion matrix).
2. Field-level exact match.
3. Design equivalence: the catalogue entry and the WLP match the gold design. The factor order counts only when the prompt sets it.
4. Silent error rate: the fraction of requests with a wrong spec and no flag on the card. This is the main safety metric.
5. Flag performance: the fraction of wrong fields that the evidence check or the agreement flags mark, and the false alarm rate.
6. Unanimous error rate: how often a spec on which all voters agree is wrong.
7. Optimality rate on S1, next to the direct-generation results of Vazquez et al. (2026).
8. Consistency across the 10 repetitions. Latency (median and p95). Cost per request.

12.3 Arms (ablations)

12.4 Rule for the production default

• Before the test-set run, the authors write a rule, for example: "Use the ensemble if it reduces the silent error rate by at least X% with a p95 latency of at most Y seconds and a cost of at most Z per request."
• The rule, with X, Y and Z, is committed to replication_materials/evaluation/ before the run.
• The test-set results then select the mode. The paper reports the rule and the result.

12.5 Validity

• Development and test split. Prompt work uses the development set only. The test set is frozen. It runs once for each final prompt version.
• Few-shot examples never come from the test set.
• Raw outputs are saved as JSONL, with the model id that each provider returned. Scoring is a separate, deterministic step. Anyone can make all tables and figures again without new API calls.
• Jev is not deterministic. The 10 repetitions report its variation with the other arms.
• The harness uses the same translator module as the app.

13.1 Operations

13.2 Configuration (.env)

13.3 Privacy notice

• Single mode: "Your request text is sent to {provider} to interpret it. Do not include confidential data. The design is made in your browser."
• Ensemble mode: "Your request text is sent to OpenAI, Anthropic, Google and TypeSafe to interpret it and to check the result. Do not include confidential data. The design is made in your browser."
• Phase 2 opt-in: the dialog of Section 9.5.1.
• The landing page has a "Privacy" block with the same text and a link to the details.
• The server keeps no request text and has no accounts. The log has no request text and no IP address.
• The rate limit counts the requests of each network address for one minute and for one day (UTC). The Redis database (Upstash) holds a keyed hash of the address (HMAC-SHA256 with RATE_LIMIT_SALT), not the address, for at most two days.
• The host, Vercel, keeps its standard request logs, which include the IP address.
• The review page (Section 14.4) sends the name and the comment of an author to a GitHub repository as an issue. The server does not log the key, the name or the comment.

14.1 Style

• About 80% ASD-STE100.
• Procedures: at most 20 words per sentence. Descriptions: at most 25 words per sentence.
• Active voice. One instruction in each sentence.
• Technical names (resolution, aberration, alias, generator) are allowed.
• Plain academic wording (Section 9.7).
• scripts/ste-check.mjs reports the percentage of sentences that meet these rules. CI fails below 80%.

14.2 Documents

14.3 Interactive architecture page

• The app page /architecture renders docs/architecture/architecture.json.
• The JSON lists components, data flows, source file paths, tests and decision records.
• The page shows clickable diagrams. A click on a component shows its purpose, inputs, outputs, files, tests and decision records.
• Drift test: a unit test fails if a path in the JSON does not exist, or if a source module is not in the JSON.
• Each phase updates the JSON.

14.4 Review questions page (alpha)

The page /review collects the answers of the authors during the alpha. Decisions A12 to A14 and AP6 apply.

• No other page links to /review. The link has the key in its fragment: /review#key=.... The page keeps the key in sessionStorage (ffd:review-key) and removes it from the address bar. It sends the key in the header x-review-key.
• The page asks 55 questions as data: the 13 guarantee regions, the class sentences, center points, replications and resolution-only entries. It also asks about entries without an MA claim, the refused defaults 26-17.1 and 27-18.1, the 35 glossary entries, R23, the references and "Other comments". The progress count has the 54 questions that have choices.
• The choices are Agree, Change and Not sure. Change needs a comment. "Other comments" has no choices and needs a comment. A comment has at most 4,000 characters and a name has at most 100 characters. The name is kept in localStorage (ffd:review-name).
• Four questions are asked again after the answers of 2026-10-19 (decision F6): the claim for 128 runs with 25 to 39 factors (guarantee.region.128.25-64), the entries without an MA claim (guarantee.no-claim, title "Claims for the other catalogue entries"), and the glossary entries glossary.all_2fis_clear and glossary.guarantee. The file data/review-asked-again.json lists them and names the last issue number before the change (after: 65). An answer in an issue up to that number shows under the question as an earlier answer and does not count. Only an answer in a later issue counts. The rule holds on the page, in the read-back and in the CSV export. "Other comments" keeps every answer. The question about the refused defaults is not asked again (decision F7).
• Each answer is one GitHub issue. The title is [alpha-review] {question id}: {choice label}. The labels are alpha-review and one topic (guarantee, glossary, r23, references or other). The first line of the body is a marker with the question id and the choice.
• GET /api/review reads the answers and needs the key. POST /api/review files one issue. The limits are 30 per minute and 300 per day for each address (prefix rv), counted for both methods together and before the key check. The body has at most 25 KB, because a comment of 4,000 control characters and a name of 100 take 6 bytes each in JSON. The server compares the key in constant time. REVIEW_KEY has at least 24 characters.
• The read-back counts only the issues that the token owner made with the label alpha-review and without the label withdrawn. The newest answer to a question wins. A closed issue still counts. The server caches the list for 60 seconds in each instance.
• The messages are fixed strings in lib/review/answer.ts, for example "The review key is not valid." and "The review service is not available now."
• node scripts/export-review.mjs writes the answers to one CSV file, one row for each issue with a marker. It reads the issues with the GitHub CLI of the author. The column counted uses the rule of the read-back. The default file is alan_feedback/alpha-review-answers.csv, which git ignores. The authors decide if the answers go into the public replication materials.

17.1 Open items

17.2 Risks