LAB / WMSI-EXP-049
Sim-to-Real Policy Testing.
How robustly do simple learned policies transfer from simulation to a low-cost physical robot?
01 / QUESTION
Robotics and Embodied AI
How robustly do simple learned policies transfer from simulation to a low-cost physical robot?
This is a protocol scaffold, not a completed experiment or a frozen execution plan. No measured effect or research conclusion is claimed here.
Hypothesis to test
The manipulation described below produces a reproducible, condition-specific change in the measured system behavior or internal state that is not explained by the ordinary controls.
Ordinary competing explanation
The observed pattern is explained by ordinary sampling variation, prompt/representation artifacts, surface lexical cues, evaluator bias, or the known product/runtime mechanics captured by the controls.
02 / PLANNED DESIGN
How we intend to test it
Train in simulation, deploy in controlled physical tasks, and quantify transfer gaps.
Controls
Multiple simulators/domain randomization; scripted baseline.
What we would measure
Success, trajectory error, interventions, safety stops.
Access and instruments
Physical audio or robotics equipment. This describes access requirements, not verified readiness on our Spark.
Simulator; open robot platform; camera/telemetry.
Before a full run
Feasibility=HARDWARE; priority=P2. Before confirmatory execution, run a small pilot to estimate variance/cost, freeze the scoring rules, and assign a run-count target.
Model revisions, final prompts, sample sizes, cost limits, and scoring rules must be settled through the pilot and protocol-freeze process.
03 / INTERPRETATION
What could fool us
Sim/real mismatch, sensor drift, uncontrolled physical environment.
What would weaken the idea
If sim policies fail systematically outside trivial tasks, transfer claims narrow.
Observation, inference, speculation, unknowns, and apparatus errors will be labeled separately. One striking transcript cannot establish a general effect.
04 / RECORD
Evidence we plan to preserve
Simulation seed, robot telemetry, video, action log, safety stops, environment state.
Planned visual evidence
Sim-vs-real trajectory overlay; error heatmap.
No research runs or raw datasets are published for this experiment. The September 26, 2026 registry and protocol compendium (version 1.0) are the sources for this planning record.
Back to all 90 experiments