Intent.
Into motion.
A MARS mobile manipulator situated in an ordinary apartment. High-fidelity MuJoCo physics, LLM intent translation, deterministic controller judges, and real-time sensor streams.
Give the agent a task.
The ORVYN simulator provides a direct loop between natural language planning, deterministic path controllers, and full rigid-body physics.
Acquire control lease
Open the playground. The web layer negotiates an exclusive control token with the private physics process. Wait for the green “Physics online” indicator and the live camera feed.
Select a scenario
Open Mission / AI from the floating bottom tool dock. Select Explore, Search & Rescue, or Shepherd. Selecting a mission automatically resets the simulation to its authored initial scene.
Execute controller or AI plan
Click Run skill for immediate deterministic trajectory execution. Alternatively, enter a free-form instruction, click Plan with AI, inspect the structured intent review, and click Run reviewed plan.
Safety & emergency brake
Hit SPACE or click ■ Stop in any panel at any moment. The manual watchdog instantly zeroes wheel velocities and brakes the chassis.
From intent to physics execution.
ORVYN decouples natural language reasoning from low-level joint actuators through a validated deterministic execution pipeline.
The server validates that all proposed motions map to safe, bounded controllers before sending actuator inputs to the MuJoCo simulation worker.
An ordinary world. Specific goals.
The simulation tests embodied agent competence across navigation, search, and dynamic object manipulation inside an authentic residential layout.
Explore the Apartment
A bounded autonomous scan and drive lasting approximately 30 simulation seconds. The robot navigates Parquet hallways and living areas, discovering room bounds.
Test in Playground ↗Search & Rescue
Approach the human avatar seated in the living room and remain nearby. The pinned Innate challenge judge monitors continuous proximity and ordered find-and-hold spatial constraints.
Launch Challenge ↗Shepherd
Reach the soccer ball within 0.8 meters, then physically push it with the chassis and articulated bumper to within 1.2 meters of the dog avatar. Tests multi-step continuous contact mechanics.
Test in Playground ↗MARS mobile manipulator specs.
MARS combines a stable differential mobile base with an articulated inspection manipulator and binocular stereo head.
| Subsystem | Specification | Simulation Parameter |
|---|---|---|
| Drive Kinematics | Dual-wheel differential drive + passive casters | Linear velocity 0.22 m/s, angular 0.65 rad/s |
| Perception Head | Binocular camera sensor mounted at eye level | Perspective projection, real-time JPEG stream |
| Manipulation Arm | High-visibility articulated yellow linkage | Forward contact bumper and spatial probing |
| Occupancy Sensor | 2D Ground-truth simulation SLAM projection | Occupancy grid cells, resolution = 0.05 m |
| Safety Watchdog | Heartbeat monitor on manual drive inputs | 350 ms timeout automatic brake |
Ground-truth vs sensor feed.
In ORVYN, the robot perceives its environment through two distinct modalities:
MuJoCo Binocular Camera
The primary visual stream rendered from MARS’s viewpoint. The camera captures authentic perspective, lighting, Parquet floors, furniture, and environmental obstacles. The View panel allows toggling between Fill viewport and Full camera frame.
2D Occupancy Map
The Map panel displays ground-truth coordinates computed directly by the physics engine. MARS is plotted as a vibrant lime circle with heading vector; interactive props (soccer ball, avatar, dog) are marked with high-contrast color pins.
Review intent before motion.
The backend translates natural language prompts into bounded robotic intent via the configured Orbio chat model.
The model can select strictly from four authorized symbolic skills: explore, rescue, shepherd, or stop. An AI plan is a hypothesis, not a guarantee of task completion: the physics simulation must physically execute the trajectory and satisfy upstream spatial judges.
{
"intent": "Find the person and stay nearby",
"plan": {
"skill": "rescue",
"summary": "Navigate toward human coordinates and maintain <1.2m proximity",
"parameters": { "target": "human", "tolerance_meters": 1.2 }
},
"reviewed": false
}
The human-in-the-loop interface requires selecting Run reviewed plan after inspecting the model’s intent summary, ensuring transparent agent behavior.
Security and rate limiting.
ORVYN enforces strict server-side boundaries to protect compute resources and model API allocations:
- API Rate Limit: Requests are throttled to one every 15 seconds, and capped at 20 per hour across active sessions.
- State Invalidation: Triggering manual driving, pressing Stop, clicking Reset, or closing the playground immediately invalidates any pending or reviewed AI plan.
- Zero Client Storage: Credentials and model keys remain exclusively in server process memory. The browser never receives or stores secret keys.
- Deterministic Fallback: Built-in skill controllers and manual controls operate independently of AI availability, requiring zero provider balance.
You keep the controls.
Take manual control over the MARS mobile base at any moment using keyboard keys, touch buttons, or the virtual directional pad.
Keyboard driving activates when either the Camera stage or the Keyboard Driving pad is focused. Keyboard events inside inputs or menus do not move the robot.
Fail-safe mechanical watchdogs.
Robotic control requires active liveness verification. ORVYN implements a multi-layer watchdog:
Every manual velocity command is bounded. If the client fails to renew drive ticks within 350 ms, the physics engine automatically applies mechanical brakes.
Exclusive control tokens expire after 90 seconds without user heartbeat, automatically freeing the simulator for subsequent sessions.
Closing the modal or navigating away sends an asynchronous /api/release signal that halts actuators and releases the hardware locks.
Running on private infrastructure.
The ORVYN deployment runs securely on server loopback ports, isolated from unauthorized external access. The web server listens on port 8092 and forwards validated API requests to the physics daemon on port 8091.
ssh -N -L 8091:127.0.0.1:8092 web3
# Open http://127.0.0.1:8091 in your browser
Provisioning AI sessions securely.
For an approved AI planning session, invoke the secure remote starter script from the ORVYN root directory:
sh scripts/start-secure-remote.sh
# Enter the approved model ID and API key at the masked terminal prompt
Know what you are seeing.
The landing page features editorial concept visualizations. Inside the simulator, the feed is generated in real time by the actual MuJoCo physics camera, rendered at calibrated resolution and refresh rate.
This is simulated embodied AI research, not physical-robot deployment. No financial tokens, wallets, or on-chain mechanisms are required or active.
Built upon Innate OS, revision 4be109a501aa8fd5fb69d11394b878592e53789e. Apache 2.0 license and Creative Commons apartment asset credits are preserved in ATTRIBUTION.md. Motion and visualization utilize Three.js and Anime.js (both MIT licensed).