Case 2 · Fitting the OpenAI brain into the android
OpenAI Multi-Agent Brain ↔ Android — Mapping
Each architecture layer becomes a concrete piece of OpenAI software and a physical touchpoint on the android.
Layer Role OpenAI implementation (likely) Where it touches the android
Director Receives the natural-language goal, reasons, decomposes, and picks the Managers. Planner agent via Responses API + function calling; reasoning model. Checks skill memory before planning. No direct contact — orchestrates the rest.
Managers Domains: navigation, manipulation, task, safety. Break responsibilities into subtasks. Specialized agents, one per domain, each with its own tools and prompt. Define what each android subsystem must do.
Execution Translate subtasks into concrete low-level commands. Deterministic tool-calls through the HAL; the LLM picks the action, code emits the command. Android SDK: motors, joints, actuators, trajectories.
Perception Turns sensors into state and closes the feedback loop. Vision models + Realtime API (audio/stream) for continuous fusion. Android cameras, IMU, force sensors, LiDAR.
Skill Memory Stores and retrieves validated skills; personalization. Vector store (embeddings) — RAG over skills. “make coffee” → “make MY coffee”. Per-user and per-android-unit profiles.
Safety Final gate before any motion. Deterministic (hard-coded) validators + a guardrail agent. Not LLM-only. Joint limits, collision, no-go zones, emergency stop.
Critical principle: the LLM plans and decides, but the safety gate and low-level control are deterministic. You never trust physical motion to the model alone — the android needs rigid validation layers between “the AI wanted” and “the body did.”
Conceptual mapping — the OpenAI stack is the brain; the HAL + deterministic validators are the spinal cord that protects the body.