EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents
Abstract
EmbodiedSkills proposes a unified framework that validates and verifies robot skill executions through a fixed interface, enabling closed-loop embodied agents with adaptable low-level vision-language-action policies.
Vision-language-action (VLA) models map visual observations and language instructions directly to robot actions, but long-horizon tasks require more than action prediction. An agent must coordinate perception, planning, execution, progress verification, and recovery as the physical state evolves. An action prediction or a model-generated skill decision does not, by itself, guarantee that the proposed operation is valid in the current state or that its outcome will be verified. We propose EmbodiedSkills, a unified framework that treats each skill decision as an execution proposal: the runtime checks its prerequisites before execution and verifies the outcome afterward. A shared executable-skill interface connects high-level skill selection, bounded low-level VLA execution, and post-action verification within a single agent loop. Because this interface remains fixed, low-level VLA policies can be replaced or adapted without changing the agent loop. The interface also records planning, execution, verification, and recovery events as structured trajectories, which provide supervision for individual components and can support optional online adaptation when interactive feedback is available. We instantiate EmbodiedSkills with Qwen3-VL and OpenPI/pi0.5 on RoboTwin 2.0 and LIBERO. Task-adapted low-level VLA policies achieve an average success rate of 86.20% across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites. These results establish the execution performance of the task-adapted low-level VLA policies used in EmbodiedSkills. On four memory-dependent RMBench tasks, the same task-adapted execution approach achieves 12.5% average success. The framework provides a trainable and inspectable agent layer for turning these policies into closed-loop embodied systems.
Community
EmbodiedSkills is a unified framework for organizing VLM planning, VLA execution, and environment feedback within a single robot agent. It connects these components through a shared executable-skill interface and a six-stage loop: Observe, Plan, Preflight, Execute, Verify, and Recover.
The loop manages how a task progresses across multiple action chunks. The planner breaks an instruction into semantic subtasks, and the VLA executes actions conditioned on the current subtask. Before execution, the system checks whether the required observations, plan, and backend are ready. After execution, fresh observations support progress verification. An unfinished subtask can continue with another action chunk, a completed subtask advances the plan, and an unsuccessful attempt can trigger recovery or replanning.
This is particularly relevant when execution duration varies. Reaching, placing, or stacking an object may take different amounts of time depending on the initial state and the motion that actually occurs. Verification lets the system adjust the number of action chunks to observed progress. Ordered interaction history preserves previous skill calls and their outcomes, giving subsequent decisions access to what has already been attempted.
The same interface also connects execution with training. Subtask-aligned demonstrations train the low-level action policy, while structured interaction trajectories provide supervision for high-level skill selection. The scheduler can be fine-tuned with the action backend frozen, using the same context construction and history compression rules as deployment. Planning, verification, and action components can therefore be adapted independently.
The paper instantiates the framework with Qwen3-VL and π₀.₅. Task-adapted low-level policies achieve 86.20% average success across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites. The reported AgentLoop ablations further examine intermediate verification, active-subtask conditioning, and repeated action chunks, highlighting how semantic planning and progress-aware execution work together.
Get this paper in your agent:
hf papers read 2609.01281 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper