SimWorld Documentation

Welcome to the SimWorld Documentation!

Realistic, open-ended, and natively LLM/VLM-compatible simulators are essential for advancing agent development in complex physical and social environments. However, most existing simulators focus on indoor or task-specific domains (e.g., household robotics or autonomous driving), limiting research into large-scale, open-ended scenarios with realistic physical and social dynamics.

SimWorld addresses these limitations by providing a comprehensive platform that combines:

1. Realistic, Open-Ended World Simulation.

2. Rich Interface for LLM/VLM Agents.

3. Diverse Physical and Social Reasoning Scenarios.

With SimWorld, researchers can explore embodied intelligence at scale, enabling agents to perceive, reason, and act in complex, dynamic worlds that closely mirror real-world scenarios.

Note

This project is under active development.

GETTING STARTED

• Introduction
  • Overview
  • Architecture
• Installation
  • Before you begin
  • Installation
• Core Configuration
  • Quick guidance
  • Global Settings (simworld)
  • More configuration
• Minimal Example
  • Prerequisites
  • 1. Import and configure
  • 2. Define an LLM agent for navigation
  • 3. Build the environment wrapper
  • 4. Run a short rollout
  • Next steps
• Base Environments
  • Usage
• Additional Environments
  • Download and Installation
  • Usage
  • Predefined Environments List

CUSTOMIZATION

• Make Your Own Pak Files
  • 1. Download the Unreal Editor
  • 2. Create a New Unreal Project
  • 3. Import Assets
  • 4. Set Up Chunk Package
  • 5. Configure Chunk Settings
  • 6. Build the Pak File
  • 7. Use the Pak File in SimWorld

NEXT STEPS

• Next Steps Roadmap

SIMWORLD MODULES

• Overview
• Unreal Engine Backend
  • Various Scenes
  • Assets
  • Sensors
  • Synchronous and Asynchronous mode
• Procedural City Generation
  • Overview
  • Procedural Generation Process
  • Using Procedural City Generation
• Waypoint System
  • Overview
  • Map
  • Granularity
• Traffic System
  • Overview
  • Architecture
  • Important Class
  • Using Traffic System
• Agent System
  • Overview
  • Base Agent
  • Base LLM
  • Get Camera Observation
  • Action Space
  • Local Planner
  • Agent Asset Paths
• UnrealCV+
  • Overview
  • Communicator
  • Using Communicator

ADDITIONAL CONFIGURATION

• Full Configuration Reference
  • Section map
  • Global Settings (simworld)
  • City Generation (citygen)
  • Asset Retrieval and Placement (assets_rp)
  • Traffic Simulation (traffic)
  • Map Settings (map)
  • User Agent Settings (user)
  • Scooter Settings (scooter)
  • Creating Your Own Configuration

PYTHON API REFERENCE

• Python API
  • simworld package

SIMWORLD ROBOTICS

• SimWorld-Robotics
  • Overview
  • Robot Type
  • Action Space
  • Observation Space
  • Potential Applications