Bevy MCP Ratatui Reference Implementation

AI-Controlled 3D Game Development in Your Terminal

A reference implementation demonstrating AI-assisted game development with Bevy game engine rendered to the terminal using TUI (Text User Interface). Control and visualize 3D scenes through AI prompts via MCP (Model Context Protocol).

🎯 Overview

This project combines three powerful technologies to create a unique AI-driven development experience:

Bevy Game Engine (0.16+) - High-performance ECS-based 3D engine
Bevy Remote Protocol (BRP) - Live entity inspection and manipulation via MCP
bevy_ratatui_camera - 3D scene rendering to terminal using Unicode characters

What Makes This Unique?

AI Prompt → 3D Visualization: Ask Claude to spawn entities, change colors, or modify transforms and see results immediately in your terminal
No Recompilation Required: Iterate on your 3D scenes in seconds, not minutes
Visual AI Feedback: AI can “see” the rendered terminal output to make intelligent decisions
Terminal-Native Development: Full 3D game development directly in your terminal with 24-bit color support

✨ Features

🎮 AI-Controlled Game Development: Natural language commands create and modify 3D entities
🖥️ Terminal 3D Rendering: Multiple rendering strategies (ASCII, color, edge detection)
🔄 Live Hot-Reloading: Modify scenes without restarting the application
🤖 MCP Integration: Full suite of tools for AI-assisted development
📊 Dual Rendering Modes: Support for both windowed and headless TUI modes
⚡ High Performance: 60 FPS rendering with optimized Unicode output
🎨 Multiple Rendering Strategies: ASCII art, 24-bit color, edge detection wireframes

🚀 Quick Start

Prerequisites

Rust (latest stable) - Install here
Claude Code CLI - Install guide
Terminal with 24-bit color support (Alacritty, Kitty, iTerm, WezTerm recommended)

Installation

# Clone the repository
git clone https://github.com/your-username/bevy-mcp-ratatui-ref.git
cd bevy-mcp-ratatui-ref
 
# Run basic example (window + terminal ASCII output)
cargo run --example tui_basic --features tui
 
# OR run with BRP for AI control
cargo run --example tui_brp --features full
 
# How it works:
# - Window displays standard 3D rendering
# - Terminal shows ASCII conversion of the 3D scene
# - BRP listens on localhost:15702 for AI commands (with --features full)
# - Press Ctrl+C or close window to exit

First AI Interaction

Once the application is running, try these prompts with Claude Code:

"Show me all entities currently in the TUI scene"
"Add a red cube at position [3, 1, 0]"
"Spawn a shiny purple sphere at [-3, 1, 0]"
"Move the red sphere up by 2 units"
"Change the green sphere color to yellow"

Custom BRP Methods

This project implements custom BRP methods that solve a key limitation: standard BRP cannot spawn entities with meshes and materials because asset handles aren’t serializable.

Available custom methods:

bevy/spawn_cube - Spawn cubes with position, scale, color, and material properties
bevy/spawn_sphere - Spawn spheres with position, radius, color, and material properties

Example usage:

// Via MCP BRP tool
mcp__brp__brp_execute({
  method: "bevy/spawn_cube",
  params: {
    position: [3.0, 1.0, 0.0],
    color: [0.8, 0.2, 0.2],
    metallic: 0.7,
    roughness: 0.3,
    name: "Red Cube"
  }
})

See Custom BRP Methods Documentation for complete API reference, AI prompt examples, and how to extend with more shapes.

📚 Documentation

Comprehensive documentation organized by topic:

Core Documentation

Research & Feasibility - Technical analysis of the integration
- Technical feasibility study (9/10 rating)
- Key integration points (Bevy ECS, BRP, Ratatui)
- Rendering pipeline architecture
- Performance benchmarks and optimization strategies
- Terminal compatibility matrix
- Prior art and related projects
System Architecture - Complete system design
- 5-layer architecture (AI → MCP → Bevy → TUI → Terminal)
- Component responsibilities and interfaces
- Data flow diagrams with mermaid visualizations
- Plugin architecture and extensibility points
- State management and synchronization
- Error handling and recovery strategies
Implementation Plan - Development roadmap
- 5-phase implementation strategy (16-21 days)
- Detailed tasks with acceptance criteria
- File structure and dependencies
- Testing strategy for each phase
- Risk mitigation plans
- Success metrics and milestones
Usage Examples - Practical guide
- Quick start guide with installation steps
- 20+ AI prompt examples with expected outputs
- MCP tool usage patterns
- Interactive workflows and advanced use cases
- Troubleshooting guide with solutions
- Multi-camera layouts and custom renderers
Custom BRP Methods - Entity spawning API
- Complete API reference for spawn_cube and spawn_sphere
- AI prompt patterns for entity creation
- JSON-RPC examples and curl commands
- Technical details on why custom methods are needed
- Guide to extending with more shapes
- Integration with standard BRP methods

Additional Resources

ARCHITECTURE_SUMMARY.md - Quick reference guide

🏗️ Architecture Overview

┌─────────────────┐
│   AI (Claude)   │  Natural language commands
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   MCP Bridge    │  Custom: spawn_cube, spawn_sphere
└────────┬────────┘  Standard: mutate_component, query, etc.
         │
         ▼
┌─────────────────┐
│  Bevy Engine    │  ECS with 3D scene management
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ TUI Rendering   │  bevy_ratatui_camera → Unicode output
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│    Terminal     │  24-bit color ANSI display
└─────────────────┘

🎯 Project Structure

bevy-mcp-ratatui-ref/
├── docs/
│   ├── research.md              # Technical feasibility & analysis (2,028 lines)
│   ├── architecture.md          # System architecture & design (1,730 lines)
│   ├── ARCHITECTURE_SUMMARY.md  # Quick reference guide
│   ├── implementation-plan.md   # Development roadmap (5 phases)
│   └── usage-examples.md        # Practical examples & tutorials
├── src/
│   └── (to be implemented)      # Rust source code
├── examples/
│   └── (to be implemented)      # Example applications
├── Cargo.toml                   # Dependencies and configuration
└── README.md                    # This file

🛠️ Technology Stack

Core Dependencies

[dependencies]
bevy = { version = "0.16", features = ["bevy_remote"] }
bevy_brp_extras = "0.2"           # Enhanced BRP with full mutation support
bevy_ratatui_camera = "latest"    # 3D → TUI rendering
bevy_ratatui = "latest"           # Bevy + ratatui integration
ratatui = "latest"                # Terminal UI framework
crossterm = "latest"              # Terminal backend

Architecture Layers

AI Layer - Claude Code with natural language processing
MCP Bridge - Protocol translation and state management
Bevy Application - ECS-based 3D scene management
TUI Rendering - bevy_ratatui_camera with multiple strategies
Terminal Display - ANSI output with 24-bit color

💡 Key Concepts

Bevy Remote Protocol (BRP)

BRP provides a JSON-RPC interface for external tools to:

Query entity-component data in real-time
Modify component values without recompilation
Spawn and destroy entities dynamically
Access global resources
Monitor changes with watchers

Terminal Rendering

bevy_ratatui_camera converts 3D scenes to terminal output:

ASCII Strategy: Character density mapping (@%#*+=-:.)
Color Strategy: RGB values with Unicode blocks (█▓▒░)
Edge Strategy: Wireframe with box drawing characters (┌─┐│└┘)

AI Integration

Claude Code can:

Understand scene composition through terminal visualization
Generate and execute MCP commands
Iterate on designs based on visual feedback
Debug issues by inspecting entity state

🎮 Example Workflows

Scene Composition

User: "Create a solar system demo in the terminal"

Claude:
1. Spawns sun (yellow sphere, position: 0,0,0)
2. Creates planets with orbital paths
3. Adds rotation animations
4. Configures camera for best view
5. Sets color rendering strategy

Result: Animated solar system in your terminal!

Live Debugging

User: "Why isn't my player moving?"

Claude:
1. Queries player entity
2. Checks Transform component
3. Inspects velocity values
4. Identifies stuck at (0,0,0)
5. Suggests fix and applies it
6. Verifies movement in TUI

Result: Bug fixed without recompilation!

📊 Performance

Expected performance on modern hardware:

Metric	Target	Typical
Frame Rate	60 FPS	60 FPS
Frame Time	<16.67ms	6-14ms
Memory Usage	<100MB	60-75MB
Entity Count	100+	50-200
Terminal Redraw	<10ms	3-8ms

🔧 Development Status

Current Phase: Documentation & Research Complete ✅

✅ Comprehensive research (2,028 lines)
✅ System architecture design (1,730 lines)
✅ Implementation roadmap (5 phases)
✅ Usage examples and tutorials
⏳ Phase 1: Foundation setup (next)
⏳ Phase 2: Core integration
⏳ Phase 3: MCP enhancement
⏳ Phase 4: Examples & docs
⏳ Phase 5: Testing & polish

See implementation-plan.md for detailed roadmap

🌟 Use Cases

Game Development

Prototype 3D games in terminal
Test mechanics without graphics overhead
Debug physics and collision visually
Rapid iteration on gameplay

Education

Learn 3D graphics concepts with instant feedback
Understand ECS architecture through visualization
Explore AI-assisted development workflows
Terminal-based game tutorials

CI/CD & Testing

Headless rendering for automated tests
Visual regression testing in terminals
Performance benchmarking with TUI output
Deployment verification visualization

Creative Coding

ASCII art generation from 3D models
Terminal-based creative installations
Retro game aesthetic development
Live coding performances

🤝 Contributing

This is a reference implementation designed to demonstrate the integration of AI, Bevy, and terminal rendering. Contributions welcome:

Implement phases from the roadmap
Add new rendering strategies
Create example scenes and prompts
Improve documentation
Report issues and bugs

📄 License

MIT License - See LICENSE file for details

🔗 Resources

Project Resources

bevy-mcp-ref - Foundation project
bevy_ratatui_camera - TUI rendering library

Technology Documentation

🎓 Learning Path

Understand the Concept - Read this README and research.md
Study Architecture - Review architecture.md and diagrams
Follow Examples - Try prompts from usage-examples.md
Build Features - Implement phases from implementation-plan.md
Experiment - Create your own AI-driven terminal applications

🌈 Vision

This project demonstrates the future of AI-assisted development:

Natural Interaction: Describe what you want, see it appear
Instant Feedback: No compile-run-debug cycles
Visual Understanding: AI sees what you see
Terminal-First: Powerful development anywhere with just a terminal

The future of game development is conversational, visual, and happens in your terminal.

Built with ❤️ using Bevy, Ratatui, and Claude Code

Ready to build 3D games with AI in your terminal? Let’s go! 🚀

Raibid Labs Documentation

Explorer

README