When Python Agents Meet 3D: Automating Blender from Natural Language

What if AI could help design 3D worlds without taking creative control away from humans?

In this talk, I will present a hosted Blender agent that creates and edits 3D scenes from natural language. The agent runs Blender headlessly inside Docker, communicates with it through a TCP socket protocol, and exposes Python-powered tools for scene creation, object manipulation, materials, screenshots, rendering, and asset import. It is built with the Microsoft Agent Framework and Azure AI Foundry, using Python as the bridge between the LLM, Blender’s scripting API, cloud storage, and the hosted runtime.

More than a technical demo, this talk explores a potential future pipeline for AI-assisted 3D creation: a workflow where artists, game designers, architects, and developers can iteratively generate, inspect, modify, and refine 3D environments while keeping human control at every stage. Instead of replacing creative tools, the agent becomes a programmable collaborator that can accelerate repetitive work, propose variations, and turn high-level intent into editable Blender scenes.

Through live examples, we will discuss what this could mean for the future of video game world-building, virtual production, and 3D architecture: faster prototyping, more accessible scene creation, and new ways to move from idea to interactive environment. Attendees will leave with a practical architecture for building Python agents that operate real creative software, and a grounded view of both the opportunities and current limitations of human-in-the-loop AI for 3D design.