FMX 430 · Spatial Computing — Course Companion

Overview

This studio offers a hands-on approach to how humans communicate with computers and inhabit physical and virtual space. You will build works of self-expression and interaction across tangible, digital, and expanded realities — and ship them to headsets and to the LED wall.

What's new this year

Engine

Unreal Engine 5.8

The final UE5 release before UE6. Megalights and Dataflow are now production-ready, Lumen Lite targets 60 fps GI, and experimental Mesh Terrain, Toon Shader, and MetaHuman Collections open new territory for environments and crowds.

New · AI

Agents in the editor

UE 5.8 ships an official in-editor MCP server, which means AI coding agents can spawn actors, wire Blueprints, build materials, and run tests inside your project. We will use Claude Code and OpenAI Codex as studio assistants all semester.

Studio

Virtual Production

The Cass Building VP Studio remains the capstone venue: LED cabinets, camera tracking, nDisplay. You'll learn to run the volume from the video server down to troubleshooting the wall.

Course objectives

Understand the phases of Unreal Engine / Twinmotion level design from concept to completion.
Become proficient in architectural modeling and level design, at true 1:1 scale with real environments.
Build interactivity into UE levels with Blueprints — and direct AI agents to help you build it faster.
Use Claude Code and Codex responsibly: prompt, review, verify, and document AI-assisted work.
Package and deploy UE5 projects to Meta Quest and HTC Vive headsets via OpenXR.
Operate the Virtual Production Studio from the video server to the LED cabinets.
Develop your own voice in virtual and mixed-reality environments.

A Friday in studio

We meet once a week for four hours. Every session follows the same rhythm:

10:00 · Brief

Demo + concepts. New engine territory introduced on the projector.

10:45 · Guided lab

You build the same thing at your station, step by step.

12:00 · AI lab

Same problem, agent-assisted: prompt Claude Code or Codex, review what it did, fix what it got wrong.

1:15 · Crit / share

Screens up. Show progress, trade techniques, plan the week's homework.

Grade breakdown

Component	Weight	Notes
Unreal guided training	30%	In-class practical exam (Week 7 · Oct 16)
Realistic Virtual Environment	10%	Weeks 3–6
Architectural Visualization	10%	Weeks 5–9
Project of Choice / VP Collaboration	10%	Weeks 10–12
Final Project	40%	Weeks 12–15 · exhibited in the VP Studio

Scale: 95–100 A · 89–94 A/B · 84–88 B · 77–83 B/C · 72–76 C · 66–71 C/D · 60–65 D · below 60 F. More than three absences: NF. Every production project includes an AI development log as a required deliverable.

Equipment & software

Laptop or desktop capable of running Unreal Engine 5.8 and Twinmotion (Epic account required).
A GitHub account — projects live in version control from Week 1; it's how you and your AI agent stay honest.
Claude Code and Codex CLI installed (setup in Week 1; see the AI Lab).
Suggested: 1 TB external SSD; flat-response studio headphones.
UT Adobe Creative Cloud is free for this class: creativecloud.adobe.com with your UT email.

University policies (Title IX, ADA, attendance, academic integrity, campus closure, disruption) are stated in full on the official syllabus in SpartanLearn / Canvas — that document governs.

Semester Map

Fourteen Friday studio sessions (Sep 4 – Dec 11, no class Nov 27) plus a finals-week exhibition, in four phases: learn the engine, make it interactive, take it into space, then ship the final. Amber notes mark each week's AI lab.

Phase 1 — World Building · Weeks 1–6

01Sep 04

Orientation: the spatial computing landscape

Course tour: VR, AR, virtual production, and where "spatial computing" is in 2026.
Install party: UE 5.8, Twinmotion, Epic launcher, GitHub, Claude Code, Codex CLI.
Create your first project; connect it to source control.

AI LabFirst contact: ask Claude Code to explain the folder anatomy of the UE project it's sitting in. Learn the golden rule — the agent proposes, you approve.

State of the Art · The 2026 hardware paradoxHeadsets are shrinking while glasses explode: standalone headset shipments are in a fifth straight year of decline (Omdia forecasts −15% in 2026), yet AR smart glasses grew 136% year-over-year and Meta and Google have pivoted toward glasses. Discussion: you're building for Quest — a shrinking but still 58%-dominant install base. Why is that still the right training bet?Omdia, June 2026 · Counterpoint tracker

02Sep 11

Editor fundamentals

Viewport navigation, actors, levels, the outliner, content browser discipline.
Quixel / Fab assets; migrating content between projects.

AI LabEnable the official Unreal MCP plugin and Epic's UE skills for Claude Code. Have the agent spawn and arrange actors from a prompt; inspect everything it did in the outliner.

State of the Art · A generational hingeUE 5.8 is the last planned major UE5 release — Epic is ramping up UE6 (early access reportedly targeted for late 2027). Meanwhile Unity's GDC 2026 roadmap headlines graphics and runtime work, not XR. Discussion: what does it mean to master a tool at the end of its major version — and who benefits from being the bridge generation?Epic announcement · Unity roadmap

03Sep 18

Blockout & true scale Project 1 launch

Greyboxing a space; 1:1 measurement between real and virtual environments.
Reference gathering; measuring a real room and rebuilding it.

AI LabFeed the agent your room measurements and have it block out the space via MCP; verify scale against your reference with a mannequin actor.

State of the Art · Living in the modelArchitecture has moved from showing a model to walking inside it at 1:1 — and manufacturing (BMW, Boeing, Siemens) mirrors entire factory floors as digital twins. The scale discipline you're practicing this week is the exact skill those sectors hire for. Discussion: what breaks — trust, safety, sales — when the virtual twin doesn't match the tape measure?Digital twins & spatial computing (2025)

04Sep 25

Materials & lighting: Megalights era

PBR materials and Substrate basics; material instances.
Megalights (now production-ready) vs. Lumen vs. Lumen Lite — when to use which.

AI LabPrompt-driven look development: ask the agent for three lighting scenarios of your blockout (noon, dusk, practical-lit night), then art-direct the best one by hand.

State of the Art · Megalights & the 60fps claimMegalights is now Production-Ready (hundreds of dynamic shadow-casting lights, Epic's stated 60fps target on current-gen consoles) and Lumen Lite delivers GI twice as fast as Lumen High Quality — fast enough for Switch 2. Discussion: "Production-Ready" and "60fps" are vendor labels, not independent benchmarks. How would you verify them in your own scene this week?UE 5.8 release notes

05Oct 02

Architectural visualization pipeline Project 2 launch

Twinmotion → UE via Datasmith; CAD/BIM import; archviz conventions.
Cameras, composition, and still/flythrough rendering with Accumulation DoF.

AI LabUse the agent to batch-fix import problems: naming, collision, lightmap UVs. Log what it fixed and what it broke.

State of the Art · Scan vs. model — Gaussian splatting3D Gaussian Splatting (Kerbl et al., SIGGRAPH 2023 — this course's landmark paper) renders captured reality at 100+ fps and is now the fastest-growing capture method, entering UE through plugins like Volinga and NanoGS (no first-party Epic support yet). LiDAR and photogrammetry still own measurement. Discussion: for an archviz client, when do you scan, when do you model, and when do you combine?Kerbl et al. project page · Splats in UE, 2026

06Oct 09

Environments & terrain Project 1 due · crit

Landscape tools vs. experimental Mesh Terrain (overhangs, tunnels, floating islands).
Foliage, PCG basics, atmosphere and skies.
Crit: Realistic Virtual Environment.

AI LabProcedural dressing: have the agent scatter and vary props with PCG, then cull its choices — taste is the part you can't delegate.

State of the Art · Terrain grows a third dimensionMesh Terrain (Experimental in 5.8) replaces the 2.5D heightfield with a true 3D mesh — overhangs, tunnels, floating islands — and PCG now allows manual edits without severing procedural logic. The old procedural-vs-art-directed tension is softening. Discussion: if the system generates and you curate, where does authorship live?Mesh Terrain docs

Phase 2 — Interactivity · Weeks 7–9

07Oct 16

Guided training practical exam Exam · 30%

In-class, hands-on: build a small scene to spec — modeling, materials, lighting, packaging.
No AI agents during the exam: this is the skill floor the rest of the course stands on.
After the break: introduction to Blueprints — nodes, events, variables.

State of the Art · Why the skill floor mattersReported hiring data says degrees alone don't land jobs — portfolios, internships, and demonstrable skills do, and AI fluency commands a premium on top of fundamentals, not instead of them. Today's no-AI exam is the floor everything else stands on.Robert Half 2026 tech hiring

08Oct 23

Blueprints I: making the world respond

Triggers, timelines, doors, pickups, simple UI widgets.
Reading a Blueprint you didn't write — the essential AI-era skill.

AI LabAgent-built interactivity: describe an interaction in plain English, let Claude Code wire the Blueprint via MCP, then walk the graph node-by-node and annotate what every node does.

State of the Art · Agents get the keys to the editorThe Unreal MCP plugin you're using this week is first-party Epic — an MCP server inside the editor that Claude Code and other agents drive over local HTTP, with tools for actors, lighting, materials, Blueprints, and automation tests. It's Experimental: local-only, no auth, incomplete docs. Discussion: what new failure modes appear when an AI can touch your Blueprint graph directly?Epic's Unreal MCP docs · VP Land analysis

09Oct 30

Blueprints II & a taste of C++ Project 2 due · crit

Blueprint communication, casting, interfaces; when a project wants C++ instead.
Crit: Architectural Visualization.

AI LabHead-to-head: give Claude Code and Codex the same interaction task. Compare approach, correctness, and how each explains itself. Write up which you'd hire and why.

State of the Art · The economics of agent-native productionIf one creative technologist directing agents can do what took a small team, studios gain speed and a brand-new QA problem: catching AI-generated errors before they cascade. Your head-to-head today is practice for a real hiring question — reported offer rates for graduates with verifiable AI-pipeline experience run well above coursework-only peers. Discussion: what does the review process for agent work look like?Claude Code + UE 5.8 guide

Phase 3 — Into Space · Weeks 10–12

10Nov 06

VR I: deploying to headsets Project 3 launch

OpenXR, project settings for VR, performance budgets, packaging for Meta Quest and HTC Vive.
Comfort and locomotion: teleport vs. smooth, vignettes, what makes people sick.

AI LabUse the agent as a performance consultant: profile the frame, ask it to find the three most expensive things in your scene, verify with stat commands before believing it.

State of the Art · The platform map you're shipping intoMeta holds 58% of the AR/VR headset market (Q1 2026) — Quest-first is justified. But the edges matter: Valve's Steam Frame targets PC-VR streaming, Samsung's Android XR devices are arriving, and Vision Pro takes UE content via streaming paths rather than native OpenXR. Discussion: one build, five ecosystems — what does shipping cross-platform actually cost?Counterpoint share data · 2026 headset preview

11Nov 13

VR II: hands, bodies, presence

Motion controllers, grabbing, spatial UI; audio as a spatial material.
MetaHumans and crowd-scale MetaHuman Collections for populated spaces.

AI LabAgent-assisted interaction polish: haptics, sound hooks, and grab affordances generated from a spec you write like a design doc.

State of the Art · Comfort is a research problemPresence, cybersickness, and spatial UI are active research frontiers — IEEE VR 2025's best-paper slate includes perception and locomotion work directly relevant to what you're building this week. Read one award paper's abstract and intro; bring one design implication to crit. Discussion: which comfort choices are settled science vs. still contested?IEEE VR 2025 awards · CHI 2025 spatial interaction

12Nov 20

The Volume: virtual production Project 3 due · crit

VP Studio operation: video server, nDisplay, camera tracking, troubleshooting LED cabinets.
Designing environments that read on camera vs. in headset.
Crit: Project of Choice / VP Collaboration. Final project proposals pitched.

State of the Art · The volume industry you're training forFuturesource counted ~200 LED ICVFX volumes worldwide (end-2023 data) and forecasts 18% annual growth in installs through 2028 — driven by small and medium stages like ours, not mega-volumes. UE 5.8 backs this week directly: Live Link Hub and Movie Render Graph are now Production-Ready, with nDisplay support. Discussion: map the VP crew roles — which one is your entry point?Futuresource LED report · VP job roles

—Nov 27

No class — Thanksgiving break

Phase 4 — The Final · Weeks 13–15

13Dec 04

Production sprint I

Studio time on finals with instructor desk-crits; sequencer and cinematics for documentation.

AI LabYour agent, your pipeline: by now the AI workflow is yours to direct. Sprint goal set at 2 pm, demo what you (and the agent) shipped at 5:15.

State of the Art · Free markerless mocap changes the sprintEpic's free MetaHuman Animator markerless plugin extracts full-body animation — fingers included — from single-camera phone video. Character performance, which used to require a mocap stage, is now a production-sprint option for your final. Discussion: what does free mocap do to the economics of indie production?MetaHuman markerless mocap

14Dec 11

Production sprint II & playtesting

Structured playtests in headsets and on the wall; triage what feedback to act on.
Packaging, install, and exhibition logistics.

State of the Art · The sectors nobody told you aboutEntertainment is the visible slice; healthcare simulation, manufacturing digital twins, and smart-city planning hire spatial computing graduates too — job-board estimates put median spatial-computing salaries around $141k (ZipRecruiter; treat as indicative). Discussion: describe your final project in the language of three different industries.ZipRecruiter data · Spatial computing 2026 guide

15Finals

Final crit & exhibition Final due · 40%

Finals week (Dec 14–17) — exhibition in the Virtual Production Studio and headsets during our scheduled exam slot; invited guests.
AI development log and process documentation submitted with the build.

State of the Art · Position your work in 2026The field you're exhibiting into: UE6 on the horizon, headsets contracting until a cheaper Vision Pro (analysts say ~2028), glasses ascendant, agents in the editor, and reality itself becoming a capture format. At crit, every project answers one question: where does this work sit in that landscape — and what would you build next?Omdia 2027–28 outlook

Projects

Four production projects, each building on the last. Every project ships with an AI development log — the record of what you asked the agents to do, what they did, and what you had to fix or decided to reject.

01 · Realistic Virtual Environment

10%Weeks 3–6

Rebuild a real space you can physically measure — a room, a corridor, a courtyard — at true 1:1 scale in UE 5.8. The test: someone who knows the real space should feel its proportions in the headset.

Blockout with verified real-world measurements
PBR materials and a considered lighting scenario (Megalights or Lumen)
60-second flythrough or walkable build

Reference sheet: photos + measurements of the real space
AI development log (what the agent blocked out vs. what you refined)

02 · Architectural Visualization

10%Weeks 5–9

An unbuilt or reimagined space, presented the way an architecture client would see it: Twinmotion/Datasmith pipeline into UE, composed cameras, stills and a flythrough with cinematic depth of field.

Imported and cleaned architectural model
Three hero stills + one flythrough (Accumulation DoF encouraged)
Day and night lighting scenarios

Pipeline notes: what Datasmith brought in cleanly, what needed repair
AI development log

03 · Project of Choice / VP Collaboration

10%Weeks 10–12

Two tracks. Choice: an interactive VR piece deployed to Quest or Vive with at least three authored interactions. VP Collaboration: partner with the video team to design and run an environment on the LED wall, live.

Playable/runnable build on target hardware
Interaction design spec written before implementation
Performance budget met (frame-time evidence)

Playtest notes from at least two classmates
AI development log

04 · Final Project

40%Weeks 12–15

Your voice in spatial computing: a complete virtual, VR, or mixed-reality experience, pitched in Week 12, produced in the sprint weeks, exhibited in the Virtual Production Studio in Week 15. Ambition is graded — so is finish.

Written + visual pitch (approved Week 12)
Exhibited build: headset, wall, or both
Documentation edit (2–3 min) suitable for your portfolio

Process book: references, iterations, playtests
Full-semester AI development log with a one-page reflection: where the agents made you faster, and where they made you worse

How production projects are assessed

Criterion	Weight	What it looks like
Craft & technical execution	30%	Clean scale, materials, lighting, performance; it runs on the target.
Concept & spatial design	25%	The space means something; composition and pacing are authored, not defaulted.
Interactivity / experience design	20%	Interactions are discoverable, comfortable, and serve the concept.
Process & AI development log	15%	Honest record of prompts, agent output, your review and fixes. You can explain every node and line in the build.
Crit participation	10%	You show up, show work, and give your peers real feedback.

AI Lab

This course treats AI coding agents the way a studio treats any powerful tool: you are the director, the agent is a fast but overconfident assistant, and nothing ships that you can't explain. Here's the toolchain, the setup, and the rules. And this site has its own AI TA — the amber button in the corner — tuned to each week's material for questions between studio sessions.

The toolchain

Agent

Claude Code

Anthropic's terminal-based coding agent. Reads your project, edits files, runs builds. With Epic's official UE skills plugin it understands engine conventions; with MCP it can operate the editor itself.

Agent

OpenAI Codex CLI

OpenAI's counterpart agent. We use both deliberately — comparing how two different agents attack the same Unreal problem is itself a course exercise (Week 9).

Bridge

Unreal MCP (built into 5.8)

An MCP server inside the Unreal Editor exposing 30+ toolsets — actors, Blueprints, materials, Niagara, Sequencer, automation tests — so agents can act in the editor, not just on files.

Skills

Epic's UE skills for Claude Code

An official Epic plugin that gives Claude Code built-in knowledge of UE workflows and documentation lookup, so its answers cite engine reality instead of guessing.

Setup (Week 1, step by step)

Install the agents
Claude Code (npm install -g @anthropic-ai/claude-code) and Codex CLI per current docs. Sign in with the accounts provided/discussed in class.
Enable Unreal MCP in the editor
In UE 5.8: Edit > Plugins, search "Unreal MCP", enable it (Toolset Registry enables automatically), restart. Start the server from the console: ModelContextProtocol.StartServer.
Connect Claude Code to the editor
Add the local MCP endpoint to Claude Code and install Epic's unreal-engine-skills-for-claude-code plugin so the agent has engine-aware skills.
Put the project under version control
Git + GitHub from day one. Before any AI session: commit. After: review the diff. The diff is your ground truth for the AI log.
Smoke test
Ask the agent to list the actors in your level and describe the project structure. If it can see your world, you're wired up.

Prompt patterns that work in Unreal

Look up before you build. Have the agent use its class/docs lookup (inheritance, properties, functions) before writing Blueprint or C++ logic — UE's API is too big to trust from memory.
Small, verifiable steps. "Add a trigger volume that opens this door" beats "build my level's interactivity." Compile and test after every change.
Spec first. Write the interaction as three plain-English sentences before prompting. If you can't spec it, the agent can't build it — it will just build something.
Make it explain. After the agent edits a Blueprint, have it walk you through the graph. Then verify in the editor that the explanation matches the nodes.
Distrust performance claims. Agents assert; profilers measure. stat unit, stat gpu, and Unreal Insights settle arguments.

Course AI policy

Allowed and expected: using Claude Code and Codex on all production projects, with every session recorded in your AI development log (prompt, what the agent did, what you kept/fixed/rejected).

Required: you must be able to explain every node, material graph, and line of code in your submitted build, live, at crit. "The AI did it" is the new "I found it on YouTube" — it's a starting point, not an answer.

Not allowed: AI agents during the Week 7 practical exam; presenting agent output as hand-authored in your process documentation; using AI to fabricate playtest notes, references, or reflection writing.

Undocumented AI use on graded work is an academic integrity matter under the University's policy.

The AI development log

A running document (Markdown in your repo is ideal) with one entry per working session: date, goal, the prompts that mattered, what the agent produced, what you changed and why, and one thing you learned about directing it. Five honest minutes after each session. It's worth 15% of every project grade and is the backbone of your final reflection.