Roblox has published a detailed look at how it evaluates Build, its new mobile-first creation tool, saying the system now helps creators turn text prompts into playable games and gives the company a way to measure quality from prompt to player session. In the post, Roblox said that since Build’s alpha launch in New Zealand, more than 100,000 players have used it to try making a game, around 9,000 Build games have been published, and 71% of creators had never used Roblox Studio before.
The company framed the work as part of a broader effort to judge AI game creation as a whole rather than by isolated technical checks. Roblox said Build has to coordinate code, visual assets, gameplay systems, UI and controls into one compound artifact, and that the game must still hold up once real players begin interacting with it in multiplayer settings.
Four signals, one creation pipeline
Roblox said its Build eval framework combines four scored signals, each tied to a different stage of creation. The first is game specification quality, which scores how well Build translates a prompt into a usable spec without inventing requirements or missing key decisions. The second is static game quality, which inspects the place in Roblox Studio edit mode for both engineering soundness and design depth. The third is a playtest score, which runs the game with player-like inputs and reports a pass rate. The fourth is visual and experiential quality, which judges how the game looks and sounds on device-sized viewports.
Roblox said the split matters because games are executable real-time systems. Responsiveness, agency, progression and game feel emerge through interaction, so the company said a game needs to clear both low-level correctness and high-level player experience. In the post, Roblox also said it tracks informative-only metrics such as token usage, tool calls, skills triggered and latency, but that those figures do not affect the eval score itself.
The company summarized the main failure modes the framework is meant to catch as follows:
- Vague loop, missing fail/retry or mobile control assumptions
- Systems present but miswired, unsafe client authority, UI covering the playfield, not mobile-ready
- Architecturally clean but shallow; no progression, variety or hook
- Game does not start, respond, score, recover or stay free of runtime errors
- Runs correctly but looks unfinished, cluttered or unreadable on a phone
Skyforge Ascent shows the gap between working and worth replaying
To show how the framework works, Roblox walked through a prompt for a 3D obby: “A true 3D obby game, with a start platform, moving hazards, checkpoints, and finish line.” Roblox said Build turned that into a game called Skyforge Ascent, with a five-section course, checkpoints, three hazard types, a defined core loop, a difficulty curve, art direction, screens, overlays and controls.
That specification scored 91/100. Roblox said it earned full marks for completeness, prompt adherence, feasibility and internal consistency, while losing points because it left out numeric constants such as spin speeds, slide distances, tile timings, platform dimensions and reset-plane height. The company also said some acceptance criteria, including “about 24 platforms” and “celestial-forge palette,” were not strictly falsifiable.
The same project produced very different results once Roblox’s other lenses were applied. The engineering check scored the build 98/100, saying it delivered five sections and three moving hazards, with server-authoritative sequential checkpoints, fall-reset systems and a UI built on the StyleSheet framework. Roblox said the only issue was two unused helper function exports.
The design score was much lower at 31/100. According to Roblox, the game was structurally clean but offered no progression, replayability, player agency or original hook. The loop was described as linear: reach each checkpoint, cross the finish, and little else. Roblox said that makes it a genre staple, not a game that gives players a reason to return.
Playtesting passed; the game still felt like a prototype
Roblox said the playtest agent ran Skyforge Ascent with real player-like input and passed all nine items on its test plan. The run covered spawning on the start platform, moving across gaps, taking hazard impacts, triggering checkpoints and automatic fall recovery, completing the course and restarting, all with zero runtime errors. That produced a 100/100 playtest score.
The company was careful to separate that from design quality. A perfect playtest score, Roblox said, only proves that the mechanics work and the game is technically playable from start to finish. It does not say whether the game is engaging, which is why the design lens can still score far lower than the playtest.
The visual pass scored 86/100 on a phone-sized viewport. Roblox described Skyforge Ascent as clean and cohesive, with readable platforming geometry, legible hazards and a modern HUD. Minor visual limits kept it from perfection, but the company said there was nothing broken in the presentation that would make a player stop scrolling.
Build feedback has already changed the product
Roblox said the evaluation system has already fed back into Build development. After the New Zealand alpha launch, users reported that Build sometimes made a 2D game when they wanted a 3D one. Roblox said the framework helped it improve 3D capabilities quickly, and that within two weeks Build became better at understanding when a user intended a 3D game even without stating it directly.
The company also said it used those evals to refine 3D playability and polish by fixing common problems with object positioning and lighting. Since those changes, Roblox said “not what I want” complaints dropped from 18% to 3% among all complaints, while overall dissatisfied reaction after playtest dropped 10% points. Roblox described those as descriptive indicators of progress rather than a causal estimate from evaluation alone.
Roblox added that offline A/B eval reports are used before changes ship to production, comparing candidate releases against baselines across all four dimensions. The post said the company uses those reports to benchmark model upgrades and agent routing decisions, and that a base model upgrade delivered consistent gains across every signal.
The company also linked the update to its wider creator and platform efforts in a previous security-first push for the platform and creator economy, though this latest post stays focused on how Build is judged rather than on policy or safety changes.
Across five stages, Roblox said Skyforge Ascent showed that Build can turn a single prompt into a complete, correct, playable and presentable Roblox experience. The remaining gap, the company argued, is that the system can still produce something that works well without producing something players would choose to replay.
That 71% of creators never used Studio before is wild. do you think Build is actually gonna make proper Roblox games easier for new phone users, or is it still gonna hit the same wall once they try making anything complex?