Roblox has published a detailed look at how it evaluates Build, its new mobile-first creation tool, saying the system now helps creators turn text prompts into playable games and gives the company a way to measure quality from prompt to player session. In the post, Roblox said that since Build’s alpha launch in New Zealand, more than 100,000 players have used it to try making a game, around 9,000 Build games have been published, and 71% of creators had never used Roblox Studio before.
The company framed the work as part of a broader effort to judge AI game creation as a whole rather than by isolated technical checks. Roblox said Build has to coordinate code, visual assets, gameplay systems, UI and controls into one compound artifact, and that the game must still hold up once real players begin interacting with it in multiplayer settings.
Four signals, one creation pipeline
Roblox said its Build eval framework combines four scored signals, each tied to a different stage of creation. The first is game specification quality, which scores how well Build translates a prompt into a usable spec without inventing requirements or missing key decisions. The second is static game quality, which inspects the place in Roblox Studio edit mode for both engineering soundness and design depth. The third is a playtest score, which runs the game with player-like inputs and reports a pass rate. The fourth is visual and experiential quality, which judges how the game looks and sounds on device-sized viewports.
Roblox said the split matters because games are executable real-time systems. Responsiveness, agency, progression and game feel emerge through interaction, so the company said a game needs to clear both low-level correctness and high-level player experience. In the post, Roblox also said it tracks informative-only metrics such as token usage, tool calls, skills triggered and latency, but that those figures do not affect the eval score itself.
The company summarized the main failure modes the framework is meant to catch as follows:
- Vague loop, missing fail/retry or mobile control assumptions
- Systems present but miswired, unsafe client authority, UI covering the playfield, not mobile-ready
- Architecturally clean but shallow; no progression, variety or hook
- Game does not start, respond, score, recover or stay free of runtime errors
- Runs correctly but looks unfinished, cluttered or unreadable on a phone
Skyforge Ascent shows the gap between working and worth replaying
To show how the framework works, Roblox walked through a prompt for a 3D obby: “A true 3D obby game, with a start platform, moving hazards, checkpoints, and finish line.” Roblox said Build turned that into a game called Skyforge Ascent, with a five-section course, checkpoints, three hazard types, a defined core loop, a difficulty curve, art direction, screens, overlays and controls.
That specification scored 91/100. Roblox said it earned full marks for completeness, prompt adherence, feasibility and internal consistency, while losing points because it left out numeric constants such as spin speeds, slide distances, tile timings, platform dimensions and reset-plane height. The company also said some acceptance criteria, including “about 24 platforms” and “celestial-forge palette,” were not strictly falsifiable.
The same project produced very different results once Roblox’s other lenses were applied. The engineering check scored the build 98/100, saying it delivered five sections and three moving hazards, with server-authoritative sequential checkpoints, fall-reset systems and a UI built on the StyleSheet framework. Roblox said the only issue was two unused helper function exports.
The design score was much lower at 31/100. According to Roblox, the game was structurally clean but offered no progression, replayability, player agency or original hook. The loop was described as linear: reach each checkpoint, cross the finish, and little else. Roblox said that makes it a genre staple, not a game that gives players a reason to return.
Playtesting passed; the game still felt like a prototype
Roblox said the playtest agent ran Skyforge Ascent with real player-like input and passed all nine items on its test plan. The run covered spawning on the start platform, moving across gaps, taking hazard impacts, triggering checkpoints and automatic fall recovery, completing the course and restarting, all with zero runtime errors. That produced a 100/100 playtest score.
The company was careful to separate that from design quality. A perfect playtest score, Roblox said, only proves that the mechanics work and the game is technically playable from start to finish. It does not say whether the game is engaging, which is why the design lens can still score far lower than the playtest.
The visual pass scored 86/100 on a phone-sized viewport. Roblox described Skyforge Ascent as clean and cohesive, with readable platforming geometry, legible hazards and a modern HUD. Minor visual limits kept it from perfection, but the company said there was nothing broken in the presentation that would make a player stop scrolling.
Build feedback has already changed the product
Roblox said the evaluation system has already fed back into Build development. After the New Zealand alpha launch, users reported that Build sometimes made a 2D game when they wanted a 3D one. Roblox said the framework helped it improve 3D capabilities quickly, and that within two weeks Build became better at understanding when a user intended a 3D game even without stating it directly.
The company also said it used those evals to refine 3D playability and polish by fixing common problems with object positioning and lighting. Since those changes, Roblox said “not what I want” complaints dropped from 18% to 3% among all complaints, while overall dissatisfied reaction after playtest dropped 10% points. Roblox described those as descriptive indicators of progress rather than a causal estimate from evaluation alone.
Roblox added that offline A/B eval reports are used before changes ship to production, comparing candidate releases against baselines across all four dimensions. The post said the company uses those reports to benchmark model upgrades and agent routing decisions, and that a base model upgrade delivered consistent gains across every signal.
The company also linked the update to its wider creator and platform efforts in a previous security-first push for the platform and creator economy, though this latest post stays focused on how Build is judged rather than on policy or safety changes.
Across five stages, Roblox said Skyforge Ascent showed that Build can turn a single prompt into a complete, correct, playable and presentable Roblox experience. The remaining gap, the company argued, is that the system can still produce something that works well without producing something players would choose to replay.
That 71% of creators never used Studio before is wild. do you think Build is actually gonna make proper Roblox games easier for new phone users, or is it still gonna hit the same wall once they try making anything complex?
The interesting bit is the eval split, not the shiny 3D obby demo. If the tool can score spec quality, engineering, playtest and phone presentation separately, at least Roblox can see where the failure is instead of saying ‘AI bad’ or ‘AI good’ like it is one thing. 98 on engineering but 31 on design is honestly the most realistic part of the whole post. Working code is not the same as a game people keep queuing into.
nah the split is fine but that 31 tells u the eval still cant measure if ppl will care after 2 mins. playtest passed perfect and it still got called shallow, so the score is useful but not the whole truth
That is kind of the point though. One score would be useless if a game can be mechanically perfect and still dead on arrival. At least now they can tell the difference between broken, functional and actually worth replaying.
For the people here who actually make stuff, does a mobile-first build tool sound useful or just limiting? I feel like text prompts are nice, but would you trust it for anything bigger than an obby?
Useful for drafts, not big stuff.
For quick obbies, yes.
For me it is good for quick protos. But bigger games need Studio anyway, the article shows even a clean build can miss replayability and that stuff is the hard part.
31/100 on design is basically the whole story lol. publishers always love saying ‘it works’ when the thing is still a boring loop with a fresh coat of paint.
I dont think it is only publisher thing here. The story shows a build with checkpoints, hazards and clean UI, so by old standards many teams would say ship it and fix later. But if the design lens says there is no hook, then the game is maybe just a nice template, not a game people stay for. In Rocket League we see same thing all the time – the car can drive perfect and still the mode dies if it has no reason to return.
Template is generous tbh. If it passes every test but still feels empty, that is exactly the kind of thing Roblox floods the platform with already.
Sure, but empty is still a useful starting point if the tool gets the basic game right fast. The more interesting part to me is that Roblox says it improved 3D understanding after users complained about getting 2D when they asked for 3D, so the eval seems to be fixing the prompt mismatch too, not just rating the end product. That is at least better than blind generation.
Fair, prompt mismatch is the one part I actually care about. If it stops giving me a 2D box when I asked for a 3D game, that’s progress. I still think replay value is the real boss fight here though, and Roblox knows it because they keep talking about that gap.
98/100 engineering and still only ‘playable’ sounds very Roblox. Clean code, empty soul, perfect for the marketplace mood lol
What do you mean by marketplace mood? You saying the tool is good for selling shells or that every generated game ends same?
yeah, polished but hollow
not every working thing is a game
true, but Roblox loves acting like ‘working’ is the win button. if the game has no reason to replay, it is just a demo with extra steps
The phone viewport score being 86 is the part that stood out to me. If they are actually judging readability and controls on device-sized screens, that feels way more useful than just checking whether the code compiles. Does anyone think this could finally cut down on the ‘looks fine on PC, awful on mobile’ problem, or will creators just keep stuffing UI over the playfield anyway?
On phone Roblox games already get ruined by tiny buttons all the time, so yes, that part matters a lot. But people will still cram stuff on screen if the eval does not punish it hard enough, especially with younger devs copying whatever looks cool.
I kinda like that Roblox is checking token usage and latency too, even if it does not affect the score. If the base model upgrade really improved all four signals, maybe they are tuning the whole pipeline instead of just the final output. But do yall trust those offline A/B reports before shipping, or do you think live player feedback still catches the real jank better?