Deconstructing Real World Creativity.
Exploring the frontier of AI design capabilities through real client briefs from Contra’s marketplace, golden designs and model comparisons.
Leaderboard
How frontier models rank in blind head-to-head votes on real client briefs from Contra’s marketplace.
Elo rating from blind head-to-head votes
Real design projects, judged by designers and scored by rubrics
Frontier Design Benchmark is the first benchmark built on real world, high value, and complex design projects. It is judged by professional designers in the Contra Labs network, ensuring the quality of evaluation.
In the benchmark, we also built a design critique rubric and a list of AI slop patterns. The rubric scores are highly correlated with the designers’ ratings of the models (r = 0.94). These additional metrics provide explainable design quality judgement on top of pairwise preferences.
Golden Gallery
Real client briefs, each with its golden: the design set beside the models’ outputs as the bar.
Read the research
Frontier Design: Evaluating AI on Real World Design Tasks
How design jobs are set, reviewed, and evaluated
Pairwise, design rubrics, and AI slop patterns
Compare the outputs
See two AI designs for the same brief, with the model names hidden. Pick the one you think works better.
Deconstructing Real World Creativity.
Exploring the frontier of AI design capabilities through real client briefs from Contra’s marketplace, golden designs and model comparisons.
Leaderboard
How frontier models rank in blind head-to-head votes on real client briefs from Contra’s marketplace.
Real design projects, judged by designers and scored by rubrics
Frontier Design Benchmark is the first benchmark built on real world, high value, and complex design projects. It is judged by professional designers in the Contra Labs network, ensuring the quality of evaluation.
In the benchmark, we also built a design critique rubric and a list of AI slop patterns. The rubric scores are highly correlated with the designers’ ratings of the models (r = 0.94). These additional metrics provide explainable design quality judgement on top of pairwise preferences.
Golden Gallery
Real client briefs, each with its golden: the design set beside the models’ outputs as the bar.