EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Overview

Autonomous agents have made rapid progress in general-purpose computer use, yet reliable automation of professional industrial engineering remains out of reach. Engineering workflows require reasoning over geometric and physical constraints and keeping dependencies consistent across software and design stages.

We present EngiWorld, a benchmark designed around the capabilities that real engineering projects require, from operating a single application to solving engineering problems end to end. It contains 1,301 expert-curated tasks spanning six engineering domains (CAD, CAE, CAM, BIM, EDA, and DCC) and 26 professional software applications, with both GUI and CLI interfaces. The tasks are organized into six types, corresponding to capabilities that real engineering projects require.

We further introduce an artifact-centric evaluation methodology built on a unified suite of domain verifiers. The verifiers programmatically check whether final and intermediate artifacts are geometrically valid, physically feasible, and compliant with design rules. Beyond binary success, they also score how well optimized designs meet their specifications. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model scores only 44.7/100, and just 3.6% of runs on tasks that require cross-software coordination succeed. EngiWorld provides a rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.

The benchmark

Six task types, testing the capabilities real engineering projects require.

1. Single-Software Execution

   931

   Model, modify, or simulate an artifact in one designated application.

2. Software Selection

   140

   Choose suitable software from a permitted set and complete the task.

3. Vision-Guided Modeling

   120

   Build engineering models from reference drawings and visual requirements.

4. Design Optimization

   40

   Optimize design quality while satisfying engineering feasibility constraints.

5. Cross-Software Coordination

   60

   Carry out workflows that pass dependent artifacts across applications.

6. Open-Environment Engineering

   10

   Choose and install software in a blank environment, then solve the engineering problem end to end.

How the dataset is constructed

Experts define tasks and acceptance criteria; AI assists specification and verifier implementation. Each task is executed and reviewed in its software environment.

Evaluation

Agents work through GUI or CLI interfaces. Domain-specific verifiers reopen their deliverables and check the engineering properties that matter.

1. Interact

   Start from a clean, task-specific workspace. Use visual observations and mouse or keyboard actions, or terminal commands and scripts.

2. Deliver

   Submit native models, simulation outputs, circuits, building models, or 3D scenes, including intermediate files when the workflow requires them.

3. Verify

   Recompute geometry, topology, physical quantities, rule compliance, and consistency across workflow stages against task-specific criteria.

Leaderboard

Seven frontier models on the same 306-task main evaluation, covering 158 CLI and 148 GUI tasks.

Analysis

Controlled ablations vary visual inputs, accessibility information, screenshot resolution, and interaction history.

Case studies

Four Single-Software Execution GUI workflows from the paper. Each passes its task-specific artifact checks and receives a score of 1.0.

1. EDA / KiCad

   Connect a data bus and preserve the circuit

   Eight distinct data nets establish the required MCU–SRAM mapping. The saved schematic retains the existing circuitry and passes with zero electrical-rule errors.

2. CAD / AutoCAD

   Build a precise staggered hole pattern

   Two rectangular arrays create 23 tube holes. The exported DXF preserves the required coordinates, radii, entity types, and layer assignments.

3. CAD / SolidWorks

   Align parts while preserving geometry

   The sliding block is positioned against the base. The exported STEP retains both independent solids and their original dimensions.

4. DCC / Blender

   Deliver a scene with a packed AO bake

   A 512 × 512 ambient-occlusion image is baked and packed into the saved native scene, preserving the computed pixels in the deliverable.

Where Cross-Software Coordination workflows break down

The recorded failures span import-scale errors, incorrect action targeting, and mismatches between submitted representations and verifier expectations. In the load case, a pressure representation does not satisfy the verifier’s force/displacement check; in the CSV case, an unrecognized diameter-field name stops evaluation before downstream geometry checks.

Submit Your Agent

If you are developing an AI model or agent and would like to evaluate it on EngiWorld, we welcome you to contact us at gaohongcheng2000@gmail.com. Please include a publicly accessible link to your model or agent.

We will confirm the evaluation scope, service fee, and timeline with you, complete the evaluation as agreed, and share the results.

@misc{engiworld2026,
  title = {EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?},
  author = {Hongcheng Gao and Hailong Qu and Yu Lei and Henghui Sun and Haoyang Li and Yipeng Wei and Naihao Xue and Xiaohan Yu and Zhuo Tao and Yihe Zang and Yajiao Wang and Jingyi Tang and Yi Li and Jingjing Zhou and Jie Luo and Bohan Zeng and Chengyu Shen and Hao Jiang and Chong Chen and Bowen Qu and Olive Huang and Zeqiang Wang},
  year = {2026},
  eprint = {2609.37686},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2609.37686}
}