Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity

Yuhan Zhang* 1,2, Long Zhuo* 2, Ziyang Chu* 2,3, Tong Wu† 4, Zhibing Li2,5,
Liang Pan† 2, Dahua Lin2,5, Ziwei Liu† 6
(* equal contributions, † corresponding authors)
1Fudan University 2Shanghai Artificial Intelligence Laboratory 3Tsinghua University 4Stanford University 5The Chinese University of Hong Kong 6S-Lab, Nanyang Technological University
Paper Leaderboard Code(Coming soon) Data(Coming soon)

Overview of Hi3DEval, a unified framework for 3D generation evaluation with three key components: 1) Hierarchical evaluation protocols that jointly assess object-level and part-level quality, with extended material evaluation via reflectance cues. 2) A large-scale benchmark> featuring a diverse set of 3D generative models, and extensive human-aligned annotations generated via a multi-agent, multi-modal LLMs pipeline. 3) A hybrid automated scoring system that integrates video-based and naive 3D-based representations to enhance evaluators' perceptions of 3D structure.

Abstract

Despite rapid advances in 3D content generation, quality assessment for the generated 3D assets remains challenging. Existing methods mainly rely on image-based metrics and operate solely at the object level, limiting their ability to capture spatial coherence, material authenticity, and high-fidelity local details.
1) To address these challenges, we introduce Hi3DEval, a hierarchical evaluation framework tailored for 3D generative content. It combines both object-level and part-level evaluation, enabling holistic assessments across multiple dimensions as well as fine-grained quality analysis. Additionally, we extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism, focusing on attributes such as albedo, saturation, and metallicness.
2) To support this framework, we construct Hi3DBench, a large-scale dataset comprising diverse 3D assets and high-quality annotations, accompanied by a reliable multi-agent annotation pipeline. We further propose a 3D-aware automated scoring system based on hybrid 3D representations. Specifically, we leverage video-based representations for object-level and material-subject evaluations to enhance modeling of spatio-temporal consistency and employ pretrained 3D features for part-level perception. Extensive experiments demonstrate that our approach outperforms existing image-based metrics in modeling 3D characteristics and achieves superior alignment with human preference, providing a scalable alternative to manual evaluations.

Leaderboard

Text-to-3D Generative Models

Image-to-3D Generative Models

Visualizations of Hierarchical Evaluation

Object-Level Evaluation

Part-Level Evaluation

Material-Subject Evaluation

BibTeX


        @misc{zhang2025hi3devaladvancing3dgeneration,
            title={Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity}, 
            author={Yuhan Zhang and Long Zhuo and Ziyang Chu and Tong Wu and Zhibing Li and Liang Pan and Dahua Lin and Ziwei Liu},
            year={2025},
            eprint={2508.05609},
            archivePrefix={arXiv},
            primaryClass={cs.CV},
            url={https://arxiv.org/abs/2508.05609}, 
        }