REVIEW 4 major objections 5 minor 1 cited by
Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that HyperScore, a hypernetwork-based evaluator, predicts alignment, geometry, texture, and overall quality of text-to-3D meshes with higher correlation to human opinion scores than all compared metrics on the new MATE-3D…
desk verdict Solid benchmark, sound metric, but the superiority claim is only shown on the authors' own benchmark. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is HyperScore, built on a frozen CLIP encoder. For each evaluation dimension it creates a condition feature by prepending the dimension's meta text (e.g., 'geometry quality') to learnable prompt tokens and passing them through the text encoder. Two mechanisms use these condition features: conditional feature fusion, which re-weights image patches by their relevance to the dimension and combines them with the text feature, and adaptive quality mapping, in which a hypernetwork (a small network that outputs the weights and biases of another network) generates the parameters of a four-layer mapping head specific to that dimension. The same architecture therefore implements several scoring functions, one per dimension, from a single set of weights plus a dimension condition. Training adds a feature-disentangling loss that pushes the condition features for different dimensions apart.
What would settle it
Build a new test set of text-to-3D outputs from generators released after 2024, using prompts written by human users rather than GPT-4, and compare HyperScore against the best fine-tuned and zero-shot baselines from the paper. If HyperScore's rank-correlation advantage over DINO v2 fine-tuned (or over ImageReward) shrinks or reverses on this out-of-distribution set, the claim that it is a generally superior multi-dimensional evaluator would be falsified. A cheaper check is to re-run the same 5-fold evaluation with a leave-one-generator-out split; the paper's appendix already shows a large performance drop on 3DTopia and One-2-3-45++ for all metrics, so a metric that generalizes should not show such a drop.
Extended reading notes
Core claim
On MATE-3D, HyperScore achieves the highest PLCC, SRCC, and KRCC among all compared metrics on each of the four evaluation dimensions. For example, its SRCC values are 0.739 for alignment, 0.782 for geometry, 0.811 for texture, and 0.792 for overall quality, while the best fine-tuned baseline, DINO v2 with regression heads, reaches 0.642, 0.739, 0.771, and 0.728, and the best zero-shot baseline, ImageReward, reaches 0.651, 0.591, 0.612, and 0.623. The paper interprets this as evidence that a single model can emulate the way human raters shift their attention and decision criteria when asked to judge different quality dimensions. It also shows that the gap over a same-backbone multi-task baseline (MultiScore) comes specifically from the conditional feature fusion and the hypernetwork-generated mapping weights, and that HyperScore even surpasses separately trained per-dimension networks.
Load-bearing premise
The paper's performance claims all rest on MATE-3D being a representative and unbiased sample of text-to-3D usage: 160 GPT-4-generated prompts, eight open-source generators, and 21 human raters per sample determine what 'correct' quality scores look like. If those prompts, generators, or raters do not reflect how people actually use text-to-3D systems, HyperScore's advantage may not transfer to other prompts or newer generators.
Editorial extensions
If this is right
- A single trained model can replace separate per-dimension expert models for text-to-3D quality assessment, cutting the cost of fine-grained evaluation.
- MATE-3D provides a reusable test bed: the per-category and per-method MOS analyses identify where generators fail, such as multiple-object prompts and Janus artifacts.
- Because HyperScore outputs absolute scores for all four dimensions, it can rank samples across different prompts, not only pairs from the same prompt.
- Automated multi-dimensional scoring makes it feasible to use quality as a training signal or selection criterion when iterating on text-to-3D generators.
Reading between the lines
- The condition-feature-plus-hypernetwork recipe is not specific to 3D meshes; the same design could produce dimension-specific quality scores for text-to-image or text-to-video outputs, though the paper only evaluates text-to-3D.
- The benchmark's prompts are GPT-4-generated by design; a natural extension is a human-prompt or user-revision split to test whether HyperScore's advantage persists on naturally typed prompts.
- Aggregating 21 raters into a single MOS hides rater disagreement; a useful stress test would be predicting per-rater scores or score distributions rather than just the mean.
- HyperScore's correlation numbers are benchmark-relative; comparing it against the same baselines on T3Bench or GPTEval3D would require those datasets' scores to be released.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript makes two contributions. First, it introduces MATE-3D, a text-to-3D quality benchmark of 1,280 textured meshes generated from 160 GPT-4 prompts across eight prompt categories (Basic, Refined, Complex, Fantastical, Grouped, Action, Spatial, Imaginative) and eight text-to-3D methods. The benchmark provides MOS labels collected from 21 subjects on four evaluation dimensions (alignment, geometry, texture, overall), with outlier screening reducing the valid ratings to 17 per sample. Second, the paper proposes HyperScore, a multi-dimensional evaluator built on CLIP features with learnable condition prompts, a conditional patch-fusion mechanism, and a hypernetwork that generates the mapping-head weights for each evaluation dimension. Experiments report 5-fold cross-validation correlations on MATE-3D; HyperScore achieves the highest PLCC, SRCC, and KRCC in every dimension, and the paper includes ablations and leave-one-category and leave-one-method analyses in the appendix.
Significance. The benchmark itself is a substantial resource: 1,280 samples, four rating dimensions, a standardized ITU-based subjective protocol, outlier screening, and a stated plan to release the MOS. The HyperScore architecture is clearly specified, and the ablation study plus leave-one-out tests are good practice. If the reported gains are statistically robust and transfer beyond the MATE-3D distribution, the paper would make a useful contribution to text-to-3D evaluation, which currently lacks a widely accepted multi-dimensional metric. The main value lies in the combination of a new benchmark and a learned metric; the metric alone is a variant of condition-based multi-task quality prediction and would be hard to evaluate without the benchmark.
major comments (4)
- [§5.1 and Table 2] All headline claims are based on 5-fold cross-validation means, but the paper reports no per-fold standard deviations, confidence intervals, or paired significance tests. For example, the alignment SRCC gap between HyperScore and DINO v2 + FT is 0.097 (0.739 vs 0.642) and the overall gap is 0.064; these differences are plausible but could be within fold-to-fold variation, especially since many category-level SRCC values in Tables S14 and S15 fall below 0.7. The authors should report per-fold results and a paired significance test (e.g., Steiger's test or a bootstrap over samples) for the differences in Table 2 and Table 3. In addition, comparing HyperScore with zero-shot metrics such as ImageReward is a trained-on-MATE-3D versus never-trained comparison; the fine-tuned baselines are the right controls, so the significance analysis should focus on those.
- [§3.1, §5, Appendix D.1, Tables S14 and S15] The entire evaluation is confined to MATE-3D, which is generated from 160 GPT-4 prompts in eight hand-defined categories and eight methods, all from the SDS/NeRF or feed-forward era and none from the recent 3D-Gaussian-splatting line. The leave-one-category-out (Table S14) and leave-one-method-out (Table S15) tests reuse the same prompt and method distribution, so they cannot detect whether HyperScore's advantage is tied to artifacts common to this specific method set. The Objaverse check in Appendix D.1 has no human MOS and no baseline comparison, so it cannot validate HyperScore's absolute-score calibration. To support the claimed generality ('a promising metric for assessing and improving text-to-3D generation'), the authors need at least one external hold-out with human ratings: for example, collect MOS on new prompts and methods not used in MATE-3D, including a 3DGS-based method, and report the same correlation metrics, or quantitatively test HyperScore on an existing externally annotated benchmark. Without this, the superiority claim is strictly a claim about the MATE-3D test distribution.
- [§5.1] The sentence 'the performance on the test set with minimal training loss is recorded' is ambiguous. If the epoch is selected by minimal training loss, this is fine, but if the test set is used for model selection, the reported 5-fold means are optimistic and the protocol must be revised. Please state explicitly which loss is monitored and whether any test-set information is used during training. Additionally, the paper should report the variance of the 5-fold procedure across random seeds, because several key comparisons in Table 3 are small (e.g., HyperScore 0.739 vs Separately Trained 0.737 for alignment SRCC).
- [§3.2] The subjective experiment section does not report inter-rater agreement or the distribution of valid ratings per sample after outlier detection. It says 'four outliers are identified and eliminated' and then 'we collect 17 scores for each sample,' but it is unclear whether four subjects or four individual ratings are removed and whether every sample ends with exactly 17 valid ratings. Reporting Krippendorff's alpha or ICC and the per-sample count distribution is important because MOS reliability sets the ceiling on the correlations reported in Table 2.
minor comments (5)
- [§3.2] The abstract states 107,520 annotations while Section 3.2 later reports that outlier screening leaves 17 valid scores per sample; please state explicitly that 107,520 is the raw annotation count and give the final number of valid scores used for MOS computation.
- [§3.3, Fig. 5d] The text says the deviations in Fig. 5d are computed 'after non-linear regression' but does not describe the regression model; please specify the procedure.
- [Appendix B.5, Table S4] There are typos in this section ('Comparion with GPTEval3D' and 'Comparion with T3Bench' should be 'Comparison'); please also align the notation in Table S4, where the GPTEval3D row reports '234×5×3' for the number of rating scores but the table header says '%' for unavailable scores.
- [Eq. (4)] The dimensions in Eq. (4) are confusing: I_{v2t} is a matrix while I_{t2c}^i is written as R^{N_t}; please clarify the shapes and the product order in Eq. (5).
- [§5.2] For zero-shot baselines the paper says scores are averaged over multiple rendered viewpoints, whereas HyperScore concatenates view features before fusion; please state whether HyperScore scores are computed per view and averaged or from the concatenated features, so that the aggregation is interpretable in Table 2.
Circularity Check
No significant circularity: HyperScore is trained on MATE-3D MOS and evaluated on held-out folds; all performance claims are explicitly benchmark-relative.
full rationale
The paper's derivation chain is a standard supervised benchmark-plus-metric pipeline. MATE-3D provides human MOS from 21 subjects across four dimensions, and HyperScore is trained with an MSE regression loss against those MOS values (Eq. 8) and evaluated with 5-fold cross-validation with no prompt overlap between training and testing sets. The appendix additionally performs leave-one-category-out and leave-one-method-out experiments to test generalization to unseen categories and generative methods. The per-dimension condition features are learnable parameters optimized against the four MOS targets, so the predicted scores are not identical to the inputs by construction. Comparisons against zero-shot metrics such as CLIPScore and ImageReward, and against fine-tuned baselines trained under the same protocol, are legitimate empirical comparisons. No load-bearing self-citation appears in the references, and no uniqueness or ansatz is imported from the authors' prior work. The absence of an external human-rated benchmark limits the generality of the superiority claim, but that is an external-validity concern, not circularity.
Assumptions & free parameters
free parameters (4)
- HyperScore network weights (CLIP visual encoder, MLP, hypernetwork, mapping head) =
trained on MATE-3D
- Learnable condition prompts (K sequences of L tokens) =
L=12, K=4
- Hyperparameters: lambda=1, epsilon=0, epochs=30, lr=2e-6/2e-4 =
as stated in Appendix C
- Number of rendered viewpoints M =
6
assumptions (5)
- domain assumption CLIP features contain enough information for multi-dimensional quality assessment.
- domain assumption Rendering a mesh into six orthographic views preserves the quality information humans use.
- domain assumption The four evaluation dimensions (alignment, geometry, texture, overall) are distinct and consistently understood by raters.
- domain assumption MOS from 21 subjects (17 after outlier removal) on an 11-level scale is a stable ground truth.
- domain assumption The 160 GPT-4 prompts and eight open-source methods cover the relevant distribution of text-to-3D use.
Cite this review
Pith. "Pith review of Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation." pith.science (2026). https://pith.science/paper/RHTL7HE7
@misc{pith2026241211170,
author = {Pith},
title = {Pith review of: Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/RHTL7HE7}},
note = {Machine review of arXiv:2412.11170}
}
read the original abstract
Text-to-3D generation has achieved remarkable progress in recent years, yet evaluating these methods remains challenging for two reasons: i) Existing benchmarks lack fine-grained evaluation on different prompt categories and evaluation dimensions. ii) Previous evaluation metrics only focus on a single aspect (e.g., text-3D alignment) and fail to perform multi-dimensional quality assessment. To address these problems, we first propose a comprehensive benchmark named MATE-3D. The benchmark contains eight well-designed prompt categories that cover single and multiple object generation, resulting in 1,280 generated textured meshes. We have conducted a large-scale subjective experiment from four different evaluation dimensions and collected 107,520 annotations, followed by detailed analyses of the results. Based on MATE-3D, we propose a novel quality evaluator named HyperScore. Utilizing hypernetwork to generate specified mapping functions for each evaluation dimension, our metric can effectively perform multi-dimensional quality assessment. HyperScore presents superior performance over existing metrics on MATE-3D, making it a promising metric for assessing and improving text-to-3D generation. The project is available at https://mate-3d.github.io/.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Point Cloud Compression and Objective Quality Assessment: A Survey
A survey of point cloud compression and objective quality assessment that benchmarks representative methods on standard datasets and distills design insights.
Reference graph
Works this paper leans on
-
[1]
Lara: Efficient large-baseline radiance fields
Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, and Andreas Geiger. Lara: Efficient large-baseline radiance fields. In European Conference on Computer Vision (ECCV),
-
[2]
Sculpt3d: Multi-view consistent text-to-3d generation with sparse 3d prior
Cheng Chen, Xiaofeng Yang, Fan Yang, Chengzeng Feng, Zhoujie Fu, Chuan-Sheng Foo, Guosheng Lin, and Fayao Liu. Sculpt3d: Multi-view consistent text-to-3d generation with sparse 3d prior. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[3]
Vp3d: Unleashing 2d visual prompt for text-to-3d gen- eration
Yang Chen, Yingwei Pan, Haibo Yang, Ting Yao, and Tao Mei. Vp3d: Unleashing 2d visual prompt for text-to-3d gen- eration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[4]
Objaverse-xl: A universe of 10m+ 3d objects
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, Eli VanderBilt, Anirud- dha Kembhavi, Carl V ondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. In Advances in Neural Infor- mation Pro...
2023
-
[5]
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsanit, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,
-
[6]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 8
2009
-
[7]
Interactive3d: Create what you want by interactive 3d generation
Shaocong Dong, Lihe Ding, Zhanpeng Huang, Zibin Wang, Tianfan Xue, and Dan Xu. Interactive3d: Create what you want by interactive 3d generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[8]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 7, 8, 16
arXiv 2010
Show all 124 references
-
[9]
Axiom-based grad-cam: To- wards accurate visualization and explanation of cnns
Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom-based grad-cam: To- wards accurate visualization and explanation of cnns. arXiv preprint arXiv:2008.02312, 2020. 8
2008 arXiv
-
[10]
Shapecrafter: A recursive text-conditioned 3d shape generation model
Rao Fu, Xiao Zhan, Yiwen Chen, Daniel Ritchie, and Sri- nath Sridhar. Shapecrafter: A recursive text-conditioned 3d shape generation model. In Advances in Neural Information Processing Systems (NIPS), 2022. 2
2022
-
[11]
Get3d: A generative model of high quality 3d tex- tured shapes learned from images
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d tex- tured shapes learned from images. In Advances in Neural Information Processing Systems (NIPS), 2022. 2
2022
-
[12]
Hypernet- works
David Ha, Andrew M Dai, and Quoc V Le. Hypernet- works. In International Conference on Learning Represen- tations (ICLR), 2022. 2
2022
-
[13]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 7, 8
2016
-
[14]
T3bench: Benchmarking current progress in text-to-3d gen- eration
Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng, Ran Yi, Juanzi Li, and Yong-Jin Liu. T3bench: Benchmarking current progress in text-to-3d gen- eration. arXiv preprint arXiv:2310.02977, 2023. 1, 3, 15
2023 arXiv
-
[15]
Mmpi: a flexible radiance field representation by multiple multi-plane images blend- ing
Yuze He, Peng Wang, Yubin Hu, Wang Zhao, Ran Yi, Yong- Jin Liu, and Wenping Wang. Mmpi: a flexible radiance field representation by multiple multi-plane images blend- ing. In International Conference on Robotics and Automa- tion (ICRA), 2024. 1
2024
-
[16]
CLIPScore: A reference-free evaluation metric for image captioning
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. In Empirical Methods in Natu- ral Language Processing (EMNLP), 2021. 2, 3, 6, 7, 8
2021
-
[17]
3dtopia: Large text-to-3d generation model with hybrid diffusion priors
Fangzhou Hong, Jiaxiang Tang, Ziang Cao, Min Shi, Tong Wu, Zhaoxi Chen, Tengfei Wang, Liang Pan, Dahua Lin, and Ziwei Liu. 3dtopia: Large text-to-3d generation model with hybrid diffusion priors. arXiv preprint arXiv:2403.02234 ,
-
[18]
Debi- asing scores and prompts of 2d diffusion for view-consistent text-to-3d generation
Susung Hong, Donghoon Ahn, and Seungryong Kim. Debi- asing scores and prompts of 2d diffusion for view-consistent text-to-3d generation. In Advances in Neural Information Processing Systems (NIPS), 2023. 1
2023
-
[19]
Lrm: Large reconstruction model for single image to 3d
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. In International Conference on Learning Representa- tions (ICLR), 2024. 2
2024
-
[20]
Towards trans- parent deep image aesthetics assessment with tag-based con- tent descriptors
Jingwen Hou, Weisi Lin, Yuming Fang, Haoning Wu, Chaofeng Chen, Liang Liao, and Weide Liu. Towards trans- parent deep image aesthetics assessment with tag-based con- tent descriptors. IEEE Transactions on Image Processing (TIP), 2023. 6
2023
-
[21]
Nerf-texture: Texture synthesis with neural radi- ance fields
Yi-Hua Huang, Yan-Pei Cao, Yu-Kun Lai, Ying Shan, and Lin Gao. Nerf-texture: Texture synthesis with neural radi- ance fields. In ACM SIGGRAPH, 2023. 1
2023
-
[22]
Make-a-shape: a ten-million-scale 3D shape model
Ka-Hei Hui, Aditya Sanghi, Arianna Rampini, Kamal Rahimi Malekshan, Zhengzhe Liu, Hooman Shayani, and Chi-Wing Fu. Make-a-shape: a ten-million-scale 3D shape model. In International Conference on Machine Learning (ICML), 2024. 1
2024
-
[23]
Ice-g: Image conditional editing of 3d gaussian splats
Vishnu Jaganathan, Hannah Hanyun Huang, Muham- mad Zubair Irshad, Varun Jampani, Amit Raj, and Zsolt Kira. Ice-g: Image conditional editing of 3d gaussian splats. arXiv preprint arXiv:2406.08488, 2024. 3
2024 arXiv
-
[24]
Shap-e: Generat- ing conditional 3d implicit functions
Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 2
2023 arXiv
-
[25]
3d gaussian splatting for real-time radiance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions On Graphics (TOG), 2023. 1
2023
-
[26]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,
-
[27]
Evaluating and improving composi- tional text-to-visual generation
Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. Evaluating and improving composi- tional text-to-visual generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[28]
Aigiqa-20k: A large database for ai- generated image quality assessment
Chunyi Li, Tengchuan Kou, Yixuan Gao, Yuqin Cao, Wei Sun, Zicheng Zhang, Yingjie Zhou, Zhichao Zhang, Weixia Zhang, Haoning Wu, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. Aigiqa-20k: A large database for ai- generated image quality assessment. In IEEE/CVF Con- ference on C...
2024
-
[29]
Agiqa-3k: An open database for ai-generated image quality assessment
Chunyi Li, Zicheng Zhang, Haoning Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, and Weisi Lin. Agiqa-3k: An open database for ai-generated image quality assessment. IEEE Transactions on Circuits and Sys- tems for Video Technology (TCSVT), 2024. 2
2024
-
[30]
Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional Conference on Machine Learning (ICML), 2022. 2, 3, 7, 8
2022
-
[31]
Instant3d: Instant text- to-3d generation
Ming Li, Pan Zhou, Jia-Wei Liu, Jussi Keppo, Min Lin, Shuicheng Yan, and Xiangyu Xu. Instant3d: Instant text- to-3d generation. International Journal of Computer Vision (IJCV), 2024. 3
2024
-
[32]
https://threejs.org
Three.js Library. https://threejs.org. 15
-
[33]
Magic3d: High-resolution text-to-3d content creation
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3, 4, 14
2023
-
[34]
Sherpa3d: Boosting high-fidelity text-to-3d genera- tion via coarse 3d prior
Fangfu Liu, Diankun Wu, Yi Wei, Yongming Rao, and Yueqi Duan. Sherpa3d: Boosting high-fidelity text-to-3d genera- tion via coarse 3d prior. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[35]
A comprehensive survey on 3d con- tent generation
Jian Liu, Xiaoshui Huang, Tianyu Huang, Lu Chen, Yuenan Hou, Shixiang Tang, Ziwei Liu, Wanli Ouyang, Wangmeng Zuo, Junjun Jiang, et al. A comprehensive survey on 3d con- tent generation. arXiv preprint arXiv:2402.01166, 2024. 1
2024 arXiv
-
[36]
One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion
Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In IEEE/CVF Conference on Computer Vision and P...
2024
-
[37]
Zero-1-to-3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In International Confer- ence on Computer Vision (ICCV), 2023. 14
2023
-
[38]
Vqa-diff: Exploit- ing vqa and diffusion for zero-shot image-to-3d vehicle asset generation in autonomous driving
Yibo Liu, Zheyuan Yang, Guile Wu, Yuan Ren, Kejian Lin, Bingbing Liu, Yang Liu, and Jinjun Shan. Vqa-diff: Exploit- ing vqa and diffusion for zero-shot image-to-3d vehicle asset generation in autonomous driving. In European Conference on Computer Vision (ECCV), 2024. 3
2024
-
[39]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 7, 8
2021
-
[40]
Scaledreamer: Scalable text-to-3d synthesis with asynchronous score distillation
Zhiyuan Ma, Yuxiang Wei, Yabin Zhang, Xiangyu Zhu, Zhen Lei, and Lei Zhang. Scaledreamer: Scalable text-to-3d synthesis with asynchronous score distillation. In European Conference on Computer Vision (ECCV), 2024. 3
2024
-
[41]
Latent-nerf for shape-guided generation of 3d shapes and textures
Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , 2023. 3, 4, 14
2023
-
[42]
Nerf: Representing scenes as neural radiance fields for view synthesis
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), 2020. 1
2020
-
[43]
Clip-mesh: Generating textured meshes from text using pretrained image-text models
Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, and Tiberiu Popa. Clip-mesh: Generating textured meshes from text using pretrained image-text models. In ACM SIG- GRAPH Asia, 2022. 2
2022
-
[44]
Point-e: A system for generat- ing 3d point clouds from complex prompts
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generat- ing 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022. 2
2022 arXiv
-
[45]
Gpt-4 system card
OpenAI. Gpt-4 system card. OpenAI, 2023. 3, 13
2023
-
[46]
Dinov2: Learning robust visual features without supervision
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 7, 8
2023 arXiv
-
[47]
Barron, and Ben Milden- hall
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In In- ternational Conference on Learning Representations (ICLR),
-
[48]
Learn- ing transferable visual models from natural language super- vision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...
2021
-
[49]
Methodologies for the subjective assessment of the quality of television images
ITU-R BT.500 Recommendation. Methodologies for the subjective assessment of the quality of television images
-
[50]
Subjective video quality as- sessment methods for multimedia applications
ITU-T P.910 Recommendation. Subjective video quality as- sessment methods for multimedia applications. 1999. 4
1999
-
[51]
Dreamgaussian4d: Genera- tive 4d gaussian splatting
Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. Dreamgaussian4d: Genera- tive 4d gaussian splatting. arXiv preprint arXiv:2312.17142,
-
[52]
Laion-5b: An open large-scale dataset for training next generation image-text models
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In Advances in Neural In...
2022
-
[53]
Retrieval-augmented score distillation for text-to-3d gener- ation
Junyoung Seo, Susung Hong, Wooseok Jang, In `es Hyeonsu Kim, Min-Seop Kwak, Doyup Lee, and Seungryong Kim. Retrieval-augmented score distillation for text-to-3d gener- ation. In International Conference on Machine Learning (ICML), 2024. 3
2024
-
[54]
Let 2d diffusion model know 3d- consistency for robust text-to-3d generation
Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Hyeonsu Kim, Jaehoon Ko, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d- consistency for robust text-to-3d generation. InInternational Conference on Learning Representations (ICLR), 2024. 3
2024
-
[55]
Mvdream: Multi-view diffusion for 3d gen- eration
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. arXiv preprint arXiv:2308.16512, 2024. 3
2024 arXiv
-
[56]
Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network
Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 7
2020
-
[57]
Lgm: Large multi-view gaussian model for high-resolution 3d content creation
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision (ECCV), 2024. 3
2024
-
[58]
Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for ef- ficient 3d content creation. In International Conference on Learning Representations (ICLR), 2024. 1, 3
2024
-
[59]
Textmesh: Gen- eration of realistic 3d meshes from text prompts
Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari. Textmesh: Gen- eration of realistic 3d meshes from text prompts. In Interna- tional Conference on 3D Vision (3DV), 2024. 1, 2, 4, 14
2024
-
[60]
Yeh, and Gregory Shakhnarovich
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Gregory Shakhnarovich. Score jacobian chaining: Lift- ing pretrained 2d diffusion models for 3d generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 4, 14
2023
-
[61]
Ex- ploring clip for assessing the look and feel of images
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Ex- ploring clip for assessing the look and feel of images. In AAAI Conference on Artificial Intelligence, 2023. 7, 8
2023
-
[62]
Prolificdreamer: high-fidelity and diverse text-to-3d generation with variational score distilla- tion
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: high-fidelity and diverse text-to-3d generation with variational score distilla- tion. In Advances in Neural Information Processing Systems (NIPS), 2024. 2, 3
2024
-
[63]
Q-align: Teaching lmms for visual scoring via discrete text-defined levels
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023. 7, 8
2023 arXiv
-
[64]
Gpt- 4v(ision) is a human-aligned evaluator for text-to-3d gener- ation
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt- 4v(ision) is a human-aligned evaluator for text-to-3d gener- ation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 3, 15
2024
-
[65]
Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,
-
[66]
Consistent3d: Towards consistent high-fidelity text-to-3d generation with deterministic sampling prior
Zike Wu, Pan Zhou, Xuanyu Yi, Xiaoding Yuan, and Han- wang Zhang. Consistent3d: Towards consistent high-fidelity text-to-3d generation with deterministic sampling prior. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 4, 14
2024
-
[67]
Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models
Jiale Xu, Xintao Wang, Weihao Cheng, Yan-Pei Cao, Ying Shan, Xiaohu Qie, and Shenghua Gao. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2
2023
-
[68]
Imagere- ward: Learning and evaluating human preferences for text- to-image generation
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In Advances in Neural Information Processing Systems (NIPS), 2024. 2, 3, 7, 8
2024
-
[69]
Dreamview: Inject- ing view-specific text guidance into text-to-3d generation
Junkai Yan, Yipeng Gao, Qize Yang, Xihan Wei, Xuansong Xie, Ancong Wu, and Wei-Shi Zheng. Dreamview: Inject- ing view-specific text guidance into text-to-3d generation. In European Conference on Computer Vision (ECCV), 2024. 3
2024
-
[70]
Consistent flow distillation for text-to-3d generation
Runjie Yan, Yinbo Chen, and Xiaolong Wang. Consistent flow distillation for text-to-3d generation. In International Conference on Learning Representations (ICLR), 2025
2025
-
[71]
Viewfusion: Towards multi-view consistency via interpolated denoising
Xianghui Yang, Yan Zuo, Sameera Ramasinghe, Loris Baz- zani, Gil Avraham, and Anton van den Hengel. Viewfusion: Towards multi-view consistency via interpolated denoising. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3
2024
-
[72]
Dream- reward: Text-to-3d generation with human preference
Junliang Ye, Fangfu Liu, Qixiu Li, Zhengyi Wang, Yikai Wang, Xinzhou Wang, Yueqi Duan, and Jun Zhu. Dream- reward: Text-to-3d generation with human preference. In European Conference on Computer Vision (ECCV), 2024. 3, 7
2024
-
[73]
Consistent-1-to-3: Consistent image to 3d view syn- thesis via geometry-aware diffusion models
Jianglong Ye, Peng Wang, Kejie Li, Yichun Shi, and Heng Wang. Consistent-1-to-3: Consistent image to 3d view syn- thesis via geometry-aware diffusion models. InInternational Conference on 3D Vision (3DV), 2024. 3
2024
-
[74]
Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models
Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xing- gang Wang. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recogn...
2024
-
[75]
Gaussiandreamerpro: Text to manipula- ble 3d gaussians with highly enhanced quality
Taoran Yi, Jiemin Fang, Zanwei Zhou, Junjie Wang, Guan- jun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Xinggang Wang, and Qi Tian. Gaussiandreamerpro: Text to manipula- ble 3d gaussians with highly enhanced quality. CoRR, 2024. 3
2024
-
[76]
3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions On Graphics (TOG), 2023
Biao Zhang, Jiapeng Tang, Matthias Nießner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions On Graphics (TOG), 2023. 2
2023
-
[77]
Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 2024. 2
2024
-
[78]
Learning multi- dimensional human preference for text-to-image generation
Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingt- ing Gao, Di Zhang, and Zhongyuan Wang. Learning multi- dimensional human preference for text-to-image generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3, 6
2024
-
[79]
Learning to prompt for vision-language models
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. Inter- national Journal of Computer Vision (IJCV), 2022. 6
2022
-
[80]
A green apple
Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High- fidelity text-to-3d generation with advanced diffusion guid- ance. In International Conference on Learning Representa- tions (ICLR), 2024. 3 Appendix A. Overview This appendix includes additional details on benchmark cons...
2024
-
[81]
Object categories: - Living Beings: Humans, animals, mythical creatures, insects, aquatic life - Plants: Trees, flowers, bushes, grass, aquatic plants - Buildings and Structures: Houses, skyscrapers, bridges, tunnels, castles - Vehicles: Cars, bikes, planes, boats, spacecraft ...
-
[82]
- Symmetry: Define if the object should be symmetrical, asymmetrical, or radially symmetrical
Geometry properties: - Volume: Specify if the object is hollow, solid, porous, or layered. - Symmetry: Define if the object should be symmetrical, asymmetrical, or radially symmetrical. - Contours: Indicate if the contours should be smooth, jagged, irregular, or undulating. - ...
-
[83]
Object interaction: - Spatial relation: the orientation or movement involved in the interaction, describing where the subject and object are in relation to each other. e.g. toward, above, below, near to, on the left/right/top/bottom of - Action relation: Specific verbs represe...
-
[84]
A green apple
Basic: descriptions about a single object without detailed geometry or appearance details, where some global properties (e.g., color , shape, material) could be included. Here are examples: "A green apple", "A humanoid robot”, ""A square table”, “A wooden chair”
-
[85]
A photo of a black bird
Refined: descriptions about a single object with one simple specification of the geometry or appearance details. Here are examples: “A photo of a black bird", “A yellow rubber duck has only one foot ", “A plastic cup with a printed logo”, “An imperial state crown of England"
-
[86]
A brown teddy bear, fur matted, one eye missing
Complex: descriptions about a single object with more than two detailed specifications of geometry or appearance details. Here are examples: “A brown teddy bear, fur matted, one eye missing ”, “A turtle standing on its hind legs, wearing a top hat and holding a cane ”, “An ove...
-
[87]
A frog with a translucent skin displaying a mechanical heart beating
Fantastical: descriptions about a single object with high creativities that are not, or are generally unlikely to be, found in the real wo rld. Here are examples: “A frog with a translucent skin displaying a mechanical heart beating ”, “A squirrel-octopus hybrid”, “A white cat...
-
[88]
A delicious hamburger and a green apple
Grouped: descriptions about multiple objects without interactions, where some global properties (e.g., color, shape, material) could b e included. Here are examples: “A delicious hamburger and a green apple”, “A red pig and a huge drum”, “A round table and a square chair”
-
[89]
A humanoid robot with a top hat is playing the cello
Action: descriptions about multiple objects with interaction about action relations, where some geometry or appearance details could be included. Here are examples: “A humanoid robot with a top hat is playing the cello”, “A black cat with white feet is sleeping peacefully besi...
-
[90]
A red apple on a round ceramic plate
Spatial: descriptions about multiple objects with interactions about spatial relations, where some geometry or appearance details coul d be included. Here are examples: “A red apple on a round ceramic plate”, “A strong football player wearing a number seven jersey next to a bl...
-
[91]
A panda with a wizard hat is reading a newspaper
Imaginative: descriptions about multiple objects with interactions, where objects or interactions are not, or are generally unlikely to be found in the real world. Here are examples: “A panda with a wizard hat is reading a newspaper ”, “A ghost is eating a golden apple ”, “A l...
-
[92]
A brown leather belt
-
[93]
A glowing light bulb
-
[94]
A white wedding dress
-
[95]
A wooden dining table
-
[97]
A red apple on a white ceramic plate
-
[98]
A black hat hanging on a brass hook
-
[99]
A glass teapot next to a porcelain cup
-
[100]
Spatial
A delicious hamburger These prompts span a variety of categories, including clothing, accessories, household items, vehicles, and natural elements, ensuring coverage of different forms, materials, and textures. Please create 100 “Spatial ” prompts about “multiple object genera...
-
[101]
A golden key inside a brown leather pouch
-
[102]
A blue backpack under a wooden bench
-
[103]
A silver spoon resting on a folded napkin
-
[104]
A cat sitting next to a pair of shoes
-
[105]
A guitar leaning against a wooden chair
-
[106]
A clock above a wooden bookshelf
-
[107]
A pair of sunglasses on top of a magazine
-
[108]
A candle placed inside a glass jar
-
[109]
A handwoven basket filled with a ripe peach
-
[110]
A set of dice beside a deck of cards
-
[111]
Imaginative
A small cactus inside a clay pot These prompts ensure spatial relationships between objects are clear, diverse, and easy to visualize while maintaining a compact, simple structure. Here are 100 “Imaginative” prompts about multiple objects, each within 20 words, ensuring divers...
-
[112]
A giraffe wearing sunglasses plays chess with a robot
-
[113]
A dragon sipping tea from a porcelain cup
-
[114]
A violin floating next to a glowing crystal ball
-
[115]
A penguin in a tuxedo is conducting an orchestra with a baton
-
[116]
A ghost riding a bicycle with a pumpkin helmet
-
[117]
A knight polishing his sword while a cat reads a map
-
[118]
A frog wearing boots and holding an umbrella
-
[119]
A unicorn carrying a glowing lantern on its horn
-
[120]
A lion rider is riding a motorcycle
-
[121]
A shark with a crown is holding a golden trident
-
[122]
A bear astronaut is planting a flag with a paw print
-
[123]
A phoenix with flaming wings is carrying a silver lantern
-
[124]
A raccoon detective is studying a map under a magnifying glass
-
[125]
Figure S9
A dog wearing backpack is walking on a rainbow These imaginative prompts feature whimsical, surreal interactions between objects and creatures, maintaining variety and unexpected combinations for creativity in 3D generation. Figure S9. Examples of guiding GPT-4 for automatic p...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.