Many 3D reconstruction capable models require very large datasets to be trained. Certain models, like, the Zero-1-to-3 suggest using a dataset that is larger than 1.5 TB in size in order for the model to have good generalization capabilities. This poses a question; can a model be effectively trained on a smaller dataset and how well could a smaller dataset be used to further finetune existing models that have been trained on larger datasets. In this research, 3 models are evaluated using 2D and 3D evaluation methods. The models include a model trained from scratch using a smaller ~50GB dataset, a model trained on a large >1.5 TB dataset and a model that has been trained on a larger dataset that is further finetuned using a smaller dataset. Evaluation results showed that the model trained solely on the smaller dataset struggled to generate objects from the right angles when performing novel view synthesis, whereas the model that was further finetuned on the smaller dataset, but trained on the larger one showed promising results, displaying increased numerical performance in 2D and 3D evaluation over the model that was trained only on the larger dataset. This shows the potential of utilizing smaller datasets to not necessarily train model from ground up, but use them to further finetune existing models to get better performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Impact of Smaller Datasets on Training and Fine-Tuning for Novel View Synthesis and 3D Reconstruction

  • Arminas Šidlauskas,
  • Andrius Kriščiūnas

摘要

Many 3D reconstruction capable models require very large datasets to be trained. Certain models, like, the Zero-1-to-3 suggest using a dataset that is larger than 1.5 TB in size in order for the model to have good generalization capabilities. This poses a question; can a model be effectively trained on a smaller dataset and how well could a smaller dataset be used to further finetune existing models that have been trained on larger datasets. In this research, 3 models are evaluated using 2D and 3D evaluation methods. The models include a model trained from scratch using a smaller ~50GB dataset, a model trained on a large >1.5 TB dataset and a model that has been trained on a larger dataset that is further finetuned using a smaller dataset. Evaluation results showed that the model trained solely on the smaller dataset struggled to generate objects from the right angles when performing novel view synthesis, whereas the model that was further finetuned on the smaller dataset, but trained on the larger one showed promising results, displaying increased numerical performance in 2D and 3D evaluation over the model that was trained only on the larger dataset. This shows the potential of utilizing smaller datasets to not necessarily train model from ground up, but use them to further finetune existing models to get better performance.