融合多特征与全局-局部协同Transformer的图像修复算法
DOI:
CSTR:
作者:
作者单位:

大连民族大学计算机科学与工程学院 大连 116600

作者简介:

通讯作者:

中图分类号:

TP391.4;TN919.8

基金项目:

辽宁省应用基础研究计划项目(2023JH2/101300191)、辽宁省自然科学基金(2023-MS-133)、辽宁省教育厅科研项目(LJKZ0024)资助


Fusion of multi-features and global-local Transformer for image inpainting
Author:
Affiliation:

College of Computer Science and Engineering, Dalian Minzu University,Dalian 116600, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对当前图像修复领域所面临的高计算复杂度以及在生成结构合理且细节丰富的图像方面的局限,提出了一种融合多尺度分层特征与全局局部协同Transformer的图像修复模型。首先提出多尺度分层特征融合模块,以实现深层特征与浅层特征细节上的有效融合,在扩大感受野的同时减少关键信息丢失情况。其次提出用于全局推理的全局-局部协同Transformer模块,它通过集成矩形窗口注意力机制和局部前馈神经网络,在降低计算复杂度的同时,提高模型对全局上下文信息的宏观理解和对局部细节特征的微观捕捉能力,增强图像的整体一致性。实验在CelebA-HQ和Places2数据集上进行了验证,在处理40%~50%掩码时,所提方法与常用的修复方法对比,PSNR平均提高了0.26~6.25 dB,SSIM平均提升了1.4%~19%,L1平均下降了0.2%~5.66%。实验证明,所提方法修复后的图像在视觉上具有更加真实和自然的效果,进一步验证了该方法的有效性。

    Abstract:

    Addressing the challenges in the domain of image inpainting, such as the high computational complexity, loss of information during feature extraction, and the blurring of textures in the inpainting images, this study proposed a image inpainting model that integrates multiscale hierarchical feature fusion with synergetic global-local Transformer. Initially, the multi-scale hierarchical feature fusion block was proposed as a means of effectively fusing deep and shallow features in detail, thereby reducing the loss of key information while expanding the sensory field. Subsequently, synergetic global-local Transformer blocks for global reasoning was proposed, featuring an integrated rectangle-window self-attention mechanism and local feed-forward neural networks. This design reduced computational complexity while enhancing the model′s macroscopic understanding of global context and microscopic grasp of local detail characteristics.The proposed method was validated on the CelebA-HQ and Places2 datasets, and the results demonstrated that it yielded improvements in PSNR by an average of 0.26~6.25 dB, SSIM by an average of 1.4%~19%, and L1 decreased by an average of 0.2%~5.66% compared to commonly used inpainting methods when dealing with 40%~50% masks. The experiments show that the inpainted images resulting from the proposed method exhibit a more realistic and natural visual effect, thereby providing further validation of the method′s effectiveness.

    参考文献
    相似文献
    引证文献
引用本文

滕诗宇,何丽君.融合多特征与全局-局部协同Transformer的图像修复算法[J].电子测量技术,2025,48(6):121-129

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2025-05-08
  • 出版日期:
文章二维码