Breast cancer has become a killer of women's health nowadays. In order to exploit the potential representational capabilities of the models more comprehensively, we propose a multi-model fusion strategy. Specifically, we combine two differently structured deep learning models, ResNet101 and Swin Transformer (SwinT), with the addition of the Convolutional Block Attention Module (CBAM) attention mechanism, which makes full use of SwinT's global context information modeling ability and ResNet101's local feature extraction ability, and additionally the cross entropy loss function is replaced by the focus loss function to solve the problem of unbalanced allocation of breast cancer data sets. The multi-classification recognition accuracies of the proposed fusion model under 40X, 100X, 200X and 400X BreakHis datasets are 97.50%, 96.60%, 96.30 and 96.10%, respectively. Compared with a single SwinT model and ResNet101 model, the fusion model has higher accuracy and better generalization ability, which provides a more effective method for screening, diagnosis and pathological classification of female breast cancer.