# Hybrid Dual-Batch and Cyclic Progressive Learning for Efficient Distributed Training of Convolutional Neural Networks

DevFeed: [Hybrid Dual-Batch and Cyclic Progressive Learning for Efficient Distributed Training of Convolutional Neural Networks](<https://devfeed.tech/articles/hybrid-dual-batch-and-cyclic-progressive-learning-for-efficient-distributed-training-of-convolutional-neural-networks-120365.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.26092>)

Author: Kuan-Wei Lu, Ding-Yong Hong, Pangfeng Liu, Jan-Jan Wu

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [cnn](<https://devfeed.tech/tags/cnn.md>), [convolutional-neural-networks](<https://devfeed.tech/tags/convolutional-neural-networks.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [framework](<https://devfeed.tech/tags/framework.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [image](<https://devfeed.tech/tags/image.md>), [image-classification](<https://devfeed.tech/tags/image-classification.md>), [large](<https://devfeed.tech/tags/large.md>), [lead](<https://devfeed.tech/tags/lead.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The proposed distributed-training approach combines dual-batch learning with cyclic progressive learning. It uses the largest batch size supported by the hardware alongside a smaller batch to improve generalization, then gradually raises image resolution in repeated cycles to reduce the added training time. In ResNet-18 experiments versus conventional training, it reportedly improved accuracy by 3.3% and cut training time by 10.1% on CIFAR-100; on ImageNet, it cut training time by 34.7%. These are abstract-level results for image classification.

## Source excerpt

arXiv:2509.26092v3 Announce Type: replace-cross Abstract: Distributed deep learning is critical for training convolutional neural network (CNN) models on large-scale image datasets with numerous parameters. Current research primarily focuses on leveraging additional hardware resources and powerful computing units to accelerate the training process. As a result, larger batch sizes are often employed to speed up training. However, training with large batch sizes can lead to lower accuracy due to poor generalization. To address this issue, we propose the dual-batch learning scheme, a distributed training method built on the parameter server framework. This approach maximizes training efficiency by utilizing the largest batch size that the hardware can support while incorporating a smaller batch size to enhance model generalization. By using two different batch sizes simultaneously, this method improves accuracy with minimal additional training time. Additionally, to mitigate the time overhead caused by dual-batch learning, we propose the cyclic progressive learning scheme. This technique repeatedly and gradually increases image resolution from low to high during training, thereby reducing training time. By combining cyclic progressive learning with dual-batch learning, our hybrid approach improves both model generalization and training efficiency for image classification tasks. Experimental results with ResNet-18 demonstrate that, compared to conventional training methods, our approach improves accuracy by 3.3% while reducing training time by 10.1% on CIFAR-100, and further achieves a 34.7% reduction in training time on ImageNet.