# Where Global Context and Weight Sharing Work in Compact Vision Backbones

DevFeed: [Where Global Context and Weight Sharing Work in Compact Vision Backbones](<https://devfeed.tech/articles/where-global-context-and-weight-sharing-work-in-compact-vision-backbones-142656.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23446>)

Author: Michael Schroeder

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [batchnorm](<https://devfeed.tech/tags/batchnorm.md>), [block](<https://devfeed.tech/tags/block.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [classification](<https://devfeed.tech/tags/classification.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [efficientnet](<https://devfeed.tech/tags/efficientnet.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [global](<https://devfeed.tech/tags/global.md>), [global-context](<https://devfeed.tech/tags/global-context.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [scale](<https://devfeed.tech/tags/scale.md>), [self-attention](<https://devfeed.tech/tags/self-attention.md>), [sharing](<https://devfeed.tech/tags/sharing.md>), [stage](<https://devfeed.tech/tags/stage.md>), [vision](<https://devfeed.tech/tags/vision.md>), [weight](<https://devfeed.tech/tags/weight.md>), [work](<https://devfeed.tech/tags/work.md>)

## AI overview

Controlled EfficientNet-B0 experiments on ImageNet-1k suggest that global context helps when added to an intact MBConv trunk: zero-gated residual attention adapters at P3 (windowed) and P4/P5 (global) scored 73.88% versus 72.92% for a matched plain-B0 control, with attention amplitude increasing at deeper stages. A BiFPN wrapper alone scored 69.99-71.81%, below plain B0's 72.86%; adding stage attention raised it to 73.15%. For weight sharing, averaging kernels collapsed accuracy to 17.03%, while tying one template across a stage reduced parameters 27.6% but reached only 66.10%. Alternating two shared templates in Stage 6 cut parameters 10.9% (582K weights) and reached 70.92% after one recovery epoch, 1.84 points below the unshared control. Findings are limited to the described experiments and training schedule.

## Source excerpt

arXiv:2611.23446v1 Announce Type: cross Abstract: Compact convolutional backbones such as EfficientNet achieve strong parameter efficiency through local depthwise separable convolutions, yet two structural questions remain central to hybrid design: where does global self-attention improve recognition over pure convolution, and can repeated blocks share weights without losing representational capacity? We investigate both questions on ImageNet-1k (50,000-image validation) using controlled EfficientNet-B0 experiments. First, we show that wrapping a classification backbone in multi-scale feature pyramids (BiFPN) fails to outperform a plain MBConv trunk (69.99% - 71.81% vs. 72.86% plain B0) until paired with stage attention (73.15%). In contrast, keeping the MBConv trunk intact and inserting lightweight zero-gated residual self-attention adapters at stages P3 (28x28 windowed), P4 (14x14 global), and P5 (7x7 global) reaches 74.02% standalone and outperforms a bit-for-bit matched plain-B0 control by +0.954 percentage points (73.88% vs. 72.92%) across an 88-epoch stateless replay schedule, with learned attention amplitude rising monotonically with semantic depth (27.2% at P3 -> 45.4% at P4 -> 66.3% at P5). Second, we evaluate recurrent weight sharing across repeated MBConv blocks with pass-specific BatchNorm statistics (+1,536 parameters per pass). While post-hoc kernel averaging degrades accuracy to 17.03% and single-template stage tying (-27.6% parameters) plateaus at 66.10%, alternating two shared block templates ([B1, A, B, A]) in Stage 6 cuts total model parameters by 10.9% (582K weights) and recovers by +10.82 points in a single epoch to 70.92% (-1.84 points vs. unshared control).