deeplabv3plus网络结构图

### DeepLabV3Plus 网络架构详解 #### 背景介绍 DeepLab系列模型旨在解决语义分割任务中的挑战，特别是处理不同尺度的对象。通过引入空洞卷积（Atrous Convolution）、ASPP模块和其他改进技术，这些模型能够在不降低特征图分辨率的情况下增加感受野。 #### 主要组件解析 #### 编码器部分：基于ResNet的Backbone 编码器采用预训练的ResNet作为骨干网来提取图像特征[^1]。为了维持较高的空间分辨率并扩展感受域，在某些层中应用了`replace_stride_with_dilation`策略，即用膨胀率大于1的空洞卷积代替标准步长为2的标准卷积操作。这使得网络可以在相同计算成本下获得更大的上下文信息量。 ```python from torchvision import models def build_backbone(output_stride): model = models.resnet50(pretrained=True) if output_stride == 8: rates = [1, 2, 4] elif output_stride == 16: rates = [1, 2, 1] for name, layer in model.named_children(): if isinstance(layer, nn.Conv2d) and 'downsample' not in name: stride = layer.stride kernel_size = layer.kernel_size if stride[0] > 1: dilation_rate = rates.pop(0) new_padding = ((kernel_size[0]-1)*dilation_rate+1)//2 setattr(model, name, nn.Conv2d(in_channels=layer.in_channels, out_channels=layer.out_channels, kernel_size=kernel_size, stride=(1, 1), padding=new_padding, dilation=dilation_rate)) return model ``` #### ASPP (Atrous Spatial Pyramid Pooling) 模块 该模块位于编码器之后，由多个平行分支组成，每个分支都执行具有不同扩张比率的空洞卷积运算。这种设计允许模型捕捉多尺度的信息，并有效地增强了对于目标物体边界的理解能力[^3]。 ```python class ASPP(nn.Module): def __init__(self, inplanes, planes, rate_list=[1, 6, 12, 18]): super().__init__() self.branches = nn.ModuleList([ nn.Sequential( nn.Conv2d(inplanes, planes, 1, bias=False), nn.BatchNorm2d(planes), nn.ReLU(inplace=True)), *[nn.Sequential( nn.Conv2d(inplanes, planes, 3, padding=r, dilation=r, bias=False), nn.BatchNorm2d(planes), nn.ReLU(inplace=True)) for r in rate_list], nn.Sequential( nn.AdaptiveAvgPool2d((1, 1)), nn.Conv2d(inplanes, planes, 1, bias=False), nn.BatchNorm2d(planes), nn.ReLU(inplace=True)) ]) self.project = nn.Sequential( nn.Conv2d(len(rate_list)+2 * planes, planes, 1, bias=False), nn.BatchNorm2d(planes), nn.ReLU(inplace=True)) def forward(self, x): size = x.shape[-2:] feats = [] for branch in self.branches[:-1]: feat = F.interpolate(branch(x).unsqueeze(-1).unsqueeze(-1), size=size, mode='bilinear', align_corners=True).squeeze(-1).squeeze(-1) feats.append(feat) global_feat = self.branches[-1](F.adaptive_avg_pool2d(x, (1, 1))) global_feat = F.interpolate(global_feat, size=x.size()[2:], mode="bilinear", align_corners=True) feats.append(global_feat.squeeze(-1).squeeze(-1)) concat_feats = torch.cat(feats, dim=-1) proj_output = self.project(concat_feats.unsqueeze(-1).unsqueeze(-1)).squeeze(-1).squeeze(-1) return proj_output.view(*proj_output.shape[:2], *size) ``` #### 解码器部分 解码阶段融合来自低层次浅层特征与经过ASPP增强后的高层抽象表示。具体来说，先利用双线性插值放大高维特征映射至原始输入尺寸的一半；接着将其同早期阶段获取到的空间细节相结合，最终得到精确度更高的预测结果。 ```python import torch.nn.functional as F class Decoder(nn.Module): def __init__(self, low_level_inplanes, num_classes): super().__init__() self.conv_lowlevel = nn.Sequential( nn.Conv2d(low_level_inplanes, 48, 1, bias=False), nn.BatchNorm2d(48), nn.ReLU() ) self.last_conv = nn.Sequential( nn.Conv2d(304, 256, kernel_size=3, stride=1, padding=1, bias=False), nn.BatchNorm2d(256), nn.ReLU(), nn.Dropout(0.5), nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1, bias=False), nn.BatchNorm2d(256), nn.ReLU(), nn.Dropout(0.1), nn.Conv2d(256, num_classes, kernel_size=1, stride=1) ) def forward(self, x, low_level_feature): low_level_feature = self.conv_lowlevel(low_level_feature) x = F.interpolate(x, scale_factor=4, mode='bilinear', align_corners=True) x = torch.cat([x, low_level_feature], dim=1) x = self.last_conv(x) return x ``` #### 总结 综上所述，DeepLabV3Plus 结合了强大的编码机制和有效的解码流程，不仅能够高效地学习丰富的场景表征，而且还能恢复细粒度的位置信息，从而显著提升了语义分割的效果。

### DeepLabV3Plus 网络架构详细解析 #### Encoder 部分 DeepLabV3Plus 的编码器部分主要依赖于改进的 Xception 或 ResNet 架构作为骨干网络。此部分大量采用空洞卷积，在不减少特征图尺寸的前提下扩大感受野，从而保留更多空间分辨率信息[^3]。 #### Atrous Spatial Pyramid Pooling (ASPP) 模块 位于编码器末端的是 ASPP 模块，它由多个并行分支组成，每个分支执行不同膨胀率的空洞卷积操作。这种设计允许模型在同一层内捕捉多种尺度下的上下文信息，对于改善边界区域以及小目标检测至关重要[^2]。 ```python class ASPP(nn.Module): def __init__(self, in_channels, out_channels, rates=[1, 6, 12, 18]): super(ASPP, self).__init__() self.aspp_block1 = nn.Sequential( nn.Conv2d(in_channels, out_channels, kernel_size=1), nn.ReLU(inplace=True) ) self.aspp_blocks = [] for rate in rates: block = nn.Sequential( nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=rate, dilation=rate), nn.ReLU(inplace=True)) self.add_module(f'aspp_block{len(self.aspp_blocks)+2}', block) self.aspp_blocks.append(block) self.global_avg_pool = nn.Sequential( nn.AdaptiveAvgPool2d((1, 1)), nn.Conv2d(in_channels, out_channels, kernel_size=1), nn.ReLU(), nn.Upsample(scale_factor=(height, width), mode='bilinear', align_corners=True)) self.conv1x1_out = nn.Conv2d(out_channels * (len(rates) + 2), out_channels, kernel_size=1) def forward(self, x): height, width = x.shape[-2:] res = [block(x) for block in self.children()] res += [F.interpolate(self.global_avg_pool(x), size=x.size()[2:], mode="bilinear", align_corners=False)] return self.conv1x1_out(torch.cat(res, dim=1)) ``` #### Decoder 组件 解码阶段则相对简单得多，仅包含少量上采样层与跳跃连接机制。这些组件负责恢复因下采样而丢失的空间维度，并融合来自浅层特征映射的数据以增强最终预测的质量。 ```python class Decoder(nn.Module): def __init__(self, low_level_inplanes, num_classes): super().__init__() self.conv1 = nn.Conv2d(low_level_inplanes, 48, 1, bias=False) self.bn1 = nn.BatchNorm2d(48) self.relu = nn.ReLU() self.last_conv = nn.Sequential( nn.Conv2d(304, 256, kernel_size=3, stride=1, padding=1, bias=False), nn.BatchNorm2d(256), nn.ReLU(), nn.Dropout(0.5), nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1, bias=False), nn.BatchNorm2d(256), nn.ReLU(), nn.Dropout(0.1), nn.Conv2d(256, num_classes, kernel_size=1, stride=1) ) def forward(self, x, low_level_feat): low_level_feat = self.conv1(low_level_feat) low_level_feat = self.bn1(low_level_feat) low_level_feat = self.relu(low_level_feat) x = F.interpolate(x, size=low_level_feat.size()[2:], mode='bilinear', align_corners=True) x = torch.cat((x, low_level_feat), dim=1) x = self.last_conv(x) return x ```