0
0

论文阅读笔记：ShuffleNet

1. 背景

由于深度学习模型结构越来越复杂，参数量也越来越大，需要大量的算力去做模型的训练和推理。然而随着移动设备的普及，将深度学习模型部署于计算资源有限基于ARM的移动设备成为了研究的热点。

ShuffleNet[1]是一种专门为计算资源有限的设备设计的神经网络结构，主要采用了pointwise group convolution 和 channel shuffle两种技术，在保留了模型精度的同时极大减少了计算开销。

[1] Zhang X, Zhou X, Lin M, et al. Shufflenet: An extremely efficient convolutional neural network for mobile devices[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6848-6856.

2. 相关工作

在论文中，提到了目前sota的两个工作，一个是谷歌的Xception，另一个是facebook推出的ResNeXt。

2.1 Xception

Xception[2]主要涉及了一个技术：深度可分离卷积，即把原本常规的卷积操作分为两步去做。
常规卷积是利用若干个多通道卷积核对输入的多通道图像进行处理，输出的是既提取了通道特征又提取了空间特征的feature map。
在这里插入图片描述
而深度可分离卷积将提取通道特征(PointWise Convolution)和空间特征(DepthWise Convolution)分为了两步去做:
首先卷积核从三维变为了二维的，每个卷积核只负责输入图像的一个通道，用于提取空间特征，这一步操作中不涉及通道和通道之间的信息交互。接着通过一维卷积来完成通道之间特征提取的工作，即一个常规的卷积操作，只不过卷积核是1*1的。
在这里插入图片描述
在这里插入图片描述
这样做的好处是降低了常规卷积时的参数量，假设输入通道为

  1.     M
  2.    
  3.   
  4.   
  5.    M
  6.   
  7.  
  8. M, 输出通道为
  10.  
  11.   
  12.    
  13.     N
  14.    
  15.   
  16.   
  17.    N
  18.   
  19.  
  20. N，卷积核大小为
  22.  
  23.   
  24.    
  25.     k
  26.    
  27.    
  28.     ×
  29.    
  30.    
  31.     k
  32.    
  33.    
  34.     ,
  35.    
  36.    
  37.     那
  38.    
  39.    
  40.     么
  41.    
  42.   
  43.   
  44.    k \times k, 那么
  45.   
  46.  
  47. k×k,那么常规卷积的参数是：
  49.  
  50.   
  51.    
  52.     N
  53.    
  54.    
  55.     ×
  56.    
  57.    
  58.     M
  59.    
  60.    
  61.     ×
  62.    
  63.    
  64.     k
  65.    
  66.    
  67.     ×
  68.    
  69.    
  70.     k
  71.    
  72.   
  73.   
  74.    N \times M \times k \times k
  75.   
  76.  
  77. N×M×k×k。而通过深度可分离卷积之后，参数量为
  79.  
  80.   
  81.    
  82.     M
  83.    
  84.    
  85.     ×
  86.    
  87.    
  88.     k
  89.    
  90.    
  91.     ×
  92.    
  93.    
  94.     k
  95.    
  96.    
  97.     +
  98.    
  99.    
  100.     N
  101.    
  102.    
  103.     ×
  104.    
  105.    
  106.     M
  107.    
  108.    
  109.     ×
  110.    
  111.    
  112.     1
  113.    
  114.    
  115.     ×
  116.    
  117.    
  118.     1
  119.    
  120.   
  121.   
  122.    M \times k \times k + N \times M \times 1 \times 1
  123.   
  124.  
  125. M×k×k+N×M×1×1。

代码如下：

  1. classSeparableConv2d(nn.Module):def__init__(self, in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1):super(SeparableConv2d, self).__init__()
  3.         self.conv1 = nn.Conv2d(
  4.             in_channels, in_channels, kernel_size, stride, padding, dilation, groups=in_channels, bias=False)
  5.         self.pointwise = nn.Conv2d(in_channels, out_channels,1,1,0,1,1, bias=False)defforward(self, x):
  6.         x = self.conv1(x)
  7.         x = self.pointwise(x)return x

[2] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258.

2.2 ResNeXt

作者灵感来源于VGG的模块化堆叠的结构，提出了一种基于分组卷积和残差连接的模块化卷积块从而降低了参数的数量。简单来说，理解了分组卷积的思想就能理解ResNeXt。
在这里插入图片描述

[3] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1492-1500.

3. ShuffleNet

由于使用

  1.     1
  2.    
  3.    
  4.     ×
  5.    
  6.    
  7.     1
  8.    
  9.   
  10.   
  11.    1 \times 1
  12.   
  13.  
  14. 1×1卷积核进行操作时的复杂度较高，因为需要和每个像素点做互相关运算，作者关注到ResNeXt的设计中，
  16.  
  17.   
  18.    
  19.     1
  20.    
  21.    
  22.     ×
  23.    
  24.    
  25.     1
  26.    
  27.   
  28.   
  29.    1 \times 1
  30.   
  31.  
  32. 1×1卷积操作的那一层需要消耗大量的计算资源，因此提出将这一层也设计为分组卷积的形式。然而，分组卷积只会在组内进行卷积，因此组和组之间不存在信息的交互，为了使得信息在组之间流动，作者提出将每次分组卷积后的结果进行组内分组，再互相交换各自的组内的子组。

在这里插入图片描述
在这里插入图片描述
上图c就是一个shufflenet块，图a是一个简单的残差连接块，区别在于，shufflenet将残差连接改为了一个平均池化的操作与卷积操作之后做cancat，并且将

  1.     1
  2.    
  3.    
  4.     ×
  5.    
  6.    
  7.     1
  8.    
  9.   
  10.   
  11.    1 \times 1
  12.   
  13.  
  14. 1×1卷积改为了分组卷积，并且在分组之后进行了channel shuffle操作。

3.1 代码讲解

代码如下：
首先定义1x1，3x3，depthwise_3x3卷积操作：

  1. import torch
  2. import torch.nn as nn
  3. import torchvision
  4. from torch.utils import data
  5. import matplotlib.pyplot as plt
  6. import copy
  7. defconv1x1(in_channels, out_channels, stride=1, groups=1, bias=False):# 1x1卷积操作return nn.Conv2d(in_channels=in_channels, out_channels=out_channels,
  8.                      kernel_size=1, stride=stride, groups=groups, bias=bias)defconv3x3(in_channels, out_channels, stride=1, padding=1, dilation=1, groups=1, bias=False):# 3x3卷积操作# 默认不是下采样return nn.Conv2d(in_channels=in_channels, out_channels=out_channels,
  9.                      kernel_size=3, stride=stride, padding=padding, dilation=dilation,
  10.                      groups=groups,bias=bias)defdepthwise_con3x3(channels, stride):# 空间特征抽取# 输入通道和输出通道相等，且分组数等于通道数return nn.Conv2d(in_channels=channels, out_channels=channels,
  11.                      kernel_size=3, stride=stride, padding=1, groups=channels,bias=False)

接着是核心的channel shuffle操作:
通过矩阵变化即可实现,此操作并不会改变通道数和图像的尺寸
在这里插入图片描述

  1. defchannel_shuffle(x, groups):# x[batch_size, channels, H, W]
  2.     batch, channels, height, width = x.size()
  3.     channels_per_group = channels // groups  # 每组通道数
  4.     x = x.view(batch, groups, channels_per_group, height, width)
  5.     x = torch.transpose(x,1,2).contiguous()
  6.     x = x.view(batch, channels, height, width)return x
  8. classChannelShuffle(nn.Module):def__init__(self, channels, groups):super(ChannelShuffle, self).__init__()if channels % groups !=0:raise ValueError("通道数必须可以整除组数")
  9.         self.groups = groups
  11.     defforward(self, x):return channel_shuffle(x, self.groups)

然后定义shufflenet块,分为下采样和不下采样:

  1. classShuffleUnit(nn.Module):def__init__(self, in_channels, out_channels, groups, downsample, ignore_group):# 如果做下采样，那么通道数翻倍，高宽减半# 如果不做下采样，那么输入输出通道数相等，高宽不变super(ShuffleUnit, self).__init__()
  2.         self.downsample = downsample
  3.         mid_channels = out_channels //4if downsample:
  4.             out_channels -= in_channels
  5.         else:assert in_channels == out_channels,"不做下采样时应该输入输出通道相等"
  7.         self.compress_conv1 = conv1x1(
  8.             in_channels=in_channels,
  9.             out_channels=mid_channels,
  10.             groups=(1if ignore_group else groups))
  12.         self.compress_bn1 = nn.BatchNorm2d(num_features=mid_channels)
  14.         self.c_shuffle = ChannelShuffle(channels=mid_channels, groups=groups)
  16.         self.dw_conv2 = depthwise_con3x3(channels=mid_channels, stride=(2if downsample else1))
  18.         self.dw_bn2 = nn.BatchNorm2d(num_features=mid_channels)
  20.         self.expand_conv3 = conv1x1(
  21.             in_channels=mid_channels,
  22.             out_channels=out_channels,
  23.             groups=groups
  24.         )
  26.         self.expand_bn3 = nn.BatchNorm2d(num_features=out_channels)if downsample:
  27.             self.avgpool = nn.AvgPool2d(kernel_size=3, stride=2, padding=1)
  29.         self.activ = nn.ReLU(inplace=True)defforward(self, x):
  30.         identity = x
  31.         x = self.compress_conv1(x)# x[batch_size, mid_channels, H, W]
  32.         x = self.compress_bn1(x)
  33.         x = self.activ(x)
  34.         x = self.c_shuffle(x)
  35.         x = self.dw_conv2(x)# x[batch_size, mid_channels, H, w]
  36.         x = self.dw_bn2(x)
  37.         x = self.expand_conv3(x)# x[batch_size, out_channels, H, W]
  38.         x = self.expand_bn3(x)if self.downsample:
  39.             identity = self.avgpool(identity)
  40.             x = torch.cat((x, identity), dim=1)# 通道维上拼接else:
  41.             x = x + identity
  42.         x = self.activ(x)return x

在进入shufflenet之前常规地做一个下采样：

  1. classShuffleInitBlock(nn.Module):def__init__(self, in_channels, out_channels):super(ShuffleInitBlock, self).__init__()
  2.         self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=2, padding=1)# 下采样
  3.         self.bn = nn.BatchNorm2d(out_channels)
  4.         self.activ = nn.ReLU(inplace=True)
  5.         self.pool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)# 下采样defforward(self, x):
  6.         x = self.conv(x)
  7.         x = self.bn(x)
  8.         x = self.activ(x)
  9.         x = self.pool(x)return x

建立shufflenet完整的流程：

  1. classShuffleNet(nn.Module):def__init__(self, channels, init_block_channels, groups, in_channels=1, in_size=(224,224), num_classes=10):super(ShuffleNet, self).__init__()
  3.         self.in_size = in_size
  4.         self.num_classes = num_classes
  6.         self.features = nn.Sequential()
  7.         self.features.add_module("init_block", ShuffleInitBlock(in_channels, init_block_channels))
  8.         in_channels = init_block_channels
  9.         for i, channels_per_stage inenumerate(channels):
  10.             stage = nn.Sequential()for j, out_channels inenumerate(channels_per_stage):
  11.                 downsample =(j ==0)
  12.                 ignore_group =(i==0)and(j==0)
  13.                 stage.add_module("unit{}".format(j +1), ShuffleUnit(
  14.                     in_channels=in_channels,
  15.                     out_channels=out_channels,
  16.                     groups=groups,
  17.                     downsample=downsample,
  18.                     ignore_group=ignore_group))
  19.                 in_channels = out_channels
  20.             self.features.add_module("stage{}".format(i +1), stage)
  21.         self.features.add_module("final_pool", nn.AvgPool2d(
  22.             kernel_size=7,
  23.             stride=1))
  25.         self.output = nn.Linear(
  26.             in_features=in_channels,
  27.             out_features=num_classes)defforward(self, x):
  28.         x = self.features(x)
  29.         x = x.view(x.size(0),-1)
  30.         x = self.output(x)return x
  32. defget_shufflenet(groups, width_scale):
  33.     init_block_channels =24
  34.     layers =[2,4,2]if groups ==1:
  35.         channels_per_layers =[144,288,576]elif groups ==2:
  36.         channels_per_layers =[200,400,800]elif groups ==3:
  37.         channels_per_layers =[240,480,960]elif groups ==4:
  38.         channels_per_layers =[272,544,1088]elif groups ==8:
  39.         channels_per_layers =[384,768,1536]else:raise ValueError("The {} of groups is not supported".format(groups))
  40.     channels =[[ci]* li for(ci, li)inzip(channels_per_layers, layers)]if width_scale !=1.0:
  41.         channels =[[int(cij * width_scale)for cij in ci]for ci in channels]
  42.         init_block_channels =int(init_block_channels * width_scale)
  44.     net = ShuffleNet(
  45.         channels=channels,
  46.         init_block_channels=init_block_channels,
  47.         groups=groups)return net

训练过程：

  1. net = get_shufflenet(groups=2, width_scale=1.0)
  2. NUM_EPOCHS =10
  3. BATCH_SIZE =64
  4. NUM_CLASSES =10
  5. LR =0.001defload_data_fashion_mnist(batch_size, resize=None):
  6.     trans =[torchvision.transforms.ToTensor()]if resize:
  7.         trans.insert(0, torchvision.transforms.Resize(resize))
  8.     trans = torchvision.transforms.Compose(trans)
  9.     mnist_train = torchvision.datasets.FashionMNIST(
  10.         root="./FashionMinist", train=True, transform=trans, download=True)
  11.     mnist_test = torchvision.datasets.FashionMNIST(
  12.         root="./FashionMinist", train=False, transform=trans, download=True)return(data.DataLoader(mnist_train, batch_size, shuffle=True,),
  13.             data.DataLoader(mnist_test, batch_size, shuffle=False,))
  14. train_loader, test_loader = load_data_fashion_mnist(BATCH_SIZE,224)
  15. device = torch.device('cuda'if torch.cuda.is_available()else'cpu')defvalidate(net, data):
  16.     total =0
  17.     correct =0
  18.     net.eval()with torch.no_grad():for i,(images, labels)inenumerate(data):
  19.             images = images.to(device)
  20.             x = net(images)
  21.             value, pred = torch.max(x,1)
  22.             pred = pred.data.cpu()
  23.             total += x.size(0)
  24.             correct += torch.sum(pred == labels)return correct*100./total
  26. deftrain(net):
  28.     lossfunc = nn.CrossEntropyLoss()
  29.     optimizer = torch.optim.Adam(net.parameters(), lr=LR)
  30.     max_accuracy =0
  31.     accuracies =[]definit_weights(m):iftype(m)== nn.Linear ortype(m)== nn.Conv2d:
  32.             nn.init.xavier_uniform_(m.weight)
  33.     net.apply(init_weights)
  34.     net = net.to(device)
  35.     net.train()for epoch inrange(NUM_EPOCHS):for i,(images, labels)inenumerate(train_loader):
  36.             images = images.to(device)
  37.             labels = labels.to(device)
  38.             optimizer.zero_grad()
  39.             out = net(images)
  40.             loss = lossfunc(out, labels)
  41.             loss_item = loss.item()
  42.             loss.backward()
  43.             optimizer.step()
  44.         accuracy =float(validate(net, test_loader))
  45.         accuracies.append(accuracy)print("Epoch %d accuracy: %f loss: %f"%(epoch, accuracy, loss_item))if accuracy > max_accuracy:
  46.             best_model = copy.deepcopy(net)
  47.             max_accuracy = accuracy
  48.             print("Saving Best Model with Accuracy: ", accuracy)
  49.     plt.plot(accuracies)return best_model
  51. shufflenet = train(net)

4. ShuffleNetV2

Ma N, Zhang X, Zheng H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C]. Proceedings of the European conference on computer vision (ECCV). 2018: 116-131.

ShuffleNetV2 这篇文章对shufflenet进行了进一步的改进，并且提出了四个设计轻量化网络的原则：

  • 输入输出通道相同时，内存访问量MAC最小
  • 分组数过大会导致MAC增加
  • 碎片化操作对并行加速不友好
  • 逐元素操作会增加内存的消耗在这里插入图片描述 可以看到，shufflenetv2替换了1x1的分组卷积，并且尽量避免了add操作。 变动很小，就直接给出代码了：
  1. classShuffleUnitV2(nn.Module):def__init__(self, in_channels, out_channels, downsample, use_residual):super(ShuffleUnitV2, self).__init__()
  2.         self.downsample = downsample
  3.         self.use_residual = use_residual
  4.         mid_channels = out_channels //2
  6.         self.compress_conv1 = conv1x1(
  7.             in_channels=(in_channels if downsample else mid_channels),
  8.             out_channels=mid_channels
  9.         )
  11.         self.compress_bn1 = nn.BatchNorm2d(num_features=mid_channels)
  13.         self.dw_conv2 = depthwise_con3x3(
  14.             channels=mid_channels,
  15.             stride=(2if downsample else1))
  17.         self.dw_bn2 = nn.BatchNorm2d(mid_channels)
  19.         self.expand_conv3 = conv1x1(
  20.             in_channels=mid_channels,
  21.             out_channels=mid_channels
  22.         )
  23.         self.expand_bn3 = nn.BatchNorm2d(num_features=mid_channels)if downsample:
  24.             self.dw_conv4 = depthwise_con3x3(
  25.                 channels=in_channels,
  26.                 stride=2)
  27.             self.dw_bn4 = nn.BatchNorm2d(num_features=in_channels)
  28.             self.expand_conv5 = conv1x1(
  29.                 in_channels=in_channels,
  30.                 out_channels=mid_channels
  31.             )
  32.             self.expand_bn5 = nn.BatchNorm2d(num_features=mid_channels)
  34.         self.activ = nn.ReLU(inplace=True)
  35.         self.c_shuffle = ChannelShuffle(
  36.             channels=out_channels,
  37.             groups=2)defforward(self, x):if self.downsample:
  38.             y1 = self.dw_conv4(x)
  39.             y1 = self.dw_bn4(y1)
  40.             y1 = self.expand_conv5(y1)
  41.             y1 = self.expand_bn5(y1)
  42.             y1 = self.activ(y1)
  43.             x2 = x
  44.         else:
  45.             y1, x2 = torch.chunk(x, chunks=2, dim=1)
  47.         y2 = self.compress_conv1(x2)
  48.         y2 = self.compress_bn1(y2)
  49.         y2 = self.activ(y2)
  50.         y2 = self.dw_conv2(y2)
  51.         y2 = self.dw_bn2(y2)
  52.         y2 = self.expand_conv3(y2)
  53.         y2 = self.expand_bn3(y2)
  54.         y2 = self.activ(y2)if self.use_residual andnot self.downsample:
  55.             y2 = y2 + x2
  56.         x = torch.cat((y1, y2), dim=1)
  57.         x = self.c_shuffle(x)return x
  1. classShuffleNetV2(nn.Module):def__init__(self, channels, init_block_channels, final_block_channels,
  2.                  use_residual=False, in_channels=1, in_size=(224,224), num_classes=10):super(ShuffleNetV2, self).__init__()
  3.         self.in_size = in_size
  4.         self.num_classes = num_classes
  6.         self.features = nn.Sequential()
  7.         self.features.add_module("init_block", ShuffleInitBlock(
  8.             in_channels=in_channels,
  9.             out_channels=init_block_channels))
  10.         in_channels = init_block_channels
  11.         for i, channels_per_stage inenumerate(channels):
  12.             stage = nn.Sequential()for j, out_channels inenumerate(channels_per_stage):
  13.                 downsample =(j==0)
  14.                 stage.add_module("unit{}".format(j+1), ShuffleUnitV2(
  15.                     in_channels=in_channels,
  16.                     out_channels=out_channels,
  17.                     downsample=downsample,
  18.                     use_residual=use_residual
  19.                 ))
  20.                 in_channels=out_channels
  21.             self.features.add_module("stage{}".format(i +1), stage)
  22.         self.features.add_module("final_block", conv1x1(
  23.             in_channels=in_channels,
  24.             out_channels=final_block_channels
  25.         ))
  26.         in_channels = final_block_channels
  27.         self.features.add_module("final_bn", nn.BatchNorm2d(num_features=in_channels))
  28.         self.features.add_module("final_pool", nn.AdaptiveAvgPool2d(output_size=(1,1)))
  29.         self.features.add_module("flatten", nn.Flatten())
  30.         self.output = nn.Linear(
  31.             in_features=in_channels,
  32.             out_features=num_classes
  33.         )defforward(self, x):
  34.         x = self.features(x)
  35.         x = x.view(x.size(0),-1)
  36.         x = self.output(x)return x
  1. defget_shufflenetv2(width_scale):
  2.     init_block_channels =24
  3.     final_block_channels =1024
  4.     layers =[4,8,4]
  5.     channels_per_layers =[116,232,464]
  6.     channels =[[ci]* li for(ci, li)inzip(channels_per_layers, layers)]if width_scale !=1.0:
  7.         channels =[[int(cij * width_scale)for cij in ci]for ci in channels]if width_scale >1.5:
  8.             final_block_channels =int(final_block_channels * width_scale)
  9.     net = ShuffleNetV2(
  10.     channels=channels,
  11.     init_block_channels=init_block_channels,
  12.     final_block_channels=final_block_channels)return net
标签： python 深度学习 人工智能
本文转载自: https://blog.csdn.net/loki2018/article/details/124077822
版权归原作者 loki2018 所有， 如有侵权，请联系我们删除。
登录后发布评论

“论文阅读笔记：ShuffleNet”的评论:

还没有评论
关于作者
...
overfit同步小助手
文章同步
相关阅读

网络安全法-网络运行安全

使用selenium/drissionpage时如何阻止chrome自动跳转http到https

docker desktop 里部署的Open WebUI 管理员密码忘记了的处理方法

在ubuntu20.04中搭建onsite比赛运行环境

利用开源的低代码表单设计器FcDesigner高效管理和渲染复杂表单结构

Kafka学习笔记

【前端】浏览器输入url到页面呈现发生了什么？