Tensorflow2.0教程-自定义层

Tensorflow2.0教程-自定义层

Tensorflow2.0教程-自定义层

tensorflow2.0建议使用tf.keras作为构建神经网络的高级API。 也就是说,大多数TensorFlow API都可用于eager执行模式。

最全Tensorflow 2.0 入门教程持续更新:
Doit:最全Tensorflow2.0 入门教程持续更新zhuanlan.zhihu.com图标


完整tensorflow2.0教程代码请看 github.com/czy36mengfei (欢迎star)
本教程主要由tensorflow2.0官方教程的个人学习复现笔记整理而来,中文讲解,方便喜欢阅读中文教程的朋友,官方教程:https://www.tensorflow.org



一、网络层layer的常见操作

通常机器学习模型可以表示为简单网络层的堆叠与组合,而tensorflow就提供了常见的网络层,为我们编写神经网络程序提供了便利。 TensorFlow2推荐使用tf.keras来构建网络层,tf.keras来自原生keras,用其来构建网络具有更好的可读性和易用性。

如,我们要构造一个简单的全连接网络,只需要指定网络的神经元个数

layer = tf.keras.layers.Dense(100)
# 也可以添加输入维度限制
layer = tf.keras.layers.Dense(100, input_shape=(None, 20))

可以在文档中查看预先存在的图层的完整列表。 它包括Dense,Conv2D,LSTM,BatchNormalization,Dropout等等。

每个层都可以当作一个函数,然后以输入的数据作为函数的输入

layer(tf.ones([6, 6]))

同时我们也可以得到网络的变量、权重矩阵、偏置等

print(layer.variables) # 包含了权重和偏置
[<tf.Variable 'dense_1/kernel:0' shape=(6, 100) dtype=float32, numpy=
array([[-0.18237606,  0.16415142,  0.20687856,  0.23396944,  0.09779547,
        -0.14794639, -0.10231382, -0.22263053, -0.0950674 ,  0.18697281,
         ...
        -0.10450925, -0.12111329, -0.2259491 ,  0.12304659, -0.04047236]],
      dtype=float32)>, <tf.Variable 'dense_1/bias:0' shape=(100,) dtype=float32, numpy=
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
      dtype=float32)>]
print(layer.kernel, layer.bias)  # 也可以分别取出权重和偏置
<tf.Variable 'dense_1/kernel:0' shape=(6, 100) dtype=float32, numpy=
array([[-0.18237606,  0.16415142,  0.20687856,  0.23396944,  0.09779547,
        -0.14794639, -0.10231382, -0.22263053, -0.0950674 ,  0.18697281,
         ...
        -0.10348159, -0.07493602, -0.1722112 , -0.23290877,  0.18784209],
       [ 0.13477843,  0.11936818, -0.21257897,  0.21244659, -0.18786472,
        -0.06494723, -0.07063387, -0.07994832, -0.11256738, -0.22335076,
        -0.02153319, -0.20943552, -0.21425952, -0.12278055, -0.00619341,
        -0.09176037, -0.1766775 , -0.21622379, -0.04250833,  0.23764552,
         0.21168886,  0.09459655, -0.07919639, -0.21559525,  0.20465617,
        -0.20613717,  0.13103445,  0.21384992,  0.04693423,  0.20122723,
         0.12190209,  0.22194327, -0.05410977, -0.1792583 , -0.03342254,
         0.09272121,  0.06039228,  0.09666802, -0.22759588, -0.14688678,
         0.12520896,  0.15474696, -0.23104139,  0.18017791, -0.02388267,
        -0.01371126,  0.2352383 , -0.10501392,  0.01626216, -0.14222105,
         0.13740788,  0.18499441,  0.03618436, -0.01862051, -0.1401035 ,
        -0.01304157, -0.04905747, -0.07051091,  0.10759439, -0.08964662,
        -0.01344521, -0.17841959, -0.17568308, -0.12892699,  0.11976974,
         0.02280475,  0.16669382,  0.21027894,  0.21428709, -0.04820213,
        -0.22136293, -0.13934767,  0.142024  , -0.07064074,  0.1470062 ,
         0.00042979, -0.2371952 , -0.06649312,  0.10123204, -0.20473264,
        -0.09161748,  0.20804678, -0.22195774, -0.09219673,  0.02311908,
         0.13456099,  0.14470674, -0.05369592,  0.02126037,  0.0682667 ,
         0.08384518,  0.17998771, -0.1927835 , -0.11473013, -0.01386146,
        -0.10450925, -0.12111329, -0.2259491 ,  0.12304659, -0.04047236]],
      dtype=float32)> <tf.Variable 'dense_1/bias:0' shape=(100,) dtype=float32, numpy=
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
      dtype=float32)>

二、实现自定义网络层

实现自己的层的最佳方法是扩展tf.keras.Layer类并实现:

  • __init__()函数,你可以在其中执行所有与输入无关的初始化
  • build()函数,可以获得输入张量的形状,并可以进行其余的初始化
  • call()函数,构建网络结构,进行前向传播

实际上,你不必等到调用build()来创建网络结构,您也可以在__init__() 中创建它们。 但是,在build()中创建它们的优点是它可以根据图层将要操作的输入的形状启用后期的网络构建。 另一方面,在__init__中创建变量意味着需要明确指定创建变量所需的形状。

class MyDense(tf.keras.layers.Layer):
    def __init__(self, n_outputs):
        super(MyDense, self).__init__()
        self.n_outputs = n_outputs

    def build(self, input_shape):
        self.kernel = self.add_variable('kernel',
                                       shape=[int(input_shape[-1]),
                                             self.n_outputs])
    def call(self, input):
        return tf.matmul(input, self.kernel)
layer = MyDense(10)
print(layer(tf.ones([6, 5])))
print(layer.trainable_variables)
tf.Tensor(
[[ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]
 [ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]
 [ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]
 [ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]
 [ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]
 [ 1.0200843  -0.42590106 -0.92992705  0.46160045  0.7518406   0.32543844
   0.34020287  0.08215448  0.22044104 -0.5337319 ]], shape=(6, 10), dtype=float32)
[<tf.Variable 'my_dense/kernel:0' shape=(5, 10) dtype=float32, numpy=
array([[ 0.54810244,  0.042225  ,  0.25634396,  0.1677258 , -0.0361526 ,
         0.32831818,  0.17709464,  0.46625894,  0.29662275, -0.32920587],
       [ 0.30925363, -0.426274  , -0.49862564,  0.3068235 ,  0.29526353,
         0.50076336,  0.17321467,  0.21151704, -0.26317668, -0.2006711 ],
       [ 0.10354012, -0.3258371 , -0.12274069, -0.33250242,  0.46343058,
        -0.45535576,  0.5332853 , -0.37351888, -0.00410944,  0.16418225],
       [-0.4515978 ,  0.04706419, -0.42583126, -0.19347438,  0.54246336,
         0.57910997,  0.01877069,  0.01255274, -0.14176458, -0.6309683 ],
       [ 0.5107859 ,  0.23692083, -0.13907343,  0.51302797, -0.5131643 ,
        -0.6273973 , -0.56216246, -0.23465535,  0.332869  ,  0.4629311 ]],
      dtype=float32)>]

三、网络层组合

机器学习模型中有很多是通过叠加不同的结构层组合而成的,如resnet的每个残差块就是“卷积+批标准化+残差连接”的组合。

在tensorflow2中要创建一个包含多个网络层的的结构,一般继承与tf.keras.Model类。

# 残差块
class ResnetBlock(tf.keras.Model):
    def __init__(self, kernel_size, filters):
        super(ResnetBlock, self).__init__(name='resnet_block')

        # 每个子层卷积核数
        filter1, filter2, filter3 = filters

        # 三个子层,每层1个卷积加一个批正则化
        # 第一个子层, 1*1的卷积
        self.conv1 = tf.keras.layers.Conv2D(filter1, (1,1))
        self.bn1 = tf.keras.layers.BatchNormalization()
        # 第二个子层, 使用特点的kernel_size
        self.conv2 = tf.keras.layers.Conv2D(filter2, kernel_size, padding='same')
        self.bn2 = tf.keras.layers.BatchNormalization()
        # 第三个子层,1*1卷积
        self.conv3 = tf.keras.layers.Conv2D(filter3, (1,1))
        self.bn3 = tf.keras.layers.BatchNormalization()

    def call(self, inputs, training=False):

        # 堆叠每个子层
        x = self.conv1(inputs)
        x = self.bn1(x, training=training)

        x = self.conv2(x)
        x = self.bn2(x, training=training)

        x = self.conv3(x)
        x = self.bn3(x, training=training)

        # 残差连接
        x += inputs
        outputs = tf.nn.relu(x)

        return outputs

resnetBlock = ResnetBlock(2, [6,4,9])
# 数据测试
print(resnetBlock(tf.ones([1,3,9,9])))
# 查看网络中的变量名
print([x.name for x in resnetBlock.trainable_variables])
tf.Tensor(
[[[[0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.83203167 0.9436392  1.0989372  1.2588525  0.8683256  1.1279813
    0.7571581  0.47963202 0.88908756]]

  [[0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.79764616 1.0550306  0.9386751  1.1079601  0.9402881  0.99479383
    0.9072118  0.5618475  0.9134829 ]
   [0.83203167 0.9436392  1.0989372  1.2588525  0.8683256  1.1279813
    0.7571581  0.47963202 0.88908756]]

  [[1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.72680396 1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.7268039  1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [1.0775117  1.1620466  0.7268039  1.0019443  1.2767658  1.1365149
    1.1792164  1.0868194  1.0623009 ]
   [0.87889266 0.9541194  0.8929231  0.96703756 1.0905087  1.0646607
    0.9235744  0.9829142  1.1302696 ]]]], shape=(1, 3, 9, 9), dtype=float32)
['resnet_block/conv2d_12/kernel:0', 'resnet_block/conv2d_12/bias:0', 'resnet_block/batch_normalization_v2_12/gamma:0', 'resnet_block/batch_normalization_v2_12/beta:0', 'resnet_block/conv2d_13/kernel:0', 'resnet_block/conv2d_13/bias:0', 'resnet_block/batch_normalization_v2_13/gamma:0', 'resnet_block/batch_normalization_v2_13/beta:0', 'resnet_block/conv2d_14/kernel:0', 'resnet_block/conv2d_14/bias:0', 'resnet_block/batch_normalization_v2_14/gamma:0', 'resnet_block/batch_normalization_v2_14/beta:0']

如果模型是线性的,可以直接用tf.keras.Sequential来构造。

seq_model = tf.keras.Sequential(
[
    tf.keras.layers.Conv2D(1, 1, input_shape=(None, None, 3)),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Conv2D(2, 1, padding='same'),
    tf.keras.layers.BatchNormalization(),
    tf.keras.layers.Conv2D(3, 1),
    tf.keras.layers.BatchNormalization(),

])
seq_model(tf.ones([1,2,3,3]))
<tf.Tensor: id=1354, shape=(1, 2, 3, 3), dtype=float32, numpy=
array([[[[-0.36850607, -0.60731524,  1.2792252 ],
         [-0.36850607, -0.60731524,  1.2792252 ],
         [-0.36850607, -0.60731524,  1.2792252 ]],

        [[-0.36850607, -0.60731524,  1.2792252 ],
         [-0.36850607, -0.60731524,  1.2792252 ],
         [-0.36850607, -0.60731524,  1.2792252 ]]]], dtype=float32)>

发布于 2019-07-08

文章被以下专栏收录