 在 TensorFlow.org 上查看 |
 在 Google Colab 中运行 |
 在 GitHub 上查看源代码 |
 下载笔记本 |
本教程通过以下三个示例介绍自编码器：基础知识、图像降噪和异常检测。
自编码器是一种特殊类型的神经网络，经过训练后可将其输入复制到其输出。例如，给定一个手写数字的图像，自编码器首先将图像编码为低维的潜在表示，然后将该潜在表示解码回图像。自编码器学习压缩数据，同时最大程度地减少重构误差。
要详细了解自编码器，请考虑阅读 Ian Goodfellow、Yoshua Bengio 和 Aaron Courville 撰写的《深度学习》一书的第 14 章。
导入 TensorFlow 和其他库
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn.metrics import accuracy_score, precision_score, recall_score
from sklearn.model_selection import train_test_split
from tensorflow.keras import layers, losses
from tensorflow.keras.datasets import fashion_mnist
from tensorflow.keras.models import Model
2023-11-07 19:41:16.708459: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2023-11-07 19:41:16.708505: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2023-11-07 19:41:16.710018: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
加载数据集
首先，您将使用 Fashion MNIST 数据集训练基本自编码器。此数据集中的每个图像均为 28x28 像素。
(x_train, _), (x_test, _) = fashion_mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
print (x_train.shape)
print (x_test.shape)
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz 29515/29515 [==============================] - 0s 0us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz 26421880/26421880 [==============================] - 0s 0us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz 5148/5148 [==============================] - 0s 0us/step Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz 4422102/4422102 [==============================] - 0s 0us/step (60000, 28, 28) (10000, 28, 28)
第一个示例：基本自编码器
定义一个具有两个密集层的自编码器：一个将图像压缩为 64 维隐向量的 encoder，以及一个从隐空间重构原始图像的 decoder。
要定义模型，请使用 Keras Model Subclassing API。
latent_dim = 64
class Autoencoder(Model):
def __init__(self, latent_dim):
super(Autoencoder, self).__init__()
self.latent_dim = latent_dim
self.encoder = tf.keras.Sequential([
layers.Flatten(),
layers.Dense(latent_dim, activation='relu'),
])
self.decoder = tf.keras.Sequential([
layers.Dense(784, activation='sigmoid'),
layers.Reshape((28, 28))
])
def call(self, x):
encoded = self.encoder(x)
decoded = self.decoder(encoded)
return decoded
autoencoder = Autoencoder(latent_dim)
autoencoder.compile(optimizer='adam', loss=losses.MeanSquaredError())
使用 x_train 作为输入和目标来训练模型。encoder 会学习将数据集从 784 个维度压缩到隐空间，而 decoder 将学习重构原始图像。
autoencoder.fit(x_train, x_train,
epochs=10,
shuffle=True,
validation_data=(x_test, x_test))
Epoch 1/10 WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1699386084.268834 448569 device_compiler.h:186] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process. 1875/1875 [==============================] - 6s 2ms/step - loss: 0.0238 - val_loss: 0.0132 Epoch 2/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0117 - val_loss: 0.0108 Epoch 3/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0102 - val_loss: 0.0098 Epoch 4/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0095 - val_loss: 0.0094 Epoch 5/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0092 - val_loss: 0.0092 Epoch 6/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0091 - val_loss: 0.0091 Epoch 7/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0090 - val_loss: 0.0090 Epoch 8/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0089 - val_loss: 0.0090 Epoch 9/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0088 - val_loss: 0.0089 Epoch 10/10 1875/1875 [==============================] - 4s 2ms/step - loss: 0.0088 - val_loss: 0.0089 <keras.src.callbacks.History at 0x7ff7fd32d7c0>
现在，模型已经训练完成，我们通过对测试集中的图像进行编码和解码来测试该模型。
encoded_imgs = autoencoder.encoder(x_test).numpy()
decoded_imgs = autoencoder.decoder(encoded_imgs).numpy()
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
# display original
ax = plt.subplot(2, n, i + 1)
plt.imshow(x_test[i])
plt.title("original")
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
# display reconstruction
ax = plt.subplot(2, n, i + 1 + n)
plt.imshow(decoded_imgs[i])
plt.title("reconstructed")
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()
第二个示例：图像降噪
经过训练后，自编码器还可以去除图像中的噪点。在以下部分中，您将通过对每个图像应用随机噪声来创建有噪版本的 Fashion MNIST 数据集。随后，您将使用有噪图像作为输入并以原始图像作为目标来训练自编码器。
我们重新导入数据集以忽略之前所做的修改：
(x_train, _), (x_test, _) = fashion_mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train[..., tf.newaxis]
x_test = x_test[..., tf.newaxis]
print(x_train.shape)
(60000, 28, 28, 1)
向图像添加随机噪声：
noise_factor = 0.2
x_train_noisy = x_train + noise_factor * tf.random.normal(shape=x_train.shape)
x_test_noisy = x_test + noise_factor * tf.random.normal(shape=x_test.shape)
x_train_noisy = tf.clip_by_value(x_train_noisy, clip_value_min=0., clip_value_max=1.)
x_test_noisy = tf.clip_by_value(x_test_noisy, clip_value_min=0., clip_value_max=1.)
绘制有噪图像：
n = 10
plt.figure(figsize=(20, 2))
for i in range(n):
ax = plt.subplot(1, n, i + 1)
plt.title("original + noise")
plt.imshow(tf.squeeze(x_test_noisy[i]))
plt.gray()
plt.show()
定义卷积自编码器
在此示例中，您将使用 encoder 中的 Conv2D 层和 decoder 中的 Conv2DTranspose 层来训练卷积自编码器。
class Denoise(Model):
def __init__(self):
super(Denoise, self).__init__()
self.encoder = tf.keras.Sequential([
layers.Input(shape=(28, 28, 1)),
layers.Conv2D(16, (3, 3), activation='relu', padding='same', strides=2),
layers.Conv2D(8, (3, 3), activation='relu', padding='same', strides=2)])
self.decoder = tf.keras.Sequential([
layers.Conv2DTranspose(8, kernel_size=3, strides=2, activation='relu', padding='same'),
layers.Conv2DTranspose(16, kernel_size=3, strides=2, activation='relu', padding='same'),
layers.Conv2D(1, kernel_size=(3, 3), activation='sigmoid', padding='same')])
def call(self, x):
encoded = self.encoder(x)
decoded = self.decoder(encoded)
return decoded
autoencoder = Denoise()
autoencoder.compile(optimizer='adam', loss=losses.MeanSquaredError())
autoencoder.fit(x_train_noisy, x_train,
epochs=10,
shuffle=True,
validation_data=(x_test_noisy, x_test))
Epoch 1/10 1875/1875 [==============================] - 10s 4ms/step - loss: 0.0162 - val_loss: 0.0093 Epoch 2/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0086 - val_loss: 0.0081 Epoch 3/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0076 - val_loss: 0.0073 Epoch 4/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0072 - val_loss: 0.0071 Epoch 5/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0070 - val_loss: 0.0070 Epoch 6/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0069 - val_loss: 0.0069 Epoch 7/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0068 - val_loss: 0.0068 Epoch 8/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0068 - val_loss: 0.0068 Epoch 9/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0067 - val_loss: 0.0067 Epoch 10/10 1875/1875 [==============================] - 7s 4ms/step - loss: 0.0067 - val_loss: 0.0067 <keras.src.callbacks.History at 0x7ff82e75c910>
我们来看一下编码器的摘要。请注意图像是如何从 28x28 像素下采样为 7x7 像素的。
autoencoder.encoder.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 14, 14, 16) 160
conv2d_1 (Conv2D) (None, 7, 7, 8) 1160
=================================================================
Total params: 1320 (5.16 KB)
Trainable params: 1320 (5.16 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
解码器将图像从 7x7 像素上采样为 28x28 像素。
autoencoder.decoder.summary()
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_transpose (Conv2DTr (None, 14, 14, 8) 584
anspose)
conv2d_transpose_1 (Conv2D (None, 28, 28, 16) 1168
Transpose)
conv2d_2 (Conv2D) (None, 28, 28, 1) 145
=================================================================
Total params: 1897 (7.41 KB)
Trainable params: 1897 (7.41 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
绘制由自编码器生成的有噪图像和去噪图像。
encoded_imgs = autoencoder.encoder(x_test_noisy).numpy()
decoded_imgs = autoencoder.decoder(encoded_imgs).numpy()
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
# display original + noise
ax = plt.subplot(2, n, i + 1)
plt.title("original + noise")
plt.imshow(tf.squeeze(x_test_noisy[i]))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
# display reconstruction
bx = plt.subplot(2, n, i + n + 1)
plt.title("reconstructed")
plt.imshow(tf.squeeze(decoded_imgs[i