卷积神经网络在猫狗图像分类及迁移学习中的应用

### 卷积神经网络在猫狗图像分类及迁移学习中的应用 #### 1. 引言 在图像分类任务中，传统神经网络在处理经过平移的图像时表现不佳，而卷积神经网络（CNNs）通过滤波器、步长和池化等机制，有效解决了这一问题。本文将详细介绍如何构建一个CNN模型来对猫狗图像进行分类，并探讨不同数量的训练数据对测试数据准确性的影响，最后引入迁移学习技术以应对训练数据不足的情况。 #### 2. 构建CNN进行猫狗图像分类 ##### 2.1 准备工作 - **导入必要的包**： ```python import torchvision import torch.nn as nn import torch import torch.nn.functional as F from torchvision import transforms,models,datasets from PIL import Image from torch import optim device = 'cuda' if torch.cuda.is_available() else 'cpu' import cv2, glob, numpy as np, pandas as pd import matplotlib.pyplot as plt %matplotlib inline from glob import glob !pip install torch_summary ``` - **下载数据集**： 1. 上传Kaggle认证文件： ```python !pip install -q aggle from google.colab import files files.upload() ``` 2. 移动到Kaggle文件夹并复制`kaggle.json`文件： ```python !mkdir -p ~/.kaggle !cp kaggle.json ~/.kaggle/ !ls ~/.kaggle !chmod 600 /root/.kaggle/kaggle.json ``` 3. 下载并解压猫狗数据集： ```python !kaggle datasets download -d tongpython/cat-and-dog !unzip cat-and-dog.zip ``` - **指定训练和测试数据集文件夹**： ```python train_data_dir = '/content/training_set/training_set' test_data_dir = '/content/test_set/test_set' ``` ##### 2.2 构建数据加载类 ```python from torch.utils.data import DataLoader, Dataset class cats_dogs(Dataset): def __init__(self, folder): cats = glob(folder+'/cats/*.jpg') dogs = glob(folder+'/dogs/*.jpg') self.fpaths = cats + dogs from random import shuffle, seed; seed(10); shuffle(self.fpaths) self.targets=[fpath.split('/')[-1].startswith('dog') for fpath in self.fpaths] # dog=1 def __len__(self): return len(self.fpaths) def __getitem__(self, ix): f = self.fpaths[ix] target = self.targets[ix] im = (cv2.imread(f)[:,:,::-1]) im = cv2.resize(im, (224,224)) return torch.tensor(im/255).permute(2,0,1).to(device).float(),\ torch.tensor([target]).float().to(device) ``` ##### 2.3 检查随机图像 ```python data = cats_dogs(train_data_dir) im, label = data[200] plt.imshow(im.permute(1,2,0).cpu()) print(label) ``` ##### 2.4 定义模型、损失函数和优化器 - **定义卷积层函数**： ```python def conv_layer(ni,no,kernel_size,stride=1): return nn.Sequential( nn.Conv2d(ni, no, kernel_size, stride), nn.ReLU(), nn.BatchNorm2d(no), nn.MaxPool2d(2) ) ``` - **定义获取模型函数**： ```python def get_model(): model = nn.Sequential( conv_layer(3, 64, 3), conv_layer(64, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), nn.Flatten(), nn.Linear(512, 1), nn.Sigmoid(), ).to(device) loss_fn = nn.BCELoss() optimizer=torch.optim.Adam(model.parameters(), lr= 1e-3) return model, loss_fn, optimizer ``` - **获取模型并总结模型架构**： ```python from torchsummary import summary model, loss_fn, optimizer = get_model() summary(model, torch.zeros(1,3, 224, 224)); ``` ##### 2.5 创建数据加载函数 ```python def get_data(): train = cats_dogs(train_data_dir) trn_dl = DataLoader(train, batch_size=32, shuffle=True, drop_last = True) val = cats_dogs(test_data_dir) val_dl = DataLoader(val,batch_size=32, shuffle=True, drop_last = True) return trn_dl, val_dl ``` ##### 2.6 定义训练和评估函数 - **训练批量数据函数**： ```python def train_batch(x, y, model, opt, loss_fn): model.train() prediction = model(x) ba ```