卷积神经网络与图像分类：从猫狗识别到迁移学习

# 卷积神经网络与图像分类：从猫狗识别到迁移学习 ## 1. 猫狗图像分类的CNN构建 ### 1.1 准备工作 在进行猫狗图像分类之前，我们需要完成一系列的准备工作。首先，导入必要的Python包： ```python import torchvision import torch.nn as nn import torch import torch.nn.functional as F from torchvision import transforms,models,datasets from PIL import Image from torch import optim device = 'cuda' if torch.cuda.is_available() else 'cpu' import cv2, glob, numpy as np, pandas as pd import matplotlib.pyplot as plt %matplotlib inline from glob import glob !pip install torch_summary ``` 接着，从Kaggle下载猫狗数据集，具体步骤如下： 1. 上传Kaggle认证文件： ```python !pip install -q aggle from google.colab import files files.upload() ``` 2. 移动到Kaggle文件夹并复制`kaggle.json`文件： ```python !mkdir -p ~/.kaggle !cp kaggle.json ~/.kaggle/ !ls ~/.kaggle !chmod 600 /root/.kaggle/kaggle.json ``` 3. 下载并解压猫狗数据集： ```python !kaggle datasets download -d tongpython/cat-and-dog !unzip cat-and-dog.zip ``` ### 1.2 数据集路径设置 提供训练集和测试集的文件夹路径： ```python train_data_dir = '/content/training_set/training_set' test_data_dir = '/content/test_set/test_set' ``` ### 1.3 数据加载类的构建 构建一个类来从上述文件夹中获取数据，并为猫狗图像分配标签。同时，对获取的图像进行归一化和通道调整： ```python from torch.utils.data import DataLoader, Dataset class cats_dogs(Dataset): def __init__(self, folder): cats = glob(folder+'/cats/*.jpg') dogs = glob(folder+'/dogs/*.jpg') self.fpaths = cats + dogs from random import shuffle, seed; seed(10); shuffle(self.fpaths) self.targets=[fpath.split('/')[-1].startswith('dog') for fpath in self.fpaths] # dog=1 def __len__(self): return len(self.fpaths) def __getitem__(self, ix): f = self.fpaths[ix] target = self.targets[ix] im = (cv2.imread(f)[:,:,::-1]) im = cv2.resize(im, (224,224)) return torch.tensor(im/255).permute(2,0,1).to(device).float(), torch.tensor([target]).float().to(device) ``` ### 1.4 随机图像检查 检查一个随机图像，确保数据加载和处理正确： ```python data = cats_dogs(train_data_dir) im, label = data[200] plt.imshow(im.permute(1,2,0).cpu()) print(label) ``` ### 1.5 模型、损失函数和优化器的定义 1. 定义卷积层函数： ```python def conv_layer(ni,no,kernel_size,stride=1): return nn.Sequential( nn.Conv2d(ni, no, kernel_size, stride), nn.ReLU(), nn.BatchNorm2d(no), nn.MaxPool2d(2) ) ``` 2. 定义获取模型的函数： ```python def get_model(): model = nn.Sequential( conv_layer(3, 64, 3), conv_layer(64, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), conv_layer(512, 512, 3), nn.Flatten(), nn.Linear(512, 1), nn.Sigmoid(), ).to(device) loss_fn = nn.BCELoss() optimizer=torch.optim.Adam(model.parameters(), lr= 1e-3) return model, loss_fn, optimizer ``` 3. 获取模型、损失函数和优化器，并总结模型结构： ```python from torchsummary import summary model, loss_fn, optimizer = get_model() summary(model, torch.zeros(1,3, 224, 224)); ``` ### 1.6 数据加载器的创建 创建`get_data`函数，用于创建训练集和验证集的数据加载器： ```python def get_data(): train = cats_dogs(train_data_dir) trn_dl = DataLoader(train, batch_size=32, shuffle=True, drop_last = True) val = cats_dogs(test_data_dir) val_dl = DataLoader(val,batch_size=32, shuffle=True, drop_last = True) return trn_dl, val_dl ``` ### 1.7 训练和评估函数的定义 1. 定义批量训练函数： ```python def train_batch(x, y, model, opt, loss_fn): model.train() prediction = model(x) batch_l ```