【NLP】文本分类TorchText实战-AG_NEWS 新闻主题分类任务（PyTorch版）

AG's News Topic Classification Dataset Version 3, Updated 09/09/2015 ORIGIN AG is a collection of more than 1 million news articles. News articles have been gathered from more than 2000 news sources by ComeToMyHead in more than 1 year of activity. ComeToMyHead is an academic news search engine which has been running since July, 2004. The dataset is provided by the academic comunity for research purposes in data mining (clustering, classification, etc), information retrieval (ranking, search, etc), xml, data compression, data streaming, and any other non-commercial activity. For more information, please refer to the link http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html . The AG's news topic classification dataset is constructed by Xiang Zhang ([email protected]) from the dataset above. It is used as a text classification benchmark in the following paper: Xiang Zhang, Junbo Zhao, Yann LeCun. Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28 (NIPS 2015). DESCRIPTION The AG's news topic classification dataset is constructed by choosing 4 largest classes from the original corpus. Each class contains 30,000 training samples and 1,900 testing samples. The total number of training samples is 120,000 and testing 7,600. The file classes.txt contains a list of classes corresponding to each label. The files train.csv and test.csv contain all the training samples as comma-sparated values. There are 3 columns in them, corresponding to class index (1 to 4), title and description. The title and description are escaped using double quotes ("), and any internal double quote is escaped by 2 double quotes (""). New lines are escaped by a backslash followed with an "n" character, that is "\n".

本篇文章将聚焦于如何利用PyTorch处理文本分类任务，特别是针对AG_News数据集的测试过程。AG_News是一个广泛使用的新闻主题分类数据集，包含四个类别：世界、体育、商业和科技，总计约12万条新闻标题。 首先，让...

文本分类的标准代码，Pytorch实现 数据集Dataset - IMDB - SST - Trec ### 模型 - FastText - BasicCNN (KimCNN,MultiLayerCNN, Multi-perspective CNN) - InceptionCNN - LSTM (BILSTM, StackLSTM) - LSTM with Attention (Self Attention / Quantum Attention) - Hybrids between CNN and RNN (RCNN, C-LSTM) - Transformer - Attention is all you need - ConS2S - Capsule - Quantum-inspired NN

文本分类是NLP领域的较为容易的入门问题，本文记录我自己在做文本分类任务以及复现相关论文时的基本流程，绝大部分操作都使用了torch和torchtext两个库。 1. 文本数据预处理 首先数据存储在三个csv文件中，分别是train.csv，valid.csv，test.csv，第一列存储的是文本数据，例如情感分类问题经常是用户的评论review，例如imdb或者amazon数据集。第二列是情感极性polarity，N分类问题的话就有N个值，假设值得范围是0~N-1。 下面是很常见的文本预处理流程，英文文本的话不需要分词，直接按空格split就行了，这里只会主要说说第4点。 1、去除非文本部分

This repo contains a PyTorch implementation of a pretrained BERT model for multi-label text classification.

文本分类数据集（共13个20ng,mr,R8,R52,ohsumed,wiki,agnews等） 数据量很多，适合进行深度学习实验