T5 模型 VS CLIP 模型

子燕若水

已于 2023-02-04 16:12:07 修改

阅读量1.9k

点赞数 1

CC 4.0 BY-SA版权

分类专栏：深度学习算法文章标签：自然语言处理人工智能深度学习

于 2023-02-02 13:45:32 首次发布

本文链接：https://blog.csdn.net/u010087338/article/details/128848883

深度学习算法专栏收录该内容

159 篇文章

订阅专栏

CLIP模型包含文本和视觉Transformer，通过预训练使相似的图像和文本在向量空间中接近。它使用大量从互联网抓取的400M图像-文本对进行训练，区别于传统分类模型。同时提及了T5模型和其训练方法，如Transformer结构、破坏策略等。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

CLIP consists of two models, as discussed in more depth in the previous chapter. The version of CLIP we use here consists of a text transformer for encoding text embeddings and a vision transformer (ViT) for encoding image embeddings.

Both CLIP models are optimized during pretraining to align similar text and images in vector space. It does this by taking image-text pairs and pushing their output vectors nearer in vector space while separating the vectors of non-pairs.

It distinguishes itself from typical classification models for several reasons. First, OpenAI trained it on a huge dataset of 400M text-image pairs that were scraped from across the internet.

T5 模型：NLP Text-to-Text 预训练模型超大规模探索 - Andy Yang的文章 - 知乎 https://zhuanlan.zhihu.com/p/88438851

T5 模型，还有它的训练方法。