Transformer：Attention is All You Need 代码实现，训练和测试 | PyTorch 深度学习实战

Chatopera 研发团队

于 2025-04-20 11:30:38 发布

阅读量472

点赞数 4

CC 4.0 BY-SA版权

分类专栏：机器学习聊天机器人文章标签：深度学习 transformer pytorch

本文链接：https://blog.csdn.net/samurais/article/details/147363060

聊天机器人同时被 2 个专栏收录

139 篇文章

订阅专栏

机器学习

74 篇文章

订阅专栏

transformer-pytorch-get-started

GitHub Repo: https://github.com/hailiang-wang/transformer-pytorch-get-started

Setup Transformer Project with annotated-transformer by upgrading library, etc.

在这里插入图片描述

Install

pip install -r requirements.txt

*Required torch==2.3.1+cu121 and torchtext==0.18.0.

Run exmaple

001. A First Example

We can begin by trying out a simple copy-task. Given a random set
of input symbols from a small vocabulary, the goal is to generate
back those same symbols.

./scripts/002.run_example_001.sh

在这里插入图片描述

002. A Real Word Example

First, run

./scripts/002.train_example_002.sh

By the first time, it comes to error, UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 37: invalid start byte, then, use this solution https://blog.csdn.net/samurais/article/details/147311770

Further, run the previous script again.

./scripts/002.train_example_002.sh

Later, get result

在这里插入图片描述

Results List - 一些执行历史

https://github.com/hailiang-wang/transformer-pytorch-get-started/issues/1

Machine GPU RTX 2050

2025_04_18_180705_no_multi30k_model_final.zip

2025/04/18 18:07:09 ~ 2025/04/20 04:44:01, Epoch 300

2025/04/20 04:43:33 [INFO] Checking Model Outputs:
2025/04/20 04:43:33 [INFO] 
Example 0 ========

2025/04/20 04:43:36 [INFO] Source Text (Input)        : <s> Mehrere Menschen stehen in einem Raum und essen . </s>
2025/04/20 04:43:36 [INFO] Target Text (Ground Truth) : <s> Several people are standing in a room eating . </s>
2025/04/20 04:43:39 [INFO] Model Output               : <s> Several people are standing in a room eating . </s>
2025/04/20 04:43:39 [INFO] 
Example 1 ========

2025/04/20 04:43:42 [INFO] Source Text (Input)        : <s> Ein junger Mann macht sich bereit , einen Fußball zu schießen . </s>
2025/04/20 04:43:42 [INFO] Target Text (Ground Truth) : <s> A young man gets ready to kick a soccer ball . </s>
2025/04/20 04:43:45 [INFO] Model Output               : <s> A young male getting ready to kick a soccer ball . </s>
2025/04/20 04:43:45 [INFO] 
Example 2 ========

2025/04/20 04:43:47 [INFO] Source Text (Input)        : <s> Ein Mann mit einem großen Hut im Gebüsch . </s>
2025/04/20 04:43:47 [INFO] Target Text (Ground Truth) : <s> A man with a large hat in the bushes . </s>
2025/04/20 04:43:50 [INFO] Model Output               : <s> A man in a large hat is in shrubbery . </s>
2025/04/20 04:43:50 [INFO] 
Example 3 ========

2025/04/20 04:43:53 [INFO] Source Text (Input)        : <s> Ein Mann in einem orangen Gewand fegt im Freien . </s>
2025/04/20 04:43:53 [INFO] Target Text (Ground Truth) : <s> A man in an orange robe sweeping outside . </s>
2025/04/20 04:43:56 [INFO] Model Output               : <s> A man in an orange robe sweeping outdoors . </s>
2025/04/20 04:43:56 [INFO] 
Example 4 ========

2025/04/20 04:43:58 [INFO] Source Text (Input)        : <s> Ein Paar geht einen Gang in einem Laden entlang , der Kunst- und <unk> verkauft . </s>
2025/04/20 04:43:58 [INFO] Target Text (Ground Truth) : <s> A couple walks down an <unk> at a store selling art and <unk> books . </s>
2025/04/20 04:44:01 [INFO] Model Output               : <s> A couple walks down a hallway in a store selling arts and crafts activities . </s>