[玩装AIGC]Llama 2官方模型下载 、转换与C++模型推理

一、前言

全程可用cpu跑，没有gpu的也没关系，跑起来爽一把也不错，没有显卡你也可以学习的，等有了一块gpu，你就可以起飞了，学起来吧！！！

二、账号注册

llama

In order to download the model weights and tokenizer, please visit the Meta AI website and accept our License.

三、模型下载

git clone https://github.com/facebookresearch/llama.git

运行./download.sh，根据提示，把URL贴到控制台上，然后选择所想要下载的模型
我这边对代码进行了修改，就是把PRESIGNED_URL，跟MODEL_SIZE写死在里面，这样不需要每次运行的时候都输入那一大串URL key

#!/bin/bash

# Copyright (c) Meta Platforms, Inc. and affiliates.
# This software may be used and distributed according to the terms of the Llama 2 Community License Agreement.

# read -p "Enter the URL from email: " PRESIGNED_URL

# read -p "Enter the list of models to download without spaces (7B,13B,70B,7B-chat,13B-chat,70B-chat), or press Enter for all: " MODEL_SIZE

PRESIGNED_URL="your URL key"
MODEL_SIZE="7B"    

TARGET_FOLDER="."             # where all files should end up
mkdir -p ${TARGET_FOLDER}

if [[ $MODEL_SIZE == "" ]]; then
    MODEL_SIZE="7B,13B,70B,7B-chat,13B-chat,70B-chat"
fi

echo "Downloading LICENSE and Acceptable Usage Policy"
wget ${PRESIGNED_URL/'*'/"LICENSE"} -O ${TARGET_FOLDER}"/LICENSE"
wget ${PRESIGNED_URL/'*'/"USE_POLICY.md"} -O ${TARGET_FOLDER}"/USE_POLICY.md"

echo "Downloading tokenizer"
wget ${PRESIGNED_URL/'*'/"tokenizer.model"} -O ${TARGET_FOLDER}"/tokenizer.model"
wget ${PRESIGNED_URL/'*'/"tokenizer_checklist.chk"} -O ${TARGET_FOLDER}"/tokenizer_checklist.chk"
(cd ${TARGET_FOLDER} && md5sum -c tokenizer_checklist.chk)

for m in ${MODEL_SIZE//,/ }
do
    if [[ $m == "7B" ]]; then
        SHARD=0
        MODEL_PATH="llama-2-7b"
    elif [[ $m == "7B-chat" ]]; then
        SHARD=0
        MODEL_PATH="llama-2-7b-chat"
    elif [[ $m == "13B" ]]; then
        SHARD=1
        MODEL_PATH="llama-2-13b"
    elif [[ $m == "13B-chat" ]]; then
        SHARD=1
        MODEL_PATH="llama-2-13b-chat"
    elif [[ $m == "70B" ]]; then
        SHARD=7
        MODEL_PATH="llama-2-70b"
    elif [[ $m == "70B-chat" ]]; then
        SHARD=7
        MODEL_PATH="llama-2-70b-chat"
    fi

    echo "Downloading ${MODEL_PATH}"
    mkdir -p ${TARGET_FOLDER}"/${MODEL_PATH}"

    for s in $(seq -f "0%g" 0 ${SHARD})
    do
        wget ${PRESIGNED_URL/'*'/"${MODEL_PATH}/consolidated.${s}.pth"} -O ${TARGET_FOLDER}"/${MODEL_PATH}/consolidated.${s}.pth"
    done

    wget ${PRESIGNED_URL/'*'/"${MODEL_PATH}/params.json"} -O ${TARGET_FOLDER}"/${MODEL_PATH}/params.json"
    wget ${PRESIGNED_URL/'*'/"${MODEL_PATH}/checklist.chk"} -O ${TARGET_FOLDER}"/${MODEL_PATH}/checklist.chk"
    echo "Checking checksums"
    (cd ${TARGET_FOLDER}"/${MODEL_PATH}" && md5sum -c checklist.chk)
done

下载后有三个文件，llama-2-7b这个路径要记住，后面模型转换要用到
![在这里插入图片描述](https://img-blog.csdnimg.cn/747ffab6fb49455aacdef8c582b0fd59.png

四、模型转换

推理代码主要采用大佬已经写好的llama2.c，关于llama2.c的使用，还可参考：[玩转AIGC]如何训练LLaMA2（模型训练、推理、代码讲解，并附可直接运行的kaggle连接）

git clone https://github.com/karpathy/llama2.c.git
cd llama2.c

python export_meta_llama_bin.py path/to/llama/model/llama-2-7b llama2_7b.bin

五、C++推理

先把run.c编译为可运行文件(编译过的话可以用不再运行编译了，关于run.c的代码解释可参考：[玩转AIGC]如何训练LLaMA2（模型训练、推理、代码讲解，并附可直接运行的kaggle连接）)

make run

运行：

./run llama2_7b.bin

跑的是故事生成（写作）模型，用CPU跑的，所以跑了很久。不知道跑了多久，睡前让它跑，早上醒来后就跑出来了，以下是跑出来的效果：