【wails】（7）：研究发现一个新的项目llama.go，使用纯golang写的代码，下载模型文件，然后可以本地执行了，可以执行，就是模型文件26G，运行速度慢 0.37 token/s

1，视频简介

https://www.bilibili.com/video/BV1Mw4m1f7qQ/

2，在autodl 上申请服务器

然后在autodl上申请服务器，选择 30G内存的服务器：

然后就可以进行 golang 环境的安装了：

cd /usr/local/
wget https://golang.google.cn/dl/go1.22.0.linux-amd64.tar.gz
export GOROOT=/usr/local/go
export PATH="$PATH:$GOPATH/bin:$GOROOT/bin"

go env -w GOPROXY=https://goproxy.cn,direct

然后克隆项目进行编译：

# 克隆项目：
git clone https://github.com/gotzmann/llama.go.git

# 然后进入 
cd llama.go/

# 执行
make

# 然后在 builds 目录里面就有了三个文件：
ls builds
llama-go-v1.4.0.exe  llama-go-v1.4.0-linux  llama-go-v1.4.0-macos

wget https://nogpu.com/llama-7b-fp32.bin

3，然后执行测试代码

./llama --model /root/autodl-tmp/llama-7b-fp32.bin --prompt hello

  /▒▒       /▒▒         /▒▒▒/▒▒▒   /▒▒/▒▒▒▒/▒▒   /▒▒▒/▒▒▒      /▒▒▒▒/▒▒   /▒▒▒/▒▒▒    
  /▒▒▒      /▒▒▒      /▒▒▒/ /▒▒▒ /▒▒▒/▒▒▒▒/▒▒▒ /▒▒▒/ /▒▒▒     /▒▒▒▒ //   /▒▒▒▒//▒▒▒  
  /▒▒▒▒/▒▒  /▒▒▒▒/▒▒  /▒▒▒▒/▒▒▒▒ /▒▒▒/▒▒▒▒/▒▒▒ /▒▒▒▒/▒▒▒▒ /▒▒ /▒▒▒▒/▒▒▒▒ /▒▒▒ /▒▒▒▒ 
  /▒▒▒▒/▒▒▒ /▒▒▒▒/▒▒▒ /▒▒▒ /▒▒▒▒ /▒▒▒//▒▒ /▒▒▒ /▒▒▒ /▒▒▒▒ /▒▒▒//▒▒▒▒/▒▒  //▒▒▒/▒▒▒
  //// ///  //// ///  ///  ////  ///  //  ///  ///  ////  ///  //// //    /// ///

   ▒▒▒▒ [ LLaMA.go v1.4.0 ] [ LLaMA GPT in pure Golang - based on LLaMA C++ ] ▒▒▒▒

[ INIT ] Loading model, please wait .............................

[ PROMPT ] hello
[ OUTPUT ]
 1. 2. 3. 4. 5. 6. 

不知道为啥返回数字。

测试这个问题就正常了：

./llama --model /root/autodl-tmp/llama-7b-fp32.bin \
 --pods 6 --threads 6 --prompt "Why Golang is so popular?"


[ PROMPT ] Why Golang is so popular?
[ OUTPUT ]
Why Golang is so popular?
I have been using Go for a while now, and I am very impressed by the language. It has many features that make it easy to use and write code in. The syntax is simple, concise, and expressive. It also has some great tools like gofmt, goimports, and go vet.
Go is an open source programming language developed at Google. Its design emphasizes simplicity, safety, and productivity. It has a clean, modern C-like syntax with static typing, garbage collection, interfaces, and modules.
The Go Programming Language is free software released under the BSD license. You can download it from https://golang.org/dl/. The following are stocks or mutual funds tagged as medical device manufacturers. When researching companies, it is important to know what companies have a similar focus. The stocks below have been categorized as "medical device manufacturers" by either user submission or electronic means. The 2019 Nissan Rogue. Smooth gearshifts are achieved thanks to the efficient 4 cylinder engine, and all wheel drive keeps this model firmly attached to the road surface. Well tuned suspension and stability control deliver a spirited, yet composed, ride and drive A wealth of standard features means that you no longer have to sacrifice. Such as remote keyless entry, delay-off headlights, front and rear reading lights, a trip computer, power door mirrors, cruise control, an overhead console, and much more. Audio features include an AM/FM radio, steering wheel mounted audio controls, and 6 well positioned speakers. With side curtain airbags supplementing the rest of the safety network, you can be assured that you and your passengers will experience top-tier protection. We pride ourselves on providing excellent customer service. Stop by our dealership or give us a call for more information. The new year brings new opportunities to learn and grow. In 2018, we’ll continue to provide you with the best content available on the topics of leadership, teamwork, and organizational excellence. Our goal is to help you become a better leader and create a high performing organization.
We’re excited about the upcoming year and look forward to sharing our thoughts and ideas with you!
What are your goals for 

=== EVAL TIME | ms ===

16181 | 1744 | 1779 | 1789 | 1853 | 1918 | 2125 | 2158 | 1971 | 1866 | 1889 | 2024 | 2500 | 2027 | 1819 | 1884 | 2384 | 2238 | 2623 | 2446 | 2596 | 2022 | 1755 | 1898 | 1910 | 1870 | 1981 | 1879 | 1961 | 2012 | 2011 | 1977 | 1964 | 1960 | 1932 | 1852 | 1880 | 1750 | 1760 | 1942 | 1954 | 1839 | 1940 | 1916 | 1848 | 2012 | 2002 | 1918 | 1895 | 1867 | 1924 | 1990 | 1983 | 1937 | 1973 | 1932 | 1975 | 1979 | 2070 | 1917 | 1891 | 1885 | 1761 | 1822 | 1932 | 1819 | 1947 | 1890 | 1935 | 1942 | 1864 | 1852 | 1898 | 1927 | 1962 | 1976 | 2036 | 1993 | 2015 | 1968 | 1966 | 2013 | 1904 | 1998 | 1984 | 2010 | 1941 | 2113 | 2045 | 2094 | 1953 | 1936 | 1955 | 1934 | 2001 | 1960 | 2132 | 2018 | 2036 | 2104 | 2106 | 1981 | 1976 | 1951 | 2023 | 1970 | 2100 | 2056 | 2187 | 2120 | 2151 | 2027 | 2062 | 2095 | 2117 | 2059 | 2031 | 2103 | 2132 | 2052 | 1994 | 2038 | 1956 | 1941 | 1940 | 1970 | 2075 | 2138 | 1993 | 2063 | 2085 | 2034 | 2078 | 2029 | 2056 | 2131 | 2046 | 2129 | 2186 | 2155 | 2137 | 2058 | 2142 | 2194 | 2180 | 2131 | 2150 | 2103 | 2139 | 2123 | 2157 | 2146 | 2158 | 2067 | 2194 | 2057 | 2055 | 2177 | 2012 | 2059 | 2062 | 2090 | 2071 | 2078 | 2125 | 2191 | 2153 | 2230 | 2327 | 2170 | 2211 | 2175 | 2212 | 2206 | 2368 | 2270 | 2219 | 2350 | 2307 | 2400 | 2414 | 2209 | 2404 | 2243 | 2319 | 2433 | 2300 | 2215 | 2280 | 2281 | 2165 | 2350 | 2219 | 2201 | 2205 | 2194 | 2226 | 2459 | 2243 | 2254 | 2261 | 2400 | 2261 | 2401 | 2277 | 2239 | 2391 | 2315 | 2320 | 2379 | 2422 | 2469 | 2251 | 2541 | 2461 | 2429 | 2433 | 2516 | 2582 | 2393 | 2308 | 2444 | 2306 | 2493 | 2459 | 2424 | 2450 | 2460 | 2451 | 2472 | 2389 | 2444 | 2431 | 2382 | 2695 | 2398 | 2367 | 2390 | 2397 | 2344 | 2317 | 2386 | 2440 | 2570 | 2621 | 2484 | 2540 | 2470 | 2463 | 2523 | 2511 | 2462 | 2539 | 2423 | 2695 | 2591 | 2487 | 2490 | 2470 | 2588 | 2575 | 2692 | 2592 | 2670 | 2587 | 2567 | 2547 | 2572 | 2553 | 2564 | 2758 | 2532 | 2827 | 2452 | 2406 | 2590 | 2499 | 2514 | 2518 | 2529 | 2549 | 2628 | 2632 | 2703 | 2608 | 2584 | 2595 | 2743 | 2808 | 2629 | 2521 | 2722 | 2650 | 2763 | 2825 | 2815 | 2701 | 2666 | 2649 | 2619 | 2585 | 2803 | 2712 | 2577 | 2810 | 2598 | 2769 | 2697 | 2799 | 2538 | 2676 | 2607 | 2539 | 2638 | 2696 | 2585 | 2621 | 2561 | 2677 | 2673 | 2630 | 2649 | 2632 | 2699 | 2764 | 2702 | 2719 | 2637 | 2700 | 2714 | 2655 | 2631 | 2784 | 2706 | 2759 | 2854 | 2863 | 2830 | 2756 | 2687 | 2789 | 2798 | 2739 | 2778 | 2777 | 2793 | 2753 | 2750 | 2820 | 2767 | 2831 | 2791 | 2685 | 2793 | 2830 | 2802 | 2846 | 2704 | 2765 | 2787 | 2764 | 2868 | 2769 | 2838 | 2806 | 2837 | 2805 | 2848 | 2798 | 2831 | 2787 | 2832 | 2928 | 2861 | 2781 | 2880 | 2874 | 2886 | 2722 | 2886 | 2782 | 2909 | 2843 | 2881 | 2878 | 2768 | 2911 | 2779 | 2939 | 2913 | 2921 | 2913 | 2886 | 2992 | 2924 | 2981 | 2944 | 2963 | 2901 | 2896 | 2901 | 2951 | 2991 | 3104 | 2956 | 2909 | 2948 | 2934 | 2891 | 2938 | 2894 | 3007 | 2938 | 2998 | 2990 | 2953 | 3021 | 2944 | 2955 | 3094 | 2966 | 3052 | 3038 | 3064 | 2998 | 3126 | 3090 | 2981 | 3016 | 3012 | 3057 | 3085 | 3163 | 3120 | 3016 | 3175 | 3004 | 3126 | 3057 | 3030 | 3119 | 3119 | 3222 | 3116 | 3192 | 3048 | 3214 | 3122 | 3120 | 3025 | 3150 | 3223 | 3283 | 3134 | 3181 | 3265 | 3248 | 3121 | 3173 | 3159 | 3217 | 3182 | 3241 | 3224 | 3178 | 3212 | 3131 | 3092 | 3099 | 3164 | 3211 | 3157 | 3239 | 3220 | 3300 | 3257 | 3292 | 3159 | 3200 | 3161 | 3203 | 3267 | 3247 | 3207 | 3257 | 3232 | 3144 | 3206 | 3260 | 3191 | 3250 | 3209 | 3088 | 3285 | 3134 | 3288 | 3265 | 3232 | 3317 | 3231 | 3288 | 3324 | 3254 | 3255 | 3235 | 3182 | 3312 | 3325 | 3289 | 3434 | 3266 | 3252 | 

=== SAMPLING TIME | ms ===

147 | 165 | 150 | 153 | 149 | 154 | 181 | 183 | 178 | 119 | 146 | 186 | 202 | 171 | 178 | 176 | 158 | 179 | 182 | 181 | 127 | 181 | 133 | 157 | 156 | 134 | 178 | 195 | 178 | 180 | 179 | 179 | 183 | 177 | 135 | 178 | 178 | 194 | 178 | 184 | 142 | 177 | 195 | 165 | 117 | 177 | 182 | 155 | 116 | 178 | 180 | 171 | 178 | 159 | 155 | 158 | 180 | 176 | 158 | 179 | 185 | 234 | 152 | 144 | 116 | 178 | 177 | 175 | 123 | 179 | 179 | 182 | 179 | 177 | 181 | 165 | 159 | 180 | 186 | 180 | 153 | 152 | 186 | 154 | 117 | 138 | 136 | 188 | 155 | 181 | 189 | 189 | 179 | 183 | 180 | 179 | 178 | 186 | 179 | 152 | 163 | 122 | 178 | 122 | 178 | 178 | 177 | 152 | 150 | 180 | 158 | 187 | 155 | 211 | 119 | 179 | 184 | 157 | 166 | 135 | 180 | 164 | 186 | 181 | 160 | 157 | 126 | 158 | 180 | 174 | 145 | 179 | 178 | 125 | 180 | 184 | 182 | 180 | 185 | 179 | 192 | 185 | 181 | 161 | 184 | 118 | 152 | 151 | 184 | 179 | 143 | 143 | 179 | 178 | 124 | 215 | 178 | 151 | 178 | 177 | 178 | 177 | 127 | 170 | 120 | 181 | 180 | 179 | 179 | 179 | 146 | 180 | 178 | 152 | 179 | 194 | 178 | 177 | 179 | 178 | 156 | 216 | 135 | 173 | 193 | 130 | 164 | 163 | 178 | 157 | 155 | 183 | 117 | 121 | 179 | 185 | 179 | 119 | 178 | 179 | 178 | 164 | 195 | 149 | 179 | 197 | 218 | 116 | 182 | 181 | 183 | 204 | 179 | 148 | 151 | 182 | 180 | 179 | 180 | 179 | 175 | 180 | 178 | 151 | 181 | 152 | 179 | 179 | 218 | 177 | 180 | 180 | 189 | 183 | 178 | 191 | 181 | 165 | 181 | 184 | 192 | 179 | 179 | 178 | 180 | 181 | 179 | 179 | 186 | 139 | 179 | 208 | 178 | 183 | 178 | 180 | 123 | 160 | 179 | 176 | 183 | 149 | 135 | 182 | 179 | 159 | 181 | 185 | 180 | 153 | 180 | 162 | 158 | 214 | 177 | 190 | 222 | 182 | 191 | 148 | 179 | 155 | 180 | 180 | 180 | 169 | 165 | 183 | 218 | 152 | 179 | 179 | 179 | 180 | 207 | 148 | 180 | 185 | 182 | 153 | 180 | 167 | 178 | 184 | 182 | 185 | 150 | 180 | 186 | 179 | 179 | 179 | 181 | 157 | 179 | 180 | 151 | 186 | 153 | 179 | 178 | 148 | 151 | 180 | 143 | 182 | 180 | 180 | 179 | 165 | 180 | 186 | 208 | 163 | 182 | 204 | 178 | 180 | 156 | 180 | 179 | 137 | 178 | 207 | 149 | 162 | 150 | 148 | 183 | 178 | 182 | 179 | 201 | 180 | 180 | 181 | 152 | 180 | 187 | 192 | 180 | 162 | 144 | 181 | 188 | 179 | 153 | 180 | 172 | 180 | 182 | 178 | 179 | 149 | 180 | 181 | 226 | 180 | 182 | 181 | 195 | 149 | 180 | 177 | 152 | 180 | 172 | 180 | 150 | 227 | 181 | 180 | 194 | 180 | 227 | 183 | 177 | 184 | 181 | 185 | 149 | 159 | 180 | 181 | 149 | 177 | 180 | 180 | 181 | 180 | 195 | 151 | 179 | 202 | 214 | 180 | 151 | 180 | 178 | 154 | 201 | 151 | 143 | 181 | 182 | 193 | 179 | 148 | 180 | 180 | 149 | 150 | 215 | 150 | 182 | 180 | 207 | 181 | 180 | 178 | 180 | 180 | 184 | 180 | 184 | 151 | 149 | 180 | 195 | 185 | 181 | 182 | 181 | 149 | 180 | 180 | 211 | 179 | 180 | 218 | 151 | 192 | 183 | 179 | 164 | 180 | 184 | 180 | 182 | 180 | 149 | 181 | 154 | 185 | 180 | 184 | 181 | 152 | 227 | 182 | 178 | 180 | 220 | 169 | 218 | 150 | 155 | 183 | 229 | 151 | 190 | 180 | 196 | 181 | 150 | 185 | 163 | 150 | 185 | 182 | 181 | 204 | 198 | 150 | 152 | 200 | 181 | 150 | 152 | 227 | 151 | 180 | 

=== FULL TIME | ms ===

0 | 16329 | 1910 | 1930 | 1942 | 2002 | 2073 | 2307 | 2342 | 2150 | 1986 | 2036 | 2210 | 2702 | 2199 | 1997 | 2060 | 2542 | 2418 | 2806 | 2627 | 2724 | 2204 | 1888 | 2056 | 2067 | 2004 | 2159 | 2074 | 2140 | 2193 | 2190 | 2156 | 2148 | 2138 | 2068 | 2031 | 2059 | 1945 | 1939 | 2126 | 2097 | 2017 | 2135 | 2081 | 1965 | 2189 | 2184 | 2074 | 2012 | 2045 | 2104 | 2162 | 2162 | 2097 | 2129 | 2090 | 2155 | 2156 | 2229 | 2096 | 2076 | 2119 | 1913 | 1967 | 2049 | 1997 | 2125 | 2065 | 2059 | 2121 | 2044 | 2035 | 2077 | 2105 | 2143 | 2142 | 2195 | 2174 | 2201 | 2148 | 2120 | 2165 | 2090 | 2152 | 2101 | 2149 | 2078 | 2301 | 2200 | 2275 | 2143 | 2125 | 2135 | 2118 | 2181 | 2139 | 2311 | 2204 | 2215 | 2256 | 2269 | 2104 | 2154 | 2073 | 2201 | 2148 | 2278 | 2209 | 2338 | 2300 | 2310 | 2215 | 2218 | 2307 | 2237 | 2238 | 2215 | 2260 | 2299 | 2187 | 2175 | 2203 | 2142 | 2122 | 2101 | 2128 | 2202 | 2297 | 2174 | 2237 | 2231 | 2214 | 2256 | 2155 | 2237 | 2315 | 2229 | 2309 | 2371 | 2334 | 2330 | 2244 | 2324 | 2356 | 2364 | 2249 | 2302 | 2255 | 2323 | 2303 | 2301 | 2289 | 2337 | 2246 | 2319 | 2273 | 2234 | 2329 | 2191 | 2237 | 2240 | 2268 | 2199 | 2248 | 2246 | 2372 | 2333 | 2410 | 2507 | 2350 | 2358 | 2355 | 2390 | 2358 | 2547 | 2465 | 2398 | 2527 | 2487 | 2578 | 2571 | 2425 | 2539 | 2417 | 2512 | 2563 | 2464 | 2378 | 2458 | 2439 | 2320 | 2534 | 2336 | 2322 | 2384 | 2380 | 2406 | 2578 | 2422 | 2433 | 2440 | 2564 | 2456 | 2550 | 2456 | 2437 | 2609 | 2432 | 2503 | 2560 | 2606 | 2674 | 2430 | 2690 | 2613 | 2611 | 2613 | 2695 | 2762 | 2572 | 2484 | 2625 | 2485 | 2645 | 2640 | 2576 | 2629 | 2640 | 2669 | 2649 | 2569 | 2624 | 2621 | 2566 | 2873 | 2589 | 2549 | 2556 | 2578 | 2528 | 2509 | 2566 | 2620 | 2749 | 2802 | 2666 | 2719 | 2650 | 2649 | 2662 | 2690 | 2670 | 2718 | 2606 | 2874 | 2772 | 2611 | 2651 | 2649 | 2764 | 2759 | 2841 | 2727 | 2852 | 2767 | 2726 | 2728 | 2758 | 2733 | 2717 | 2939 | 2694 | 2985 | 2667 | 2584 | 2780 | 2722 | 2696 | 2710 | 2678 | 2728 | 2784 | 2812 | 2884 | 2788 | 2753 | 2761 | 2927 | 3026 | 2781 | 2701 | 2902 | 2829 | 2943 | 3033 | 2964 | 2882 | 2852 | 2831 | 2773 | 2765 | 2970 | 2891 | 2762 | 2992 | 2783 | 2920 | 2877 | 2986 | 2718 | 2856 | 2787 | 2721 | 2795 | 2875 | 2766 | 2772 | 2748 | 2830 | 2852 | 2808 | 2798 | 2783 | 2879 | 2908 | 2885 | 2900 | 2818 | 2879 | 2880 | 2835 | 2817 | 2993 | 2870 | 2942 | 3059 | 3042 | 3011 | 2913 | 2867 | 2968 | 2936 | 2918 | 2985 | 2927 | 2956 | 2904 | 2899 | 3003 | 2946 | 3013 | 2970 | 2886 | 2973 | 3010 | 2984 | 2999 | 2884 | 2952 | 2979 | 2944 | 3030 | 2914 | 3019 | 2994 | 3017 | 2959 | 3028 | 2970 | 3011 | 2969 | 3011 | 3107 | 3010 | 2961 | 3061 | 3100 | 3067 | 2905 | 3067 | 2978 | 3058 | 3024 | 3058 | 3031 | 2948 | 3084 | 2960 | 3090 | 3140 | 3103 | 3094 | 3081 | 3172 | 3151 | 3164 | 3122 | 3148 | 3083 | 3081 | 3051 | 3110 | 3172 | 3285 | 3105 | 3086 | 3129 | 3115 | 3073 | 3119 | 3090 | 3158 | 3118 | 3200 | 3204 | 3134 | 3173 | 3125 | 3134 | 3249 | 3167 | 3203 | 3181 | 3246 | 3181 | 3319 | 3269 | 3130 | 3196 | 3193 | 3206 | 3236 | 3379 | 3270 | 3199 | 3356 | 3212 | 3308 | 3237 | 3209 | 3299 | 3299 | 3407 | 3296 | 3377 | 3200 | 3364 | 3302 | 3315 | 3211 | 3332 | 3405 | 3464 | 3284 | 3362 | 3446 | 3460 | 3301 | 3353 | 3377 | 3369 | 3375 | 3424 | 3404 | 3343 | 3392 | 3315 | 3272 | 3281 | 3345 | 3361 | 3338 | 3393 | 3406 | 3480 | 3441 | 3473 | 3311 | 3428 | 3343 | 3381 | 3447 | 3467 | 3376 | 3475 | 3382 | 3299 | 3389 | 3489 | 3343 | 3440 | 3389 | 3284 | 3466 | 3284 | 3473 | 3428 | 3383 | 3503 | 3413 | 3469 | 3528 | 3452 | 3405 | 3387 | 3382 | 3494 | 3475 | 3442 | 3661 | 3418 | 3432 | 

[ HALT ] Time per token: 2720 ms | Tokens per second: 0.37

内存使用 29G，cpu已经 700% 了。

如果没有报错，说明运行成功，但是速度超级慢，第一就是模型比较大。
go-chatglm.cpp 项目使用的 gguf 文件才 3G，这个就已经是 26G

-rw-r–r-- 1 root root 26G Apr 27 2023 llama-7b-fp32.bin

4，总结

确实是能运行，目前看不支持最新的 GGUF 文件格式。
同时是非量化版本 f32 的，造成模型比较慢。而且这个是 llama的第一代的产品。
要是有二代的模型且 int4 版本的就好了。