发现只需要11G显存！只使用一张rtx3060 12G的就搞定了。速度和答案质量还不错，太令人惊喜了！
nvidia-smi -l
Thu Feb 13 10:22:25 2025
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:02:00.0 Off | N/A |
| 31% 59C P2 66W / 170W | 10899MiB / 12288MiB | 51% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2154 G /usr/lib/xorg/Xorg 14MiB |
| 0 N/A N/A 23463 C ...1_avx/ollama_llama_server 10880MiB |
+-----------------------------------------------------------------------------+
7B只需要5GB的显存
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:02:00.0 Off | N/A |
| 30% 58C P2 168W / 170W | 5689MiB / 12288MiB | 84% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2154 G /usr/lib/xorg/Xorg 14MiB |
| 0 N/A N/A 19963 C ...1_avx/ollama_llama_server 5670MiB |
+-----------------------------------------------------------------------------+
搭建步骤：
1. 安装ollma
curl -fsSL https://ollama.com/install.sh | sh
2.下载安装14B模型
ollama pull deepseek-r1:14B
3.安装open-webui
conda create -n open-webui python=3.11
conda activate open-webui
pip install open-webu
4.启动open-webui
open-webui serve