Ollama如何構建自己的Llama3中文模型模型

2024-05-16碼農

Ollama

Ollama 是一個開源的大型語言模型（LLM）服務工具，它允許使用者在本地機器上執行和部署大型語言模型。Ollama 設計為一個框架，旨在簡化在 Docker 容器中部署和管理大型語言模型的過程，使得這一過程變得簡單快捷。使用者可以透過簡單的命令列操作，快速在本地執行如 Llama 3 這樣的開源大型語言模型。

套用模型

「

註意：推薦下載 GGUF檔格式的模型，可以快速簡潔的匯入 Ollama 中

0X01 下載模型檔

「

下載地址： https://huggingface.co/shenzhi-wang/Llama3-8B-Chinese-Chat-GGUF-8bit/tree/main

下載 Llama3-8B-Chinese-Chat-q8_0-v2_1.gguf 模型檔

0X02 編寫模型檔

「

Modelfile 檔內容如下：

# FROM 指定 GGUF 檔的路徑 FROM D:/AI/Download/Llama3-8B-Chinese-Chat-q8_0-v2_1.gguf

0X03 建立 Ollama 模型

「

使用 ollama create 命令可以根據 Modelfile 建立一個新的模型

ollama create tinywan-Llama3-8B-Chinese -f ./Modelfile transferring model data using existing layer sha256:ea6e8d5cda0fc798898b67f6e728eb2d02877a2599aa3c8290aaa6f681c2bb9e creating new layer sha256:e6e86a160950a92b7b32fcd6fcbc830e38634be6d616ec57452561bfe2a243fb writing manifest success

這個命令會讀取 Modelfile 中的配置，並建立一個名為 tinywan-Llama3-8B-Chinese 的新模型。

「

透過命令 ollama list 檢視模型列表

> ollama list NAME ID SIZE MODIFIED tinywan-Llama3-8B-Chinese:latest adcb30feaee5 16 GB About a minute ago llama3:8b a6990ed6be41 4.7 GB 2 weeks ago

0X04 執行 Ollama 模型

使用 ollama run 命令來驗證 tinywan-Llama3-8B-Chinese 新模型

ollama run tinywan-Llama3-8B-Chinese:latest

模型測試

>>> 寫一首詩吧。"我說。他眨了眨眼，微笑著回答道：「好啊，我試試看。」然後，他閉上眼睛，開始吟唱：在星空之下，月光灑銀輝。風輕輕地舞，樹葉低語。山川河流靜默，夜晚籠罩大地。但就在這寧靜中，我感受到生命的脈動，它如同詩歌般跳躍，穿梭於每一顆心中。

REST API

除了命令列界面， OLlama 還提供了 REST API ，使得您可以透過HTTP請求與模型互動。這對於在Web應用程式中整合ollama尤其有用。

請求

要生成模型的響應，您可以發送一個POST請求到 /api/generate

curl -X POST http://localhost:11434/api/generate -d '{ "model": "llama3:8b", "prompt": "中文回答。你是什麽大模型？", "stream": false }'

響應

返回一個JSON物件流：

{ "model": "llama3:8b", "created_at": "2024-05-12T09:00:01.9513668Z", "response": "I'm LLaMA, a large language model developed by Meta AI that can understand and respond to human input in a conversational manner. I've been trained on a massive dataset of text from the internet and can generate human-like responses to a wide range of topics and questions. My training data includes but is not limited to:\n\n* Web pages: articles, blogs, forums\n* Books: fiction and non-fiction\n* Research papers: academic journals, research articles\n* User-generated content: social media, comments, reviews\n\nI'm able to generate responses that are contextualized to the conversation I'm having with you. This means I can recall previous statements or questions in our conversation and respond accordingly.\n\nMy capabilities include:\n\n* Answering questions on a wide range of topics, from science and history to entertainment and culture\n* Generating text summaries of long pieces of content\n* Translating text from one language to another (in this case, Chinese to English)\n* Responding to natural language input in a conversational manner\n\nI'm constantly learning and improving my responses based on the conversations I have with users like you. So feel free to ask me anything, and I'll do my best to provide helpful and accurate information!", "done": true, "done_reason": "stop", "context": [ 128006, ... 128009 ], "total_duration": 37185731000, "load_duration": 10876300, "prompt_eval_count": 13, "prompt_eval_duration": 1058480000, "eval_count": 248, "eval_duration": 36115711000 }