本地部署AI大模型 —— Ollama文档中文翻译

写在前面

来自Ollama GitHub项目的README.md 文档。文档中涉及的其它文档未翻译，但是对于本地部署大模型而言足够了。

Ollama

开始使用大模型。

macOS

Download

Windows 预览版

Download

Linux

curl -fsSL https://ollama.com/install.sh | sh

手动安装说明

Docker

官方 Ollama Docker 镜像

ollama/ollama

已在 Docker Hub 上可用.

库资源

• ollama-python
• ollama-js

快速启动

使用 Llama 3 本地大模型:

ollama run llama3

模型库

查询 Ollama 支持的可用大模型列表 ollama.com/library

这里是一些可以下载的大模型的例子:
模型参数大小下载Llama 38B4.7GB

ollama run llama3

Llama 370B40GB

ollama run llama3:70b

Phi 3 Mini3.8B2.3GB

ollama run phi3

Phi 3 Medium14B7.9GB

ollama run phi3:medium

Gemma2B1.4GB

ollama run gemma:2b

Gemma7B4.8GB

ollama run gemma:7b

Mistral7B4.1GB

ollama run mistral

Moondream 21.4B829MB

ollama run moondream

Neural Chat7B4.1GB

ollama run neural-chat

Starling7B4.1GB

ollama run starling-lm

Code Llama7B3.8GB

ollama run codellama

Llama 2 Uncensored7B3.8GB

ollama run llama2-uncensored

LLaVA7B4.5GB

ollama run llava

Solar10.7B6.1GB

ollama run solar

Note: 你需要至少8GB RAM 来运行7B 参数的模型, 16GB 来运行 13B 大模型, 32GB 来运行33B.

自定义模型

从 GGUF 引入

Ollama支持在Modelfile中导入GGUF模型:

1. 创建一个名为 Modelfile 的文件, 使用带有要导入的模型的本地文件路径的“FROM”指令。FROM ./vicuna-33b.Q4_0.gguf
2. 在 Ollama 里创建模型ollama create example -f Modelfile
3. 运行模型ollama run example

从 PyTorch 或 Safetensors 引入

检查 引导 来获得关于引入模型的更多信息. (中文版不可用)

自定义 prompt

从Ollama 库下载的大模型可以用prompt 自定义. 例如, 要自定义

llama3

模型:

ollama pull llama3

创建

Modelfile

:

FROM llama3

# 将参数设置为1[越高越有创意，越低越连贯]
PARAMETER temperature 1

# 设置系统信息
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""

下一步, 创建并运行模型:

ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.

有关更多示例，请参阅examples目录。有关使用模型文件的更多信息，请参阅Modelfile文档。(中文版未翻译)

命令参考

创建模型

ollama create

用于通过Modelfile 来创建模型.

ollama create mymodel -f ./Modelfile

下载一个模型

ollama pull llama3

这个命令也可以用来更新本地模型。只有不同的部分会被下载。

删除模型

ollama rm llama3

复制模型

ollama cp llama3 my-model

多行输入

要实现多行输入, 你可以用

"""

包围它们:

>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.

多模式模型

>>> What's in this image? /Users/jmorgan/Desktop/smile.png
The image features a yellow smiley face, which is likely the central focus of the picture.

将Prompt 作为参数传递

$ ollama run llama3 "Summarize this file: $(cat README.md)"
 Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.

列出你电脑上的模型

ollama list

启动Ollama

ollama serve

用于在不运行桌面应用程序的情况下启动ollama.

构建

检查 开发者引导

运行本地构建

随后，启动服务:

./ollama serve

最后，在一个单独的shell中，运行一个模型:

./ollama run llama3

REST API

Ollama有一个用于运行和管理模型的REST API.

生成回应

curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt":"Why is the sky blue?"
}'

和模型对话

curl http://localhost:11434/api/chat -d '{
  "model": "llama3",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

检查 API documentation 得到所有终端.

社区整合

Web & Desktop

• Open WebUI
• Enchanted (macOS native)
• Hollama
• Lollms-Webui
• LibreChat
• Bionic GPT
• HTML UI
• Saddle
• Chatbot UI
• Chatbot UI v2
• Typescript UI
• Minimalistic React UI for Ollama Models
• Ollamac
• big-AGI
• Cheshire Cat assistant framework
• Amica
• chatd
• Ollama-SwiftUI
• Dify.AI
• MindMac
• NextJS Web Interface for Ollama
• Msty
• Chatbox
• WinForm Ollama Copilot
• NextChat with Get Started Doc
• Alpaca WebUI
• OllamaGUI
• OpenAOE
• Odin Runes
• LLM-X (Progressive Web App)
• AnythingLLM (Docker + MacOs/Windows/Linux native app)
• Ollama Basic Chat: Uses HyperDiv Reactive UI
• Ollama-chats RPG
• QA-Pilot (Chat with Code Repository)
• ChatOllama (Open Source Chatbot based on Ollama with Knowledge Bases)
• CRAG Ollama Chat (Simple Web Search with Corrective RAG)
• RAGFlow (Open-source Retrieval-Augmented Generation engine based on deep document understanding)
• StreamDeploy (LLM Application Scaffold)
• chat (chat web app for teams)
• Lobe Chat with Integrating Doc
• Ollama RAG Chatbot (Local Chat with multiple PDFs using Ollama and RAG)
• BrainSoup (Flexible native client with RAG & multi-agent automation)
• macai (macOS client for Ollama, ChatGPT, and other compatible API back-ends)
• Olpaka (User-friendly Flutter Web App for Ollama)
• OllamaSpring (Ollama Client for macOS)
• LLocal.in (Easy to use Electron Desktop Client for Ollama)

Terminal

• oterm
• Ellama Emacs client
• Emacs client
• gen.nvim
• ollama.nvim
• ollero.nvim
• ollama-chat.nvim
• ogpt.nvim
• gptel Emacs client
• Oatmeal
• cmdh
• ooo
• shell-pilot
• tenere
• llm-ollama for Datasette’s LLM CLI.
• typechat-cli
• ShellOracle
• tlm
• podman-ollama
• gollama

Database

• MindsDB (Connects Ollama models with nearly 200 data platforms and apps)
• chromem-go with example

Package managers

• Pacman
• Helm Chart
• Guix channel

Libraries

• LangChain and LangChain.js with example
• LangChainGo with example
• LangChain4j with example
• LangChainRust with example
• LlamaIndex
• LiteLLM
• OllamaSharp for .NET
• Ollama for Ruby
• Ollama-rs for Rust
• Ollama4j for Java
• ModelFusion Typescript Library
• OllamaKit for Swift
• Ollama for Dart
• Ollama for Laravel
• LangChainDart
• Semantic Kernel - Python
• Haystack
• Elixir LangChain
• Ollama for R - rollama
• Ollama for R - ollama-r
• Ollama-ex for Elixir
• Ollama Connector for SAP ABAP
• Testcontainers
• Portkey
• PromptingTools.jl with an example
• LlamaScript

Mobile

• Enchanted
• Maid

Extensions & Plugins

• Raycast extension
• Discollama (Discord bot inside the Ollama discord channel)
• Continue
• Obsidian Ollama plugin
• Logseq Ollama plugin
• NotesOllama (Apple Notes Ollama plugin)
• Dagger Chatbot
• Discord AI Bot
• Ollama Telegram Bot
• Hass Ollama Conversation
• Rivet plugin
• Obsidian BMO Chatbot plugin
• Cliobot (Telegram bot with Ollama support)
• Copilot for Obsidian plugin
• Obsidian Local GPT plugin
• Open Interpreter
• Llama Coder (Copilot alternative using Ollama)
• Ollama Copilot (Proxy that allows you to use ollama as a copilot like Github copilot)
• twinny (Copilot and Copilot chat alternative using Ollama)
• Wingman-AI (Copilot code and chat alternative using Ollama and HuggingFace)
• Page Assist (Chrome Extension)
• AI Telegram Bot (Telegram bot using Ollama in backend)
• AI ST Completion (Sublime Text 4 AI assistant plugin with Ollama support)
• Discord-Ollama Chat Bot (Generalized TypeScript Discord Bot w/ Tuning Documentation)
• Discord AI chat/moderation bot Chat/moderation bot written in python. Uses Ollama to create personalities.
• Headless Ollama (Scripts to automatically install ollama client & models on any OS for apps that depends on ollama server)

Supported backends

• llama.cpp project founded by Georgi Gerganov.