【AI实践】Ollama本地安装大模型服务

Ollama安装运行

安装与配置

Download Ollama

安装默认在C盘，成功后，window任务栏图标会有Ollama Logo

为了不占用C盘更大的空间，修改模型下载路径，修改环境变量

下载模型

由于我电脑是第六代Intel，集显，没有独立显卡；这里选择3B比较小的模型，防止模型太大响应太慢；

在Window系统的PowerShell软件，或者Windows Terminal中输入命令

ollama run phi3

此处使用微软的phi3模型，3B的版本，2.4GB大小；

下载完成

首次安装完成，就进入了，运行模型的窗口

后续运行模型

PS C:\Users\iamlv> ollama run phi3
>>> 

回复速度如下图

另外安装运行模型llama3, 7B版本

PS C:\Users\iamlv> ollama run llama3
>>> 

网页界面交互

open WebUI提供web网页

open WebUI的安装有很多方法，如下文，推荐docker容器安装，

🚀 Getting Started | Open WebUI

安装docker Windows 桌面版

PS C:\Users\iamlv> docker version
Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:48:43 2024
 OS/Arch:           windows/amd64
 Context:           default
Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:28 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

在docker下载open Web UI的镜像，注意Ollama不在你电脑和使用Nvidia GPU驱动Open WebUI的情况；

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

PS C:\Users\iamlv> docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
Unable to find image 'ghcr.io/open-webui/open-webui:main' locally
main: Pulling from open-webui/open-webui
2cc3ae149d28: Pull complete
87c0edd565e2: Pull complete
3df7545512d5: Pull complete
8f26d42ebf67: Pull complete
8951c7adce45: Pull complete
e68b3f3b28fc: Pull complete
4f4fb700ef54: Pull complete
04910e925ba2: Pull complete
e00576178709: Pull complete
7b74341f8bd7: Pull complete
46aff8bb649e: Pull complete
d2f7110849c1: Pull complete
672fa8e030d8: Pull complete
90d4d9484fcc: Pull complete
4889567b5c13: Pull complete
4ec90e217655: Pull complete
Digest: sha256:0fa56a9d947413cba22e4029df88e8c47acc78c013f68df303619ed05e45d9cf
Status: Downloaded newer image for ghcr.io/open-webui/open-webui:main
47b3c067e2d2b254336194add3431dd151664c8d4095f3ae4f5b3b5fb7acf139
PS C:\Users\iamlv>

测试一个英语题和翻译解读

总结

1. 纯CPU运行（8核），3B速度较快（和人阅读接近），7B左右的模型，延迟可以接受；
2. 3B 7B，指令遵循能力更准确；7B 的对话承前启后的接话能力更好
   1. 以英文完型填空为例，3B模型默认会给答案，让他不要给，还是会给；
   2. 7B对话前会有一个偏感性的反馈；最后可能会追问 

 ollama run phi3
>>> help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option
 The cat sat patiently on the mat, waiting for its owner to return home. What does "patiently" mean?
A) Without any noise
B) With great speed
C) In a calm and still manner
D) In an aggressive way
The correct answer is: C) In a calm and still manner
>>> help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option. do
... not give me the answer. let me answer first.
 During their weekend retreat, the family decided to engage in an activity that would foster unity and encourage
open communication among its members. What does "foster" mean?
A) To hinder or prevent
B) To promote growth or development
C) To disconnect or distance oneself from
D) To repeat actions without variation
The correct answer is: B) To promote growth or development
>>> help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option. ddo
... not give me the correct answer. let me answer first.
 The philosopher's writings were renowned for their ability to provoke profound reflection and challenge
conventional wisdom. What does "provocative" mean?
A) Tending to soothe or pacify
B) Having a tendency to cause annoyance or irritation
C) Capable of inducing deep thought or strong emotions, often controversially
D) Lacking in stimulation or interest
The correct answer is: C) Capable of inducing deep thought or strong emotions, often controversially
>>> give me another sentence, donot tell the correct answer
 The diplomat's speech was lauded for its eloquence and persuasive power in swaying international opinions. What
does "eloquent" mean?
A) Lacking clarity or precision in expression
B) Characterized by a high level of articulation and effectiveness in communication
C) Filled with errors and unclear language
D) Of little importance or influence
The correct answer is: B) Characterized by a high level of articulation and effectiveness in communication
>>> /bye
PS C:\Users\iamlv> ollama run llama3
>>> help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option. ddo
... not give me the correct answer. let me answer first.
I'd be happy to help you study vocabulary. Here's your sentence:
The new employee was _______________________ by her coworkers from day one.
You can choose from the following options:
A) intimidated
B) impressed
C) confused
D) frustrated
Go ahead and fill in the blank with the option you think is correct!
>>> D
Interesting choice!
Here's another sentence for you to try:
The tourists got _______________________ when they saw the famous landmark looming above them.
You can choose from the following options:
A) awestruck
B) annoyed
C) exhausted
D) excited
What's your answer?
>>> D
Another choice!
Here's another sentence for you to try:
The teacher encouraged her students to _______________________ their creative writing skills.
You can choose from the following options:
A) develop
B) demonstrate
C) illustrate
D) improvise
>>> D. till now I answered 3 question, are these answer all correct ?
Let me check!
For the first sentence, you chose D) frustrated, but the correct answer is actually A) intimidated.
For the second sentence, you chose D) excited, which is... CORRECT! Well done!
And for the third sentence, you also chose D) improvise, but unfortunately, that's not the correct answer either.
The correct answer is actually A) develop.
So, out of three questions, one was correct and two were incorrect. Not bad, though! You're still learning, and
that's what matters!
Keep going?
>>> for the first sentence, I still donot understanding, please teach me step by step
I'd be happy to help you understand the first sentence.
Here's the original sentence:
The new employee was _______________________ by her coworkers from day one.
You chose D) frustrated, but let's break it down step by step:
1. **"was"**: This is a linking verb that connects the subject (the new employee) with the predicate nominative
(the adjectival phrase).
2. **"intimidated"**: This is an adjective that describes how the coworkers treated the new employee.
3. **"by her coworkers"**: This phrase indicates who did the intimidating (her coworkers).
4. **"from day one"**: This phrase provides context about when the intimidation started.
Now, think about what it means to be "intimidated"...
* When someone is intimidated by another person or group of people, they feel a sense of fear, anxiety, or
apprehension.
* The other person or group might be making them feel small, weak, or uncertain about themselves.
In this sentence, the new employee was made to feel this way by her coworkers from the very beginning. It's not
that the coworkers were frustrated with the new employee; rather, they were having an adverse effect on her,
making her feel uncomfortable or uneasy.
So, the correct answer is A) intimidated.
Does this explanation help clarify things for you?
>>> Send a message (/? for help)

附录

PS C:\Users\jacob> ollama -h
Large language model runner
Usage:
  ollama [flags]
  ollama [command]
Available Commands:
  serve       Start ollama
  create      Create a model from a Modelfile
  show        Show information for a model
  run         Run a model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  ps          List running models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command
Flags:
  -h, --help      help for ollama
  -v, --version   Show version information
Use "ollama [command] --help" for more information about a command.