The official Docker image for “Ollama”, an application that allows you to easily run multiple chat AIs in the local environment, is now available – GIGAZINE

「Mistral」「Llama 2」「VicunaThe official Docker image of “Ollama” has been released, which allows you to easily run large-scale open source language models like “Ollama” locally, so I immediately experienced its ease of use.

Ollama is now available as an official Docker image · Ollama blog
https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image

Below are representative examples of large-scale language models that can be run on Olma.For the full listOlama’s official websiteYou can check it here.

The model with parameters 3B (3 billion) requires at least 8 GB of memory, the model 7B (7 billion) requires 16 GB, and the model 13B (13 billion) requires 32 GB of memory.

So, let’s start preparing the environment to run Olama. We will be using Docker this time, so install Docker using the method that suits your environment from the link below.

Install Docker engine Docker documentation
https://docs.docker.com/engine/install/

This time, to use Debian, I entered the following command.

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Once Docker is installed, start Ollama. There is a CPU-only mode, and a mode that uses NVIDIA’s GPU to speed up the process, but this time we’ll be starting in CPU-only mode, so execute the command below.

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

After starting the Ollama container, you can run Ollama using the command below.

docker exec -it ollama [実行したいコマンド]

This time we’ll be using the 7B model of Llama 2, soThe table mentioned aboveWhen I executed the command “docker exec -it ollama ollama run llama2” pointing to the “Download” column, the form was downloaded and executed automatically. When it becomes possible to enter, “>>>” will be displayed, so just enter the chat here. At first, I greeted them with “Hello!”

When I tried Japanese, he said, “I can communicate in several languages!” Regardless of the Llama 2’s performance, I found that I could use Japanese without any problems even when interacting with the models on the Olma.

In addition, GIGAZINE wrote an article about models that can be used with Ollama as follows. Please check it out if you want to know more about the model.

Mistral
A large-scale language model “Mistral 7B” has emerged that can be used and verified with a truly open source license, making it possible to develop artificial intelligence with performance beyond “Llama 2 13B” and “Llama 1 34B” – GIGAZINE

◆ Llama 2
Meta releases commercially available large-scale language model “Llama 2” for free, partners with Microsoft and Qualcomm to improve smartphones and PCs – GIGAZINE

◆Llama 2 is uncensored
What are the benefits of creating an unsupervised model that removes the “censorship” of large-scale language models? -Gegazin

◆Vocna
Japanese Chat Data AI “Vicuna-13B” Released with Performance Similar to ChatGPT and Can Run on Regular Home PCs – GIGAZINE