‘Web LLM’ that can run chat AI only with a browser’s WebGPU and can use Japanese, how to actually try it like this – GIGAZINE

Interactive AI systems such as ChatGPT and Bard have emerged that can talk to humans based on large-scale language models, but interactive AI systems require huge computing resources to operate, and usually AI can work. It allows interaction by accessing the server. Finely tuned LLaMA, a large scale language model developed by MetaVicuna-7bInteractive AI basedWeb MasterEnables GPU computing on the browserWebGPUWith it, it has the advantage of being able to access the interactive AI on the browser without accessing the server. Since it is also said to support Japanese, I have already tried to interact with Web LLM in Japanese.

GitHub – mlc-ai/web-llm: Bringing large language models and chat to web browsers. Everything works inside the browser without server support.
https://github.com/mlc-ai/web-llm

To experience Web LLM, you need to use a “WebGPU compatible browser” on a “PC with NVIDIA GPU”. However, WebGPU has just been reflected in Chrome, and at the time of writingChrome 113 betaYou’ll need to access it later, or from Chrome Canary, which is an upgraded version.

This time it will be accessed from Chrome Canary. Chrome Canary is distributed on the next page.

Chrome Canary Developer Features – Google Chrome
https://www.google.com/intl/ja/chrome/canary/

Click “Download Chrome Canary” on the distribution page to download the EXE format installer (file size 1.30MB).

When you start the downloaded installer, the installation will be done automatically.

When the installation is finished, Chrome Canary will start.

A demo of the Web LLM can be accessed here.

WebLLM | house
https://mlc.ai/web-llm/

The “Chat Demo” at the bottom of the page above is a Web LLM demo. First, click “Submit” without entering anything.

Then the following message is displayed and the form download starts.

「[System Initialize]Configuration completed”, setup completed. I entered “Hi, can you speak Japanese?” in the input field, clicked “submit”, and suddenly spoke in Japanese.

“Yes, I can speak Japanese. Please listen to me.”

First of all, I asked what kind of site GIGAZINE is. They seem to realize it’s a website, but I worry about the nonsensical “This is Japan’s primary online commenting scene.”

Then, when I asked how the curry was made, the Japanese answer was clearly broken, and words like German and Russian could be seen here and there.

Furthermore, as a result of continuing to answer in broken Japanese, I stopped at “Kari.”

next timeAnswers to the ultimate questions about life, the universe and everythingWhen I asked about it, the answer came in suspicious Japanese. Apparently, he refuses to answer, saying, “I cannot answer such questions because I am an AI.”

next timeCompetitive Programming IssuesI entered and checked if I could generate a code. However, it seems that I can’t understand the terms of the question text I entered, and the content of the affirmation at the beginning of the answer is completely different from the question, and the code is also completely different.

Explaining the contents of the code afterwards is also a sentence that makes no sense at all.

Finally, when I asked about Momotaro’s content, not only was the content of the fairy tale very different, but the grammar of the Japanese language was also quite unstable.

Web LLM is certainly able to talk to the AI ​​on the browser, but the accuracy of the large-scale language model is still sweeter than ChatGPT’s GPT-3.5, and in Japanese it returned a nonsense conversation. However, I even worked with WebGPU processing, so I would like to expect further development of the model.