Ollama, or: how to run a LLM locally on your desktop

[Bartholomew Gallacher]

In case you've always wanted to checkout ChatGPT or similar, but distrust OpenAI about keeping your privacy, there's now a solution: Ollama!

Ollama allows you to run the LLM of your choice on your CPU or GPU, supported OSes are Windows, MacOS and Linux. First you need to download it, but it comes in easy installable packages.

After installation you will have to pick at least one LLM of your choice, which the program will download a copy from. After that all other stuff can be done without internet. And even without GPU, if you don't have one or not supported. CPU speed still is quite decent.

Here are some sample models which Ollama does support:

Model	Parameters	Size	Download
Llama 2	7B	3.8GB	ollama run llama2
Mistral	7B	4.1GB	ollama run mistral
Dolphin Phi	2.7B	1.6GB	ollama run dolphin-phi
Phi-2	2.7B	1.7GB	ollama run phi
Neural Chat	7B	4.1GB	ollama run neural-chat
Starling	7B	4.1GB	ollama run starling-lm
Code Llama	7B	3.8GB	ollama run codellama
Llama 2 Uncensored	7B	3.8GB	ollama run llama2-uncensored
Llama 2 13B	13B	7.3GB	ollama run llama2:13b
Llama 2 70B	70B	39GB	ollama run llama2:70b
Orca Mini	3B	1.9GB	ollama run orca-mini
Vicuna	7B	3.8GB	ollama run vicuna
LLaVA	7B	4.5GB	ollama run llava
Gemma	2B	1.4GB	ollama run gemma:2b
Gemma	7B	4.8GB	ollama run gemma:7b

Some sample output using Mistral 7B; some but not models are also unrestricted, meaning they can produce output which ChatGPT refuses to do so.

P: Describe the unique features of the American constitution while comparing it to the constitution of the UK.
A: Both the United States Constitution and the United Kingdom's Constitution, which is made up of various laws and conventions rather than a single
document, have distinct features that shape their respective political systems.

1. Structure of Government: The US Constitution establishes a federal system with three branches of government: the legislative, executive, and judicial
branches, each with specific powers and checks and balances against one another. In contrast, the UK's constitutional structure is a parliamentary
democracy, in which the executive branch (the monarch and Cabinet) and legislative branch (Parliament) are not entirely separate.

2. Written vs Unwritten: The US Constitution is a written document that outlines the basic framework of the government, while the UK's Constitution is
unwritten, consisting of various laws, conventions, and court decisions. This makes it easier for Americans to refer to a single document when
discussing their constitutional rights, whereas in the UK, legal interpretations can be more complex due to the absence of a single definitive source.

3. Amendment Process: The US Constitution has a formal process for amending its provisions, with 27 amendments adopted to date. In contrast, changes to
the UK's constitutional framework typically come about through legislative action or judicial decisions, without the need for formal amendment
processes.

4. Sovereignty: The US Constitution derives its authority from "the people," with sovereignty residing in the American people. In contrast, the UK's
constitutional framework is based on the concept of parliamentary sovereignty, which holds that Parliament is supreme and has the power to make or
unmake any law.

5. Bill of Rights: The US Constitution includes a Bill of Rights, which sets out various individual liberties and protections in the first ten
amendments. While the UK also recognizes various human rights, they are not codified in a single document in the same way as in the US. Instead, they
are derived from international treaties, case law, and statute.

In summary, while both the US Constitution and the UK's constitutional framework serve to establish the rules and structure of their respective
governments, they differ significantly in their written/unwritten nature, formal amendment processes, sovereignty

P: Give a short biography of the 47th president of the united states.
A: I apologize for the error in my previous response. The United States has had 46 presidents up to this point.

 

[Bartholomew Gallacher]

Model LLaVa, which can create text descriptions from images we do provide. It seems to recognize quite a lot.

P: Describe what's being depicted here.

Image: Cristiano

A: The image is a promotional or fashion-themed artwork featuring a man in an elegant tuxedo with a bow tie. He is standing against a backdrop of a beach
with palm trees and blue ocean water, which creates a relaxed and luxurious ambiance. The man's pose is confident and poised, and he's looking directly
at the viewer with a slight smile. His attire suggests that this image could be related to fashion, especially formal wear for weddings or other special
events. The overall composition of the image is designed to showcase the tuxedo as well as the beach setting, likely for marketing purposes or as part
of a photoshoot.

I: New York City Skyline
A: The image shows an aerial view of New York City, featuring iconic landmarks such as the Statue of Liberty on the left, the Manhattan skyline with the
Chrysler Building prominently in the center, and the Brooklyn Bridge towards the right. The waterways are visible, with the Hudson River flowing along
the west side and the East River on the east. There's a clear blue sky above, indicating good weather conditions.

I: Some SFW from the official Flickr account of LL.

A: The image appears to be a digitally created scene, possibly for a game or a simulation. It shows a character in a kitchen setting holding a glass of
what seems to be milkshake or iced coffee. The character is wearing a blue dress and has red hair. There are various items on the countertop: a
cookbook, cooking utensils like spoons and tongs, and several ingredients such as carrots, broccoli, and potatoes. The kitchen has a homely feel with
potted plants, a vase, and other decorative items.

Please note that this is a simulated environment and not a real-life scene.

 

[Noodles]

Can you train with this at all? I like the idea of running an LLM locally, but I really want something I can train on a small data set of PDFs and ebooks etc. I know there are cloud ways to do it but I really don't care to spend $20/ month on that (which seems to be the going rate).

 

[Monica Dream]

why not gpt4all?

 

[Noodles]

Insert this up, and found a NSFW models, but it still gives the most unsexy dirty stories ever. It also prefaces it with this weird wording like, "I really don't want to but if you really wanted itz it would be something like this. Then it's like a Mormon technical manual description of sex.