If you are looking for additional security and do not want your AI conversations to be processed in the cloud, you can try out one of the models we are running locally. We have a few computers in the IdeaLab (Weitz 026) that are using Ollama via an interface called Open WebUI1 to locally run both a LLama and DeepSeekV2 model.
What are Ollama and Open WebUI?
Ollama is a platform that lets you run Large Language Models (LLMs) directly on your computer rather than in a cloud service. It provides access to generative AI similar to ChatGPT and Claude in a more secure way.
Open WebUI is a platform that runs in a browser and generates an interface to allow users to interact with the LLMs. While Ollama runs the model, Open WebUI allows you to communicate with Ollama. The Open WebUI platform looks similar to any other generative AI platform you might have come across, such as Gemini.
Why is Ollama useful?
Ollama is useful in a number of ways:
- Privacy and Security: Keeps all data processing and model interactions on your local machine, eliminating the need to send information to external servers. This is valuable for sensitive data or when privacy is a primary concern.
- Offline Access: Use LLMs even without an internet connection. The models and their dependencies are stored locally.
- Customization and Fine-tuning: Run fine-tuned language models locally, making it ideal for custom chatbots or research projects with specific model needs.
- Open Source Focus: By prioritizing open-source models, Ollama makes it easy to experiment with and explore various AI options.
- Cost Efficiency: Runs models locally with no subscription fees or API costs.
How to get started
Ollama and Open WebUI are installed on PCs in the IdeaLab (Weitz 026), where users currently have access to both a LLama and DeepSeek models. To get started, email Academic Technology (at@carleton.edu) to set up an appointment. During your first appointment, you will be required to register an account with Open WebUI on the PC you will be using and have that account approved. Your Open WebUI account will be tied to that specific computer. Your username and password will not be visible to anyone after you enter them. This first step will allow you to sign into the Open WebUI platform as an approved user, and be able to start interacting with the AI models. Consider reserving the computer you are using for the times you are coming back.