Tag: open source

[fusion_builder_container type=”flex” hundred_percent=”no” hundred_percent_height=”no” min_height_medium=”” min_height_small=”” min_height=”” hundred_percent_height_scroll=”no” align_content=”stretch” flex_align_items=”flex-start” flex_justify_content=”flex-start” flex_wrap_medium=”” flex_wrap_small=”” flex_wrap=”wrap” flex_column_spacing=”” hundred_percent_height_center_content=”yes” equal_height_columns=”no” container_tag=”div” menu_anchor=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” status=”published” publish_date=”” class=”” id=”” spacing_medium=”” margin_top_medium=”” margin_bottom_medium=”” spacing_small=”” margin_top_small=”” margin_bottom_small=”” margin_top=”” margin_bottom=”” padding_dimensions_medium=”” padding_top_medium=”” padding_right_medium=”” padding_bottom_medium=”” padding_left_medium=”” padding_dimensions_small=”” padding_top_small=”” padding_right_small=”” padding_bottom_small=”” padding_left_small=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” link_hover_color=”” link_color=”” border_sizes=”” border_sizes_top=”” border_sizes_right=”” border_sizes_bottom=”” border_sizes_left=”” border_color=”” border_style=”solid” border_radius_top_left=”” border_radius_top_right=”” border_radius_bottom_right=”” border_radius_bottom_left=”” box_shadow=”no” box_shadow_vertical=”” box_shadow_horizontal=”” box_shadow_blur=”0″ box_shadow_spread=”0″ box_shadow_color=”” box_shadow_style=”” z_index=”” overflow=”” background_color_medium=”” background_color_small=”” background_color=”” gradient_start_color=”” gradient_end_color=”” gradient_start_position=”0″ gradient_end_position=”100″ gradient_type=”linear” radial_direction=”center center” linear_angle=”180″ background_image_medium=”” background_image_small=”” background_image=”” skip_lazy_load=”” background_position_medium=”” background_position_small=”” background_position=”center center” background_repeat_medium=”” background_repeat_small=”” background_repeat=”no-repeat” background_size_medium=”” background_size_small=”” background_size=”” background_custom_size=”” background_custom_size_medium=”” background_custom_size_small=”” fade=”no” background_parallax=”none” enable_mobile=”no” parallax_speed=”0.3″ background_blend_mode_medium=”” background_blend_mode_small=”” background_blend_mode=”none” background_slider_images=”” background_slider_position=”” background_slider_skip_lazy_loading=”no” background_slider_loop=”yes” background_slider_pause_on_hover=”no” background_slider_slideshow_speed=”5000″ background_slider_animation=”fade” background_slider_direction=”up” background_slider_animation_speed=”800″ background_slider_blend_mode=”” video_mp4=”” video_webm=”” video_ogv=”” video_url=”” video_aspect_ratio=”16:9″ video_loop=”yes” video_mute=”yes” video_preview_image=”” pattern_bg=”none” pattern_custom_bg=”” pattern_bg_color=”” pattern_bg_style=”default” pattern_bg_opacity=”100″ pattern_bg_size=”” pattern_bg_blend_mode=”normal” mask_bg=”none” mask_custom_bg=”” mask_bg_color=”” mask_bg_accent_color=”” mask_bg_style=”default” mask_bg_opacity=”100″ mask_bg_transform=”left” mask_bg_blend_mode=”normal” render_logics=”” logics=”” absolute=”off” absolute_devices=”small,medium,large” sticky=”off” sticky_devices=”small-visibility,medium-visibility,large-visibility” sticky_background_color=”” sticky_height=”” sticky_offset=”” sticky_transition_offset=”0″ scroll_offset=”0″ animation_type=”” animation_direction=”left” animation_color=”” animation_speed=”0.3″ animation_delay=”0″ animation_offset=”” filter_hue=”0″ filter_saturation=”100″ filter_brightness=”100″ filter_contrast=”100″ filter_invert=”0″ filter_sepia=”0″ filter_opacity=”100″ filter_blur=”0″ filter_hue_hover=”0″ filter_saturation_hover=”100″ filter_brightness_hover=”100″ filter_contrast_hover=”100″ filter_invert_hover=”0″ filter_sepia_hover=”0″ filter_opacity_hover=”100″ filter_blur_hover=”0″][fusion_builder_row][fusion_builder_column type=”1_1″ layout=”1_1″ align_self=”auto” content_layout=”column” align_content=”flex-start” valign_content=”flex-start” content_wrap=”wrap” spacing=”” center_content=”no” column_tag=”div” link=”” target=”_self” link_description=”” min_height=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” sticky_display=”normal,sticky” class=”” id=”” type_medium=”” type_small=”” flex_grow_medium=”” flex_grow_small=”” flex_grow=”” flex_shrink_medium=”” flex_shrink_small=”” flex_shrink=”” order_medium=”0″ order_small=”0″ dimension_spacing_medium=”” dimension_spacing_small=”” dimension_spacing=”” dimension_margin_medium=”” dimension_margin_small=”” margin_top=”” margin_bottom=”” padding_medium=”” padding_small=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” hover_type=”none” border_sizes=”” border_color_hover=”” border_color=”” border_style=”solid” border_radius=”” box_shadow=”no” dimension_box_shadow=”” box_shadow_blur=”0″ box_shadow_spread=”0″ box_shadow_color=”” box_shadow_style=”” z_index_hover=”” z_index=”” overflow=”” background_type=”single” background_color_medium=”” background_color_small=”” background_color_medium_hover=”” background_color_small_hover=”” background_color_hover=”” background_color=”” gradient_start_color=”” gradient_end_color=”” gradient_start_position=”0″ gradient_end_position=”100″ gradient_type=”linear” radial_direction=”center center” linear_angle=”180″ background_image_medium=”” background_image_small=”” background_image=”” background_image_id_medium=”” background_image_id_small=”” background_image_id=”” lazy_load=”none” skip_lazy_load=”” background_position_medium=”” background_position_small=”” background_position=”left top” background_repeat_medium=”” background_repeat_small=”” background_repeat=”no-repeat” background_size_medium=”” background_size_small=”” background_size=”” background_custom_size=”” background_custom_size_medium=”” background_custom_size_small=”” background_blend_mode_medium=”” background_blend_mode_small=”” background_blend_mode=”none” background_slider_images=”” background_slider_position=”” background_slider_skip_lazy_loading=”no” background_slider_loop=”yes” background_slider_pause_on_hover=”no” background_slider_slideshow_speed=”5000″ background_slider_animation=”fade” background_slider_direction=”up” background_slider_animation_speed=”800″ background_slider_blend_mode=”” render_logics=”” sticky=”off” sticky_devices=”small-visibility,medium-visibility,large-visibility” sticky_offset=”” absolute=”off” absolute_props=”” filter_type=”regular” filter_hover_element=”self” filter_hue=”0″ filter_saturation=”100″ filter_brightness=”100″ filter_contrast=”100″ filter_invert=”0″ filter_sepia=”0″ filter_opacity=”100″ filter_blur=”0″ filter_hue_hover=”0″ filter_saturation_hover=”100″ filter_brightness_hover=”100″ filter_contrast_hover=”100″ filter_invert_hover=”0″ filter_sepia_hover=”0″ filter_opacity_hover=”100″ filter_blur_hover=”0″ transform_type=”regular” transform_hover_element=”self” transform_scale_x=”1″ transform_scale_y=”1″ transform_translate_x=”0″ transform_translate_y=”0″ transform_rotate=”0″ transform_skew_x=”0″ transform_skew_y=”0″ transform_scale_x_hover=”1″ transform_scale_y_hover=”1″ transform_translate_x_hover=”0″ transform_translate_y_hover=”0″ transform_rotate_hover=”0″ transform_skew_x_hover=”0″ transform_skew_y_hover=”0″ transform_origin=”” transition_duration=”300″ transition_easing=”ease” transition_custom_easing=”” motion_effects=”” scroll_motion_devices=”small-visibility,medium-visibility,large-visibility” animation_type=”” animation_direction=”left” animation_color=”” animation_speed=”0.3″ animation_delay=”0″ animation_offset=”” last=”true” border_position=”all” first=”true”][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”” rule_size=”” rule_color=”” hue=”” saturation=”” lightness=”” alpha=”” user_select=”” awb-switch-editor-focus=”” content_alignment_medium=”” content_alignment_small=”” content_alignment=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” sticky_display=”normal,sticky” class=”” id=”” margin_top=”” margin_right=”” margin_bottom=”” margin_left=”” fusion_font_family_text_font=”” fusion_font_variant_text_font=”” font_size=”” line_height=”” letter_spacing=”” text_transform=”” text_color=”” animation_type=”” animation_direction=”left” animation_color=”” animation_speed=”0.3″ animation_delay=”0″ animation_offset=”” logics=””]

How to Set Up and Expand Open-Source DeepSeek for a Growing Private Knowledge Base at Home

Introduction
DeepSeek is an open-source, AI-powered search solution that can be self-hosted on your home computer to create a robust and fully private knowledge base. Beyond simple text searches, DeepSeek can be extended to function as specialized GPT-like assistants—such as a “Doctor GPT,” “Lawyer GPT,” or any other expert system—by feeding it a domain-specific dataset. Thanks to its capacity to learn from ongoing interactions, DeepSeek grows smarter and more intuitive the more you use it, ultimately providing tailored answers that adapt to your communication style and specialized knowledge needs.

In this expanded guide, we’ll cover everything from the basic installation process to the advanced use of custom knowledge bases for specialized GPTs. By the end, you’ll have a comprehensive overview of how to harness DeepSeek’s powerful AI features to continually refine and scale your self-hosted knowledge base at home.

1. What Is DeepSeek?

DeepSeek is an open-source project offering a lightweight, high-performance interface for document indexing and AI-assisted search. Its core capabilities include:

1. Document Indexing: Quickly indexes large collections of local files (PDFs, Word documents, text files, and more).
2. AI-Based Search & Summaries: Uses NLP (Natural Language Processing) to present more relevant or summarized results.
3. Adaptive Learning: Learns from user interactions, improving the accuracy and relevancy of search results over time.
4. Customizable GPT-Like Assistants: Allows you to configure specialized “expert modes” or GPTs (e.g., medical, legal, technical), each utilizing a curated subset of your knowledge base.

Many of these features are built around open-source models or frameworks like GPT-NeoX or Hugging Face Transformers, offering an extendable foundation for custom AI applications.

2. Why Self-Host DeepSeek and Create Specialized GPT Assistants?

1. Data Privacy: All your documents and user interactions remain on your local machine.
2. Cost Savings: Eliminate recurring fees of cloud-based AI services.
3. Domain-Specific Expertise: Train specialized GPT modes (e.g., “Doctor GPT” trained on medical knowledge or “Lawyer GPT” trained on legal documents) for better context-aware responses.
4. Adaptive Learning: DeepSeek learns from your ongoing usage—improving its responses and search relevancy over time.
5. Scalability: Control your own hardware upgrades and scale your AI services as needed.

3. Core Features of DeepSeek and How They Evolve With Usage

3.1. Adaptive Knowledge Base

• Personal Growth: Every document you add or conversation you have helps refine DeepSeek’s language model. Over time, it better understands your queries and communication style.
• Contextual Expansion: The more you interact or upload specialized documents, the richer and more context-specific the knowledge base becomes.

3.2. Specialized GPT Training

• Custom Domains: Set up multiple “expert modes” by categorizing documents. For instance, legal documents can train a “Lawyer GPT,” while medical literature can inform a “Doctor GPT.”
• Improved Accuracy: Because the system has direct, local access to highly relevant domain texts, it can generate more accurate responses in the chosen field.

3.3. Intelligent Search and Summaries

• Relevancy Ranking: Deploy advanced NLP algorithms to retrieve highly relevant documents.
• Summaries and Insights: DeepSeek can generate concise summaries of a set of documents, helping you quickly grasp key points.

4. Prerequisites

1. Hardware:
   • A modern multi-core CPU (Intel i5/i7, Ryzen 5/7, or above).
   • 16 GB RAM or more is recommended for larger knowledge bases.
   • An SSD for fast read/write operations.
2. Software:
   • Operating System: Windows 10/11, macOS, or Linux (Ubuntu, Fedora, etc.).
   • Python 3.8+ (for manual installation and advanced customization).
   • Docker (for containerized deployment).
   • Git (for cloning repositories and version control).

5. Installation Methods

5.1. Docker-Based Installation

1. Install Docker:
   • Docker for Windows
   • Docker for macOS
   • Docker for Linux
2. Pull the DeepSeek Image
   docker pull deepseeklab/deepseek:latest

3. Run the Container
   bash

   docker run -d \
--name deepseek \
-p 8080:8080 \
-v /path/to/your/documents:/app/documents \
deepseeklab/deepseek:latest

   • Access the interface at: http://localhost:8080

5.2. Manual Installation (Python Environment)

1. Clone the Repository
   git clone https://github.com/deepseek-lab/DeepSeek.git
cd DeepSeek

2. Set Up a Virtual Environment
   python -m venv venv
source venv/bin/activate # On Linux/macOS
# or on Windows:
# venv\Scripts\activate

3. Install Dependencies
   pip install -r requirements.txt

4. Configure DeepSeek
   • Modify config.yaml or .env to specify your document paths, indexing parameters, and any optional AI integrations for GPT-based features.
5. Run DeepSeek
   python main.py

   • Visit http://localhost:8080 to start using DeepSeek.

6. Indexing Your Documents

Once installed, you can point DeepSeek to any directory on your local machine:

1. Default Folder: By mounting a folder via Docker (-v /path/to/your/documents:/app/documents) or specifying a directory in config.yaml.
2. Supported Formats: .txt, .pdf, .docx, .md, and potentially others depending on the modules installed.
3. Incremental Indexing: New documents can be auto-detected and indexed periodically. Adjust scheduling in the config file to avoid slowing down your machine during peak usage.

7. Building Specialized GPT Assistants

To create specialized GPT-like modes—such as a “Doctor GPT,” “Lawyer GPT,” or even a “Coder GPT” for technical tasks—you can segment your knowledge base and configure AI modules accordingly:

1. Document Segmentation:
   • Organize your documents by topic or domain (e.g., medical_docs, legal_docs, tech_docs).
   • Each folder can represent a specialized corpus for the AI.
2. Configuration for Domain-Specific GPT:
   • In the DeepSeek configuration, define multiple “knowledge contexts” for the AI.
   • Assign each context to a set of folders (e.g., “medical_docs” for Doctor GPT).
   • Enable or disable certain features (like summarization or advanced NLP) on a per-context basis.
3. Additional NLP Models or Plugins:
   • Integrate large language models via Hugging Face Transformers or GPT-NeoX.
   • Fine-tune or instruct these models to provide more specialized advice based on the uploaded documents.
4. Adaptive Learning:
   • Each GPT-like assistant “learns” from both the documents and your interactions.
   • Frequent Q&A sessions in the “Doctor GPT” context help it refine medical language and improve response accuracy over time.

8. How DeepSeek Learns and Grows with You

1. User Interactions:
   • Every question, correction, or feedback you provide helps the system better understand your tone, jargon, and preferred answer style.
2. Relevance Feedback:
   • If you mark a search result as particularly relevant or not relevant, the AI fine-tunes its internal ranking and prioritization.
3. Ongoing Index Updates:
   • As you add or remove documents, the index evolves.
   • Older documents are re-contextualized in light of newly added information.
4. Conversational Memory:
   • Over time, each GPT-like context or mode builds a conversational memory that enables more natural dialogues.
   • The system starts to anticipate your needs, offering suggestions or clarifications proactively.

9. Best Practices and Maintenance

1. Regular Backups:
   • Store copies of your indexed data and any specialized AI model checkpoints.
   • Keep your configuration files in a version control system like Git.
2. Scheduled Re-Indexing:
   • Schedule indexing during low-usage hours to minimize performance impact.
3. Frequent Updates:
   • Stay current with the latest Docker image or manual installation updates to benefit from new features and security patches.
4. Monitor Resource Usage:
   • Specialized GPTs can be resource-intensive; monitor CPU, RAM, and storage usage.
   • Consider hardware upgrades if you scale up your knowledge base or add more advanced AI models.
5. Document Management:
   • Organize and label documents carefully for maximum accuracy.
   • Use consistent naming conventions and metadata to make the best use of advanced search and GPT modes.

10. Useful Resources & Citations

• Docker Installation Docs:
  https://docs.docker.com/get-docker/
• Python 3.8+ Downloads:
  https://www.python.org/downloads/
• Git Official Website:
  https://git-scm.com/
• Hugging Face Transformers (For Advanced GPT Integration):
  https://huggingface.co/docs/transformers
• GPT-NeoX by EleutherAI:
  https://github.com/EleutherAI/gpt-neox
• Haystack (Alternative AI Search Framework):
  https://github.com/deepset-ai/haystack
• MeiliSearch (Lightweight Alternative):
  https://github.com/meilisearch/meilisearch

Conclusion

By deploying DeepSeek on your home computer, you gain unmatched control over your data and the ability to build domain-specific GPT-like assistants. Whether you’re crafting a legal research assistant, a medical reference tool, or a personal code helper, DeepSeek’s adaptive learning capability ensures it becomes more accurate and personalized with each interaction. As you continually refine your document organization and user feedback, DeepSeek transforms into a powerful, specialized partner that grows alongside your evolving knowledge needs.

A custom GPT offers remarkable flexibility and control, letting you integrate vast amounts of data—including entire book collections, specialized databases, or niche archives—that standard AI models might be restricted from accessing due to commercial licenses or copyright constraints. By hosting the GPT model locally, you can fine-tune it on specific texts and domains to produce context-rich responses far beyond what publicly available AI typically provides. This personalized approach also means fewer default safety restrictions, giving you the freedom to experiment with unconventional or cutting-edge topics without the usual usage barriers. Want to explore a delicate historical event in extreme detail or dissect a rare technical document from the 1960s? A custom GPT can handle it, delivering deep insights tailored to your personal or professional interests—free from the constraints of general-purpose AI filters.

[/fusion_text][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”” rule_size=”” rule_color=”” hue=”” saturation=”” lightness=”” alpha=”” user_select=”” awb-switch-editor-focus=”” content_alignment_medium=”” content_alignment_small=”” content_alignment=”center” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” sticky_display=”normal,sticky” class=”” id=”” margin_top=”” margin_right=”” margin_bottom=”” margin_left=”” fusion_font_family_text_font=”” fusion_font_variant_text_font=”” font_size=”” line_height=”” letter_spacing=”” text_transform=”” text_color=”” animation_type=”” animation_direction=”left” animation_color=”” animation_speed=”0.3″ animation_delay=”0″ animation_offset=”” logics=””]

Run Deepseek Locally for Free!  Video

[/fusion_text][fusion_youtube id=”https://www.youtube.com/watch?v=pbCQnDDj-bo” alignment=”center” width=”” height=”” start_time=”” end_time=”” autoplay=”false” mute=”false” loop=”false” controls=”true” api_params=”” title_attribute=”” video_facade=”” thumbnail_size=”auto” margin_top=”50px” margin_bottom=”50px” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” css_id=”” structured_data=”off” video_upload_date=”” video_duration=”” video_title=”” video_desc=”” /][fusion_accordion type=”” inactive_icon=”fa-question-circle fas” active_icon=”” margin_top=”” margin_bottom=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” boxed_mode=”” border_size=”1″ border_color=”” hue=”” saturation=”” lightness=”” alpha=”” hover_color=”” background_color=”” divider_line=”” divider_hover_color=”” divider_color=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” title_tag=”h4″ fusion_font_family_title_font=”” fusion_font_variant_title_font=”” title_font_size=”” title_line_height=”var(–awb-typography1-line-height)” title_letter_spacing=”var(–awb-typography1-letter-spacing)” title_text_transform=”” title_color=”var(–awb-color4)” icon_size=”” icon_color=”” icon_boxed_mode=”” icon_box_color=”” icon_alignment=”” fusion_font_family_content_font=”” fusion_font_variant_content_font=”” content_font_size=”” content_line_height=”” content_letter_spacing=”” content_text_transform=”” content_color=”var(–awb-color4)” toggle_hover_accent_color=”” toggle_active_accent_color=”” parent_dynamic_content=””][fusion_toggle title=”Run Your Own ChatGPT Locally for Free!” open=”no” awb-switch-editor-focus=”” class=”” id=”” fusion_font_family_title_font=”” fusion_font_variant_title_font=”” title_font_size=”” title_line_height=”” title_letter_spacing=”” title_text_transform=”” title_color=”var(–awb-color4)” hue=”” saturation=”” lightness=”” alpha=”” fusion_font_family_content_font=”” fusion_font_variant_content_font=”” content_font_size=”” content_line_height=”” content_letter_spacing=”” content_text_transform=”” content_color=”var(–awb-color4)”]

In this post I’m going to show you just how easy it is to have local large language models, or LLM’s, also known as AI, running on your own Windows PC – for free.  Basically it’s like having a local copy of ChatGPT, but you can also pick and choose from tons of different AI models such as Meta’s Llama, Google’s Gemma, and even the recently popular and controversial DeepSeek R1.  But – since these large language models are running locally on your own computer, they aren’t phoning home and giving all of your data to large corporations or International governments.  And you can even run uncensored models that break free from the guardrails put in place by their online versions.

This is an awesome project that takes no more than 10 to 15 minutes until you have your very own powerful ChatGPT-style local AI interface, so let’s dig in and get started!

Hardware Requirements

Large language models can run on pretty much any Windows PC, but they’re going to be much faster with a good CPU, a good GPU, or if you have one of the new CPUs that come with an embedded NPU, or Neural Processing Unit designed for AI-intensive applications such as Intel’s Lunar Lake, Qualcomm’s Snapdragon 8 Gen 1, Nvidia’s Jetson AGX Xavier, or AMD’s AMD Ryzen AI.

But, they also run just fine without a super powerful PC or a special processor.  I’m going to be using a Minisforum MS-A1 which has an AMD Ryzen 7 8700G with built-in Radeon 780M graphics and 32 gigs of RAM.  You can pick one of these up brand new for just over $700 bucks.  The point being is that you don’t need a super powerful GPU like a 4000-series NVIDIA to run a large language model locally.

Project Overview

This project consists of 3 main components.  We’re first going to install Ollama, which is an open source project that allows users to run large language models directly on their own hardware.  Then we’ll be installing Docker Desktop for Windows so that we can finally run Open-WebUI which is a graphical user interface for the large language models that looks and feels very much like ChatGPT.

Prepare Windows

The first thing we want to do is enable a Windows Feature called Virtual Machine Platform.  This installs what’s known as WSL2 or Windows Subsystem for Linux 2, and it works with Windows 11 and Windows 10 version 1903 build 18362 or higher.

To install WSL2, hit the Windows or start button and type windows features.  Then scroll down until you see Virtual Machine Platform.  Check the box, and click OK.  This is gonna apply that feature to Windows and then ask you to reboot.

Ollama Install

Once rebooted, head over to ollama.com/download and click Download for Windows.  The setup file for Ollama is about 750 megs, so it may take a while to download depending on your Internet connection.  Once downloaded, run the setup for Ollama and click ‘Install.’

One thing that might throw you off about Ollama is that there’s no GUI at all.  Ollama lets you run large language models in your Windows CLI, or command prompt – so you can actually start working with models immediately…but we want this to function like ChatGPT with a nice user interface that saves our previous conversations.

Docker Desktop Install

So the next thing we’re going to do is install Docker Desktop for Windows.  Head on over to docs.docker.com/get-started, click on Get Started, and then click on Docker Desktop for Windows.  From here you can download the x86_64 version of Docker Desktop.  Once downloaded, click on the downloaded file and Docker Desktop will install onto your PC.

Once installed, reboot your computer and when you’re back up and running, you may get a CMD prompt pop-up telling you to press any button to update to the latest version of WSL2 – go ahead and click that.

Then you can finish going through the Docker wizard and then you’ll see the Docker Desktop interface which should have no containers in it already if you’re just installing Docker Desktop for the first time.

Open-WebUI Install

Next we’re going to install Open-WebUI which is a ChatGPT-style interface for Ollama.  Open-WebUI service that runs as a container in Docker.  Navigate to https://docs.openwebui.com and click ‘Getting Started’ followed by ‘Quick Start.’

Docker is selected by default, so just copy the first command which will pull Open-WebUI down to our local machine.

docker pull ghcr.io/open-webui/open-webui:main

Hit Windows Key + R to run a new program and type CMD to open command prompt. Then paste in the first command and hit ENTER.

Next we want to run the Docker container, but first we have to take a look at the various options.  For my PC, I’m just gonna grab this first command that has default settings, but if you’re running an NVIDIA GPU and you want to take advantage of the GPU processing power, then you’ll want to copy the command that has NVIDIA GPU support down below.

Run without NVIDIA GPU support:

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Run WITH NVIDIA GPU support:

docker run -d -p 3000:8080 --gpus all -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:cuda

I’m going to run the container by pasting in the default settings into my CMD prompt.  If you get any pop-ups warning you about the Windows Firewall, just click Allow.

If we go back to Docker Desktop, we should now see that our container is running since we have the green light.  At this point, you can start and stop the Open-WebUI container at any time – I like to disable the container when I’m not actively using the LLM.  Just hit the ‘STOP’ button to stop, and then the ‘PLAY’ button to start it back up.

If you click the link under ‘Port(s)’ this will bring up Open-WebUI in a browser window.

When you first log in, Open-WebUI will prompt you to create an Admin account – create your admin credentials and then you should be brought to a very ChatGPT-looking interface.

Success! The next thing we need to do is to start downloading some large language models.

First, let’s find an LLM to install – if you are unsure which to pick, Llama 3.2 is a good starter pick. Navigate back to Ollama.com and click on ‘Models.’

Models are sorted by popularity by default. The blue numbers underneath the various models are variations with more or less parameters (how much data the model was trained on). For Llama 3.2 we can see 1b and 3b. Click into that model and we can get even more detail.

To install the models into Open-WebUI, we need to just grab the name. In this case, for the 3B (default) model, the name is simply ‘llama3.2’ – but for the 1b variation, the name would be ‘llama3.2:1b’ – for all of these models on Ollama.com, the model name is what comes after ‘ollama run.’

Copy the name you want and go back to Open-WebUI. Click the drop-down next to ‘Select a model’ and paste the name into the search bar. Since Open-WebUI doesn’t have that model installed, it gives you the option to pull the model from Ollama.com.

Models can be HUGE, so keep that in mind. The Deepseek 70B model is around 20GB to download. The smaller models are much more reasonable.

Once downloaded, you can start making queries against that model!

Keep playing around with various models and you’re sure to find ones that you like. You can even run 2 models side by side with the same query, but keep in mind that your processing power will be doubled when doing that.

If you enjoyed this tutorial, be sure to subscribe to Crosstalk Solutions on YouTube, and be sure to buy me a coffee!

[/fusion_toggle][/fusion_accordion][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”” rule_size=”” rule_color=”” hue=”” saturation=”” lightness=”” alpha=”” user_select=”” awb-switch-editor-focus=”” content_alignment_medium=”” content_alignment_small=”” content_alignment=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” sticky_display=”normal,sticky” class=”” id=”” margin_top=”50px” margin_right=”” margin_bottom=”” margin_left=”” fusion_font_family_text_font=”” fusion_font_variant_text_font=”” font_size=”” line_height=”” letter_spacing=”” text_transform=”” text_color=”” animation_type=”” animation_direction=”left” animation_color=”” animation_speed=”0.3″ animation_delay=”0″ animation_offset=”” logics=””]

Related Videos:

Related Posts:

DeepSeek vs. ChatGPT: Which AI Dominates in 2024 — Enterprise Efficiency or Global Creativity?

Should I Choose a Hosted or Non-hosted Blogging Platform?

How To Start a Blog – Beginner’s Guide for 2024

10 Visual Studio Code extensions for every developer

DeepSeek AI vs. ChatGPT-4 Plus: Why GPT-4 Plus is the Superior Choice

Revitalizing Canada’s Manufacturing Sector: Addressing Profit-Driven Decline and Foreign Interests

Nvidia CEO Jensen Huang: Specialize, Don’t Just Learn to Code, in the AI Era

The importance of Computing power development

[/fusion_text][/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]