OpenWebUI notes ...

Open WebUI installer: https://github.com/freeload101/SCRIPTS/blob/master/Bash/OpenWebUI_Fast.bash

Older Professor synapse prompt you can use: https://raw.githubusercontent.com/freeload101/SCRIPTS/refs/heads/master/Prof%20Synapse%20Old.txt

Fabric prompts you can import into openwebui !!! ( https://github.com/danielmiessler/fabric/tree/main/patterns

) https://github.com/freeload101/SCRIPTS/blob/master/MISC/Fabric_Prompts_Open_WebUI_OpenWebUI_20241112.json

Example AT windows task startup script to make it start and not

die on boot https://github.com/freeload101/SCRIPTS/blob/master/MISC/StartKokoro.xml

Open WebUI RAG fail sause ... https://youtu.be/CfnLrTcnPtY

Open registration 

Model list / order

NAME                                                   ID              SIZE      MODIFIED

hf.co/mradermacher/L3-8B-Stheno-v3.2-i1-GGUF:Q4_K_S    017d7a278e7e    4.7 GB    2 days ago

qwen2.5:32b                                            9f13ba1299af    19 GB     3 days ago

deepsex:latest                                         c83a52741a8a    20 GB     3 days ago

HammerAI/openhermes-2.5-mistral:latest                 d98003b83e17    4.4 GB    2 weeks ago

Sweaterdog/Andy-3.5:latest                             d3d9dc04b65a    4.7 GB    2 weeks ago

nomic-embed-text:latest                                0a109f422b47    274 MB    2 weeks ago

deepseek-r1:32b                                        38056bbcbb2d    19 GB     4 weeks ago

psyfighter2:latest                                     c1b3d5e5be73    7.9 GB    2 months ago

CognitiveComputations/dolphin-llama3.1:latest          ed9503dedda9    4.7 GB    2 months ago

Disable Arena models

Documents WIP RAG is not good .

Discord notes;

https://discord.com/channels/1170866489302188073/1340112218808909875

Abhi Chaturvedi:  @(Operat0r) try this To reduce latency

and improve accuracy, modify the .env file: Enable RAG

ENABLE_RAG=true

Use Hybrid Mode (Retrieval + Reranking for better context)

RAG_MODE=hybrid

Reduce the number of retrieved documents (default: 5)

RETRIEVAL_TOP_K=3

Use a Fast Embedding Model (instead of OpenAI's Ada-002)

EMBEDDING_MODEL=all-MiniLM-L6-v2 # Faster and lightweight .

Optimize the Vector Database VECTOR_DB_TYPE=chroma

CHROMA_DB_IMPL=hnsw # Faster search

CHROMA_DB_PATH=/root/open-webui/backend/data/vector_db.

Optimize Backend Performance # Increase Uvicorn worker count

(improves concurrency) UVICORN_WORKERS=4

Increase FastAPI request timeout (prevents RAG failures)

FASTAPI_TIMEOUT=60

Optimize database connection pool (for better query

performance)

SQLALCHEMY_POOL_SIZE=10

So probably the first thing to do is increase the top K value

in admin -> settings -> documents, or you could try the

new "full context mode" for rag documents. You may also need

to increase the context size on the model, but it will make it

slower, so you probably don't want to do that unless you start

seeing the "truncating input" warnings.

@JamesK

So probably the first thing to do is increase the top K value

in admin -> settings -> documents, or you could try the

new "full context mode" for rag documents. You may also need

to increase the context size on the model, but it will make it

slower, so you probably don't want to do that unless you start

seeing the "truncating input" warnings.

M]

JamesK:  Ah, I see. The rag didn't work great for you in

this prompt. There are three hits and the first two are

duplicates, so there isn't much data for the model to work

with

[9:12 PM] JamesK:  context section

I see a message warning that you are using the default 2048

context length, but not the message saying you've hit that

limit (from my logs the warning looks like

level=WARN source=runner.go:126 msg="truncating input prompt"

limit=32768 prompt=33434 numKeep=5

[6:06 AM] JamesK:  If you set the env var OLLAMA_DEBUG=1

before running ollama serve it will dump the full prompt being

sent to the model, that should let you confirm what the rag

has put in the prompt

JamesK: Watch the console output from ollama and check for

warnings about overflowing the context. If you have the

default 2k context you may need to increase it until the

warnings go away

[8:58 PM] JamesK:  But also, if you're using the default

rag, it chunks the input into small fragments, then matches

the fragments against your prompt and only inserts a few

fragments into the context, not the entire document. So it's

easily possible for the information you want to not be

present.

Auto updates

echo '0,12 */4 * * * docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui' >> /etc/crontab

Search

red note for API keys 

Go to Google Developers, use

Programmable Search Engine

, and log on or create account.

Go to

control panel

and click

Add

button

Enter a search engine name, set the other properties to suit

your needs, verify you're not a robot and click

Create

button.

Generate

API key

and get the

Search engine ID

. (Available after the engine is created)

With

API key

and

Search engine ID

, open

Open WebUI Admin panel

and click

Settings

tab, and then click

Web Search

Enable

Web search

and Set

Web Search Engine

to

google_pse

Fill

Google PSE API Key

with the

API key

and

Google PSE Engine Id

(# 4)

Click

Save

Note

​

You have to enable

Web search

in the prompt field, using plus (

+

) button. Search the web ;-)

Kokoro / Open Webui 

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

https://github.com/remsky/Kokoro-FastAPI?tab=readme-ov-file

apt update

apt upgrade

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg   && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list |     sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' |     sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

apt install docker.io -y

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:v0.2.2

http://localhost:8880/v1

af_bella

Import fabric prompts

https://raw.githubusercontent.com/freeload101/Python/46317dee34ebb83b01c800ce70b0506352ae2f3c/Fabric_Prompts_Open_WebUI_OpenWebUI.py