Commit Graph

398 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek 7f488b3754 feat: experimental pgvector pgcrypto support
Deploy to HuggingFace Spaces / check-secret (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64, ubuntu-latest) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64, ubuntu-latest) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64, ubuntu-latest) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64, ubuntu-latest) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled Details
Python CI / Format Backend (3.11.x) (push) Has been cancelled Details
Python CI / Format Backend (3.12.x) (push) Has been cancelled Details
Frontend Build / Format & Build Frontend (push) Has been cancelled Details
Frontend Build / Frontend Unit Tests (push) Has been cancelled Details
Deploy to HuggingFace Spaces / deploy (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-main-images (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-cuda126-images (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Has been cancelled Details
2025-06-09 18:14:33 +04:00
Timothy Jaeryang Baek 7953aa5763 refac 2025-06-09 17:43:22 +04:00
Timothy Jaeryang Baek d1be1de013 fix: typo 2025-06-09 01:50:10 +04:00
Timothy Jaeryang Baek 0cd400f5ee refac: docling picture describe params 2025-06-08 20:02:14 +04:00
Tim Jaeryang Baek 6bf393a480
Merge pull request #14787 from vaclcer/vaclavs-custom-docling
feat: Customize Docling's "Describe Pictures" feature
2025-06-08 19:02:36 +04:00
Vaclav Cerny 99f05561f8 Add configuration options for picture description modes and update related components 2025-06-08 16:30:26 +02:00
Duncan Ogilvie 7010393a41 Change default CORS_ALLOW_ORIGIN to '*'
The local development setup defaults do not actually work currently.
2025-06-08 15:16:51 +02:00
Timothy Jaeryang Baek f17bfb3adb refac 2025-06-05 01:11:31 +04:00
Timothy Jaeryang Baek 93236cead0 refac 2025-06-05 00:46:39 +04:00
Tim Jaeryang Baek 7c4f261aa2
Merge pull request #14616 from Davixk/feat/new-perplexity-options
feat: add Perplexity AI model and search context usage configuration options
2025-06-05 00:28:00 +04:00
Timothy Jaeryang Baek 185249623b feat: follow ups backend integration 2025-06-03 18:47:49 +04:00
Timothy Jaeryang Baek 9e49fbc8bf feat: follow ups 2025-06-03 18:07:29 +04:00
Dave 96e9bfe0e5 feat: add Perplexity model and search context usage configuration options 2025-06-03 00:19:08 +02:00
Timothy Jaeryang Baek 2c15f8e676 refac
Deploy to HuggingFace Spaces / check-secret (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Has been cancelled Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Has been cancelled Details
Python CI / Format Backend (3.11.x) (push) Has been cancelled Details
Python CI / Format Backend (3.12.x) (push) Has been cancelled Details
Frontend Build / Format & Build Frontend (push) Has been cancelled Details
Frontend Build / Frontend Unit Tests (push) Has been cancelled Details
Deploy to HuggingFace Spaces / deploy (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-main-images (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Has been cancelled Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Has been cancelled Details
2025-05-31 15:07:28 +04:00
Timothy Jaeryang Baek e1e2c096e2 refac: PLEASE follow existing convention 2025-05-30 00:34:18 +04:00
Tim Jaeryang Baek ff353578db
Merge pull request #14370 from daw/feat/add-azure-openai-embeddings-option
feat:Add Azure OpenAI embedding support
2025-05-30 00:18:55 +04:00
Timothy Jaeryang Baek cb4299eb98 refac 2025-05-29 02:33:40 +04:00
Hisma a9405cc101 feat: Marker api content extraction support 2025-05-27 00:44:07 -04:00
Tim Jaeryang Baek e663b90a9f
Merge pull request #14069 from Ithanil/bm25_weight
feat: Configurable weight for BM25Retriever during hybrid search
2025-05-24 01:13:03 +04:00
Jan Kessler e70dd33233
rename BM25_WEIGHT -> HYBRID_BM25_WEIGHT 2025-05-23 22:06:44 +02:00
Timothy Jaeryang Baek 2eca6f6414 feat: bypass web loader in web search
Co-Authored-By: Perry Li <peiyaoli@mail.nankai.edu.cn>
Co-Authored-By: WilliamGates <3852641+williamgateszhao@users.noreply.github.com>
2025-05-23 02:30:35 +04:00
Jan Kessler b5ddaf6417
make weight for bm25 retriever in hybrid search ui-configurable 2025-05-20 10:39:31 +02:00
Derek Wischusen 42be1f956a Add Azure OpenAI embedding support 2025-05-19 22:58:04 -04:00
Timothy Jaeryang Baek 2bd7db12a2 enh: ALLOWED_FILE_EXTENSIONS ui 2025-05-16 21:05:52 +04:00
Kiet Trinh 418ac1a8da refac: Rename Qdrant multi-tenancy variable for improved clarity and consistency 2025-05-15 09:09:24 +00:00
Kiet Trinh 485bd7666c fix: Update Qdrant multi-tenancy variable name for consistency in configuration 2025-05-15 08:02:58 +00:00
LoiTra 184d8dfd7e
feat: Implement Qdrant multi-tenancy support with collection management and tenant isolation 2025-05-15 11:28:06 +07:00
Timothy Jaeryang Baek 0a8cecfbfa feat: response watermark
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Python CI / Format Backend (3.11.x) (push) Waiting to run Details
Python CI / Format Backend (3.12.x) (push) Waiting to run Details
Frontend Build / Format & Build Frontend (push) Waiting to run Details
Frontend Build / Frontend Unit Tests (push) Waiting to run Details
2025-05-14 23:53:28 +04:00
Timothy Jaeryang Baek 39b06dc3b6 refac 2025-05-14 23:13:06 +04:00
Tim Jaeryang Baek e358a66fb1
Merge pull request #13852 from fizyr-forks/self-signed-cert
feat: Add option to disable certificate validation for LDAP certificates.
2025-05-14 23:12:24 +04:00
Timothy Jaeryang Baek a6624a4b16 feat: `RAG_ALLOWED_FILE_EXTENSIONS` 2025-05-14 23:06:33 +04:00
Timothy Jaeryang Baek 8732b64b6b feat: external document loader support 2025-05-14 22:28:40 +04:00
Timothy Jaeryang Baek 74ccec2812 refac 2025-05-14 21:39:17 +04:00
Tim Jaeryang Baek 29ae231ee0
Merge pull request #13889 from Classic298/pendingtext
feat: Custom Title and Text for Account Activation Screen for pending users
2025-05-14 21:28:16 +04:00
Timothy Jaeryang Baek de70d0cb64 feat: docling do picture description support 2025-05-14 21:26:49 +04:00
DrMelone bdb3778f8b fix 2025-05-14 18:10:23 +02:00
DrMelone 143cb4cc34 feat custom text and title 2025-05-14 17:58:20 +02:00
Hans Gaiser f63e76e653 Add option to disable certificate validation. 2025-05-13 17:38:06 +02:00
Timothy Jaeryang Baek d5fd3b3600 feat: external reranker
Co-Authored-By: Brendan Campbell <20541191+bcambs09@users.noreply.github.com>
2025-05-10 18:25:20 +04:00
Timothy Jaeryang Baek cc1dc6cb78 refac: config 2025-05-10 17:08:09 +04:00
Tim Jaeryang Baek e60fa7927d
Merge pull request #13604 from Acters/dev
feat: Improve Default Code Interpreter Prompt
2025-05-08 22:14:29 +04:00
Tim Jaeryang Baek 1fea4f794f
Merge pull request #13670 from HarrisonConsulting/fix/milvus-standalone-index
fix: enhance MilvusClient with dynamic index type and improved logging
2025-05-08 22:08:23 +04:00
Timothy Jaeryang Baek 08b6ea0e17 refac 2025-05-08 20:47:41 +04:00
Tim Jaeryang Baek 9509528deb
Merge pull request #13703 from hurxxxx/fix/onedrive-org-picker-error
fix: Add SharePoint tenant ID to OneDrive settings
2025-05-08 20:41:06 +04:00
hurxxxx 4ca0ea1e55 Add SharePoint tenant ID to OneDrive settings 2025-05-09 00:35:23 +09:00
Matt Harrison 4abe2cfa7b fix: Specified "HNSW" for backwards compatibility prefering cluster environments over standalone. 2025-05-08 07:21:34 -04:00
silentoplayz 994f17b430
Update config.py
Update DEFAULT_TITLE_GENERATION_PROMPT_TEMPLATE
2025-05-08 07:31:51 +00:00
Matt Harrison 2b8be86f23 feat: add Milvus configuration options for index and metric types 2025-05-07 23:32:58 -04:00
Tim Jaeryang Baek 410af53eca
Merge pull request #13581 from kaytwo/refreshOauthPfp
feat: refresh oauth profile picture
2025-05-07 22:06:43 +04:00
Suleiman Elkhoury 7620a6bf8c
Update config.py
Fixed S3_USE_ACCELERATE_ENDPOINT variable from "False" to "false" since boto3 expects a Python bool, not a string.
Added S3_ENABLE_TAGGING env variable to avoid incompatiblilty with Storages that does not support tagging (like Cloudflare R2)
2025-05-07 14:46:58 +02:00
acters 9c760ace80
Improve Default Code Interpreter Prompt
The change to the default "Code Interpreter" prompt makes it more clear to the LLM to not wrap the code in markdown's code block formatting. This improves the reliability of having code generated in the XML tags that will run properly.
2025-05-06 15:36:17 -06:00
Timothy Jaeryang Baek ace3303ff5 enh: `TOOL_SERVER_CONNECTIONS` env var support 2025-05-07 01:11:28 +04:00
Chris Kanich 500f4d73e1 refresh oauth profile picture 2025-05-06 11:00:35 -05:00
Bryan Berns 5aabe21cbe Add Custom Azure TTS URL 2025-05-05 22:08:48 -04:00
Timothy Jaeryang Baek 84a05bec7b enh: notes user group permission 2025-05-04 17:22:51 +04:00
Athanasios Oikonomou 657162e96d feat(ocr): add support for Docling OCR engine and language configuration
This commit adds support for configuring the OCR engine and language(s) for Docling.
Configuration can be set via the environment variables `DOCLING_OCR_ENGINE` and `DOCLING_OCR_LANG`, or through the UI.

Fixes #13133
2025-05-03 00:32:06 +03:00
Timothy Jaeryang Baek 754f631a07 feat: OAUTH_BLOCKED_GROUPS support 2025-05-02 14:47:02 +04:00
Tim Jaeryang Baek 7b014e44ee
Merge pull request #13376 from Thaniel94/add-whisper-language-constraint
feat: Added WHISPER_LANGUAGE env variable
2025-05-02 03:08:00 -07:00
nathaniel cc14aacaaa Improvements to parsing of WHISPER_LANGUAGE environment variable (Setting as empty string now equivalent to unsetting/removing WHISPER_LANGUAGE). Resolved crash caused when unset 2025-05-01 22:03:49 +01:00
nathaniel ef7acfbf3d WHISPER_LANGUAGE no longer a "PersistentConfig" variable (Was not appropriate with how WHISPER_LANGUAGE is currently configured). 2025-05-01 21:33:57 +01:00
Timothy Jaeryang Baek 5d5e351937 feat: notes scaffolding
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Python CI / Format Backend (3.11.x) (push) Waiting to run Details
Python CI / Format Backend (3.12.x) (push) Waiting to run Details
Frontend Build / Format & Build Frontend (push) Waiting to run Details
Frontend Build / Frontend Unit Tests (push) Waiting to run Details
2025-05-01 16:39:36 +04:00
Bryan Berns 6c8a9d000e Azure STT Allow Base URL & Max Speaker Setting 2025-04-30 08:51:01 -04:00
Tim Jaeryang Baek 7b863465a9
Merge pull request #13311 from stephen304/yacy-support
feat: Yacy search support
2025-04-29 08:35:10 -07:00
Timothy Jaeryang Baek 63fe426a52 chore: format 2025-04-28 17:46:05 +04:00
noexcs 4b9b24ae78 fix: The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'. 2025-04-27 21:29:13 +08:00
nathaniel 1efa708f83 Added WHISPER_LANGUAGE env variable. If set to a country's ISO2, constrains Whisper's stt to that language. Detects language as normal if unset 2025-04-27 05:58:06 +01:00
Stephen Smith 240d91d38d Add yacy config for user/pass, automatically add yacy json api path 2025-04-26 22:28:30 -04:00
Stephen Smith 0f73b96616 first pass at yacy support copied from searxng 2025-04-26 14:07:13 -04:00
Tim Jaeryang Baek d0a1a0f16b
Merge pull request #13119 from jarrod-lowe/suggestions
feat: Set default prompt suggestions from environment variable DEFAULT_PROMPT_SUGGESTIONS
2025-04-23 09:55:57 -07:00
Timothy Jaeryang Baek 2f7b5acdf8 enh: chat share & export permissions 2025-04-23 14:43:33 +09:00
Jarrod Lowe af5efe1580 Set default prompt suggestions from environment variable 2025-04-22 11:39:01 +12:00
Athanasios Oikonomou e000c56ef7 feat(vector-db): add support for Pinecone client
Adds Pinecone as a supported vector database option.

- Implements `PineconeClient` with support for common operations: `add`, `query`, `delete`, `reset`.
- Emulates namespace support using metadata filtering (`collection_name` prefix).
- Dynamically configures Pinecone via the following env vars:
  - `PINECONE_API_KEY`
  - `PINECONE_ENVIRONMENT`
  - `PINECONE_INDEX_NAME`
  - `PINECONE_DIMENSION`
  - `PINECONE_METRIC`
  - `PINECONE_CLOUD`
- Integrates cleanly with the vector DB abstraction layer.
- Includes markdown documentation under `docs/getting-started/env-configuration.md`.

BREAKING CHANGE: None
2025-04-20 11:08:51 +03:00
Tim Jaeryang Baek 87844a8042
Merge pull request #12822 from tth37/feat_external_search_loader
feat: Support for Self-Hosted/External Web Search/Loader Engines
2025-04-18 23:51:27 -07:00
Timothy Jaeryang Baek 1c18282d19 fix 2025-04-18 18:21:49 -07:00
Juan Calderon-Perez 62f87c2e38
Use default AnyIO pool size 2025-04-18 19:21:54 -04:00
Juan Calderon-Perez 19b3d88737 Add support for configuring FastAPI/AnyIO Thread Pool Size 2025-04-18 09:22:23 -04:00
Tim Jaeryang Baek fa8132903e
Merge pull request #12904 from hurxxxx/feat/onedrive-orgs
feat: support Organization and School accounts in OneDrive
2025-04-18 04:03:54 -07:00
Juan Calderon-Perez 6188c0c5b7 Add suport for Qdrant GRPC 2025-04-17 01:13:49 -04:00
Athanasios Oikonomou 575c12f80c feat: add QDRANT_ON_DISK configuration option for Qdrant integration
This commit will allow configuring the on_disk client parameter, to reduce the memory usage.
https://qdrant.tech/documentation/concepts/storage/?q=mmap#configuring-memmap-storage
Default is false, keeping vectors in memory.
2025-04-15 01:40:57 +03:00
hurxxxx 2d7062fc99 fix: onedrive orgs selection 2025-04-15 00:27:59 +09:00
Timothy Jaeryang Baek d932fc555d enh: tts/stt user permissions 2025-04-14 01:40:22 -07:00
Timothy Jaeryang Baek e7332fd6fe refac 2025-04-13 23:39:38 -07:00
tth37 839ba22c90 feat: Backend for Self-Hosted/External Web Search/Loader Engines 2025-04-14 01:49:05 +08:00
Timothy Jaeryang Baek 79dff685ea feat: multiple models chat user permissions 2025-04-12 17:37:30 -07:00
Timothy Jaeryang Baek 77b25ae36a refac 2025-04-12 16:34:20 -07:00
Timothy Jaeryang Baek 48a23ce3fe refac: web/rag config 2025-04-12 16:33:36 -07:00
tth37 5eac5960ef feat: Add frontend configuration for web loader 2025-04-12 17:13:30 +08:00
Tim Jaeryang Baek 36ac81b229
Merge pull request #12727 from decent-engineer-decent-datascientist/main
feat: add Azure AI Speech STT provider
2025-04-10 16:50:40 -07:00
priten 9a55257c5b feat: add Azure AI Speech STT provider
- Add Azure STT configuration variables for API key, region and locales
- Implement Azure STT transcription endpoint with 200MB file size limit
- Update Audio settings UI to include Azure STT configuration fields
- Handle Azure API responses and error cases consistently
2025-04-10 15:38:59 -05:00
Tim Jaeryang Baek 7aab8b9307
Merge pull request #12567 from aleprj/fix/ms-sso
fix: Make Microsoft SSO work even when custom signing keys. #12475
2025-04-10 10:11:49 -07:00
Tim Jaeryang Baek b40b50fb2d
Merge pull request #12563 from IONOS-Productivity/oidc-add-support-for-pkce
OIDC add support for PKCE
2025-04-10 09:03:31 -07:00
Youggls 3e2a6df1fb feat: Add sougou web search API for backend, add config panel in for frontend. 2025-04-10 14:51:44 +08:00
Miroslav Valcicak a5faaf388e refactor: rename persistent config variable 2025-04-09 21:22:25 +02:00
Miroslav Valcicak 68aea6bf10 feat: disable persistent config 2025-04-09 16:18:58 +02:00
aleprj 818d24e672
fix: add appid to the URL to ensure we fetch the correct parameters in all scenarios 2025-04-07 11:15:47 -03:00
Thomas Lehmann 5c658a4879 feat(config): add config OAUTH_CODE_CHALLENGE_METHOD
Add support to enable OIDC code challenge method (PKCE).
2025-04-07 14:03:52 +02:00
Timothy Jaeryang Baek 579aca65bb
Merge pull request #12477 from Ithanil/improved_rag_template
fix: more coherent default RAG template
2025-04-05 03:22:41 -07:00
Timothy Jaeryang Baek 0c0505e1cd refac 2025-04-05 04:05:52 -06:00
Jan Kessler e0da6003f1
tweak default rag template to be more coherent and improve consistency with citations 2025-04-05 11:40:48 +02:00
Timothy Jaeryang Baek c0711ba0c9 refac 2025-04-02 20:07:09 -07:00