Commit Graph

274 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek 5e35aab292 chore: format
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda126-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda126-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda126-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Frontend Build / Format & Build Frontend (push) Waiting to run Details
Frontend Build / Frontend Unit Tests (push) Waiting to run Details
Python CI / Format Backend (3.11.x) (push) Has been cancelled Details
Python CI / Format Backend (3.12.x) (push) Has been cancelled Details
2025-06-05 01:12:28 +04:00
Tim Jaeryang Baek 7c4f261aa2
Merge pull request #14616 from Davixk/feat/new-perplexity-options
feat: add Perplexity AI model and search context usage configuration options
2025-06-05 00:28:00 +04:00
Vaclav Cerny 9772c18b20 fix(loader): remove deprecated picture description configuration 2025-06-04 17:21:44 +02:00
Vaclav Cerny c71236ba07 feat(loader): enhance picture description prompt for improved detail and clarity 2025-06-04 14:25:31 +02:00
Vaclav Cerny c4278f4784 fix description vs classification mismatch 2025-06-04 14:13:00 +02:00
Vaclav Cerny 8644e81a1c feat(loader): add picture description configuration for DoclingLoader 2025-06-04 12:34:39 +02:00
Timothy Jaeryang Baek 4d364e2967 refac: remove msg from known type 2025-06-03 16:27:28 +04:00
Dave 77b357c73b fix: update label for search context usage to clarify its purpose 2025-06-03 00:27:07 +02:00
Dave 96e9bfe0e5 feat: add Perplexity model and search context usage configuration options 2025-06-03 00:19:08 +02:00
Tim Jaeryang Baek 3c32d2cada
Merge pull request #14539 from PVBLIC-F/refac/mistral
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Python CI / Format Backend (3.11.x) (push) Waiting to run Details
Python CI / Format Backend (3.12.x) (push) Waiting to run Details
perf mistral.py Enhance for Overall Speed and Efficiency
2025-06-02 23:52:59 +04:00
PVBLIC Foundation cf3635ba25
Update mistral.py
1. Intelligent Error Handling
Added _is_retryable_error() method to distinguish retryable vs non-retryable errors
Prevents unnecessary retries on client errors (4xx) that won't succeed
Caps retry delay at 30 seconds to prevent excessive waiting
2. Optimized Timeout Configuration
Upload: Capped at 2 minutes (was using full 5-minute timeout)
URL requests: 30 seconds (should be fast)
OCR processing: Full timeout (can take time)
Cleanup: 30 seconds (should be quick)
3. Enhanced Connection Pool
Increased connection limits: 20 total, 10 per host
Longer DNS cache TTL (10 minutes vs 5 minutes)
Increased keepalive timeout (60s vs 30s)
Added async DNS resolver for better performance
Granular timeout controls (connect, read, total)
4. Concurrency Control for Batch Processing
Added semaphore-based concurrency control (default: 5 concurrent)
Prevents API overwhelming while maintaining throughput
Configurable concurrency limit per workload
5. Memory Efficient Result Processing
Early exit for empty content validation
Better error metadata for debugging
Added content length tracking
Streamlined page processing logic
6. General Performance Improvements
Better error logging with truncated responses
Optimized metadata creation
Improved debug logging efficiency
2025-05-30 20:06:29 -07:00
PVBLIC Foundation 66bde32623
Update pinecone.py 2025-05-30 18:47:23 -07:00
PVBLIC Foundation 4ecf2a8685
Update pinecone.py
May 2025 Latest Pinecone Best Practices
2025-05-30 09:33:57 -07:00
Timothy Jaeryang Baek 9306ae5972 refac 2025-05-30 01:19:56 +04:00
Timothy Jaeryang Baek e1e2c096e2 refac: PLEASE follow existing convention 2025-05-30 00:34:18 +04:00
Tim Jaeryang Baek ff353578db
Merge pull request #14370 from daw/feat/add-azure-openai-embeddings-option
feat:Add Azure OpenAI embedding support
2025-05-30 00:18:55 +04:00
Timothy Jaeryang Baek 7dc7d5c028 refac: PLEASE FOLLOW EXISTING CONVENTION 2025-05-29 03:47:02 +04:00
Timothy Jaeryang Baek 551597b9cc chore: format 2025-05-29 02:36:33 +04:00
Hisma e12a79c0e2 fix: handle json output format correctly 2025-05-27 01:12:03 -04:00
Hisma a9405cc101 feat: Marker api content extraction support 2025-05-27 00:44:07 -04:00
Timothy Jaeryang Baek da75d0ca1e chore: format
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Python CI / Format Backend (3.11.x) (push) Waiting to run Details
Python CI / Format Backend (3.12.x) (push) Waiting to run Details
Frontend Build / Format & Build Frontend (push) Waiting to run Details
Frontend Build / Frontend Unit Tests (push) Waiting to run Details
2025-05-24 02:13:54 +04:00
Tim Jaeryang Baek e663b90a9f
Merge pull request #14069 from Ithanil/bm25_weight
feat: Configurable weight for BM25Retriever during hybrid search
2025-05-24 01:13:03 +04:00
Timothy Jaeryang Baek 8b5e89eada chore: format 2025-05-24 00:43:38 +04:00
Jan Kessler e70dd33233
rename BM25_WEIGHT -> HYBRID_BM25_WEIGHT 2025-05-23 22:06:44 +02:00
Tim Jaeryang Baek c8f1bdf928
Merge pull request #14245 from PVBLIC-F/dev
perf Update mistral.py
2025-05-23 21:57:16 +04:00
PVBLIC Foundation bf193dfb5d
Update mistral.py 2025-05-23 10:00:19 -07:00
Timothy Jaeryang Baek aac25eac9e refac: reranker
Co-Authored-By: Tornike Gurgenidze <togurg14@freeuni.edu.ge>
2025-05-23 01:29:48 +04:00
Tim Jaeryang Baek da4aa5f08b
Merge pull request #14152 from U8F69/fix_user_auth
fix(auth): correctly use password hash when duplicate email records exist
2025-05-22 14:58:10 +04:00
U8F69 dd6124a84f
fix(auth): fix invalid password use in auth 2025-05-22 11:03:43 +08:00
PVBLIC Foundation 86e24bb4aa
Update pinecone.py
I've improved the pinecone.py file by:
Updated from the deprecated PineconeGRPC client to the newer Pinecone client
Modified the client initialization code to match the new API requirements
Added better response handling with getattr() to safely access attributes from response objects
Removed the streaming_upsert method which is not available in the newer client
Added safer attribute access with fallbacks throughout the code
Updated the close method to reflect that the newer client doesn't need explicit closing
These changes ensure the code is compatible with the latest Pinecone Python SDK and will be more robust against future changes. The key improvement is migrating away from the deprecated gRPC client which will eventually stop working.
2025-05-21 15:28:42 -07:00
Tim Jaeryang Baek d3c7628092
Merge pull request #14059 from sreesdas/main
fix: resolve issue where external document loader was not invoked
2025-05-20 17:43:06 +04:00
Tim Jaeryang Baek fac5884d8c
Merge pull request #14073 from tth37/fix_default_web_loader_verify_ssl
fix: Default web loader fail silently when `verify_ssl=False`
2025-05-20 17:24:22 +04:00
tth37 78befd5a2f fix: Default web loader fail when `verify_ssl=False` 2025-05-20 19:44:18 +08:00
Jan Kessler 308d8ac04a
make bm25_weight a regular parameter of query_doc.. / get_sources_from_files functions 2025-05-20 11:46:32 +02:00
Jan Kessler b5ddaf6417
make weight for bm25 retriever in hybrid search ui-configurable 2025-05-20 10:39:31 +02:00
sree f408b08965 minor bug fix for external document loader not working 2025-05-20 11:10:23 +05:30
Derek Wischusen 42be1f956a Add Azure OpenAI embedding support 2025-05-19 22:58:04 -04:00
Marcelo Mendoza d6ad96affb fix: use get method for title and snippet in search results 2025-05-19 17:24:47 +02:00
Timothy Jaeryang Baek 6692fb2181 chore: format 2025-05-17 01:00:37 +04:00
Kiet Trinh 418ac1a8da refac: Rename Qdrant multi-tenancy variable for improved clarity and consistency 2025-05-15 09:09:24 +00:00
Kiet Trinh 485bd7666c fix: Update Qdrant multi-tenancy variable name for consistency in configuration 2025-05-15 08:02:58 +00:00
LoiTra 184d8dfd7e
feat: Implement Qdrant multi-tenancy support with collection management and tenant isolation 2025-05-15 11:28:06 +07:00
Timothy Jaeryang Baek b143c71da2 refac: AIOHTTP_CLIENT_SESSION_SSL 2025-05-14 23:33:52 +04:00
Timothy Jaeryang Baek 42382b5167 fix 2025-05-14 22:46:01 +04:00
Timothy Jaeryang Baek 8732b64b6b feat: external document loader support 2025-05-14 22:28:40 +04:00
Timothy Jaeryang Baek de70d0cb64 feat: docling do picture description support 2025-05-14 21:26:49 +04:00
hwzhuhao 6f869ded43 feat:Add vector type and vector factory class for vector database integration 2025-05-14 21:30:50 +08:00
Timothy Jaeryang Baek 6b5f99bf66 fix: external reranker
Deploy to HuggingFace Spaces / check-secret (push) Waiting to run Details
Deploy to HuggingFace Spaces / deploy (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / build-main-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-main-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-cuda-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/amd64) (push) Waiting to run Details
Create and publish Docker images with specific build args / build-ollama-image (linux/arm64) (push) Waiting to run Details
Create and publish Docker images with specific build args / merge-main-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-cuda-images (push) Blocked by required conditions Details
Create and publish Docker images with specific build args / merge-ollama-images (push) Blocked by required conditions Details
Frontend Build / Format & Build Frontend (push) Waiting to run Details
Frontend Build / Frontend Unit Tests (push) Waiting to run Details
Python CI / Format Backend (3.11.x) (push) Has been cancelled Details
Python CI / Format Backend (3.12.x) (push) Has been cancelled Details
2025-05-10 19:33:34 +04:00
Timothy Jaeryang Baek c61790b355 chore: format 2025-05-10 19:00:01 +04:00
Timothy Jaeryang Baek d5fd3b3600 feat: external reranker
Co-Authored-By: Brendan Campbell <20541191+bcambs09@users.noreply.github.com>
2025-05-10 18:25:20 +04:00