IEEE Computer Architecture Letters· 2025Q2
Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
- 7citations
- Q2SCImago
- 2025year
Short summary
COSMOS offloads Approximate Nearest Neighbor Search (ANNS) entirely to CXL memory devices, achieving up to 6.72x higher throughput than baseline CXL systems by integrating general-purpose cores and using rank-level parallel distance computation.
AI-generated from the title and abstract; the full text is not read.
Key points
- COSMOS integrates general-purpose cores within CXL memory devices for full ANNS offload.
- It utilizes rank-level parallel distance computation to maximize memory bandwidth.
- Adjacency-aware data placement balances search loads across CXL devices.
- Evaluations show COSMOS achieves up to 6.72x higher throughput than baseline CXL systems on SIFT1B and DEEP1B traces.
AI-generated from the title and abstract; the full text is not read.
Abstract
Retrieval-Augmented Generation (RAG) is crucial for improving the quality of large language models by injecting proper contexts extracted from external sources. RAG requires high-throughput, low-latency Approximate Nearest Neighbor Search (ANNS) over billion-scale vector databases. Conventional DRAM/SSD solutions face capacity/latency limits, whereas specialized hardware or RDMA clusters lack flexibility or incur network overhead. We presentCOSMOS, integrating general-purpose cores within CXL memory devices for full ANNS offload and introducing rank-level parallel distance computation to maximize memory bandwidth. We also propose an adjacency-aware data placement that balances search loads across CXL devices based on inter-cluster proximity. Evaluations on SIFT1B and DEEP1B traces show thatCOSMOSachieves up to 6.72× higher throughput than the baseline CXL system and 2.35× over a state-of-the-art CXL-based solution, demonstrating scalability for RAG pipelines.
The authors' abstract, as published at the source. IEEE Computer Architecture Letters, 2025 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputer Science