first draft

[codeperfectplus]

1. Use intent detection to classify incoming queries as either general conversation or knowledge-based (requiring RAG).
2. implement ranking for the data retrieval
3. Improve context retrieval and find out the right text size of chunks.
4. After retrieving the top-k documents using an embedding-based search, assign scores to each document to assess relevance further.
5. Fine-tune models (e.g., BERT) to assign scores based on semantic similarity and contextual relevance.

Source :

1. RAGFlow