RAG Node Documentation

The RAG Node generates more relevant responses from a Language Learning Model (LLM) by using Retrieval Augmented Generation (RAG). This approach incorporates additional context retrieved from a vector database to improve the accuracy and relevance of the generated text.

The node allows users to configure parameters such as the embedding model, generative model, and query specifics to tailor the response generation process to their needs. By leveraging RAG, users can efficiently create context-aware, concise, and helpful responses, making it a valuable tool for projects requiring advanced AI-driven text generation.

Features

What can I build?

1. Develop AI-driven chatbots that provide context-aware responses using RAG for improved user interactions.
2. Create automated customer support systems that deliver precise answers based on user queries and contextual data retrieval.
3. Build educational tools that generate tailored learning content and explanations by leveraging context from a vector database.
4. Design content creation platforms that use RAG to produce more relevant and concise articles or reports based on user input.

Setup

Select the RAG Node

1. Fill in the required parameters.
2. Build the desired flow
3. Deploy the Project
4. Click Setup on the workflow editor to get the automatically generated instruction and add it in your application.

Configuration Reference

Low-Code Example

nodes:
  - nodeId: RAGNode_666
    nodeType: RAGNode
    nodeName: RAG
    values:
      systemPrompt: >-
        You are a helpful AI assistant that answers user queries based on the
        context provided to you.
      vectorDB: IndexCrawler
      queryField: "{{triggerNode.output.topic}}"
      limit: "3"
      certainty: "0.7"
      filters: |-
        {
          "operator": "And",
          "operands": [
            {
              "path": [
                "title"
              ],
              "operator": "Equal",
              "valueText": "Test"
            }
          ]
        }
      messages: "[]"
      embeddingModelName:
        provider_name: mistral
        type: embedder/text
        credential_name: Mistral API
        credentialId: 32bf5e3b-a8fc-4697-b95a-b1af3dcf7498
        model_name: mistral/mistral-embed
      generativeModelName:
        provider_name: mistral
        type: generator/text
        credential_name: Mistral API
        credentialId: 32bf5e3b-a8fc-4697-b95a-b1af3dcf7498
        model_name: mistral/mistral-large-2402
    needs:
      - triggerNode

Output

modelResponse

• The AI-generated text response based on the retrieved information and reasoning.

references

• A list of source documents or data points used to generate the response.
  • _additional: Contains extra metadata about the reference.
  • certainty: A confidence score (float) indicating how relevant the reference is.
  • data: The extracted content from the reference (can be a string or null).
  • rr: An additional attribute or identifier, which can be null.
  • certainty (duplicate at the top level): Same as the one inside _additional.

_meta

• Metadata about the model’s processing of the request.

  • prompt_tokens: Number of tokens in the input prompt.

  • completion_tokens: Number of tokens in the AI’s response.

  • total_tokens: Sum of prompt_tokens and completion_tokens.

  • prompt_tokens_details: Breakdown of token usage in the prompt.

    • cached_tokens: Tokens reused from cache.
    • audio_tokens: Tokens from audio input (if applicable).

  • completion_tokens_details: Breakdown of token usage in the generated output.

    • reasoning_tokens: Tokens used for reasoning.
    • audio_tokens: Tokens from audio output (if applicable).
    • accepted_prediction_tokens: Tokens from accepted predictions.
    • rejected_prediction_tokens: Tokens from rejected predictions.

  • model_name: Name of the AI model used.

  • model_provider: The provider or organization supplying the model.

Example Output

{
    "modelResponse": "Response",
    "references": [
      {
        "_additional": {
          "certainty": 0.8730262517929077
        },
        "data": null,
        "rr": "wr",
        "certainty": 0.8730262517929077
      },
      {
        "_additional": {
          "certainty": 0.8614917993545532
        },
        "data": "info",
        "rr": null,
        "certainty": 0.8614917993545532
      }
    ],
    "_meta": {
      "prompt_tokens": 112,
      "completion_tokens": 111,
      "total_tokens": 223,
      "prompt_tokens_details": {
        "cached_tokens": 0,
        "audio_tokens": 0
      },
      "completion_tokens_details": {
        "reasoning_tokens": 0,
        "audio_tokens": 0,
        "accepted_prediction_tokens": 0,
        "rejected_prediction_tokens": 0
      },
      "model_name": "gpt-4-turbo",
      "model_provider": "openai"
    }
  }

Troubleshooting

Common Issues

Debugging

1. Check Lamatic Flow logs for error details.
2. Verify API Key.