Generative AI (GenAI) is transforming industries by enabling machines to create human-like content across text, images, and beyond. This revolutionary technology is not only powering innovative applications in creative fields but is also making significant inroads into domains such as finance, healthcare, and customer service. With GenAI, organizations can harness advanced models that generate context-rich outputs, augment traditional data retrieval methods, and deliver personalized insights, fundamentally reshaping how we interact with information and make decisions.
Relying on substandard or poorly curated datasets to train LLMs poses significant risks. Without a solid foundation of accurate, representative data, these models may produce outputs that are both factually incorrect and misleading—a phenomenon known as hallucinations. Hallucinations occur when the model invents details or delivers confidently erroneous information that isn’t grounded in its training data. Such inaccuracies can drive poor decision-making, particularly in high-stakes fields like finance, healthcare, or legal services, where precision and trust are critical. Therefore, maintaining dataset integrity is essential for mitigating these risks and building reliable AI systems.
Mitigating Risks with GenAI Patterns: Leveraging Embeddings, RAG, and Query Transformation to Prevent Hallucinations and Ensure Reliable Outputs
Mitigating risks in GenAI systems is achievable by leveraging key design patterns that address the inherent challenges of training on unreliable data. By utilizing Embedding Generation, raw financial text is transformed into meaningful vector representations, ensuring that the semantic nuances of the data are captured accurately. In parallel, Retrieval Augmented Generation (RAG) enriches LLM prompts with context retrieved from a vector store, thereby bridging the gap in static knowledge bases and reducing the likelihood of hallucinations. Additionally, Query Transformation refines user queries to align seamlessly with the underlying document embeddings, enhancing the retrieval of relevant information. Together, these patterns work synergistically to prevent the generation of misleading outputs and to ensure the delivery of reliable, context-aware responses.
Embeddings: From Raw Data to Meaningful Vectors
One of the foundational patterns discussed by Fowler is the use of embeddings. In his article, he explains that embeddings transform large data blocks (whether text or images) into numeric vectors in a high-dimensional space so that items with similar semantic meaning end up near each other. In our application, this concept is critical for enabling similarity search across financial documents and news.
Embeddings implementation with SpringAI
Tokenization & Splitting
The saveVectorDb method begins by using a TokenTextSplitter to break down a list of content strings into smaller chunks, converting each into a Document instance:
List<Document> documentList = textSplitter.apply( contentList.stream().map(Document::new).collect(Collectors.toList());
This aligns with the idea of chunking large documents into manageable pieces—a common practice to preserve contextual details in embeddings.
Embedding Generation
For each document, the method calls:
embeddingModel.embed(document);
This invocation transforms the document text into an embedding vector. The vector is then stored in the vector store for later similarity searches. This step reflects the core purpose of embeddings in our GenAI architecture.
Metadata Association
Beyond generating embeddings, the method attaches additional metadata:
document.getMetadata().put("metadata", metadata);
Instead of redundantly storing the embedding in the metadata, this field is used to store auxiliary information (such as document type or indexing timestamp) that can be used to filter or display results later. Fowler’s discussion on structuring the knowledge base stresses the importance of separating data (embeddings) from contextual information (metadata).
Retrieval Augmented Generation (RAG): Enriching Prompts with Relevant Context
Martin Fowler’s article describes RAG as a way to overcome the limitations of LLMs by augmenting user prompts with relevant document fragments retrieved from an external knowledge base. This approach helps the LLM produce answers that are factually grounded and contextually appropriate.
Implementing RAG with SpringAI
Constructing an Advised Request
The method retrieveRelevantDocuments builds an AdvisedRequest using the user query and a configured chat model:
AdvisedRequest request = AdvisedRequest.builder()
.chatModel(chatModel)
.userText(query)
.build();
This request serves as the input for the retrieval augmentation process.
Using the RetrievalAugmentationAdvisor
The request is then passed to the advisor:
AdvisedRequest advisedRequest = retrievalAugmentationAdvisor.before(request);
Under the hood, this advisor orchestrates a series of operations:
- Query Transformation:
The advisor uses a RewriteQueryTransformer (created in the private method createQueryTransformers()) to refine or rephrase the query. This transformation is crucial for overcoming the “minimalistic user query” problem discussed by Fowler.
- Document Retrieval:
A VectorStoreDocumentRetriever (built in createDocumentRetriever()) searches the vector store using the refined query’s embedding. It applies a similarity threshold (set to 0.65 in our implementation) and limits the number of retrieved documents to the top 5.
- Query Augmentation:
Finally, a ContextualQueryAugmenter (from createQueryAugmenter()) supplements the original query with the context retrieved from the vector store. In our implementation, the augmenter is configured with allowEmptyContext(true), meaning that even if no documents are deemed sufficiently similar, the original query is preserved instead of being overridden by a fallback message.
- Extracting the Retrieved Documents:
After the advisor has processed the request, the method extracts the retrieved documents from the context:
Object contextValue = advisedRequest.adviseContext().get(RetrievalAugmentationAdvisor.DOCUMENT_CONTEXT);
if (contextValue instanceof List<?> documents) {
return (List<Document>) documents;
}
return List.of();
This final step directly implements the RAG pattern: enriching the query with retrieved context before passing it on to the LLM for answer generation.
Hybrid Retriever and Query Transformation: Enhancing Search Efficiency
Fowler’s discussion on the Hybrid Retriever pattern emphasizes that while dense embeddings are powerful, they can be complemented by traditional keyword or BM25 searches to overcome their limitations. Although our implementation in AIFinancialRepository currently focuses on vector-based retrieval, the integration of a query transformer shows an appreciation for combining multiple retrieval strategies.
Query Transformer
The private method createQueryTransformers() builds a RewriteQueryTransformer using a ChatClient. This transformer adapts user queries to better match the representation of the documents in the vector store:
private QueryTransformer createQueryTransformers() {
ChatClient chatClient = ChatClient.builder(chatModel).build();
return RewriteQueryTransformer.builder()
.chatClientBuilder(chatClient.mutate())
.build();
}
By rewriting the query, the system addresses ambiguities and refines the search input, increasing the chances of retrieving relevant documents.
VectorStoreDocumentRetriever
The method createDocumentRetriever() constructs a document retriever that searches the vector store using similarity thresholds:
private VectorStoreDocumentRetriever createDocumentRetriever() {
return VectorStoreDocumentRetriever.builder()
.vectorStore(vectorStore)
.similarityThreshold(0.65)
.topK(5)
.build();
}
This component implements the core of the hybrid retrieval strategy by ensuring that only the most semantically similar document fragments are returned.
AI Financial Analyst Design System
Comparative Analysis of AI Financial Analyst Implementations
In our journey to develop a robust AI Financial Analyst, we experimented with two distinct implementations that leverage different generative AI models and embedding algorithms. This section compares these two approaches in light of Martin Fowler’s GenAI patterns and our architectural goals.
Version 1: OpenAI’s GPT4o with text-embedding-ada-002
The first version of our application was built around OpenAI’s GPT4o model—a general-purpose large language model designed for a wide range of natural language tasks. GPT4o excels in generating coherent, contextually rich responses across various domains. Its strength lies in its versatility and the ability to handle diverse queries, making it a strong candidate for customer-facing applications where broad knowledge and adaptability are critical.
To complement GPT4o, we employed OpenAI’s text-embedding-ada-002 algorithm for generating embeddings. This algorithm transforms raw financial text into dense, high-dimensional vectors that capture semantic meaning. Text-embedding-ada-002 is known for its cost efficiency, robust performance, and general applicability across tasks such as similarity search, clustering, and semantic comparison. However, while it performs admirably in generating embeddings for general-purpose retrieval, its design is not explicitly optimized for deep reasoning or domain-specific nuances.
Version 2: DeepSeek R1 with nomic-embed-text on Ollama
The second implementation pivots to a locally hosted solution using DeepSeek R1 a model tailored for enhanced reasoning capabilities. Running on the Ollama platform, DeepSeek R1 was selected for scenarios where precise, domain-specific reasoning is paramount, such as detailed financial analysis. Unlike the more general-purpose GPT4o, DeepSeek R1 is fine-tuned to process complex queries and provide reasoned responses that align closely with the intricate requirements of financial analytics.
For embedding generation in this version, we use the nomic-embed-text algorithm. This embedding algorithm is designed to complement the reasoning strengths of DeepSeek R1, focusing on extracting the nuanced relationships within financial texts. While text-embedding-ada-002 offers broad semantic representation, nomic-embed-text is optimized for scenarios where contextual reasoning is essential, ensuring that the resultant embeddings preserve subtle semantic details crucial for informed decision-making.
Benchmark Summary:
- Generality vs. Specialization
GPT4o is a general-purpose model suited for a wide array of tasks, making it versatile for various applications. In contrast, DeepSeek R1 is specialized for reasoning, offering enhanced performance in scenarios that demand a higher level of analytical depth.
- Embedding Algorithms
Text-embedding-ada-002 provides robust and cost-effective embeddings that work well for generic semantic similarity tasks. On the other hand, nomic-embed-text is tailored to support DeepSeek R1’s reasoning abilities, potentially offering richer representations in context-specific financial analysis.
- Deployment Considerations
The GPT4o-based implementation leverages cloud-based, managed services from OpenAI, which can simplify scaling and maintenance. The DeepSeek R1 solution, running locally via Ollama, offers greater control over data and model tuning, which is beneficial for organizations that require enhanced privacy or need to fine-tune the model for domain-specific tasks.
- Database Setup
Both implementations leverage PGVector as the underlying vector database but differ in key configuration parameters that directly impact the effectiveness of the retrieval process.
- Distance Type (COSINE_DISTANCE)
This setting indicates that the similarity between vectors is measured using cosine distance, which evaluates the cosine of the angle between two vectors. It is a widely used metric in embedding spaces because it emphasizes the directional similarity rather than the magnitude of the vectors.
Index Type (HNSW)
The Hierarchical Navigable Small World (HNSW) algorithm is employed for indexing. HNSW is an approximate nearest neighbor search method optimized for high-dimensional spaces. It provides fast and scalable retrieval, making it ideal for applications that require efficient similarity search among thousands or millions of vectors.
- Dimensions
A crucial difference between the two versions is the dimensionality of the embeddings.
The DeepSeek R1 implementation uses .dimensions(1024), reflecting that the nomic-embed-text embedding algorithm produces 1024-dimensional vectors.
In contrast, the GPT4o implementation uses .dimensions(1536), as the text-embedding-ada-002 algorithm generates higher-dimensional (1536) embeddings.
This difference affects the granularity and capacity of the embeddings to capture semantic nuances, with the higher dimension potentially encoding more detail but also demanding stricter similarity criteria.
- Similarity Threshold
The document retriever’s configuration also varies between the two implementations:
For DeepSeek R1, the similarity threshold is set at .similarityThreshold(0.65), which allows documents with a cosine similarity above 0.65 to be considered relevant.
For GPT4o, the threshold is higher at .similarityThreshold(0.82), demanding a closer match between the query embedding and stored document embeddings.
This stricter threshold in GPT4o likely compensates for the richer, more detailed 1536-dimensional embeddings, ensuring that only highly relevant documents are retrieved.
- Output Results
After we did the ingestion data related News and Sentimentals about bitcoin we did this question.
{
"question": "What the sentiment and price expectations for Bitcoin in the next days. Make a deep analysis."
}
Response for GPT4o
Analysis Type: Market Trend and Sentiment Analysis for Bitcoin
Key Metric/Insight: The primary focus is on Bitcoin's price movement, trading volume, and market sentiment indicators. Key metrics include recent price changes, trading volume fluctuations, and sentiment scores derived from social media and news sources.
Source & Reliability: Data is sourced from cryptocurrency exchanges for price and volume, and sentiment data is aggregated from social media platforms and news articles. The reliability of price and volume data is high due to real-time updates, while sentiment data is moderately reliable, subject to rapid changes in public opinion.
Reasoning/Impact: Bitcoin's price is highly sensitive to market sentiment, which can drive significant volatility. Positive sentiment often leads to increased buying pressure, potentially driving prices up, while negative sentiment can result in selling pressure and price declines. Understanding sentiment helps anticipate short-term price movements.
Data Comparisons: Recent data shows Bitcoin's price has been fluctuating within a specific range, with a slight upward trend over the past week. Trading volume has been consistent, indicating steady interest. Sentiment analysis reveals a neutral to slightly positive sentiment, with increased mentions of Bitcoin in the context of potential regulatory developments.
Historical Context/Comparative Insight: Historically, Bitcoin's price tends to experience increased volatility during periods of heightened regulatory discussions or macroeconomic changes. Comparatively, the current sentiment is more positive than during previous regulatory scares, suggesting a potential for price stability or moderate growth.
Final Opinion: Based on the current metrics and sentiment analysis, the short-term outlook for Bitcoin appears cautiously optimistic. If sentiment remains stable or improves, there is potential for a modest price increase. However, investors should remain vigilant for any sudden changes in sentiment or regulatory news. Recommendation: Hold, with a watchful eye on sentiment shifts and regulatory announcements.
Data Patterns & Observations: A recurring pattern is the correlation between sentiment spikes and short-term price movements. Positive news or sentiment often precedes price increases, while negative sentiment can lead to declines. Monitoring sentiment trends can provide early indicators of potential price changes.
Additional Considerations: Keep an eye on liquidity metrics, as they can impact price volatility. Additionally, consider macroeconomic factors such as interest rates and inflation, which can influence investor behavior in the cryptocurrency market.
Response for Deepseek R1
Everything inside was how Deepseek build reasoning the answer:
<think>
Alright, so I need to figure out how to analyze Bitcoin's sentiment and price expectations for the next few days. Hmm, where do I start? Well, first off, I know that Bitcoin's price is influenced by various factors like market trends, news, regulations, and even macroeconomic indicators. But since the question is about sentiment and price expectations in the next day or two, maybe looking at technical analysis would be helpful.
I remember hearing about things like moving averages, RSI (Relative Strength Index), and Volume as part of technical indicators. If Bitcoin's price crosses above a significant moving average, that might indicate positive sentiment because it could signal strength in the market. Conversely, failing to do so might mean negative sentiment.
RSI is another one I'm somewhat familiar with. It measures how overbought or oversold an asset is. An RSI below 30 usually suggests buying pressure, while above 70 indicates selling pressure. If Bitcoin's RSI is rising, that could be a good sign for price movement in the coming days.
Volume also plays a role. High trading volume can indicate strong interest and confidence in the market. If Bitcoin sees a surge in volume alongside positive indicators like moving averages or buying signals, it might signal bullish momentum.
I should consider looking at historical data too—maybe see how Bitcoin's price has moved when these indicators were signaling positive or negative trends. This could give me some patterns to predict future movements.
Wait, but sentiment analysis can also involve news events. Recent major news about regulations in countries like the U.S. or Europe regarding Bitcoin could impact its price and sentiment. If there are upcoming significant events, that's something I should monitor.
I'm a bit unsure about how reliable these indicators are on their own. Sometimes technical indicators can give false signals, so relying solely on them might not be enough. Maybe combining them with other factors like market news or geopolitical events would provide a more accurate picture.
Also, the behavioral aspect—how retail investors behave in the markets—could influence short-term price movements. If there's a lot of buying activity due to fear or excitement, it could drive prices up or down regardless of technical signals.
Looking at specific timeframes might help too. For instance, if Bitcoin shows bullish patterns during certain times of the day, that could predict its behavior in the coming hours.
I think I need to check some sources for reliable data—maybe websites like CoinMarketCap, CoinDesk, or financial news sites that track Bitcoin trends. Also, looking into Twitter sentiment for retail investors might provide real-time insights on market-moving news.
Wait, but sentiment analysis can be subjective. How do you quantify something as intangible as market sentiment? Maybe using sentiment scores from social media posts or news articles could give a gauge of overall feelings towards Bitcoin in the market.
I should also consider the correlation between Bitcoin's price and other assets like Ethereum or traditional cryptocurrencies such as gold, silver, or Bitcoin itself through time-series analysis. This might help identify if there are any patterns or dependencies that can predict future moves.
Another point is to look at the correlation with macroeconomic indicators—like inflation rates or interest rates—if they have an impact on Bitcoin's price due to its status as a hedge against inflation.
But I'm not entirely sure how all these pieces fit together. Maybe starting with a simple moving average analysis would be a good first step, then incorporating RSI and volume data for more depth. Combining that with recent news or geopolitical developments could provide a comprehensive outlook.
I think I'll outline the steps: First, check Bitcoin's current price and key technical indicators like MA (moving averages), RSI, and volume. Then, look at any upcoming significant events that might affect its market. Analyze how sentiment is shaping up on social media or news feeds. Finally, see if there are patterns in historical data correlating these factors with future price movements.
So, putting it all together, I can predict whether Bitcoin's price is expected to go up or down based on the combination of these indicators and current market sentiments.
</think>
Answer:
**Bitcoin Sentiment and Price Expectations Analysis (Next 7 Days)**
**1. Technical Indicators:**
- **Moving Averages:** Bitcoin has recently crossed above a significant moving average, indicating bullish sentiment due to potential strength in market sentiment.
- **RSI (Relative Strength Index):** The RSI is currently below 30, signaling buying pressure and positive sentiment among investors.
- **Volume:** High trading volume indicates strong market interest and confidence, supporting the bullish trend.
**2. News and Events:**
- Monitor recent news from major countries like the U.S., U.K., and European Union regarding Bitcoin regulations, as these could impact sentiment and price movements.
**3. Behavioral Analysis:**
- Consider retail investor behavior—patterns of buying during specific times of the day that might influence short-term price actions.
**4. Sentiment Sources:**
- Analyze social media sentiment, particularly from platforms like Twitter where retail investors often share market-moving news.
**5. Correlation with Other Assets and Indicators:**
- Examine correlations between Bitcoin's price and other assets (Ethereum, gold) or macroeconomic indicators (inflation rates) for potential predictive patterns.
**6. Historical Context:**
- Review historical data to identify patterns where these indicators coincided with future price movements, enhancing the reliability of predictions.
**Conclusion:**
Combining these factors—positive technical signals, bullish sentiment, and supportive news—it is recommended that Bitcoin's price may experience an upward trend in the next 7 days. However, this analysis should be closely monitored as market dynamics can change rapidly due to external shocks or shifts in investor behavior.
Final Consideration
After gathering responses from each AI system, I posed the same question to OpenAI O3-mini, DeepSeek R1, and Gemini from Google. I selected the answer from OpenAI for the OpenAI O3-mini system, and for both DeepSeek R1 and Gemini, I opted for the DeepSeek response. In effect, this results in a 2-to-1 preference in favor of DeepSeek.
Observation
Since the DeepSeek R1 model is running locally via Ollama, both the process of converting data into embedding vectors and the retrieval process experienced higher latency compared to cloud-based solutions like OpenAI’s GPT4o implementation.
Top comments (0)