The generative AI tech stack is a comprehensive collection of tools, technologies, and frameworks widely used in building AI systems. It acts as the backbone for developing generative AI, providing essential guidance in transforming theoretical ideas into practical and innovative results.
Generative AI Stack are divided into 4 categories.
- Models & APIs
- Vector Databases
- LLM Frameworks
- Deployment
1. Models & APIs
2. Vector Databases
Vector databases provide a new way of structuring data that improves the performance of the database in multiple ways.
In a nutshell, vector databases store the data in the form of high-dimensional vectors which can be achieved using a transformation or an embedding function. These high-dimensional vectors allow for a faster search using a technique called semantic similarity. Instead of relying on conventional approaches such as querying databases using precise matches or predetermined criteria, a vector database allows you to locate data that is most similar or relevant by considering their semantic or contextual significance.
Some examples of vector databases are given below:
- Pinecone
- Chroma
- Qdrant
3. LLM Frameworks
LLM frameworks allow developers to build complex applications using LLMs that can execute a series of tasks and access external tools to complete complex tasks accurately. Specifically, they allow large language models to access other data sources and APIs and allow them to interact with the software’s environment.
As large language models are trained on text data, and as they don’t really have built-in capabilities to solve complex maths problems, we may want to use an external API such as WolframAlpha to solve the maths problem.
Some examples of LLM frameworks are given below:
- Langchain
- LlamaIndex
- Anarchy
4. Deployment
Finally, deploying generative AI applications such that they are scalable. As an example of such deployment infrastructure, the suite of tools provided by Microsoft in its Azure OpenAI Services. This product by Microsoft provides a single place to deploy your application in the cloud and to easily access the ChatGPT API.
Other examples of deployment infrastructure are Vertex AI by Google and HuggingFace Inference Endpoints.
To summarize, the four levels in the generative AI software stack that are given below.
Top comments (0)