DEV Community

Cover image for Llama2 Chatbot esay-to-build ? ๐Ÿฆ™๐Ÿ˜ฑ
Baivab Sarkar
Baivab Sarkar

Posted on • Edited on

Llama2 Chatbot esay-to-build ? ๐Ÿฆ™๐Ÿ˜ฑ

It's a repository for a medical chatbot that uses the Llama-2-7B-Chat-GGML model and the pdf The Gale Encyclopedia of Medicine. The chatbot is still under development, but it has the potential to be a valuable tool for patients, healthcare professionals, and researchers. The chatbot can be used to answer questions about medical topics, provide summaries of medical articles, and generate medical text. However, it is important to note that the chatbot is not a substitute for medical advice from a qualified healthcare professional.

Image description

Try it for FREE๐Ÿ‘‰๐Ÿ‘‰๐Ÿ‘‰: huggingface.co/Llama-2-GGML-Medical-Chatbot

Image description

๐Ÿ“š Here are some of the features of the Llama-2-7B-Chat-GGML-Medical-Chatbot:

  • It uses the Llama-2-7B-Chat-GGML model, which is a large language model (LLM) that has been fine-tuned.
    • Name - llama-2-7b-chat.ggmlv3.q2_K.bin
    • Quant method - q2_K
    • Bits - 2
    • Size - 2.87 GB
    • Max RAM required - 5.37 GB
    • Use case - New k-quant method. Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors.
    • Model: Know more about model Llama-2-7B-Chat-GGML
  • It is trained on the pdf The Gale Encyclopedia of Medicine, Volume 1, 2nd Edition, 637-page PDF, which is a comprehensive medical reference that provides information on a wide range of medical topics. This means that the chatbot is able to answer questions about a variety of medical topics.
  • This is a sophisticated medical chatbot, developed using Llama-2 7B and Sentence Transformers. Powered by Langchain and Chainlit, This bot operates on a powerful CPU computer that boasts a minimum of
    • Operating system: Linux, macOS, or Windows
    • CPU: Intelยฎ Coreโ„ข i3
    • RAM: 8 GB
    • Disk space: 7 GB
    • GPU: None (CPU only)
  • It is still under development, but it has the potential to be a valuable tool for patients, healthcare professionals, and researchers.

Image description

๐Ÿš€ Quickstart

  1. Open Git Bash.
  2. Change the current working directory to the location where you want the cloned directory.
  3. Type git clone, and then paste the URL you copied earlier. ```

git clone https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot.git

Press Enter to create your local clone.
4. Install the pip packages in requirements.txt
Enter fullscreen mode Exit fullscreen mode

pip install -r requirements.txt

5. Now run it!
Enter fullscreen mode Exit fullscreen mode

chainlit run model.py -w



![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ff39l5g3if62ftm0lr9.jpeg)
## ๐Ÿ“– ChatBot Conversession
### โ›“๏ธChainlit ver. on [#v1.0.1.dev20230913](https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/releases/tag/v1.0.1.dev20230913)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rlafb9c4mxegzahjrbxg.png)

### โšกStreamlit ver. on [#v2.0.1.dev20231230](https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/releases/tag/v2.0.1.dev20231230)
![ChatBot Conversession img-4](https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/assets/109382325/583002bd-22a7-4ff0-8982-6067c1bcaade)

### Video of Conversation
https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/assets/109382325/6756fb69-40c0-4d49-b392-aa6906dca786

### Long Chats
![ChatBot Conversession img-2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7vidnv2djynyvkzxhmkh.png)

![ChatBot Conversession img-3](https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/assets/109382325/d10d949f-37e5-4ec4-868d-2e62d8ad69dc)

### PDF 
https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot/blob/main/conversession%20e.g/ChatBot%20Conversession%20img-3.pdf


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g0fttj6hsaebhvht0h61.gif)
## ๐Ÿ“Œ Important Notes

- While powerful, this chatbot isn't a substitute for professional medical advice.


Enter fullscreen mode Exit fullscreen mode

Top comments (0)