We want to cook by loading recipe data, right? You can easily create a bot using LLM.
This time, we will make cooking easier by loading recipes from ABC Cooking Studio, a shareholder of our company.
Eggplant and Ground Meat Bolognese Rice
Link to ABC Cooking Studio's recipe
https://www.abc-cooking.co.jp/plus/recipe/detail/?id=44750
Even if you don't know what to focus on just by looking at the website, you can have a conversation with AI like below, and it will tell you only the information you want from the website.
The necessary technologies are:
- Langchain (to create a general AI workflow)
- LLM (this can be a service like ChatGPT, or it can be hosted on your own local PC)
- Web crawler (this time, we will directly retrieve information from websites)
- Gradio (you can easily create web interfaces)
First, we have already checked how to create basic responses with LLM in another article.
https://blueqat.com/yuichiro_minato2/128b8855-7689-4ce1-8a4b-f10da27b8aa2
First, for this project, we loaded the model from Hugging Face. It's Mistral7B.
pip install transformers accelerate langchain langchain-community sentence-transformers faiss-gpu pypdf
from transformers import AutoTokenizer, pipeline
model_id = "mistralai/Mistral-7B-Instruct-v0.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model_id, tokenizer=tokenizer, device=0, max_new_tokens=300)
Next, although we used PDFLoader last time, we will change this part for the current project.
from langchain_community.document_loaders import PyPDFLoader
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings.huggingface import HuggingFaceEmbeddings
from langchain_text_splitters import CharacterTextSplitter
loader = PyPDFLoader("https://arxiv.org/pdf/2312.03982.pdf")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
embeddings = HuggingFaceEmbeddings(
model_name="intfloat/multilingual-e5-large"
)
db = FAISS.from_documents(docs, embeddings)
Let's change it into web based loader
from langchain_community.document_loaders import WebBaseLoader
loader = WebBaseLoader("https://www.abc-cooking.co.jp/plus/recipe/detail/?id=44750")
It's incredibly easy, isn't it? With this, the setup for the Loader is complete. Finally, regarding the chat part, including the prompts in the responses,
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain.llms import HuggingFacePipeline
llm = HuggingFacePipeline(pipeline=pipe)
template = template = """Answer the question in English based only on the following context. If there is Japanese language please translate it into English:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
def format_docs(docs):
return "\n\n".join([d.page_content for d in docs])
chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
answer = chain.invoke(query)
print(answer)
This was the basis. Next we will make web interface. To make the web interface we adopt gradio.
First, install Gradio.
!pip install gradio
And by writing Python as shown below, a web server will start up.
It launches locally, but if you set parameters inside `.launch()`, it can also generate a web address for external access.
The key point is to replace the response part in the middle with the RAG constructed with the current LLM, and that's it.
import gradio as gr
import os
def add_text(history, text):
history = history + [(text, None)]
return history, gr.Textbox(value="", interactive=False)
def bot(history):
query = history[-1][0]
response = chain.invoke(query)
history[-1][1] = ""
for character in response:
history[-1][1] += character
yield history
with gr.Blocks() as demo:
chatbot = gr.Chatbot([])
with gr.Row():
txt = gr.Textbox(
scale=4,
show_label = False,
container = False
)
clear = gr.Button("Clear")
txt\_msg = txt.submit(add\_text, \[chatbot, txt\], \[chatbot, txt\], queue = False).then(bot, chatbot, chatbot)
txt\_msg.then(lambda: gr.Textbox(interactive = True), None, \[txt\], queue = False)
clear.click(lambda: None, None, chatbot, queue=False)
demo.queue()
demo.launch()
We quickly created a RAG with an easy-to-use interface!