Hello, let's use RAG today. We're working on report creation. It's tough to keep up with recent articles on quantum computing and generative AI, especially when you need to refer back to past articles or revisit previous knowledge. This time, let's see if we can use RAG for creating reports.
We'll be using Mistral, Langchain, and Gradio for this project.
pip install --quiet transformers accelerate langchain langchain-community sentence-transformers faiss-gpu pypdf gradio
from transformers import AutoTokenizer, pipeline
model_id = "mistralai/Mistral-7B-Instruct-v0.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model_id, tokenizer=tokenizer, device=0, max_new_tokens=300)
query = 'what is sandbox and softbank doing on quantum business?'
pipe(query)
Sandbox is a subsidiary of SoftBank, and they are indeed working on quantum computing. They have a quantum computing division called Quantum Matter Inc.
It gave a completely nonsensical answer, clearly wrong and full of hallucinations. Next, as a practice, we will load a website.
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings.huggingface import HuggingFaceEmbeddings
from langchain_text_splitters import CharacterTextSplitter
loader = WebBaseLoader(["https://www.digicert.com/jp/faq/cryptography/what-is-post-quantum-cryptography#:~:text=耐量子暗号方式（量子,指している用語です。", "https://www.sbbit.jp/article/cont1/85249", "https://quantumcomputingreport.com/softbank-leverages-sandboxaqs-aqtive-guard-to-identify-it-infrastructure-vulnerabilities/"])
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
embeddings = HuggingFaceEmbeddings(
model_name="intfloat/multilingual-e5-large"
)
db = FAISS.from_documents(docs, embeddings)
retriever = db.as_retriever()
print(db.index.ntotal)
We will set it up to retrieve relevant information from several websites.
The index number has become 33.
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain.llms import HuggingFacePipeline
llm = HuggingFacePipeline(pipeline=pipe)
template = """次のコンテキストを踏まえた上で日本語で答えて下さい:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
def format_docs(docs):
return "\n\n".join([d.page_content for d in docs])
chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
query = 'What is the name of quantum computing product provided by sandbox in this article?'
response = chain.invoke(query)
if "Answer:" in response:
response = response.split("Answer: ")[1]
if "Question:" in response:
response = response.split("Question: ")[0]
if "Japanese Translation:" in response:
response = response.split("Japanese Translation: ")[1]
response = response.replace("\n\n", "")
response
I have tried changing the query.
'AQtive Guard is the name of the quantum computing product provided by Sandbox in this article. It is a cryptography management platform that helps identify IT infrastructure vulnerabilities and supports compliance with NIST initiatives on post-quantum cryptography.'
It provided a very detailed explanation.
import gradio as gr
import os
def add_text(history, text):
history = history + [(text, None)]
return history, gr.Textbox(value="", interactive=False)
def bot(history):
query = history[-1][0]
response = chain.invoke(query)
if "Answer:" in response:
response = response.split("Answer:")[1]
if "Question:" in response:
response = response.split("Question:")[0]
if "Japanese Translation:" in response:
response = response.split("Japanese Translation:")[1]
response = response.replace("\n\n", "")
history[-1][1] = ""
for character in response:
history[-1][1] += character
yield history
with gr.Blocks() as demo:
chatbot = gr.Chatbot([])
with gr.Row():
txt = gr.Textbox(
scale=4,
show_label = False,
container = False
)
clear = gr.Button("Clear")
txt_msg = txt.submit(add_text, [chatbot, txt], [chatbot, txt], queue = False).then(bot, chatbot, chatbot)
txt_msg.then(lambda: gr.Textbox(interactive = True), None, [txt], queue = False)
clear.click(lambda: None, None, chatbot, queue=False)
demo.queue()
demo.launch(share=True)
Gradio is quite simple, isn't it?
We have created a nice interface. It looks like we'll be able to handle full-fledged report creation in the future. It's helpful that it even explains related terms thoroughly. That's all.