์ฝ˜ํ…์ธ  ๋Œ€ํ‘œ ์ด๋ฏธ์ง€ - ๐Ÿค– ํŒŒ์ด์ฌ + LangChain์œผ๋กœ ๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ ๊ตฌํ˜„

๐Ÿค– ํŒŒ์ด์ฌ + LangChain์œผ๋กœ ๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ ๊ตฌํ˜„

๋‚˜๋งŒ์˜ ๋˜‘๋˜‘ํ•œ AI ๋น„์„œ ๋งŒ๋“ค๊ธฐ ์™„์ „์ •๋ณต ๊ฐ€์ด๋“œ ใ…‹ใ…‹ใ…‹

๐ŸŽฏ ์š”์ฆ˜ AI ๋น„์„œ ๋งŒ๋“œ๋Š” ๊ฑฐ ์™„์ „ ํ•ซํ•˜์ž–์•„์š”?

์†”์งํžˆ ChatGPT ๋‚˜์˜ค๊ณ  ๋‚˜์„œ ๋‹ค๋“ค "๋‚˜๋„ ์ด๋Ÿฐ ๊ฑฐ ๋งŒ๋“ค ์ˆ˜ ์žˆ์„๊นŒ?" ์ƒ๊ฐํ•ด๋ดค์„ ๊ฑฐ์˜ˆ์š” ใ…‹ใ…‹ใ…‹ ๊ทผ๋ฐ ๋ง‰์ƒ ์‹œ์ž‘ํ•˜๋ ค๋‹ˆ๊นŒ ๋ญ๋ถ€ํ„ฐ ํ•ด์•ผ ํ• ์ง€ ๋ชจ๋ฅด๊ฒ ๊ณ ... ๊ทธ๋ƒฅ OpenAI API ํ˜ธ์ถœํ•˜๋Š” ๊ฒƒ๋งŒ์œผ๋กœ๋Š” ๋ญ”๊ฐ€ 2% ๋ถ€์กฑํ•˜์ž–์•„์š”? ๐Ÿค”

ํŠนํžˆ ํšŒ์‚ฌ ๋‚ด๋ถ€ ๋ฌธ์„œ ๊ฒ€์ƒ‰ํ•˜๊ฑฐ๋‚˜, ํŠน์ • ๋„๋ฉ”์ธ ์ง€์‹์„ ๊ฐ€์ง„ AI ๋น„์„œ๋ฅผ ๋งŒ๋“ค๊ณ  ์‹ถ์„ ๋•Œ๋Š” ๋‹จ์ˆœ GPT ํ˜ธ์ถœ๋งŒ์œผ๋กœ๋Š” ํ•œ๊ณ„๊ฐ€ ๋ช…ํ™•ํ•ด์š”. ์—ฌ๊ธฐ์„œ ๋“ฑ์žฅํ•˜๋Š” ๊ฒŒ ๋ฐ”๋กœ **๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ(RAG: Retrieval-Augmented Generation)** ์‹œ์Šคํ…œ์ด์—์š”! ๐Ÿ’ก

์ด ๊ธ€์—์„œ๋Š” ํŒŒ์ด์ฌ๊ณผ LangChain์„ ํ™œ์šฉํ•ด์„œ ์‹ค์ œ๋กœ ์ž‘๋™ํ•˜๋Š” ๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ๋ฅผ ์ฒ˜์Œ๋ถ€ํ„ฐ ๋๊นŒ์ง€ ๊ตฌํ˜„ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•Œ๋ ค๋“œ๋ฆด๊ฒŒ์š”. ์ฝ”๋“œ ์˜ˆ์ œ๋„ ํŒํŒ ๋„ฃ๊ณ , ์‹ค์ „ ํŒ๋„ ๊ฐ€๋“ ๋‹ด์•˜์œผ๋‹ˆ๊นŒ ๋๊นŒ์ง€ ์ง‘์ค‘ํ•ด์ฃผ์„ธ์š”~ ๐Ÿ˜Ž

์‚ฌ์šฉ์ž ์งˆ๋ฌธ LangChain์œผ๋กœ ๋ฌธ์„œ ๊ฒ€์ƒ‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์€? ๋ฒกํ„ฐ DB ๊ฒ€์ƒ‰ ๊ด€๋ จ ๋ฌธ์„œ 3๊ฐœ ์ถ”์ถœ ์™„๋ฃŒ AI ์‘๋‹ต ์ƒ์„ฑ ๊ฒ€์ƒ‰๋œ ๋ฌธ์„œ ๊ธฐ๋ฐ˜์œผ๋กœ ์ •ํ™•ํ•œ ๋‹ต๋ณ€ ์ œ๊ณต ๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ ์ž‘๋™ ํ๋ฆ„

๐Ÿ” RAG๊ฐ€ ๋ญ”๋ฐ? ์™œ ํ•„์š”ํ•œ ๊ฑฐ์•ผ?

RAG(Retrieval-Augmented Generation)๋Š” ๋ง ๊ทธ๋Œ€๋กœ "๊ฒ€์ƒ‰์œผ๋กœ ์ฆ๊ฐ•๋œ ์ƒ์„ฑ" ๊ธฐ์ˆ ์ด์—์š”. ์‰ฝ๊ฒŒ ๋งํ•˜๋ฉด AI๊ฐ€ ๋‹ต๋ณ€ํ•˜๊ธฐ ์ „์— ๋จผ์ € ๊ด€๋ จ ์ž๋ฃŒ๋ฅผ ์ฐพ์•„๋ณด๊ณ , ๊ทธ ์ž๋ฃŒ๋ฅผ ์ฐธ๊ณ ํ•ด์„œ ๋‹ต๋ณ€ํ•˜๋Š” ๋ฐฉ์‹์ด์ฃ  ๐Ÿ“š

์ผ๋ฐ˜ GPT ๋ชจ๋ธ์€ ํ•™์Šต๋œ ๋ฐ์ดํ„ฐ๊นŒ์ง€๋งŒ ์•Œ๊ณ  ์žˆ์–ด์š”. 2023๋…„ ์ดํ›„ ์ •๋ณด๋‚˜ ํšŒ์‚ฌ ๋‚ด๋ถ€ ๋ฌธ์„œ ๊ฐ™์€ ๊ฑด ์ „ํ˜€ ๋ชจ๋ฅด์ฃ . ๊ทผ๋ฐ RAG๋ฅผ ์“ฐ๋ฉด ์ด๋Ÿฐ ๋ฌธ์ œ๊ฐ€ ์‹น ํ•ด๊ฒฐ๋ผ์š”!

๐ŸŽฏ RAG์˜ ํ•ต์‹ฌ ์žฅ์  1. ์ตœ์‹  ์ •๋ณด ํ™œ์šฉ - ์‹ค์‹œ๊ฐ„์œผ๋กœ ์—…๋ฐ์ดํŠธ๋˜๋Š” ๋ฌธ์„œ๋‚˜ ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ๊ฒ€์ƒ‰ ๊ฐ€๋Šฅ
2. ํ™˜๊ฐ(Hallucination) ๊ฐ์†Œ - ์‹ค์ œ ๋ฌธ์„œ ๊ธฐ๋ฐ˜์œผ๋กœ ๋‹ต๋ณ€ํ•˜๋‹ˆ๊นŒ ๊ฑฐ์ง“๋ง ํ™• ์ค„์–ด๋“ฆ ใ…‹ใ…‹ใ…‹
3. ์ถœ์ฒ˜ ๋ช…์‹œ ๊ฐ€๋Šฅ - "์ด ์ •๋ณด๋Š” XX ๋ฌธ์„œ 3ํŽ˜์ด์ง€์—์„œ ๊ฐ€์ ธ์™”์–ด์š”" ์ด๋Ÿฐ ์‹์œผ๋กœ ์‹ ๋ขฐ๋„ UP
4. ๋„๋ฉ”์ธ ํŠนํ™” - ๋ฒ•๋ฅ , ์˜๋ฃŒ, ๊ธˆ์œต ๋“ฑ ์ „๋ฌธ ๋ถ„์•ผ์— ํŠนํ™”๋œ AI ๋น„์„œ ๊ตฌ์ถ• ๊ฐ€๋Šฅ
5. ๋น„์šฉ ํšจ์œจ์  - ๋ชจ๋ธ ์žฌํ•™์Šต ์—†์ด๋„ ์ง€์‹ ์—…๋ฐ์ดํŠธ ๊ฐ€๋Šฅ

๐Ÿ› ๏ธ LangChain์ด ๋ญ๊ธธ๋ž˜ ๋‹ค๋“ค ์“ฐ๋Š” ๊ฑฐ์•ผ?

LangChain์€ LLM(Large Language Model) ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์„ ์‰ฝ๊ฒŒ ๋งŒ๋“ค ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ๋Š” ํ”„๋ ˆ์ž„์›Œํฌ์˜ˆ์š”. ์†”์งํžˆ ์ฒ˜์Œ์—๋Š” "์ด๊ฑฐ ์™œ ํ•„์š”ํ•ด? ๊ทธ๋ƒฅ OpenAI API ์“ฐ๋ฉด ๋˜๋Š” ๊ฑฐ ์•„๋ƒ?" ํ–ˆ๋Š”๋ฐ... ์จ๋ณด๋‹ˆ๊นŒ ์™„์ „ ๊ฟ€์ด๋”๋ผ๊ณ ์š” ๐Ÿฏ

LangChain ์—†์ด RAG ์‹œ์Šคํ…œ ๋งŒ๋“ค๋ ค๋ฉด ์ง„์งœ ํ•  ๊ฒŒ ์‚ฐ๋”๋ฏธ์˜ˆ์š”. ๋ฌธ์„œ ๋กœ๋”ฉํ•˜๊ณ , ์ฒญํฌ๋กœ ๋‚˜๋ˆ„๊ณ , ์ž„๋ฒ ๋”ฉ ๋งŒ๋“ค๊ณ , ๋ฒกํ„ฐ DB์— ์ €์žฅํ•˜๊ณ , ๊ฒ€์ƒ‰ํ•˜๊ณ , ํ”„๋กฌํ”„ํŠธ ๋งŒ๋“ค๊ณ ... ํ—‰ํ—‰ ๐Ÿ˜ต

๊ทผ๋ฐ LangChain ์“ฐ๋ฉด? ์ด ๋ชจ๋“  ๊ฒŒ ๋ช‡ ์ค„ ์ฝ”๋“œ๋กœ ๋๋‚˜์š” ใ…‹ใ…‹ใ…‹

๐Ÿ“ฆ LangChain์˜ ์ฃผ์š” ์ปดํฌ๋„ŒํŠธ โ€ข Document Loaders - PDF, Word, ์›นํŽ˜์ด์ง€ ๋“ฑ ๋‹ค์–‘ํ•œ ํ˜•์‹์˜ ๋ฌธ์„œ ๋กœ๋”ฉ
โ€ข Text Splitters - ๊ธด ๋ฌธ์„œ๋ฅผ ์ ์ ˆํ•œ ํฌ๊ธฐ๋กœ ๋ถ„ํ• 
โ€ข Embeddings - ํ…์ŠคํŠธ๋ฅผ ๋ฒกํ„ฐ๋กœ ๋ณ€ํ™˜
โ€ข Vector Stores - ๋ฒกํ„ฐ ๋ฐ์ดํ„ฐ ์ €์žฅ ๋ฐ ๊ฒ€์ƒ‰
โ€ข Retrievers - ์งˆ๋ฌธ๊ณผ ๊ด€๋ จ๋œ ๋ฌธ์„œ ๊ฒ€์ƒ‰
โ€ข Chains - ์—ฌ๋Ÿฌ ์ปดํฌ๋„ŒํŠธ๋ฅผ ์—ฐ๊ฒฐํ•ด์„œ ๋ณต์žกํ•œ ์›Œํฌํ”Œ๋กœ์šฐ ๊ตฌ์„ฑ
โ€ข Agents - ๋„๊ตฌ๋ฅผ ์‚ฌ์šฉํ•ด์„œ ์ž์œจ์ ์œผ๋กœ ์ž‘์—… ์ˆ˜ํ–‰

๐Ÿ’ป ๊ฐœ๋ฐœ ํ™˜๊ฒฝ ์„ธํŒ…๋ถ€ํ„ฐ ์‹œ์ž‘ํ•ด๋ณผ๊นŒ์š”?

์ž, ์ด์ œ ๋ณธ๊ฒฉ์ ์œผ๋กœ ์ฝ”๋”ฉ ๋“ค์–ด๊ฐ‘๋‹ˆ๋‹ค! ๋จผ์ € ํ•„์š”ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋“ค ์„ค์น˜ํ•ด์•ผ์ฃ . ๊ฐ€์ƒํ™˜๊ฒฝ ๋งŒ๋“œ๋Š” ๊ฑฐ ์žŠ์ง€ ๋งˆ์‹œ๊ณ ์š”~ ๐Ÿ

1
๊ฐ€์ƒํ™˜๊ฒฝ ์ƒ์„ฑ ๋ฐ ํ™œ์„ฑํ™”
# ๊ฐ€์ƒํ™˜๊ฒฝ ์ƒ์„ฑ
python -m venv langchain_env

# Windows
langchain_env\Scripts\activate

# Mac/Linux
source langchain_env/bin/activate
2
ํ•„์ˆ˜ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์„ค์น˜
pip install langchain
pip install openai
pip install chromadb
pip install tiktoken
pip install pypdf
pip install python-dotenv
pip install langchain-community
pip install langchain-openai
๐Ÿ’ก ๊ฟ€ํŒ! requirements.txt ํŒŒ์ผ ๋งŒ๋“ค์–ด์„œ ๊ด€๋ฆฌํ•˜๋ฉด ๋‚˜์ค‘์— ๋ฐฐํฌํ•  ๋•Œ ์—„์ฒญ ํŽธํ•ด์š”. ๋ฒ„์ „ ๋ช…์‹œ๋„ ์žŠ์ง€ ๋งˆ์„ธ์š”! ํŠนํžˆ langchain์€ ์—…๋ฐ์ดํŠธ๊ฐ€ ๋นจ๋ผ์„œ ๋ฒ„์ „ ์•ˆ ๋งž์œผ๋ฉด ์—๋Ÿฌ ํญํƒ„ ๋งž์„ ์ˆ˜ ์žˆ์–ด์š” ใ…‹ใ…‹ใ…‹
3
API ํ‚ค ์„ค์ •
ํ”„๋กœ์ ํŠธ ๋ฃจํŠธ์— .env ํŒŒ์ผ ๋งŒ๋“ค๊ณ  OpenAI API ํ‚ค ๋„ฃ์–ด์ฃผ์„ธ์š”.
# .env ํŒŒ์ผ
OPENAI_API_KEY=sk-your-api-key-here
โš ๏ธ ์ฃผ์˜์‚ฌํ•ญ! .env ํŒŒ์ผ์€ ์ ˆ๋Œ€ Git์— ์ปค๋ฐ‹ํ•˜๋ฉด ์•ˆ ๋ผ์š”! .gitignore์— ๊ผญ ์ถ”๊ฐ€ํ•˜์„ธ์š”. API ํ‚ค ์œ ์ถœ๋˜๋ฉด ๊ณผ๊ธˆ ํญํƒ„ ๋งž์„ ์ˆ˜ ์žˆ์–ด์š”... ์ง„์งœ๋กœ์š” ๐Ÿ˜ฑ

๐Ÿ“„ ๋ฌธ์„œ ๋กœ๋”ฉ๊ณผ ์ „์ฒ˜๋ฆฌ - ์ฒซ ๋ฒˆ์งธ ๊ด€๋ฌธ

์ด์ œ ๋ณธ๊ฒฉ์ ์œผ๋กœ RAG ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค์–ด๋ณผ ๊ฑฐ์˜ˆ์š”. ์ฒซ ๋‹จ๊ณ„๋Š” ๋ฌธ์„œ๋ฅผ ๋กœ๋”ฉํ•˜๊ณ  ์ „์ฒ˜๋ฆฌํ•˜๋Š” ๊ฑฐ์ฃ . PDF ํŒŒ์ผ์„ ์˜ˆ์‹œ๋กœ ๋“ค์–ด๋ณผ๊ฒŒ์š” ๐Ÿ“‘

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from dotenv import load_dotenv
import os

# ํ™˜๊ฒฝ๋ณ€์ˆ˜ ๋กœ๋“œ
load_dotenv()

# PDF ํŒŒ์ผ ๋กœ๋”ฉ
loader = PyPDFLoader("company_manual.pdf")
documents = loader.load()

print(f"์ด {len(documents)}๊ฐœ์˜ ํŽ˜์ด์ง€๋ฅผ ๋กœ๋”ฉํ–ˆ์Šต๋‹ˆ๋‹ค!")
print(f"์ฒซ ๋ฒˆ์งธ ํŽ˜์ด์ง€ ๋‚ด์šฉ ๋ฏธ๋ฆฌ๋ณด๊ธฐ: {documents[0].page_content[:200]}...")

๊ทผ๋ฐ ์—ฌ๊ธฐ์„œ ๋์ด ์•„๋‹ˆ์—์š”! ๋ฌธ์„œ๋ฅผ ๊ทธ๋ƒฅ ํ†ต์งธ๋กœ ๋„ฃ์œผ๋ฉด ์•ˆ ๋˜๊ณ , ์ ์ ˆํ•œ ํฌ๊ธฐ๋กœ ์ชผ๊ฐœ์•ผ ํ•ด์š”. ์ด๊ฑธ "์ฒญํ‚น(Chunking)"์ด๋ผ๊ณ  ํ•˜๋Š”๋ฐ, ์ด๊ฒŒ ์ง„์งœ ์ค‘์š”ํ•ด์š” ๐Ÿ”ช

๐ŸŽฏ ์ฒญํ‚น์ด ์™œ ์ค‘์š”ํ•œ๊ฐ€? ์ฒญํฌ ํฌ๊ธฐ๊ฐ€ ๋„ˆ๋ฌด ํฌ๋ฉด โ†’ ๊ฒ€์ƒ‰ ์ •ํ™•๋„ ๋–จ์–ด์ง€๊ณ , ํ† ํฐ ๋‚ญ๋น„
์ฒญํฌ ํฌ๊ธฐ๊ฐ€ ๋„ˆ๋ฌด ์ž‘์œผ๋ฉด โ†’ ๋ฌธ๋งฅ ์ •๋ณด ์†์‹ค, ๊ฒ€์ƒ‰ ๊ฒฐ๊ณผ ๋„ˆ๋ฌด ๋งŽ์Œ

๋ณดํ†ต 500~1000 ํ† ํฐ ์ •๋„๊ฐ€ ์ ๋‹นํ•˜๋‹ค๊ณ  ์•Œ๋ ค์ ธ ์žˆ์–ด์š”. ๊ทผ๋ฐ ์ด๊ฒƒ๋„ ๋„๋ฉ”์ธ๋งˆ๋‹ค ๋‹ค๋ฅด๋‹ˆ๊นŒ ์‹คํ—˜ํ•ด๋ณด๋ฉด์„œ ์ตœ์ ๊ฐ’ ์ฐพ์•„์•ผ ํ•ด์š”!
# ํ…์ŠคํŠธ ๋ถ„ํ• ๊ธฐ ์„ค์ •
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,  # ์ฒญํฌ ํฌ๊ธฐ
    chunk_overlap=200,  # ์ฒญํฌ ๊ฐ„ ๊ฒน์น˜๋Š” ๋ถ€๋ถ„
    length_function=len,
    separators=["\n\n", "\n", " ", ""]
)

# ๋ฌธ์„œ ๋ถ„ํ• 
splits = text_splitter.split_documents(documents)
print(f"์ด {len(splits)}๊ฐœ์˜ ์ฒญํฌ๋กœ ๋ถ„ํ• ๋˜์—ˆ์Šต๋‹ˆ๋‹ค!")

# ์ฒญํฌ ์˜ˆ์‹œ ํ™•์ธ
for i, split in enumerate(splits[:3]):
    print(f"\n=== ์ฒญํฌ {i+1} ===")
    print(f"๋‚ด์šฉ: {split.page_content[:150]}...")
    print(f"๋ฉ”ํƒ€๋ฐ์ดํ„ฐ: {split.metadata}")
๐Ÿ’ก RecursiveCharacterTextSplitter์˜ ๋น„๋ฐ€ ์ด ์นœ๊ตฌ๋Š” ๋˜‘๋˜‘ํ•˜๊ฒŒ ๋ฌธ๋‹จ, ๋ฌธ์žฅ, ๋‹จ์–ด ์ˆœ์„œ๋กœ ๋‚˜๋ˆ ์š”. ๊ทธ๋ž˜์„œ ๋ฌธ๋งฅ์ด ์ตœ๋Œ€ํ•œ ๋ณด์กด๋˜์ฃ . chunk_overlap์„ ์„ค์ •ํ•˜๋ฉด ์ฒญํฌ ๊ฐ„ ๊ฒฝ๊ณ„์—์„œ ์ •๋ณด ์†์‹ค์„ ๋ง‰์„ ์ˆ˜ ์žˆ์–ด์š”. ๋ณดํ†ต chunk_size์˜ 10~20% ์ •๋„๋กœ ์„ค์ •ํ•˜๋Š” ๊ฒŒ ์ข‹์•„์š”!

๐Ÿงฎ ์ž„๋ฒ ๋”ฉ๊ณผ ๋ฒกํ„ฐ ์Šคํ† ์–ด - AI์˜ ๊ธฐ์–ต ์ €์žฅ์†Œ

์ด์ œ ๋ถ„ํ• ๋œ ํ…์ŠคํŠธ๋ฅผ ๋ฒกํ„ฐ๋กœ ๋ณ€ํ™˜ํ•ด์•ผ ํ•ด์š”. ๋ฒกํ„ฐ๋กœ ๋ณ€ํ™˜ํ•œ๋‹ค๋Š” ๊ฒŒ ๋ญ๋ƒ๊ณ ์š”? ์‰ฝ๊ฒŒ ๋งํ•˜๋ฉด ํ…์ŠคํŠธ๋ฅผ ์ˆซ์ž ๋ฐฐ์—ด๋กœ ๋ฐ”๊พธ๋Š” ๊ฑฐ์˜ˆ์š”. ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ์ปดํ“จํ„ฐ๊ฐ€ "์˜๋ฏธ์ ์œผ๋กœ ๋น„์Šทํ•œ" ํ…์ŠคํŠธ๋ฅผ ์ฐพ์„ ์ˆ˜ ์žˆ๊ฑฐ๋“ ์š” ๐Ÿ”ข

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

# OpenAI ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ ์ดˆ๊ธฐํ™”
embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small"  # ์ตœ์‹  ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ
)

# ๋ฒกํ„ฐ ์Šคํ† ์–ด ์ƒ์„ฑ ๋ฐ ๋ฌธ์„œ ์ €์žฅ
vectorstore = Chroma.from_documents(
    documents=splits,
    embedding=embeddings,
    persist_directory="./chroma_db"  # ๋กœ์ปฌ์— ์ €์žฅ
)

print("๋ฒกํ„ฐ ์Šคํ† ์–ด ์ƒ์„ฑ ์™„๋ฃŒ! ๐ŸŽ‰")
print(f"์ด {vectorstore._collection.count()}๊ฐœ์˜ ๋ฒกํ„ฐ๊ฐ€ ์ €์žฅ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.")

ChromaDB๋Š” ์˜คํ”ˆ์†Œ์Šค ๋ฒกํ„ฐ ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค์ธ๋ฐ, ๋กœ์ปฌ์—์„œ ๋Œ๋ฆด ์ˆ˜ ์žˆ์–ด์„œ ๊ฐœ๋ฐœํ•  ๋•Œ ์™„์ „ ํŽธํ•ด์š”. ํ”„๋กœ๋•์…˜์—์„œ๋Š” Pinecone์ด๋‚˜ Weaviate ๊ฐ™์€ ๊ฑธ ์“ฐ๋Š” ๊ฒŒ ์ข‹์ง€๋งŒ, ์ผ๋‹จ ๋ฐฐ์šฐ๋Š” ๋‹จ๊ณ„์—์„œ๋Š” Chroma๊ฐ€ ์ตœ๊ณ ์ฃ  ใ…‹ใ…‹ใ…‹

๐Ÿ—„๏ธ ์ฃผ์š” ๋ฒกํ„ฐ ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ๋น„๊ต ChromaDB - ๋กœ์ปฌ ๊ฐœ๋ฐœ์šฉ์œผ๋กœ ์ตœ์ , ๋ฌด๋ฃŒ, ์„ค์ • ๊ฐ„๋‹จ
Pinecone - ํด๋ผ์šฐ๋“œ ๊ธฐ๋ฐ˜, ํ™•์žฅ์„ฑ ์ข‹์Œ, ์œ ๋ฃŒ
Weaviate - ์˜คํ”ˆ์†Œ์Šค, ์ž์ฒด ํ˜ธ์ŠคํŒ… ๊ฐ€๋Šฅ, ๊ธฐ๋Šฅ ๋งŽ์Œ
FAISS - Facebook์ด ๋งŒ๋“  ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ, ๋น ๋ฆ„, ํด๋ผ์šฐ๋“œ ๊ธฐ๋Šฅ ์—†์Œ
Milvus - ์—”ํ„ฐํ”„๋ผ์ด์ฆˆ๊ธ‰, ๋Œ€์šฉ๋Ÿ‰ ์ฒ˜๋ฆฌ ๊ฐ•์ 
# ๋ฒกํ„ฐ ์Šคํ† ์–ด ๊ฒ€์ƒ‰ ํ…Œ์ŠคํŠธ
query = "ํšŒ์‚ฌ์˜ ํœด๊ฐ€ ์ •์ฑ…์€ ์–ด๋–ป๊ฒŒ ๋˜๋‚˜์š”?"
docs = vectorstore.similarity_search(query, k=3)

print(f"\n์งˆ๋ฌธ: {query}")
print("\n=== ๊ฒ€์ƒ‰ ๊ฒฐ๊ณผ ===")
for i, doc in enumerate(docs):
    print(f"\n[๋ฌธ์„œ {i+1}]")
    print(f"๋‚ด์šฉ: {doc.page_content[:200]}...")
    print(f"์ถœ์ฒ˜: {doc.metadata.get('source', 'Unknown')}, ํŽ˜์ด์ง€: {doc.metadata.get('page', 'N/A')}")
๐Ÿ’ก ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ ์„ ํƒ ํŒ OpenAI์˜ text-embedding-3-small์€ ๊ฐ€์„ฑ๋น„๊ฐ€ ์ข‹๊ณ , text-embedding-3-large๋Š” ์ •ํ™•๋„๊ฐ€ ๋” ๋†’์•„์š”. ํ•œ๊ตญ์–ด ๋ฌธ์„œ๊ฐ€ ๋งŽ๋‹ค๋ฉด multilingual-e5-large ๊ฐ™์€ ๋‹ค๊ตญ์–ด ๋ชจ๋ธ๋„ ๊ณ ๋ คํ•ด๋ณผ ๋งŒํ•ด์š”. ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ ๋‹ค์–‘ํ•œ ์–ธ์–ด๋ฅผ ๋‹ค๋ฃฌ๋‹ค๋ฉด ํŠนํžˆ ์ค‘์š”ํ•˜์ฃ !
์›๋ณธ ํ…์ŠคํŠธ LangChain์€ ๊ฐ•๋ ฅํ•œ ๋„๊ตฌ๋‹ค ์ž„๋ฒ ๋”ฉ ๋ณ€ํ™˜ [0.23, -0.45, 0.78, 0.12, -0.89, ...] ๋ฒกํ„ฐ DB ์ €์žฅ ๊ฒ€์ƒ‰ ๊ฐ€๋Šฅ ์ƒํƒœ ์˜๋ฏธ์ ์œผ๋กœ ์œ ์‚ฌํ•œ ํ…์ŠคํŠธ๋ฅผ ๋น ๋ฅด๊ฒŒ ๊ฒ€์ƒ‰! ๋ฒกํ„ฐ ๊ฐ„ ๊ฑฐ๋ฆฌ ๊ณ„์‚ฐ์œผ๋กœ ๊ด€๋ จ๋„ ์ธก์ •

๐Ÿ”— Retriever์™€ Chain ๊ตฌ์„ฑ - ์ง„์งœ ๋งˆ๋ฒ•์ด ์‹œ์ž‘๋˜๋Š” ๊ณณ

์ด์ œ ๊ฒ€์ƒ‰๊ธฐ(Retriever)๋ฅผ ๋งŒ๋“ค๊ณ , LLM๊ณผ ์—ฐ๊ฒฐํ•ด์„œ ์‹ค์ œ๋กœ ์งˆ๋ฌธ์— ๋‹ต๋ณ€ํ•˜๋Š” ์‹œ์Šคํ…œ์„ ๊ตฌ์ถ•ํ•  ๊ฑฐ์˜ˆ์š”. ์—ฌ๊ธฐ์„œ๋ถ€ํ„ฐ๊ฐ€ ์ง„์งœ ์žฌ๋ฐŒ์–ด์š” ใ…‹ใ…‹ใ…‹ ๐ŸŽช

from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

# LLM ์ดˆ๊ธฐํ™”
llm = ChatOpenAI(
    model_name="gpt-4o-mini",  # ๋˜๋Š” gpt-4
    temperature=0.3  # ์ฐฝ์˜์„ฑ vs ์ผ๊ด€์„ฑ ์กฐ์ ˆ
)

# Retriever ์ƒ์„ฑ
retriever = vectorstore.as_retriever(
    search_type="similarity",  # ์œ ์‚ฌ๋„ ๊ฒ€์ƒ‰
    search_kwargs={"k": 4}  # ์ƒ์œ„ 4๊ฐœ ๋ฌธ์„œ ๊ฒ€์ƒ‰
)

# ์ปค์Šคํ…€ ํ”„๋กฌํ”„ํŠธ ํ…œํ”Œ๋ฆฟ
template = """๋‹น์‹ ์€ ํšŒ์‚ฌ ๋‚ด๋ถ€ ๋ฌธ์„œ๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ๋‹ต๋ณ€ํ•˜๋Š” AI ๋น„์„œ์ž…๋‹ˆ๋‹ค.
์ฃผ์–ด์ง„ ์ปจํ…์ŠคํŠธ๋ฅผ ๋ฐ”ํƒ•์œผ๋กœ ์ •ํ™•ํ•˜๊ณ  ์นœ์ ˆํ•˜๊ฒŒ ๋‹ต๋ณ€ํ•ด์ฃผ์„ธ์š”.
๋งŒ์•ฝ ์ปจํ…์ŠคํŠธ์— ๋‹ต์ด ์—†๋‹ค๋ฉด, "์ œ๊ณต๋œ ๋ฌธ์„œ์—์„œ ํ•ด๋‹น ์ •๋ณด๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค"๋ผ๊ณ  ๋‹ต๋ณ€ํ•˜์„ธ์š”.

์ปจํ…์ŠคํŠธ: {context}

์งˆ๋ฌธ: {question}

๋‹ต๋ณ€:"""

PROMPT = PromptTemplate(
    template=template,
    input_variables=["context", "question"]
)

# RetrievalQA ์ฒด์ธ ์ƒ์„ฑ
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",  # ๋ชจ๋“  ๋ฌธ์„œ๋ฅผ ํ•œ ๋ฒˆ์— ๋„ฃ๋Š” ๋ฐฉ์‹
    retriever=retriever,
    return_source_documents=True,
    chain_type_kwargs={"prompt": PROMPT}
)

print("QA ์ฒด์ธ ์ƒ์„ฑ ์™„๋ฃŒ! ์ด์ œ ์งˆ๋ฌธ์„ ๋ฐ›์„ ์ค€๋น„๊ฐ€ ๋์–ด์š” ๐Ÿš€")
๐ŸŽฏ Chain Type ์ข…๋ฅ˜์™€ ํŠน์ง• stuff - ๋ชจ๋“  ๋ฌธ์„œ๋ฅผ ํ•œ ๋ฒˆ์— ํ”„๋กฌํ”„ํŠธ์— ๋„ฃ์Œ. ๊ฐ„๋‹จํ•˜๊ณ  ๋น ๋ฅด์ง€๋งŒ ํ† ํฐ ์ œํ•œ ์žˆ์Œ
map_reduce - ๊ฐ ๋ฌธ์„œ๋ฅผ ๊ฐœ๋ณ„ ์ฒ˜๋ฆฌ ํ›„ ๊ฒฐ๊ณผ ํ†ตํ•ฉ. ๋งŽ์€ ๋ฌธ์„œ ์ฒ˜๋ฆฌ ๊ฐ€๋Šฅ
refine - ์ˆœ์ฐจ์ ์œผ๋กœ ๋‹ต๋ณ€์„ ๊ฐœ์„ . ์ •ํ™•๋„ ๋†’์ง€๋งŒ ๋А๋ฆผ
map_rerank - ๊ฐ ๋ฌธ์„œ๋ณ„ ๋‹ต๋ณ€์— ์ ์ˆ˜ ๋งค๊ฒจ์„œ ์ตœ๊ณ  ์ ์ˆ˜ ์„ ํƒ
# ์‹ค์ œ ์งˆ๋ฌธ ํ…Œ์ŠคํŠธ
questions = [
    "์—ฐ์ฐจ ํœด๊ฐ€๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?",
    "์žฌํƒ๊ทผ๋ฌด ์ •์ฑ…์— ๋Œ€ํ•ด ์•Œ๋ ค์ฃผ์„ธ์š”",
    "ํšŒ์‚ฌ ๋ณต์ง€ ์ œ๋„๋Š” ์–ด๋–ค ๊ฒŒ ์žˆ๋‚˜์š”?"
]

for question in questions:
    print(f"\n{'='*50}")
    print(f"์งˆ๋ฌธ: {question}")
    print(f"{'='*50}")
    
    result = qa_chain.invoke({"query": question})
    
    print(f"\n๋‹ต๋ณ€: {result['result']}")
    print(f"\n์ฐธ๊ณ  ๋ฌธ์„œ:")
    for i, doc in enumerate(result['source_documents']):
        print(f"  [{i+1}] {doc.metadata.get('source', 'Unknown')} - ํŽ˜์ด์ง€ {doc.metadata.get('page', 'N/A')}")
๐Ÿ’ก Temperature ์„ค์ •์˜ ๋น„๋ฐ€ Temperature๋Š” 0~2 ์‚ฌ์ด ๊ฐ’์ธ๋ฐ์š”:
โ€ข 0์— ๊ฐ€๊นŒ์šธ์ˆ˜๋ก โ†’ ์ผ๊ด€๋˜๊ณ  ์˜ˆ์ธก ๊ฐ€๋Šฅํ•œ ๋‹ต๋ณ€ (์‚ฌ์‹ค ๊ธฐ๋ฐ˜ QA์— ์ ํ•ฉ)
โ€ข 2์— ๊ฐ€๊นŒ์šธ์ˆ˜๋ก โ†’ ์ฐฝ์˜์ ์ด๊ณ  ๋‹ค์–‘ํ•œ ๋‹ต๋ณ€ (๋ธŒ๋ ˆ์ธ์Šคํ† ๋ฐ์— ์ ํ•ฉ)
๋ณดํ†ต RAG ์‹œ์Šคํ…œ์—์„œ๋Š” 0.1~0.3 ์ •๋„๊ฐ€ ์ ๋‹นํ•ด์š”!

๐ŸŽจ ๊ณ ๊ธ‰ ๊ธฐ๋Šฅ - ๋Œ€ํ™” ๊ธฐ๋ก ์œ ์ง€ํ•˜๊ธฐ

์ง€๊ธˆ๊นŒ์ง€ ๋งŒ๋“  ๊ฑด ์ผํšŒ์„ฑ ์งˆ๋ฌธ-๋‹ต๋ณ€ ์‹œ์Šคํ…œ์ด์—์š”. ๊ทผ๋ฐ ์ง„์งœ ๋น„์„œ์ฒ˜๋Ÿผ ๋งŒ๋“ค๋ ค๋ฉด ๋Œ€ํ™” ๋งฅ๋ฝ์„ ๊ธฐ์–ตํ•ด์•ผ๊ฒ ์ฃ ? ์ด์ „ ๋Œ€ํ™” ๋‚ด์šฉ์„ ๊ธฐ์–ตํ•˜๋Š” ๊ธฐ๋Šฅ์„ ์ถ”๊ฐ€ํ•ด๋ณผ๊ฒŒ์š” ๐Ÿง 

from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferMemory

# ๋Œ€ํ™” ๊ธฐ๋ก ์ €์žฅ์šฉ ๋ฉ”๋ชจ๋ฆฌ
memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True,
    output_key="answer"
)

# ๋Œ€ํ™”ํ˜• Retrieval ์ฒด์ธ
conversational_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    memory=memory,
    return_source_documents=True,
    verbose=True  # ๋””๋ฒ„๊น…์šฉ
)

# ๋Œ€ํ™” ์‹œ๋ฎฌ๋ ˆ์ด์…˜
print("=== ๋Œ€ํ™”ํ˜• AI ๋น„์„œ ํ…Œ์ŠคํŠธ ===\n")

# ์ฒซ ๋ฒˆ์งธ ์งˆ๋ฌธ
response1 = conversational_chain.invoke({
    "question": "์šฐ๋ฆฌ ํšŒ์‚ฌ ์—ฐ์ฐจ๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?"
})
print(f"์‚ฌ์šฉ์ž: ์šฐ๋ฆฌ ํšŒ์‚ฌ ์—ฐ์ฐจ๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?")
print(f"AI: {response1['answer']}\n")

# ๋‘ ๋ฒˆ์งธ ์งˆ๋ฌธ (์ด์ „ ๋งฅ๋ฝ ์ฐธ์กฐ)
response2 = conversational_chain.invoke({
    "question": "๊ทธ๋Ÿผ ์‹ ์ž…์‚ฌ์›๋„ ๋™์ผํ•œ๊ฐ€์š”?"
})
print(f"์‚ฌ์šฉ์ž: ๊ทธ๋Ÿผ ์‹ ์ž…์‚ฌ์›๋„ ๋™์ผํ•œ๊ฐ€์š”?")
print(f"AI: {response2['answer']}\n")

# ์„ธ ๋ฒˆ์งธ ์งˆ๋ฌธ (๋” ์ด์ „ ๋งฅ๋ฝ๊นŒ์ง€ ์ฐธ์กฐ)
response3 = conversational_chain.invoke({
    "question": "๊ทธ ์—ฐ์ฐจ๋ฅผ ์–ธ์ œ๋ถ€ํ„ฐ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์ฃ ?"
})
print(f"์‚ฌ์šฉ์ž: ๊ทธ ์—ฐ์ฐจ๋ฅผ ์–ธ์ œ๋ถ€ํ„ฐ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์ฃ ?")
print(f"AI: {response3['answer']}")
๐Ÿง  Memory ํƒ€์ž…๋ณ„ ํŠน์ง• ConversationBufferMemory - ๋ชจ๋“  ๋Œ€ํ™” ์ €์žฅ, ๊ฐ„๋‹จํ•˜์ง€๋งŒ ํ† ํฐ ๋งŽ์ด ์”€
ConversationBufferWindowMemory - ์ตœ๊ทผ N๊ฐœ ๋Œ€ํ™”๋งŒ ์ €์žฅ
ConversationSummaryMemory - ๋Œ€ํ™”๋ฅผ ์š”์•ฝํ•ด์„œ ์ €์žฅ, ํ† ํฐ ์ ˆ์•ฝ
ConversationSummaryBufferMemory - ์ตœ๊ทผ ๋Œ€ํ™”๋Š” ๊ทธ๋Œ€๋กœ, ์˜ค๋ž˜๋œ ๊ฑด ์š”์•ฝ
VectorStoreBackedMemory - ๋ฒกํ„ฐ DB์— ์ €์žฅ, ๊ด€๋ จ ๋Œ€ํ™”๋งŒ ๊ฒ€์ƒ‰
โš ๏ธ ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ ์ฃผ์˜์‚ฌํ•ญ ๋Œ€ํ™”๊ฐ€ ๊ธธ์–ด์ง€๋ฉด ํ† ํฐ์ด ๊ธฐํ•˜๊ธ‰์ˆ˜์ ์œผ๋กœ ๋Š˜์–ด๋‚˜์š”! ํ”„๋กœ๋•์…˜์—์„œ๋Š” ๋ฐ˜๋“œ์‹œ ํ† ํฐ ์ œํ•œ์„ ์„ค์ •ํ•˜๊ณ , ์˜ค๋ž˜๋œ ๋Œ€ํ™”๋Š” ์š”์•ฝํ•˜๊ฑฐ๋‚˜ ์‚ญ์ œํ•˜๋Š” ๋กœ์ง์ด ํ•„์š”ํ•ด์š”. ์•ˆ ๊ทธ๋Ÿฌ๋ฉด API ๋น„์šฉ ํญํƒ„ ๋งž์•„์š” ใ… ใ… 

๐Ÿš€ ์‹ค์ „ ์ตœ์ ํ™” ํ…Œํฌ๋‹‰

์ด์ œ ๊ธฐ๋ณธ์€ ๋‹ค ๋ฐฐ์› ์œผ๋‹ˆ๊นŒ, ์‹ค์ „์—์„œ ์“ธ ์ˆ˜ ์žˆ๋Š” ์ตœ์ ํ™” ๊ธฐ๋ฒ•๋“ค์„ ์•Œ๋ ค๋“œ๋ฆด๊ฒŒ์š”. ์ด๊ฑฐ ์•Œ๊ณ  ๋ชจ๋ฅด๊ณ ์˜ ์ฐจ์ด๊ฐ€ ์—„์ฒญ๋‚˜์š” ใ…‹ใ…‹ใ…‹

1
ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ๊ฒ€์ƒ‰ (Hybrid Search)
๋ฒกํ„ฐ ๊ฒ€์ƒ‰๋งŒ ์“ฐ๋ฉด ํ‚ค์›Œ๋“œ ๋งค์นญ์ด ์•ฝํ•ด์š”. BM25 ๊ฐ™์€ ํ‚ค์›Œ๋“œ ๊ฒ€์ƒ‰๊ณผ ๊ฒฐํ•ฉํ•˜๋ฉด ์ •ํ™•๋„๊ฐ€ ํ™• ์˜ฌ๋ผ๊ฐ€์š”!
from langchain.retrievers import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever

# BM25 ํ‚ค์›Œ๋“œ ๊ฒ€์ƒ‰๊ธฐ
bm25_retriever = BM25Retriever.from_documents(splits)
bm25_retriever.k = 4

# ๋ฒกํ„ฐ ๊ฒ€์ƒ‰๊ธฐ
vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# ์•™์ƒ๋ธ” ๊ฒ€์ƒ‰๊ธฐ (๋‘ ๋ฐฉ์‹ ๊ฒฐํ•ฉ)
ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever],
    weights=[0.4, 0.6]  # ๋ฒกํ„ฐ ๊ฒ€์ƒ‰์— ๋” ๋†’์€ ๊ฐ€์ค‘์น˜
)

# ์ด์ œ ensemble_retriever๋ฅผ ์ฒด์ธ์— ์‚ฌ์šฉ
hybrid_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=ensemble_retriever,
    memory=memory
)
2
๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋ง
ํŠน์ • ๋ถ€์„œ, ๋‚ ์งœ, ์นดํ…Œ๊ณ ๋ฆฌ ๋ฌธ์„œ๋งŒ ๊ฒ€์ƒ‰ํ•˜๊ณ  ์‹ถ์„ ๋•Œ ์œ ์šฉํ•ด์š”!
# ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์ถ”๊ฐ€ํ•ด์„œ ๋ฌธ์„œ ์ €์žฅ
from datetime import datetime

for split in splits:
    split.metadata["department"] = "HR"
    split.metadata["date"] = "2024-01-15"
    split.metadata["category"] = "policy"

# ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋ง ๊ฒ€์ƒ‰
filtered_retriever = vectorstore.as_retriever(
    search_kwargs={
        "k": 4,
        "filter": {
            "department": "HR",
            "category": "policy"
        }
    }
)
3
Re-ranking์œผ๋กœ ์ •ํ™•๋„ ํ–ฅ์ƒ
๊ฒ€์ƒ‰๋œ ๋ฌธ์„œ๋ฅผ ๋‹ค์‹œ ํ•œ๋ฒˆ ์ •๋ ฌํ•ด์„œ ๊ฐ€์žฅ ๊ด€๋ จ์„ฑ ๋†’์€ ๊ฒƒ๋งŒ LLM์— ์ „๋‹ฌํ•ด์š”.
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

# LLM ๊ธฐ๋ฐ˜ ์••์ถ•๊ธฐ
compressor = LLMChainExtractor.from_llm(llm)

# ์••์ถ• ๊ฒ€์ƒ‰๊ธฐ
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=vector_retriever
)

# ์ด์ œ ๋” ์ •ํ™•ํ•œ ๋ฌธ์„œ๋งŒ ๊ฐ€์ ธ์™€์š”!
compressed_docs = compression_retriever.get_relevant_documents(
    "์—ฐ์ฐจ ํœด๊ฐ€ ์ •์ฑ…"
)
4
์บ์‹ฑ์œผ๋กœ ๋น„์šฉ ์ ˆ๊ฐ
๊ฐ™์€ ์งˆ๋ฌธ์ด ๋ฐ˜๋ณต๋˜๋ฉด ์บ์‹œ์—์„œ ๋ฐ”๋กœ ๋‹ต๋ณ€ํ•ด์š”. API ํ˜ธ์ถœ ์ค„์ด๊ณ  ์†๋„๋„ ๋นจ๋ผ์ ธ์š”!
from langchain.cache import InMemoryCache
from langchain.globals import set_llm_cache

# ์บ์‹œ ์„ค์ •
set_llm_cache(InMemoryCache())

# ์ด์ œ ๊ฐ™์€ ์งˆ๋ฌธ์€ ์บ์‹œ์—์„œ ์ฆ‰์‹œ ์‘๋‹ต!
# ์ฒซ ๋ฒˆ์งธ ํ˜ธ์ถœ - API ํ˜ธ์ถœ (๋А๋ฆผ)
result1 = qa_chain.invoke({"query": "์—ฐ์ฐจ๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?"})

# ๋‘ ๋ฒˆ์งธ ํ˜ธ์ถœ - ์บ์‹œ์—์„œ ์‘๋‹ต (๋น ๋ฆ„!)
result2 = qa_chain.invoke({"query": "์—ฐ์ฐจ๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?"})

๐Ÿ”ง ์‹ค์ „ ํ”„๋กœ์ ํŠธ ๊ตฌ์กฐ ์˜ˆ์‹œ

์‹ค์ œ ํ”„๋กœ์ ํŠธ์—์„œ๋Š” ์ฝ”๋“œ๋ฅผ ๋ชจ๋“ˆํ™”ํ•ด์„œ ๊ด€๋ฆฌํ•ด์•ผ ํ•ด์š”. ์ œ๊ฐ€ ์ถ”์ฒœํ•˜๋Š” ๊ตฌ์กฐ๋ฅผ ๋ณด์—ฌ๋“œ๋ฆด๊ฒŒ์š” ๐Ÿ“

rag_chatbot/
โ”‚
โ”œโ”€โ”€ .env                      # API ํ‚ค ๋“ฑ ํ™˜๊ฒฝ๋ณ€์ˆ˜
โ”œโ”€โ”€ requirements.txt          # ์˜์กด์„ฑ ํŒจํ‚ค์ง€
โ”œโ”€โ”€ config.py                 # ์„ค์ • ํŒŒ์ผ
โ”‚
โ”œโ”€โ”€ data/                     # ์›๋ณธ ๋ฌธ์„œ
โ”‚   โ”œโ”€โ”€ pdfs/
โ”‚   โ”œโ”€โ”€ docs/
โ”‚   โ””โ”€โ”€ txts/
โ”‚
โ”œโ”€โ”€ vectorstore/              # ๋ฒกํ„ฐ DB ์ €์žฅ์†Œ
โ”‚   โ””โ”€โ”€ chroma_db/
โ”‚
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ”œโ”€โ”€ document_loader.py    # ๋ฌธ์„œ ๋กœ๋”ฉ ๋กœ์ง
โ”‚   โ”œโ”€โ”€ embeddings.py         # ์ž„๋ฒ ๋”ฉ ์ƒ์„ฑ
โ”‚   โ”œโ”€โ”€ retriever.py          # ๊ฒ€์ƒ‰๊ธฐ ์„ค์ •
โ”‚   โ”œโ”€โ”€ chain.py              # ์ฒด์ธ ๊ตฌ์„ฑ
โ”‚   โ””โ”€โ”€ utils.py              # ์œ ํ‹ธ๋ฆฌํ‹ฐ ํ•จ์ˆ˜
โ”‚
โ”œโ”€โ”€ app.py                    # ๋ฉ”์ธ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜
โ””โ”€โ”€ streamlit_app.py          # Streamlit UI (์„ ํƒ์‚ฌํ•ญ)
# config.py ์˜ˆ์‹œ
import os
from dotenv import load_dotenv

load_dotenv()

class Config:
    # API ์„ค์ •
    OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
    
    # ๋ชจ๋ธ ์„ค์ •
    LLM_MODEL = "gpt-4o-mini"
    EMBEDDING_MODEL = "text-embedding-3-small"
    TEMPERATURE = 0.3
    
    # ๊ฒ€์ƒ‰ ์„ค์ •
    CHUNK_SIZE = 1000
    CHUNK_OVERLAP = 200
    SEARCH_K = 4
    
    # ๋ฒกํ„ฐ ์Šคํ† ์–ด ์„ค์ •
    VECTORSTORE_PATH = "./vectorstore/chroma_db"
    
    # ๋ฉ”๋ชจ๋ฆฌ ์„ค์ •
    MAX_HISTORY_LENGTH = 10
# document_loader.py ์˜ˆ์‹œ
from langchain_community.document_loaders import (
    PyPDFLoader, 
    TextLoader, 
    DirectoryLoader
)
from langchain.text_splitter import RecursiveCharacterTextSplitter
from config import Config

class DocumentProcessor:
    def __init__(self):
        self.text_splitter = RecursiveCharacterTextSplitter(
            chunk_size=Config.CHUNK_SIZE,
            chunk_overlap=Config.CHUNK_OVERLAP
        )
    
    def load_pdfs(self, directory_path):
        """PDF ํŒŒ์ผ๋“ค์„ ๋กœ๋”ฉ"""
        loader = DirectoryLoader(
            directory_path,
            glob="**/*.pdf",
            loader_cls=PyPDFLoader
        )
        documents = loader.load()
        return self.text_splitter.split_documents(documents)
    
    def load_texts(self, directory_path):
        """ํ…์ŠคํŠธ ํŒŒ์ผ๋“ค์„ ๋กœ๋”ฉ"""
        loader = DirectoryLoader(
            directory_path,
            glob="**/*.txt",
            loader_cls=TextLoader
        )
        documents = loader.load()
        return self.text_splitter.split_documents(documents)
๐Ÿ’ก ๋ชจ๋“ˆํ™”์˜ ์žฅ์  ์ฝ”๋“œ๋ฅผ ๋ชจ๋“ˆํ™”ํ•˜๋ฉด ์œ ์ง€๋ณด์ˆ˜๊ฐ€ ์—„์ฒญ ์‰ฌ์›Œ์ ธ์š”. ๋‚˜์ค‘์— ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ๋งŒ ๋ฐ”๊พธ๊ณ  ์‹ถ์œผ๋ฉด embeddings.py๋งŒ ์ˆ˜์ •ํ•˜๋ฉด ๋˜๊ณ , ๊ฒ€์ƒ‰ ๋กœ์ง๋งŒ ๊ฐœ์„ ํ•˜๊ณ  ์‹ถ์œผ๋ฉด retriever.py๋งŒ ๊ฑด๋“œ๋ฆฌ๋ฉด ๋ผ์š”. ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•  ๋•Œ๋„ ์ด๋ ‡๊ฒŒ ๊น”๋”ํ•œ ๊ตฌ์กฐ๊ฐ€ ์ค‘์š”ํ•˜์ฃ !

๐ŸŽจ Streamlit์œผ๋กœ UI ๋งŒ๋“ค๊ธฐ

CLI๋กœ๋งŒ ์“ฐ๊ธฐ์—” ์ข€ ๊ทธ๋ ‡์ฃ ? Streamlit์œผ๋กœ ๊ฐ„๋‹จํ•œ ์›น UI ๋งŒ๋“ค์–ด๋ณผ๊ฒŒ์š”. ์ง„์งœ 5๋ถ„์ด๋ฉด ๋ผ์š” ใ…‹ใ…‹ใ…‹

# streamlit_app.py
import streamlit as st
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferMemory
from config import Config

# ํŽ˜์ด์ง€ ์„ค์ •
st.set_page_config(
    page_title="AI ๋ฌธ์„œ ๋น„์„œ",
    page_icon="๐Ÿค–",
    layout="wide"
)

# ์ œ๋ชฉ
st.title("๐Ÿค– AI ๋ฌธ์„œ ๊ฒ€์ƒ‰ ๋น„์„œ")
st.markdown("---")

# ์„ธ์…˜ ์ƒํƒœ ์ดˆ๊ธฐํ™”
if "messages" not in st.session_state:
    st.session_state.messages = []
if "chain" not in st.session_state:
    # ์ฒด์ธ ์ดˆ๊ธฐํ™”
    embeddings = OpenAIEmbeddings(model=Config.EMBEDDING_MODEL)
    vectorstore = Chroma(
        persist_directory=Config.VECTORSTORE_PATH,
        embedding_function=embeddings
    )
    
    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
        output_key="answer"
    )
    
    llm = ChatOpenAI(
        model_name=Config.LLM_MODEL,
        temperature=Config.TEMPERATURE
    )
    
    st.session_state.chain = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=vectorstore.as_retriever(search_kwargs={"k": Config.SEARCH_K}),
        memory=memory,
        return_source_documents=True
    )

# ์ฑ„ํŒ… ๊ธฐ๋ก ํ‘œ์‹œ
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

# ์‚ฌ์šฉ์ž ์ž…๋ ฅ
if prompt := st.chat_input("์งˆ๋ฌธ์„ ์ž…๋ ฅํ•˜์„ธ์š”..."):
    # ์‚ฌ์šฉ์ž ๋ฉ”์‹œ์ง€ ์ถ”๊ฐ€
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)
    
    # AI ์‘๋‹ต ์ƒ์„ฑ
    with st.chat_message("assistant"):
        with st.spinner("์ƒ๊ฐ ์ค‘..."):
            response = st.session_state.chain.invoke({"question": prompt})
            answer = response["answer"]
            sources = response["source_documents"]
            
            st.markdown(answer)
            
            # ์ถœ์ฒ˜ ํ‘œ์‹œ
            if sources:
                with st.expander("๐Ÿ“š ์ฐธ๊ณ  ๋ฌธ์„œ"):
                    for i, doc in enumerate(sources):
                        st.markdown(f"**[{i+1}]** {doc.metadata.get('source', 'Unknown')}")
                        st.caption(doc.page_content[:200] + "...")
    
    # AI ๋ฉ”์‹œ์ง€ ์ถ”๊ฐ€
    st.session_state.messages.append({"role": "assistant", "content": answer})

# ์‚ฌ์ด๋“œ๋ฐ”
with st.sidebar:
    st.header("โš™๏ธ ์„ค์ •")
    st.markdown(f"**๋ชจ๋ธ:** {Config.LLM_MODEL}")
    st.markdown(f"**Temperature:** {Config.TEMPERATURE}")
    st.markdown(f"**๊ฒ€์ƒ‰ ๋ฌธ์„œ ์ˆ˜:** {Config.SEARCH_K}")
    
    if st.button("๐Ÿ—‘๏ธ ๋Œ€ํ™” ๊ธฐ๋ก ์‚ญ์ œ"):
        st.session_state.messages = []
        st.session_state.chain.memory.clear()
        st.rerun()
# ์‹คํ–‰ ๋ฐฉ๋ฒ•
streamlit run streamlit_app.py

์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ๋ธŒ๋ผ์šฐ์ €์—์„œ ๋ฐ”๋กœ ์ฑ„ํŒ…ํ•˜๋ฉด์„œ ํ…Œ์ŠคํŠธํ•  ์ˆ˜ ์žˆ์–ด์š”! ์™„์ „ ๊ฐ„๋‹จํ•˜์ฃ ? ใ…‹ใ…‹ใ…‹

๐Ÿค– AI ๋ฌธ์„œ ๊ฒ€์ƒ‰ ๋น„์„œ ์‚ฌ์šฉ์ž: ์šฐ๋ฆฌ ํšŒ์‚ฌ ์—ฐ์ฐจ ์ •์ฑ… ์•Œ๋ ค์ค˜ AI ๋น„์„œ: ์šฐ๋ฆฌ ํšŒ์‚ฌ์˜ ์—ฐ์ฐจ ์ •์ฑ…์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค: โ€ข ์ž…์‚ฌ 1๋…„์ฐจ: 15์ผ โ€ข 3๋…„ ์ด์ƒ ๊ทผ์†: ๋งค๋…„ 1์ผ ์ถ”๊ฐ€ โš™๏ธ ์„ค์ • ๋ชจ๋ธ: GPT-4 Temperature: 0.3 ๊ฒ€์ƒ‰ ๋ฌธ์„œ: 4๊ฐœ ๋Œ€ํ™” ๊ธฐ๋ก ์‚ญ์ œ โœ“

โšก ์„ฑ๋Šฅ ์ตœ์ ํ™”์™€ ๋น„์šฉ ๊ด€๋ฆฌ

์‹ค์ œ ์„œ๋น„์Šค์— ๋ฐฐํฌํ•˜๋ฉด ์„ฑ๋Šฅ๊ณผ ๋น„์šฉ์ด ์ง„์งœ ์ค‘์š”ํ•ด์ ธ์š”. ๋ช‡ ๊ฐ€์ง€ ๊ฟ€ํŒ ์•Œ๋ ค๋“œ๋ฆด๊ฒŒ์š” ๐Ÿ’ฐ

๐Ÿ’ธ ๋น„์šฉ ์ ˆ๊ฐ ์ „๋žต 1. ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ ์„ ํƒ
text-embedding-3-small์ด large๋ณด๋‹ค 5๋ฐฐ ์ €๋ ดํ•ด์š”. ๋Œ€๋ถ€๋ถ„์˜ ๊ฒฝ์šฐ small์œผ๋กœ ์ถฉ๋ถ„!

2. ์บ์‹ฑ ์ ๊ทน ํ™œ์šฉ
Redis๋‚˜ Memcached๋กœ ์ž์ฃผ ๋ฌป๋Š” ์งˆ๋ฌธ ์บ์‹ฑํ•˜๋ฉด API ํ˜ธ์ถœ 80% ์ด์ƒ ์ค„์ผ ์ˆ˜ ์žˆ์–ด์š”.

3. ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ
๋ฌธ์„œ ์ž„๋ฒ ๋”ฉ ์ƒ์„ฑํ•  ๋•Œ ๋ฐฐ์น˜๋กœ ์ฒ˜๋ฆฌํ•˜๋ฉด ๋น„์šฉ ์ ˆ๊ฐ ๊ฐ€๋Šฅ.

4. ํ† ํฐ ์ œํ•œ
max_tokens ํŒŒ๋ผ๋ฏธํ„ฐ๋กœ ์‘๋‹ต ๊ธธ์ด ์ œํ•œํ•˜๋ฉด ๋ถˆํ•„์š”ํ•œ ๋น„์šฉ ๋ฐฉ์ง€.

5. ์ €๋ ดํ•œ ๋ชจ๋ธ ํ™œ์šฉ
๊ฐ„๋‹จํ•œ ์ž‘์—…์€ gpt-3.5-turbo, ๋ณต์žกํ•œ ์ž‘์—…๋งŒ gpt-4 ์‚ฌ์šฉ
# ๋น„์šฉ ์ตœ์ ํ™” ์„ค์ • ์˜ˆ์‹œ
from langchain.callbacks import get_openai_callback

# ํ† ํฐ ์‚ฌ์šฉ๋Ÿ‰ ์ถ”์ 
with get_openai_callback() as cb:
    response = qa_chain.invoke({"query": "์—ฐ์ฐจ ์ •์ฑ…์€?"})
    print(f"์ด ํ† ํฐ: {cb.total_tokens}")
    print(f"ํ”„๋กฌํ”„ํŠธ ํ† ํฐ: {cb.prompt_tokens}")
    print(f"์™„๋ฃŒ ํ† ํฐ: {cb.completion_tokens}")
    print(f"์ด ๋น„์šฉ: ${cb.total_cost:.4f}")

# ํ† ํฐ ์ œํ•œ ์„ค์ •
llm = ChatOpenAI(
    model_name="gpt-4o-mini",
    temperature=0.3,
    max_tokens=500  # ์‘๋‹ต ๊ธธ์ด ์ œํ•œ
)

# ๊ฒ€์ƒ‰ ๋ฌธ์„œ ์ˆ˜ ์ตœ์ ํ™”
retriever = vectorstore.as_retriever(
    search_kwargs={
        "k": 3  # 4๊ฐœ ๋Œ€์‹  3๊ฐœ๋กœ ์ค„์ด๋ฉด ํ† ํฐ ์ ˆ์•ฝ
    }
)
๐Ÿš€ ์„ฑ๋Šฅ ์ตœ์ ํ™” ํŒ โ€ข ๋ฒกํ„ฐ DB ์ธ๋ฑ์‹ฑ - HNSW ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์‚ฌ์šฉํ•˜๋ฉด ๊ฒ€์ƒ‰ ์†๋„ 10๋ฐฐ ํ–ฅ์ƒ
โ€ข ๋น„๋™๊ธฐ ์ฒ˜๋ฆฌ - asyncio๋กœ ์—ฌ๋Ÿฌ ๊ฒ€์ƒ‰ ๋™์‹œ ์ฒ˜๋ฆฌ
โ€ข ๋ฌธ์„œ ์ „์ฒ˜๋ฆฌ - ๋ถˆํ•„์š”ํ•œ ๊ณต๋ฐฑ, ํŠน์ˆ˜๋ฌธ์ž ์ œ๊ฑฐ๋กœ ์ž„๋ฒ ๋”ฉ ํ’ˆ์งˆ ํ–ฅ์ƒ
โ€ข ๋กœ๋“œ ๋ฐธ๋Ÿฐ์‹ฑ - ์—ฌ๋Ÿฌ API ํ‚ค ๋กœํ…Œ์ด์…˜์œผ๋กœ Rate Limit ํšŒํ”ผ
โ€ข ์ŠคํŠธ๋ฆฌ๋ฐ ์‘๋‹ต - ๊ธด ๋‹ต๋ณ€์€ ์ŠคํŠธ๋ฆฌ๋ฐ์œผ๋กœ ์‚ฌ์šฉ์ž ๊ฒฝํ—˜ ๊ฐœ์„ 
# ์ŠคํŠธ๋ฆฌ๋ฐ ์‘๋‹ต ์˜ˆ์‹œ
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = ChatOpenAI(
    model_name="gpt-4o-mini",
    temperature=0.3,
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()]
)

# ์ด์ œ ๋‹ต๋ณ€์ด ์ƒ์„ฑ๋˜๋Š” ๋Œ€๋กœ ์‹ค์‹œ๊ฐ„์œผ๋กœ ์ถœ๋ ฅ๋ผ์š”!
response = qa_chain.invoke({"query": "ํšŒ์‚ฌ ๋ณต์ง€ ์ œ๋„ ์„ค๋ช…ํ•ด์ค˜"})

๐Ÿ› ์ž์ฃผ ๋งŒ๋‚˜๋Š” ์—๋Ÿฌ์™€ ํ•ด๊ฒฐ๋ฒ•

๊ฐœ๋ฐœํ•˜๋‹ค ๋ณด๋ฉด ์—๋Ÿฌ ์ง„์งœ ๋งŽ์ด ๋งŒ๋‚˜์ฃ  ใ…‹ใ…‹ใ…‹ ์ œ๊ฐ€ ๊ฒช์—ˆ๋˜ ์ฃผ์š” ์—๋Ÿฌ๋“ค๊ณผ ํ•ด๊ฒฐ๋ฒ• ์ •๋ฆฌํ•ด๋“œ๋ฆด๊ฒŒ์š”!

โŒ Error 1: "RateLimitError: Rate limit exceeded" ์›์ธ: OpenAI API ํ˜ธ์ถœ ์ œํ•œ ์ดˆ๊ณผ
ํ•ด๊ฒฐ:
โ€ข ์š”์ฒญ ๊ฐ„ time.sleep() ์ถ”๊ฐ€
โ€ข ๋ฐฐ์น˜ ํฌ๊ธฐ ์ค„์ด๊ธฐ
โ€ข ์œ ๋ฃŒ ํ”Œ๋žœ์œผ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œ
โ€ข ์—ฌ๋Ÿฌ API ํ‚ค ๋กœํ…Œ์ด์…˜
โŒ Error 2: "InvalidRequestError: This model's maximum context length is..." ์›์ธ: ํ† ํฐ ์ œํ•œ ์ดˆ๊ณผ
ํ•ด๊ฒฐ:
โ€ข chunk_size ์ค„์ด๊ธฐ
โ€ข ๊ฒ€์ƒ‰ ๋ฌธ์„œ ์ˆ˜(k) ์ค„์ด๊ธฐ
โ€ข map_reduce chain type ์‚ฌ์šฉ
โ€ข ๊ธด ๋Œ€ํ™” ๊ธฐ๋ก ์š”์•ฝ ๋˜๋Š” ์‚ญ์ œ
โŒ Error 3: "ChromaDB: Collection already exists" ์›์ธ: ๊ฐ™์€ ์ด๋ฆ„์˜ ์ปฌ๋ ‰์…˜์ด ์ด๋ฏธ ์กด์žฌ
ํ•ด๊ฒฐ:
โ€ข ๊ธฐ์กด ์ปฌ๋ ‰์…˜ ์‚ญ์ œ ํ›„ ์žฌ์ƒ์„ฑ
โ€ข ๋‹ค๋ฅธ ์ปฌ๋ ‰์…˜ ์ด๋ฆ„ ์‚ฌ์šฉ
โ€ข get_or_create ๋ฉ”์„œ๋“œ ์‚ฌ์šฉ
# ์—๋Ÿฌ ์ฒ˜๋ฆฌ ์˜ˆ์‹œ
import time
from openai import RateLimitError

def safe_query(chain, question, max_retries=3):
    """์žฌ์‹œ๋„ ๋กœ์ง์ด ํฌํ•จ๋œ ์•ˆ์ „ํ•œ ์ฟผ๋ฆฌ ํ•จ์ˆ˜"""
    for attempt in range(max_retries):
        try:
            response = chain.invoke({"query": question})
            return response
        except RateLimitError:
            if attempt < max_retries - 1:
                wait_time = (attempt + 1) * 2  # ์ง€์ˆ˜ ๋ฐฑ์˜คํ”„
                print(f"Rate limit ๋„๋‹ฌ. {wait_time}์ดˆ ๋Œ€๊ธฐ ์ค‘...")
                time.sleep(wait_time)
            else:
                raise
        except Exception as e:
            print(f"์—๋Ÿฌ ๋ฐœ์ƒ: {str(e)}")
            if attempt < max_retries - 1:
                time.sleep(1)
            else:
                raise

# ์‚ฌ์šฉ ์˜ˆ์‹œ
try:
    result = safe_query(qa_chain, "์—ฐ์ฐจ ์ •์ฑ…์€?")
    print(result["answer"])
except Exception as e:
    print(f"์ตœ์ข… ์‹คํŒจ: {str(e)}")

๐Ÿ” ๋ณด์•ˆ๊ณผ ํ”„๋ผ์ด๋ฒ„์‹œ ๊ณ ๋ ค์‚ฌํ•ญ

ํšŒ์‚ฌ ๋‚ด๋ถ€ ๋ฌธ์„œ ๋‹ค๋ฃจ๋Š” ๊ฑฐ๋‹ˆ๊นŒ ๋ณด์•ˆ ์ง„์งœ ์ค‘์š”ํ•ด์š”! ์ด๊ฑฐ ๋†“์น˜๋ฉด ํฐ์ผ ๋‚˜์š” ๐Ÿ˜ฑ

๐Ÿ”’ ๋ณด์•ˆ ์ฒดํฌ๋ฆฌ์ŠคํŠธ โœ“ API ํ‚ค ๊ด€๋ฆฌ
โ€ข ํ™˜๊ฒฝ๋ณ€์ˆ˜ ์‚ฌ์šฉ, ์ ˆ๋Œ€ ์ฝ”๋“œ์— ํ•˜๋“œ์ฝ”๋”ฉ ๊ธˆ์ง€
โ€ข .env ํŒŒ์ผ .gitignore์— ์ถ”๊ฐ€
โ€ข ํ”„๋กœ๋•์…˜์—์„œ๋Š” AWS Secrets Manager ๊ฐ™์€ ์„œ๋น„์Šค ์‚ฌ์šฉ

โœ“ ๋ฐ์ดํ„ฐ ์•”ํ˜ธํ™”
โ€ข ๋ฒกํ„ฐ DB ์ €์žฅ ์‹œ ์•”ํ˜ธํ™”
โ€ข ์ „์†ก ์ค‘ ๋ฐ์ดํ„ฐ๋Š” HTTPS ์‚ฌ์šฉ
โ€ข ๋ฏผ๊ฐ ์ •๋ณด๋Š” ๋งˆ์Šคํ‚น ์ฒ˜๋ฆฌ

โœ“ ์ ‘๊ทผ ์ œ์–ด
โ€ข ์‚ฌ์šฉ์ž ์ธ์ฆ/์ธ๊ฐ€ ๊ตฌํ˜„
โ€ข ๋ถ€์„œ๋ณ„, ๊ถŒํ•œ๋ณ„ ๋ฌธ์„œ ์ ‘๊ทผ ์ œํ•œ
โ€ข ๊ฐ์‚ฌ ๋กœ๊ทธ ๊ธฐ๋ก

โœ“ ๋ฐ์ดํ„ฐ ๋ณด์กด
โ€ข ๊ฐœ์ธ์ •๋ณด ํฌํ•จ ๋ฌธ์„œ๋Š” ๋ณด์กด ๊ธฐ๊ฐ„ ์„ค์ •
โ€ข ์ •๊ธฐ์ ์ธ ๋ฐ์ดํ„ฐ ์‚ญ์ œ ์ •์ฑ…
โ€ข GDPR, ๊ฐœ์ธ์ •๋ณด๋ณดํ˜ธ๋ฒ• ์ค€์ˆ˜
# ๋ฏผ๊ฐ ์ •๋ณด ๋งˆ์Šคํ‚น ์˜ˆ์‹œ
import re

def mask_sensitive_info(text):
    """๋ฏผ๊ฐ ์ •๋ณด ๋งˆ์Šคํ‚น ํ•จ์ˆ˜"""
    # ์ฃผ๋ฏผ๋“ฑ๋ก๋ฒˆํ˜ธ ๋งˆ์Šคํ‚น
    text = re.sub(r'\d{6}-\d{7}', '******-*******', text)
    
    # ์ด๋ฉ”์ผ ๋งˆ์Šคํ‚น
    text = re.sub(r'[\w\.-]+@[\w\.-]+', '***@***.com', text)
    
    # ์ „ํ™”๋ฒˆํ˜ธ ๋งˆ์Šคํ‚น
    text = re.sub(r'\d{2,3}-\d{3,4}-\d{4}', '***-****-****', text)
    
    # ์นด๋“œ๋ฒˆํ˜ธ ๋งˆ์Šคํ‚น
    text = re.sub(r'\d{4}-\d{4}-\d{4}-\d{4}', '****-****-****-****', text)
    
    return text

# ๋ฌธ์„œ ์ „์ฒ˜๋ฆฌ ์‹œ ์ ์šฉ
for doc in documents:
    doc.page_content = mask_sensitive_info(doc.page_content)
๐Ÿ’ก ํ”„๋ผ์ด๋ฒ„์‹œ ํŒ OpenAI API๋Š” ๊ธฐ๋ณธ์ ์œผ๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ํ•™์Šต์— ์‚ฌ์šฉํ•˜์ง€ ์•Š์ง€๋งŒ, ํ™•์‹คํ•˜๊ฒŒ ํ•˜๋ ค๋ฉด Enterprise ํ”Œ๋žœ์„ ์‚ฌ์šฉํ•˜๊ฑฐ๋‚˜ Azure OpenAI Service๋ฅผ ๊ณ ๋ คํ•ด๋ณด์„ธ์š”. ๋˜๋Š” ์˜คํ”ˆ์†Œ์Šค LLM(LLaMA, Mistral ๋“ฑ)์„ ์ž์ฒด ์„œ๋ฒ„์— ํ˜ธ์ŠคํŒ…ํ•˜๋Š” ๊ฒƒ๋„ ๋ฐฉ๋ฒ•์ด์—์š”!

๐ŸŒŸ ์‹ค์ „ ํ™œ์šฉ ์‚ฌ๋ก€

์ด๋ก ๋งŒ ๋ฐฐ์šฐ๋ฉด ์žฌ๋ฏธ์—†์ž–์•„์š”? ์‹ค์ œ๋กœ ์–ด๋–ป๊ฒŒ ํ™œ์šฉ๋˜๋Š”์ง€ ์‚ฌ๋ก€ ๋ช‡ ๊ฐœ ๋ณด์—ฌ๋“œ๋ฆด๊ฒŒ์š” ๐ŸŽฏ

1
๊ณ ๊ฐ ์ง€์› ์ฑ—๋ด‡
์ œํ’ˆ ๋งค๋‰ด์–ผ, FAQ, ์ด์ „ ์ƒ๋‹ด ๋‚ด์—ญ์„ ํ•™์Šตํ•ด์„œ 24/7 ๊ณ ๊ฐ ์‘๋Œ€. ์ƒ๋‹ด์‚ฌ ์—…๋ฌด ๋ถ€๋‹ด 70% ๊ฐ์†Œ ํšจ๊ณผ!
2
์‚ฌ๋‚ด ์ง€์‹ ๊ด€๋ฆฌ ์‹œ์Šคํ…œ
ํšŒ์‚ฌ ๊ทœ์ •, ์—…๋ฌด ๋งค๋‰ด์–ผ, ํ”„๋กœ์ ํŠธ ๋ฌธ์„œ ๊ฒ€์ƒ‰. ์‹ ์ž…์‚ฌ์› ์˜จ๋ณด๋”ฉ ์‹œ๊ฐ„ 50% ๋‹จ์ถ•. ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ๋„ ์ด๋Ÿฐ ์‹œ์Šคํ…œ ๊ตฌ์ถ• ์žฌ๋Šฅ์„ ๊ฑฐ๋ž˜ํ•  ์ˆ˜ ์žˆ์–ด์š”!
3
๋ฒ•๋ฅ  ๋ฌธ์„œ ๋ถ„์„
ํŒ๋ก€, ๋ฒ•๋ น, ๊ณ„์•ฝ์„œ ๊ฒ€์ƒ‰ ๋ฐ ๋ถ„์„. ๋ณ€ํ˜ธ์‚ฌ๋“ค์˜ ๋ฆฌ์„œ์น˜ ์‹œ๊ฐ„ 60% ์ ˆ๊ฐ.
4
์˜๋ฃŒ ์ •๋ณด ์–ด์‹œ์Šคํ„ดํŠธ
์˜ํ•™ ๋…ผ๋ฌธ, ์ง„๋ฃŒ ๊ฐ€์ด๋“œ๋ผ์ธ ๊ฒ€์ƒ‰. ์˜์‚ฌ๋“ค์˜ ์˜์‚ฌ๊ฒฐ์ • ์ง€์› (์ง„๋‹จ/์ฒ˜๋ฐฉ ์•„๋‹˜!).
5
์—ฐ๊ตฌ ๋…ผ๋ฌธ ์š”์•ฝ
์ˆ˜์ฒœ ๊ฐœ์˜ ๋…ผ๋ฌธ์—์„œ ๊ด€๋ จ ์—ฐ๊ตฌ ์ฐพ๊ณ  ์š”์•ฝ. ์—ฐ๊ตฌ์ž๋“ค์˜ ๋ฌธํ—Œ ์กฐ์‚ฌ ์‹œ๊ฐ„ ๋Œ€ํญ ๋‹จ์ถ•.

๐Ÿš€ ๋‹ค์Œ ๋‹จ๊ณ„ - ๋” ๋ฐœ์ „์‹œํ‚ค๊ธฐ

๊ธฐ๋ณธ์€ ๋‹ค ๋ฐฐ์› ์œผ๋‹ˆ ์ด์ œ ๋” ๊ณ ๊ธ‰ ๊ธฐ๋Šฅ๋“ค์„ ์ถ”๊ฐ€ํ•ด๋ณผ ์ฐจ๋ก€์˜ˆ์š”! ๐ŸŽ“

๐ŸŽฏ ๊ณ ๊ธ‰ ๊ธฐ๋Šฅ ์•„์ด๋””์–ด 1. Multi-Modal RAG
ํ…์ŠคํŠธ๋ฟ๋งŒ ์•„๋‹ˆ๋ผ ์ด๋ฏธ์ง€, ํ‘œ, ๊ทธ๋ž˜ํ”„๋„ ์ดํ•ดํ•˜๋Š” ์‹œ์Šคํ…œ. GPT-4 Vision ํ™œ์šฉ!

2. Agent ๊ธฐ๋ฐ˜ ์‹œ์Šคํ…œ
์—ฌ๋Ÿฌ ๋„๊ตฌ(์›น ๊ฒ€์ƒ‰, ๊ณ„์‚ฐ๊ธฐ, ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ๋“ฑ)๋ฅผ ์ž์œจ์ ์œผ๋กœ ์‚ฌ์šฉํ•˜๋Š” AI ์—์ด์ „ํŠธ.

3. ๋‹ค๊ตญ์–ด ์ง€์›
ํ•œ๊ตญ์–ด, ์˜์–ด, ์ผ๋ณธ์–ด ๋“ฑ ์—ฌ๋Ÿฌ ์–ธ์–ด ๋ฌธ์„œ ํ†ตํ•ฉ ๊ฒ€์ƒ‰.

4. ์‹ค์‹œ๊ฐ„ ์—…๋ฐ์ดํŠธ
์ƒˆ ๋ฌธ์„œ ์ถ”๊ฐ€๋˜๋ฉด ์ž๋™์œผ๋กœ ๋ฒกํ„ฐ DB ์—…๋ฐ์ดํŠธ.

5. ํ”ผ๋“œ๋ฐฑ ๋ฃจํ”„
์‚ฌ์šฉ์ž ํ”ผ๋“œ๋ฐฑ ๋ฐ›์•„์„œ ๊ฒ€์ƒ‰ ํ’ˆ์งˆ ์ง€์†์ ์œผ๋กœ ๊ฐœ์„ .

6. ์Œ์„ฑ ์ธํ„ฐํŽ˜์ด์Šค
Whisper API๋กœ ์Œ์„ฑ ์งˆ๋ฌธ ๋ฐ›๊ณ , TTS๋กœ ๋‹ต๋ณ€ ์ถœ๋ ฅ.
# Agent ์˜ˆ์‹œ (๊ฐ„๋‹จ ๋ฒ„์ „)
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType

# ๋„๊ตฌ ์ •์˜
tools = [
    Tool(
        name="Document Search",
        func=qa_chain.invoke,
        description="ํšŒ์‚ฌ ๋‚ด๋ถ€ ๋ฌธ์„œ๋ฅผ ๊ฒ€์ƒ‰ํ•  ๋•Œ ์‚ฌ์šฉ. ์งˆ๋ฌธ์„ ์ž…๋ ฅ์œผ๋กœ ๋ฐ›์Œ."
    ),
    Tool(
        name="Calculator",
        func=lambda x: eval(x),
        description="์ˆ˜ํ•™ ๊ณ„์‚ฐ์ด ํ•„์š”ํ•  ๋•Œ ์‚ฌ์šฉ. ์ˆ˜์‹์„ ์ž…๋ ฅ์œผ๋กœ ๋ฐ›์Œ."
    )
]

# Agent ์ดˆ๊ธฐํ™”
agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)

# Agent ์‹คํ–‰
result = agent.run("์—ฐ์ฐจ๊ฐ€ 15์ผ์ธ๋ฐ, ์ด๋ฏธ 5์ผ ์ผ์–ด. ๋‚จ์€ ์—ฐ์ฐจ๋Š”?")
print(result)

๐Ÿ“Š ์„ฑ๋Šฅ ์ธก์ •๊ณผ ํ‰๊ฐ€

๋งŒ๋“ค๊ธฐ๋งŒ ํ•˜๊ณ  ๋์ด ์•„๋‹ˆ์—์š”! ์–ผ๋งˆ๋‚˜ ์ž˜ ์ž‘๋™ํ•˜๋Š”์ง€ ์ธก์ •ํ•ด์•ผ์ฃ . ํ‰๊ฐ€ ์ง€ํ‘œ ์•Œ๋ ค๋“œ๋ฆด๊ฒŒ์š” ๐Ÿ“ˆ

๐Ÿ“ ์ฃผ์š” ํ‰๊ฐ€ ์ง€ํ‘œ โ€ข Retrieval Accuracy - ๊ด€๋ จ ๋ฌธ์„œ๋ฅผ ์–ผ๋งˆ๋‚˜ ์ •ํ™•ํ•˜๊ฒŒ ์ฐพ๋Š”๊ฐ€?
โ€ข Answer Relevance - ๋‹ต๋ณ€์ด ์งˆ๋ฌธ๊ณผ ์–ผ๋งˆ๋‚˜ ๊ด€๋ จ์žˆ๋Š”๊ฐ€?
โ€ข Faithfulness - ๋‹ต๋ณ€์ด ๊ฒ€์ƒ‰๋œ ๋ฌธ์„œ์— ์ถฉ์‹คํ•œ๊ฐ€? (ํ™˜๊ฐ ๋ฐฉ์ง€)
โ€ข Response Time - ๋‹ต๋ณ€ ์ƒ์„ฑ ์†๋„
โ€ข User Satisfaction - ์‚ฌ์šฉ์ž ๋งŒ์กฑ๋„ (ํ”ผ๋“œ๋ฐฑ ๊ธฐ๋ฐ˜)
# ํ‰๊ฐ€ ๋ฐ์ดํ„ฐ์…‹ ์˜ˆ์‹œ
evaluation_dataset = [
    {
        "question": "์—ฐ์ฐจ๋Š” ๋ช‡ ์ผ์ธ๊ฐ€์š”?",
        "expected_answer": "15์ผ",
        "relevant_docs": ["hr_policy.pdf"]
    },
    {
        "question": "์žฌํƒ๊ทผ๋ฌด ๊ฐ€๋Šฅํ•œ๊ฐ€์š”?",
        "expected_answer": "์ฃผ 2ํšŒ ๊ฐ€๋Šฅ",
        "relevant_docs": ["work_policy.pdf"]
    }
]

# ๊ฐ„๋‹จํ•œ ํ‰๊ฐ€ ํ•จ์ˆ˜
def evaluate_rag_system(chain, dataset):
    results = []
    
    for item in dataset:
        response = chain.invoke({"query": item["question"]})
        
        # ๋‹ต๋ณ€ ๊ด€๋ จ์„ฑ ์ฒดํฌ (๊ฐ„๋‹จ ๋ฒ„์ „)
        answer_contains_expected = item["expected_answer"].lower() in response["answer"].lower()
        
        # ๊ฒ€์ƒ‰ ์ •ํ™•๋„ ์ฒดํฌ
        retrieved_sources = [doc.metadata.get("source") for doc in response["source_documents"]]
        retrieval_correct = any(doc in retrieved_sources for doc in item["relevant_docs"])
        
        results.append({
            "question": item["question"],
            "answer_relevant": answer_contains_expected,
            "retrieval_correct": retrieval_correct
        })
    
    # ์ „์ฒด ์ •ํ™•๋„ ๊ณ„์‚ฐ
    answer_accuracy = sum(r["answer_relevant"] for r in results) / len(results)
    retrieval_accuracy = sum(r["retrieval_correct"] for r in results) / len(results)
    
    print(f"๋‹ต๋ณ€ ์ •ํ™•๋„: {answer_accuracy:.2%}")
    print(f"๊ฒ€์ƒ‰ ์ •ํ™•๋„: {retrieval_accuracy:.2%}")
    
    return results

# ํ‰๊ฐ€ ์‹คํ–‰
eval_results = evaluate_rag_system(qa_chain, evaluation_dataset)
๐Ÿ’ก A/B ํ…Œ์ŠคํŒ… ์ถ”์ฒœ ์—ฌ๋Ÿฌ ์„ค์ •(chunk_size, ๊ฒ€์ƒ‰ ์•Œ๊ณ ๋ฆฌ์ฆ˜, ํ”„๋กฌํ”„ํŠธ ๋“ฑ)์„ ๋น„๊ต ํ…Œ์ŠคํŠธํ•ด๋ณด์„ธ์š”. ์ž‘์€ ์ฐจ์ด๊ฐ€ ํฐ ์„ฑ๋Šฅ ์ฐจ์ด๋ฅผ ๋งŒ๋“ค์–ด์š”! ์‹ค์ œ ์‚ฌ์šฉ์ž ํ”ผ๋“œ๋ฐฑ์ด ๊ฐ€์žฅ ์ค‘์š”ํ•œ ์ง€ํ‘œ์˜ˆ์š”.

๐ŸŽ“ ํ•™์Šต ๋ฆฌ์†Œ์Šค์™€ ์ปค๋ฎค๋‹ˆํ‹ฐ

ํ˜ผ์ž ๊ณต๋ถ€ํ•˜๊ธฐ ํž˜๋“ค์ฃ ? ๋„์›€ ๋ฐ›์„ ์ˆ˜ ์žˆ๋Š” ๊ณณ๋“ค ์ •๋ฆฌํ•ด๋“œ๋ฆด๊ฒŒ์š” ๐Ÿ“š

๐Ÿ“– ์ถ”์ฒœ ํ•™์Šต ์ž๋ฃŒ ๊ณต์‹ ๋ฌธ์„œ
โ€ข LangChain ๊ณต์‹ ๋ฌธ์„œ (python.langchain.com)
โ€ข OpenAI API ๋ฌธ์„œ (platform.openai.com/docs)

์˜จ๋ผ์ธ ๊ฐ•์˜
โ€ข DeepLearning.AI - LangChain ๊ฐ•์ขŒ
โ€ข Udemy - RAG ์‹œ์Šคํ…œ ๊ตฌ์ถ• ์‹ค์ „

์ปค๋ฎค๋‹ˆํ‹ฐ
โ€ข LangChain Discord
โ€ข Reddit r/LangChain
โ€ข Stack Overflow

๋ธ”๋กœ๊ทธ/๋‰ด์Šค๋ ˆํ„ฐ
โ€ข LangChain ๊ณต์‹ ๋ธ”๋กœ๊ทธ
โ€ข Towards Data Science
โ€ข The Batch (Andrew Ng)
๐ŸŒŸ ์žฌ๋Šฅ๋„ท์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ ๊ณต์œ ํ•˜๊ธฐ ์ด ์ •๋„ ์‹ค๋ ฅ์ด๋ฉด ์žฌ๋Šฅ๋„ท(https://www.jaenung.net)์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•  ์ˆ˜ ์žˆ์–ด์š”!
โ€ข "RAG ์‹œ์Šคํ…œ ๊ตฌ์ถ• ์ปจ์„คํŒ…"
โ€ข "๋งž์ถคํ˜• AI ์ฑ—๋ด‡ ๊ฐœ๋ฐœ"
โ€ข "LangChain ํ”„๋กœ์ ํŠธ ๋ฉ˜ํ† ๋ง"
๊ฐ™์€ ์žฌ๋Šฅ์„ ๋“ฑ๋กํ•ด์„œ ์ˆ˜์ต๋„ ์ฐฝ์ถœํ•˜๊ณ , ๋‹ค๋ฅธ ๊ฐœ๋ฐœ์ž๋“ค๊ณผ ์ง€์‹๋„ ๋‚˜๋ˆ ๋ณด์„ธ์š” ใ…‹ใ…‹ใ…‹

๐ŸŽฌ ๋งˆ๋ฌด๋ฆฌํ•˜๋ฉฐ

์™€... ์—ฌ๊ธฐ๊นŒ์ง€ ์ฝ์œผ์…จ๋‹ค๋ฉด ์ง„์งœ ๋Œ€๋‹จํ•˜์„ธ์š”! ๐Ÿ‘๐Ÿ‘๐Ÿ‘

ํŒŒ์ด์ฌ๊ณผ LangChain์œผ๋กœ ๊ฒ€์ƒ‰ ๊ธฐ๋ฐ˜ AI ๋น„์„œ๋ฅผ ๋งŒ๋“œ๋Š” ๋ฐฉ๋ฒ•, ์ฒ˜์Œ๋ถ€ํ„ฐ ๋๊นŒ์ง€ ๋‹ค ์•Œ๋ ค๋“œ๋ ธ์–ด์š”. ๋ฌธ์„œ ๋กœ๋”ฉ๋ถ€ํ„ฐ ์ž„๋ฒ ๋”ฉ, ๋ฒกํ„ฐ ๊ฒ€์ƒ‰, ์ฒด์ธ ๊ตฌ์„ฑ, UI ๋งŒ๋“ค๊ธฐ, ์ตœ์ ํ™”, ๋ณด์•ˆ๊นŒ์ง€... ์ง„์งœ ์•Œ์ฐจ๊ฒŒ ๋‹ด์•˜์ฃ ? ใ…‹ใ…‹ใ…‹

์†”์งํžˆ ์ฒ˜์Œ์—๋Š” ์–ด๋ ค์›Œ ๋ณด์ผ ์ˆ˜ ์žˆ์–ด์š”. ๊ทผ๋ฐ ํ•˜๋‚˜์”ฉ ๋”ฐ๋ผํ•˜๋‹ค ๋ณด๋ฉด "์–ด? ์ƒ๊ฐ๋ณด๋‹ค ํ•  ๋งŒํ•œ๋ฐ?" ํ•˜๋Š” ์ˆœ๊ฐ„์ด ์™€์š”. ๊ทธ๋•Œ์˜ ํฌ์—ด์ด ์ง„์งœ ์งœ๋ฆฟํ•˜๊ฑฐ๋“ ์š” ๐ŸŽ‰

์ค‘์š”ํ•œ ๊ฑด ์ผ๋‹จ ์‹œ์ž‘ํ•˜๋Š” ๊ฑฐ์˜ˆ์š”. ์™„๋ฒฝํ•œ ์ฝ”๋“œ ์งœ๋ ค๊ณ  ํ•˜์ง€ ๋ง๊ณ , ์ผ๋‹จ ๋Œ์•„๊ฐ€๋Š” ๊ฑฐ ๋งŒ๋“ค์–ด๋ณด์„ธ์š”. ๊ทธ๋‹ค์Œ์— ์กฐ๊ธˆ์”ฉ ๊ฐœ์„ ํ•˜๋ฉด ๋ผ์š”. ์—๋Ÿฌ ๋‚˜๋„ ๋‹นํ™ฉํ•˜์ง€ ๋ง๊ณ  ๊ตฌ๊ธ€๋งํ•˜๋ฉด์„œ ํ•˜๋‚˜์”ฉ ํ•ด๊ฒฐํ•ด๋‚˜๊ฐ€๋ฉด ๋ฉ๋‹ˆ๋‹ค!

AI ๊ธฐ์ˆ ์€ ๊ณ„์† ๋ฐœ์ „ํ•˜๊ณ  ์žˆ์–ด์š”. LangChain๋„ ์—…๋ฐ์ดํŠธ๊ฐ€ ์—„์ฒญ ๋น ๋ฅด๊ณ , ์ƒˆ๋กœ์šด ๊ธฐ๋Šฅ๋“ค์ด ๊ณ„์† ๋‚˜์˜ค๊ณ  ์žˆ์ฃ . ๊ทธ๋Ÿฌ๋‹ˆ๊นŒ ์ด ๊ธ€์„ ๊ธฐ๋ฐ˜์œผ๋กœ ๊ณ„์† ๊ณต๋ถ€ํ•˜๊ณ  ์‹คํ—˜ํ•ด๋ณด์„ธ์š”. ๊ณต์‹ ๋ฌธ์„œ๋„ ์ž์ฃผ ํ™•์ธํ•˜๊ณ ์š”!

๋งˆ์ง€๋ง‰์œผ๋กœ, ํ˜ผ์ž ํ•˜๊ธฐ ํž˜๋“ค๋ฉด ์ปค๋ฎค๋‹ˆํ‹ฐ์˜ ๋„์›€์„ ๋ฐ›์œผ์„ธ์š”. ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ ๋ฉ˜ํ† ๋ฅผ ์ฐพ๊ฑฐ๋‚˜, ๋ฐ˜๋Œ€๋กœ ์—ฌ๋Ÿฌ๋ถ„์ด ๋ฐฐ์šด ๊ฑธ ๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์—๊ฒŒ ์•Œ๋ ค์ฃผ๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์—์š”. ๊ฐ€๋ฅด์น˜๋‹ค ๋ณด๋ฉด ๋” ๊นŠ์ด ์ดํ•ดํ•˜๊ฒŒ ๋˜๊ฑฐ๋“ ์š” ๐Ÿ˜Š

์ž, ์ด์ œ ์—ฌ๋Ÿฌ๋ถ„ ์ฐจ๋ก€์˜ˆ์š”! ์ด ๊ธ€ ๋‹ซ๊ณ  ๋ฐ”๋กœ ์ฝ”๋“œ ์—๋””ํ„ฐ ์ผœ์„œ ์ฒซ RAG ์‹œ์Šคํ…œ ๋งŒ๋“ค์–ด๋ณด์„ธ์š”. ํ™”์ดํŒ…! ๐Ÿš€๐Ÿ’ช

๐ŸŽฏ ํ•ต์‹ฌ ์š”์•ฝ

1. RAG๋Š” ๊ฒ€์ƒ‰ + ์ƒ์„ฑ์„ ๊ฒฐํ•ฉํ•œ ๊ฐ•๋ ฅํ•œ AI ์‹œ์Šคํ…œ
2. LangChain์œผ๋กœ ๋ณต์žกํ•œ ์›Œํฌํ”Œ๋กœ์šฐ๋ฅผ ๊ฐ„๋‹จํ•˜๊ฒŒ ๊ตฌํ˜„
3. ๋ฌธ์„œ ๋กœ๋”ฉ โ†’ ์ฒญํ‚น โ†’ ์ž„๋ฒ ๋”ฉ โ†’ ๋ฒกํ„ฐ ์ €์žฅ โ†’ ๊ฒ€์ƒ‰ โ†’ ๋‹ต๋ณ€ ์ƒ์„ฑ
4. ์ตœ์ ํ™”์™€ ๋ณด์•ˆ์„ ์žŠ์ง€ ๋ง ๊ฒƒ
5. ์‹ค์ „ ๊ฒฝํ—˜์ด ๊ฐ€์žฅ ์ค‘์š”! ์ผ๋‹จ ๋งŒ๋“ค์–ด๋ณด๊ธฐ ๐Ÿ”ฅ

๋Œ“๊ธ€ ์ž‘์„ฑ

์ด ๊ธ€์— ๋Œ€ํ•œ ์—ฌ๋Ÿฌ๋ถ„์˜ ์ƒ๊ฐ์„ ๋“ค๋ ค์ฃผ์„ธ์š”

๋Œ“๊ธ€ 0
๋Œ“๊ธ€์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ์ค‘์ž…๋‹ˆ๋‹ค.

์•„์ง ๋Œ“๊ธ€์ด ์—†์Šต๋‹ˆ๋‹ค.