์ฝ˜ํ…์ธ  ๋Œ€ํ‘œ ์ด๋ฏธ์ง€ - ๐Ÿง  AI์˜ ์žฅ๊ธฐ๊ธฐ์–ต(Long-Term Memory) ๊ตฌํ˜„ ์‹คํ—˜: ๊ธฐ๊ณ„์—๊ฒŒ ๊ธฐ์–ต์„ ์„ ๋ฌผํ•˜๋‹ค

๐Ÿง  AI์˜ ์žฅ๊ธฐ๊ธฐ์–ต(Long-Term Memory) ๊ตฌํ˜„ ์‹คํ—˜: ๊ธฐ๊ณ„์—๊ฒŒ ๊ธฐ์–ต์„ ์„ ๋ฌผํ•˜๋‹ค

์ธ๊ณต์ง€๋Šฅ์ด ์–ด์ œ์˜ ๋Œ€ํ™”๋ฅผ ๊ธฐ์–ตํ•œ๋‹ค๋ฉด? ์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์˜ ์›๋ฆฌ์™€ ์‹ค์ „ ๊ตฌํ˜„ ๋ฐฉ๋ฒ•
Short-Term Memory Long-Term Memory DB Vector Cache AI Memory Architecture

๐ŸŽฏ AI์—๊ฒŒ ๊ธฐ์–ต์ด๋ž€ ๋ฌด์—‡์ธ๊ฐ€?

์นœ๊ตฌ์•ผ, ์ƒ์ƒํ•ด๋ด. ๋งค์ผ ์•„์นจ ๋ˆˆ์„ ๋œจ๋ฉด ์–ด์ œ์˜ ๊ธฐ์–ต์ด ๋ชจ๋‘ ์‚ฌ๋ผ์ง„๋‹ค๋ฉด? ๐Ÿ˜ฑ ๋”์ฐํ•˜์ง€? ๊ทธ๋Ÿฐ๋ฐ ์‚ฌ์‹ค ๋Œ€๋ถ€๋ถ„์˜ AI ์ฑ—๋ด‡์ด ๋ฐ”๋กœ ์ด๋Ÿฐ ์ƒํƒœ์•ผ. ๋Œ€ํ™”๋ฅผ ์‹œ์ž‘ํ•˜๋ฉด ์ด์ „ ๋Œ€ํ™” ๋‚ด์šฉ์„ ์ „ํ˜€ ๊ธฐ์–ตํ•˜์ง€ ๋ชปํ•˜๋Š” ๊ฑฐ์ง€.

ChatGPT๋‚˜ Claude ๊ฐ™์€ ์ตœ์‹  AI๋“ค๋„ ์‚ฌ์‹ค์€ ์„ธ์…˜ ๊ธฐ๋ฐ˜์˜ ๋‹จ๊ธฐ๊ธฐ์–ต๋งŒ ๊ฐ€์ง€๊ณ  ์žˆ์–ด. ๋ธŒ๋ผ์šฐ์ €๋ฅผ ๋‹ซ๊ฑฐ๋‚˜ ์ƒˆ๋กœ์šด ๋Œ€ํ™”๋ฅผ ์‹œ์ž‘ํ•˜๋ฉด? ์ง ! ๋ชจ๋“  ๊ฒŒ ๋ฆฌ์…‹๋˜๋Š” ๊ฑฐ์•ผ. ๋งˆ์น˜ ์˜ํ™” '๋ฉ”๋ฉ˜ํ† '์˜ ์ฃผ์ธ๊ณต์ฒ˜๋Ÿผ ๋ง์ด์•ผ. ๐ŸŽฌ

๊ทธ๋ž˜์„œ ๋“ฑ์žฅํ•œ ๊ฐœ๋…์ด ๋ฐ”๋กœ AI์˜ ์žฅ๊ธฐ๊ธฐ์–ต(Long-Term Memory)์ด์•ผ. ์ธ๊ฐ„์ฒ˜๋Ÿผ ๊ณผ๊ฑฐ์˜ ๊ฒฝํ—˜์„ ์ €์žฅํ•˜๊ณ , ํ•„์š”ํ•  ๋•Œ ๊บผ๋‚ด ์“ธ ์ˆ˜ ์žˆ๋Š” ์‹œ์Šคํ…œ์„ ๋งŒ๋“œ๋Š” ๊ฑฐ์ง€. ์ด๊ฑด ๋‹จ์ˆœํžˆ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•˜๋Š” ๊ฒƒ ์ด์ƒ์˜ ์˜๋ฏธ๋ฅผ ๊ฐ€์ ธ.

๐Ÿ’ก ์™œ ์žฅ๊ธฐ๊ธฐ์–ต์ด ์ค‘์š”ํ• ๊นŒ? 1. ๊ฐœ์ธํ™”๋œ ๊ฒฝํ—˜: ์‚ฌ์šฉ์ž์˜ ์„ ํ˜ธ๋„, ์Šต๊ด€, ๊ณผ๊ฑฐ ๋Œ€ํ™” ๋‚ด์šฉ์„ ๊ธฐ์–ตํ•ด์„œ ๋งž์ถคํ˜• ์‘๋‹ต ์ œ๊ณต
2. ์ปจํ…์ŠคํŠธ ์œ ์ง€: ๋ฉฐ์น , ๋ช‡ ์ฃผ ์ „์˜ ๋Œ€ํ™” ๋งฅ๋ฝ์„ ์ด์–ด๊ฐˆ ์ˆ˜ ์žˆ์Œ
3. ํ•™์Šต ํšจ์œจ: ๋ฐ˜๋ณต๋˜๋Š” ์งˆ๋ฌธ์— ๋Œ€ํ•œ ๋‹ต๋ณ€์„ ๊ธฐ์–ตํ•ด์„œ ๋” ๋น ๋ฅด๊ณ  ์ •ํ™•ํ•œ ์‘๋‹ต
4. ๊ด€๊ณ„ ๊ตฌ์ถ•: ๋งˆ์น˜ ์˜ค๋žœ ์นœ๊ตฌ์ฒ˜๋Ÿผ ์‚ฌ์šฉ์ž๋ฅผ ์ดํ•ดํ•˜๋Š” AI ๊ตฌํ˜„

๐Ÿ—๏ธ ์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์˜ ํ•ต์‹ฌ ์•„ํ‚คํ…์ฒ˜

์ž, ์ด์ œ ๋ณธ๊ฒฉ์ ์œผ๋กœ ์–ด๋–ป๊ฒŒ AI์—๊ฒŒ ์žฅ๊ธฐ๊ธฐ์–ต์„ ์‹ฌ์–ด์ค„ ์ˆ˜ ์žˆ๋Š”์ง€ ์•Œ์•„๋ณด์ž. ๐Ÿ”ง ๊ธฐ๋ณธ์ ์œผ๋กœ ์„ธ ๊ฐ€์ง€ ํ•ต์‹ฌ ์ปดํฌ๋„ŒํŠธ๊ฐ€ ํ•„์š”ํ•ด.

1๏ธโƒฃ ๋ฉ”๋ชจ๋ฆฌ ์ €์žฅ์†Œ (Memory Storage)

๊ฐ€์žฅ ๊ธฐ๋ณธ์ด ๋˜๋Š” ๋ถ€๋ถ„์ด์•ผ. ๋Œ€ํ™” ๋‚ด์šฉ, ์‚ฌ์šฉ์ž ์ •๋ณด, ํ•™์Šต๋œ ํŒจํ„ด ๋“ฑ์„ ์–ด๋”˜๊ฐ€์— ์ €์žฅํ•ด์•ผ ํ•˜์ž–์•„? ์—ฌ๋Ÿฌ ๊ฐ€์ง€ ์˜ต์…˜์ด ์žˆ์–ด:

์ €์žฅ์†Œ ํƒ€์ž… ํŠน์ง• ์ ํ•ฉํ•œ ์šฉ๋„
๊ด€๊ณ„ํ˜• DB
(PostgreSQL, MySQL)
๊ตฌ์กฐํ™”๋œ ๋ฐ์ดํ„ฐ ์ €์žฅ
ํŠธ๋žœ์žญ์…˜ ์ง€์›
์‚ฌ์šฉ์ž ํ”„๋กœํ•„, ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ
๋Œ€ํ™” ์„ธ์…˜ ์ •๋ณด
๋ฒกํ„ฐ DB
(Pinecone, Weaviate)
์˜๋ฏธ ๊ธฐ๋ฐ˜ ๊ฒ€์ƒ‰
์ž„๋ฒ ๋”ฉ ์ €์žฅ
๋Œ€ํ™” ๋‚ด์šฉ ๊ฒ€์ƒ‰
์œ ์‚ฌ ์ปจํ…์ŠคํŠธ ์ฐพ๊ธฐ
NoSQL DB
(MongoDB, Redis)
์œ ์—ฐํ•œ ์Šคํ‚ค๋งˆ
๋น ๋ฅธ ์ฝ๊ธฐ/์“ฐ๊ธฐ
์„ธ์…˜ ์บ์‹œ
์ž„์‹œ ๋ฉ”๋ชจ๋ฆฌ
๊ทธ๋ž˜ํ”„ DB
(Neo4j)
๊ด€๊ณ„ ์ค‘์‹ฌ ์ €์žฅ
๋ณต์žกํ•œ ์—ฐ๊ฒฐ ํ‘œํ˜„
์ง€์‹ ๊ทธ๋ž˜ํ”„
๊ฐœ๋… ๊ฐ„ ๊ด€๊ณ„

์‹ค์ „์—์„œ๋Š” ๋ณดํ†ต ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ์ ‘๊ทผ๋ฒ•์„ ์‚ฌ์šฉํ•ด. ์˜ˆ๋ฅผ ๋“ค์–ด, ์‚ฌ์šฉ์ž ๊ธฐ๋ณธ ์ •๋ณด๋Š” PostgreSQL์—, ๋Œ€ํ™” ๋‚ด์šฉ์˜ ์ž„๋ฒ ๋”ฉ์€ Pinecone์—, ์„ธ์…˜ ๋ฐ์ดํ„ฐ๋Š” Redis์— ์ €์žฅํ•˜๋Š” ์‹์ด์ง€. ๐ŸŽฏ

2๏ธโƒฃ ์ž„๋ฒ ๋”ฉ & ๋ฒกํ„ฐํ™” (Embedding & Vectorization)

์—ฌ๊ธฐ๊ฐ€ ์ง„์งœ ๋งˆ๋ฒ•์ด ์ผ์–ด๋‚˜๋Š” ๊ณณ์ด์•ผ! โœจ ํ…์ŠคํŠธ๋ฅผ ๊ทธ๋ƒฅ ์ €์žฅํ•˜๋ฉด ๋‚˜์ค‘์— ์ฐพ๊ธฐ๊ฐ€ ์–ด๋ ค์›Œ. "์–ด์ œ ํ”ผ์ž ์–˜๊ธฐํ–ˆ๋˜ ๊ฑฐ ๊ธฐ์–ต๋‚˜?"๋ผ๊ณ  ๋ฌผ์–ด๋ดค์„ ๋•Œ, ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค์—์„œ ์ •ํ™•ํžˆ "ํ”ผ์ž"๋ผ๋Š” ๋‹จ์–ด๊ฐ€ ๋“ค์–ด๊ฐ„ ๋Œ€ํ™”๋งŒ ์ฐพ์œผ๋ฉด ์•ˆ ๋˜์ž–์•„?

๊ทธ๋ž˜์„œ ์ž„๋ฒ ๋”ฉ(Embedding)์„ ์‚ฌ์šฉํ•ด. ํ…์ŠคํŠธ๋ฅผ ๊ณ ์ฐจ์› ๋ฒกํ„ฐ ๊ณต๊ฐ„์˜ ์ ์œผ๋กœ ๋ณ€ํ™˜ํ•˜๋Š” ๊ฑฐ์•ผ. ๋น„์Šทํ•œ ์˜๋ฏธ๋ฅผ ๊ฐ€์ง„ ๋ฌธ์žฅ๋“ค์€ ๋ฒกํ„ฐ ๊ณต๊ฐ„์—์„œ๋„ ๊ฐ€๊นŒ์ด ์œ„์น˜ํ•˜๊ฒŒ ๋ผ.

๐Ÿ”ฌ ์ž„๋ฒ ๋”ฉ ์ž‘๋™ ์›๋ฆฌ "๋‚˜๋Š” ํ”ผ์ž๋ฅผ ์ข‹์•„ํ•ด" โ†’ [0.23, -0.45, 0.67, ..., 0.12] (1536์ฐจ์› ๋ฒกํ„ฐ)
"ํ”ผ์ž ๋ง›์žˆ์–ด" โ†’ [0.25, -0.43, 0.69, ..., 0.15] (์œ ์‚ฌํ•œ ๋ฒกํ„ฐ ๊ฐ’)
"๋‚ ์”จ๊ฐ€ ์ข‹๋„ค" โ†’ [-0.12, 0.78, -0.34, ..., 0.89] (์ „ํ˜€ ๋‹ค๋ฅธ ๋ฒกํ„ฐ ๊ฐ’)

์ด๋ ‡๊ฒŒ ๋ณ€ํ™˜ํ•˜๋ฉด ์ฝ”์‚ฌ์ธ ์œ ์‚ฌ๋„(Cosine Similarity) ๊ฐ™์€ ์ˆ˜ํ•™์  ๋ฐฉ๋ฒ•์œผ๋กœ ์˜๋ฏธ์ ์œผ๋กœ ์œ ์‚ฌํ•œ ๋Œ€ํ™”๋ฅผ ์ฐพ์„ ์ˆ˜ ์žˆ์–ด! ๐ŸŽ“

์ฃผ๋กœ ์‚ฌ์šฉํ•˜๋Š” ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ๋“ค:

โ€ข OpenAI text-embedding-ada-002: 1536์ฐจ์›, ๋ฒ”์šฉ์„ฑ ์ข‹์Œ, API ํ˜ธ์ถœ ๋ฐฉ์‹
โ€ข Sentence-BERT: ์˜คํ”ˆ์†Œ์Šค, ๋กœ์ปฌ ์‹คํ–‰ ๊ฐ€๋Šฅ, ๋‹ค์–‘ํ•œ ํฌ๊ธฐ ๋ชจ๋ธ
โ€ข Cohere Embed: ๋‹ค๊ตญ์–ด ์ง€์› ๊ฐ•๋ ฅ, 4096์ฐจ์› ์˜ต์…˜
โ€ข Google PaLM Embeddings: ๊ตฌ๊ธ€ ์ƒํƒœ๊ณ„์™€ ํ†ตํ•ฉ ์šฉ์ด

3๏ธโƒฃ ๊ฒ€์ƒ‰ & ๋ฆฌํŠธ๋ฆฌ๋ฒŒ (Retrieval)

์ €์žฅ๋งŒ ํ•˜๋ฉด ๋ญํ•ด? ํ•„์š”ํ•  ๋•Œ ๊บผ๋‚ด ์“ธ ์ˆ˜ ์žˆ์–ด์•ผ์ง€! ๐ŸŽฃ ์ด ๋ถ€๋ถ„์ด ์ƒ๊ฐ๋ณด๋‹ค ๊นŒ๋‹ค๋กœ์›Œ. ๋‹จ์ˆœํžˆ ๊ฐ€์žฅ ์œ ์‚ฌํ•œ ๋ฒกํ„ฐ๋ฅผ ์ฐพ๋Š” ๊ฒƒ๋งŒ์œผ๋กœ๋Š” ๋ถ€์กฑํ•ด.

๐ŸŽฏ ํšจ๊ณผ์ ์ธ ๊ฒ€์ƒ‰ ์ „๋žต

1. ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ๊ฒ€์ƒ‰ (Hybrid Search)
๋ฒกํ„ฐ ์œ ์‚ฌ๋„ ๊ฒ€์ƒ‰ + ํ‚ค์›Œ๋“œ ๊ธฐ๋ฐ˜ ๊ฒ€์ƒ‰์„ ๊ฒฐํ•ฉํ•ด. ์˜ˆ๋ฅผ ๋“ค์–ด, "์ง€๋‚œ์ฃผ ํ™”์š”์ผ์— ์–˜๊ธฐํ–ˆ๋˜ ํ”„๋กœ์ ํŠธ"๋ผ๊ณ  ํ•˜๋ฉด, ์‹œ๊ฐ„ ์ •๋ณด(ํ™”์š”์ผ)๋Š” ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋กœ, "ํ”„๋กœ์ ํŠธ"๋Š” ๋ฒกํ„ฐ ๊ฒ€์ƒ‰์œผ๋กœ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ฑฐ์ง€.

2. ๋ฆฌ๋žญํ‚น (Re-ranking)
์ฒซ ๋ฒˆ์งธ ๊ฒ€์ƒ‰์—์„œ ์ƒ์œ„ 20๊ฐœ๋ฅผ ๋ฝ‘๊ณ , ๋” ์ •๊ตํ•œ ๋ชจ๋ธ๋กœ ๋‹ค์‹œ ์ˆœ์œ„๋ฅผ ๋งค๊ฒจ. ์ •ํ™•๋„๋Š” ์˜ฌ๋ผ๊ฐ€์ง€๋งŒ ์†๋„๋Š” ์กฐ๊ธˆ ๋А๋ ค์ ธ. ํŠธ๋ ˆ์ด๋“œ์˜คํ”„๊ฐ€ ์žˆ์–ด! โš–๏ธ

3. ์‹œ๊ฐ„ ๊ฐ€์ค‘์น˜ (Temporal Weighting)
์ตœ๊ทผ ๋Œ€ํ™”์— ๋” ๋†’์€ ๊ฐ€์ค‘์น˜๋ฅผ ์ค˜. 3๊ฐœ์›” ์ „ ๋Œ€ํ™”๋ณด๋‹ค ์–ด์ œ ๋Œ€ํ™”๊ฐ€ ๋” ๊ด€๋ จ์„ฑ์ด ๋†’์„ ๊ฐ€๋Šฅ์„ฑ์ด ํฌ๋‹ˆ๊นŒ.

4. ์ปจํ…์ŠคํŠธ ์œˆ๋„์šฐ ๊ด€๋ฆฌ
๊ฒ€์ƒ‰๋œ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ AI์—๊ฒŒ ์ „๋‹ฌํ•  ๋•Œ, ํ† ํฐ ์ œํ•œ์„ ๊ณ ๋ คํ•ด์•ผ ํ•ด. GPT-4์˜ ๊ฒฝ์šฐ 8K~32K ํ† ํฐ ์ œํ•œ์ด ์žˆ์œผ๋‹ˆ, ๊ฐ€์žฅ ๊ด€๋ จ์„ฑ ๋†’์€ ๋ฉ”๋ชจ๋ฆฌ๋งŒ ์„ ๋ณ„ํ•ด์„œ ๋„ฃ์–ด์ค˜์•ผ ํ•ด. ๐Ÿ“

๐Ÿ’ป ์‹ค์ „ ๊ตฌํ˜„: ๋‹จ๊ณ„๋ณ„ ๊ฐ€์ด๋“œ

์ด๋ก ์€ ์ถฉ๋ถ„ํžˆ ๋ดค์œผ๋‹ˆ, ์ด์ œ ์‹ค์ œ๋กœ ์ฝ”๋“œ๋ฅผ ์งœ๋ณด์ž! ๐Ÿš€ Python ๊ธฐ๋ฐ˜์œผ๋กœ ๊ฐ„๋‹จํ•œ ์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค์–ด๋ณผ ๊ฑฐ์•ผ. ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•˜๋Š” ๊ฐœ๋ฐœ์ž๋“ค๋„ ์ด๋Ÿฐ ์‹œ์Šคํ…œ์„ ๋งŽ์ด ๊ตฌํ˜„ํ•˜๋”๋ผ๊ณ .

๐Ÿ”ง Step 1: ํ™˜๊ฒฝ ์„ค์ •

๋จผ์ € ํ•„์š”ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋“ค์„ ์„ค์น˜ํ•ด์•ผ ํ•ด:

pip install openai pinecone-client langchain chromadb sentence-transformers

๊ฐ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์—ญํ• :
openai: GPT ๋ชจ๋ธ ๋ฐ ์ž„๋ฒ ๋”ฉ API
pinecone-client: ๋ฒกํ„ฐ ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค (๋Œ€์•ˆ: Weaviate, Qdrant)
langchain: LLM ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ํ”„๋ ˆ์ž„์›Œํฌ
chromadb: ๋กœ์ปฌ ๋ฒกํ„ฐ DB (๊ฐœ๋ฐœ/ํ…Œ์ŠคํŠธ์šฉ)
sentence-transformers: ๋กœ์ปฌ ์ž„๋ฒ ๋”ฉ ๋ชจ๋ธ

๐Ÿ—๏ธ Step 2: ๋ฉ”๋ชจ๋ฆฌ ํด๋ž˜์Šค ์„ค๊ณ„

๊ธฐ๋ณธ์ ์ธ ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ ํด๋ž˜์Šค๋ฅผ ๋งŒ๋“ค์–ด๋ณด์ž:

import openai
from datetime import datetime
import json
from typing import List, Dict, Optional

class LongTermMemory:
    def __init__(self, user_id: str, api_key: str):
        self.user_id = user_id
        openai.api_key = api_key
        self.memories = []  # ์‹ค์ œ๋กœ๋Š” DB ์—ฐ๊ฒฐ
        
    def add_memory(self, content: str, metadata: Dict = None):
        """์ƒˆ๋กœ์šด ๊ธฐ์–ต ์ถ”๊ฐ€"""
        embedding = self._get_embedding(content)
        
        memory = {
            'id': self._generate_id(),
            'user_id': self.user_id,
            'content': content,
            'embedding': embedding,
            'timestamp': datetime.now().isoformat(),
            'metadata': metadata or {},
            'access_count': 0
        }
        
        self.memories.append(memory)
        return memory['id']
    
    def _get_embedding(self, text: str) -> List[float]:
        """ํ…์ŠคํŠธ๋ฅผ ๋ฒกํ„ฐ๋กœ ๋ณ€ํ™˜"""
        response = openai.Embedding.create(
            model="text-embedding-ada-002",
            input=text
        )
        return response['data'][0]['embedding']
    
    def search_memories(self, query: str, top_k: int = 5) -> List[Dict]:
        """์ฟผ๋ฆฌ์™€ ์œ ์‚ฌํ•œ ๊ธฐ์–ต ๊ฒ€์ƒ‰"""
        query_embedding = self._get_embedding(query)
        
        # ์ฝ”์‚ฌ์ธ ์œ ์‚ฌ๋„ ๊ณ„์‚ฐ
        similarities = []
        for memory in self.memories:
            similarity = self._cosine_similarity(
                query_embedding, 
                memory['embedding']
            )
            similarities.append((similarity, memory))
        
        # ์ƒ์œ„ k๊ฐœ ๋ฐ˜ํ™˜
        similarities.sort(reverse=True, key=lambda x: x[0])
        return [mem for _, mem in similarities[:top_k]]
    
    def _cosine_similarity(self, vec1: List[float], vec2: List[float]) -> float:
        """์ฝ”์‚ฌ์ธ ์œ ์‚ฌ๋„ ๊ณ„์‚ฐ"""
        import numpy as np
        return np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))
    
    def _generate_id(self) -> str:
        """๊ณ ์œ  ID ์ƒ์„ฑ"""
        import uuid
        return str(uuid.uuid4())

์ด ์ฝ”๋“œ๋Š” ๊ธฐ๋ณธ ๊ณจ๊ฒฉ์ด์•ผ. ์‹ค์ œ ํ”„๋กœ๋•์…˜์—์„œ๋Š” ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์—ฐ๊ฒฐ, ์—๋Ÿฌ ํ•ธ๋“ค๋ง, ๋น„๋™๊ธฐ ์ฒ˜๋ฆฌ ๋“ฑ์ด ์ถ”๊ฐ€๋˜์–ด์•ผ ํ•ด. ๐Ÿ› ๏ธ

๐ŸŽจ Step 3: ๋ฒกํ„ฐ DB ํ†ตํ•ฉ

์ด์ œ Pinecone ๊ฐ™์€ ์ „๋ฌธ ๋ฒกํ„ฐ DB๋ฅผ ์—ฐ๊ฒฐํ•ด๋ณด์ž. ๋กœ์ปฌ ํ…Œ์ŠคํŠธ์—๋Š” ChromaDB๊ฐ€ ์ข‹์•„:

import chromadb
from chromadb.config import Settings

class VectorMemoryStore:
    def __init__(self, collection_name: str = "memories"):
        self.client = chromadb.Client(Settings(
            chroma_db_impl="duckdb+parquet",
            persist_directory="./chroma_db"
        ))
        
        self.collection = self.client.get_or_create_collection(
            name=collection_name,
            metadata={"hnsw:space": "cosine"}  # ์ฝ”์‚ฌ์ธ ์œ ์‚ฌ๋„ ์‚ฌ์šฉ
        )
    
    def add(self, id: str, embedding: List[float], 
            document: str, metadata: Dict):
        """๋ฒกํ„ฐ์™€ ๋ฌธ์„œ ์ €์žฅ"""
        self.collection.add(
            ids=[id],
            embeddings=[embedding],
            documents=[document],
            metadatas=[metadata]
        )
    
    def query(self, query_embedding: List[float], 
              n_results: int = 5, 
              where: Dict = None) -> Dict:
        """์œ ์‚ฌ ๋ฒกํ„ฐ ๊ฒ€์ƒ‰"""
        results = self.collection.query(
            query_embeddings=[query_embedding],
            n_results=n_results,
            where=where  # ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋ง
        )
        return results
    
    def update_metadata(self, id: str, metadata: Dict):
        """๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์—…๋ฐ์ดํŠธ (์ ‘๊ทผ ํšŸ์ˆ˜ ๋“ฑ)"""
        self.collection.update(
            ids=[id],
            metadatas=[metadata]
        )

ChromaDB๋Š” ์„ค์ •์ด ๊ฐ„๋‹จํ•˜๊ณ  ๋กœ์ปฌ์—์„œ ๋Œ์•„๊ฐ€์„œ ๊ฐœ๋ฐœํ•  ๋•Œ ์ •๋ง ํŽธํ•ด. ํ•˜์ง€๋งŒ ํ”„๋กœ๋•์…˜์—์„œ๋Š” Pinecone์ด๋‚˜ Weaviate ๊ฐ™์€ ํด๋ผ์šฐ๋“œ ์„œ๋น„์Šค๋ฅผ ์ถ”์ฒœํ•ด. ํ™•์žฅ์„ฑ๊ณผ ์„ฑ๋Šฅ์ด ํ›จ์”ฌ ์ข‹๊ฑฐ๋“ ! โ˜๏ธ

๐Ÿงฉ Step 4: LangChain๊ณผ ํ†ตํ•ฉ

LangChain์„ ์‚ฌ์šฉํ•˜๋ฉด ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ๊ฐ€ ํ›จ์”ฌ ์‰ฌ์›Œ์ ธ. ์ด๋ฏธ ๋งŒ๋“ค์–ด์ง„ ์ปดํฌ๋„ŒํŠธ๋“ค์„ ์กฐํ•ฉํ•˜๋ฉด ๋˜๊ฑฐ๋“ :

from langchain.memory import VectorStoreRetrieverMemory
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import ConversationChain
from langchain.llms import OpenAI

# ๋ฒกํ„ฐ ์Šคํ† ์–ด ์„ค์ •
embeddings = OpenAIEmbeddings()
vectorstore = Chroma(
    collection_name="conversation_memory",
    embedding_function=embeddings,
    persist_directory="./chroma_langchain"
)

# ๋ฉ”๋ชจ๋ฆฌ ๋ฆฌํŠธ๋ฆฌ๋ฒ„ ์ƒ์„ฑ
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
memory = VectorStoreRetrieverMemory(retriever=retriever)

# ๋Œ€ํ™” ์ฒด์ธ ๊ตฌ์„ฑ
llm = OpenAI(temperature=0.7)
conversation = ConversationChain(
    llm=llm,
    memory=memory,
    verbose=True
)

# ์‚ฌ์šฉ ์˜ˆ์‹œ
response1 = conversation.predict(input="๋‚ด ์ด๋ฆ„์€ ์ฒ ์ˆ˜์•ผ")
# AI: "์•ˆ๋…•ํ•˜์„ธ์š” ์ฒ ์ˆ˜๋‹˜! ๋งŒ๋‚˜์„œ ๋ฐ˜๊ฐ‘์Šต๋‹ˆ๋‹ค."

response2 = conversation.predict(input="๋‚ด ์ด๋ฆ„์ด ๋ญ์˜€์ง€?")
# AI: "์ฒ ์ˆ˜๋‹˜์ด๋ผ๊ณ  ๋ง์”€ํ•˜์…จ์Šต๋‹ˆ๋‹ค!"

LangChain์˜ ์žฅ์ ์€ ๋ฉ”๋ชจ๋ฆฌ ํƒ€์ž…์„ ์‰ฝ๊ฒŒ ๋ฐ”๊ฟ€ ์ˆ˜ ์žˆ๋‹ค๋Š” ๊ฑฐ์•ผ. ConversationBufferMemory, ConversationSummaryMemory, VectorStoreRetrieverMemory ๋“ฑ ์ƒํ™ฉ์— ๋งž๊ฒŒ ์„ ํƒํ•  ์ˆ˜ ์žˆ์–ด. ๐ŸŽ›๏ธ

๐ŸŽญ ๊ณ ๊ธ‰ ๊ธฐ๋ฒ•: ๋ฉ”๋ชจ๋ฆฌ ๊ณ„์ธตํ™”

์ธ๊ฐ„์˜ ๊ธฐ์–ต๋„ ๋‹จ๊ธฐ๊ธฐ์–ต, ์žฅ๊ธฐ๊ธฐ์–ต, ์ž‘์—…๊ธฐ์–ต ๋“ฑ์œผ๋กœ ๋‚˜๋‰˜์ž–์•„? AI๋„ ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ ๊ณ„์ธตํ™”๋œ ๋ฉ”๋ชจ๋ฆฌ ์‹œ์Šคํ…œ์„ ๊ตฌํ˜„ํ•˜๋ฉด ํ›จ์”ฌ ํšจ์œจ์ ์ด์•ผ. ๐Ÿง 

๐Ÿ“Š 3-Tier ๋ฉ”๋ชจ๋ฆฌ ์•„ํ‚คํ…์ฒ˜

Tier 1: Working Memory (Redis) ํ˜„์žฌ ์„ธ์…˜, ์ฆ‰๊ฐ ์ ‘๊ทผ, 1-2์‹œ๊ฐ„ ์œ ์ง€ ์šฉ๋Ÿ‰: ~100 ๋ฉ”์‹œ์ง€ | ์†๋„: <10ms Tier 2: Recent Memory (Vector DB) ์ตœ๊ทผ ๋Œ€ํ™”, ๋น ๋ฅธ ๊ฒ€์ƒ‰, 1์ฃผ์ผ ์œ ์ง€ ์šฉ๋Ÿ‰: ~1000 ๋ฉ”์‹œ์ง€ | ์†๋„: ~50ms Tier 3: Long-Term Memory (PostgreSQL) ์ „์ฒด ํžˆ์Šคํ† ๋ฆฌ, ์••์ถ• ์ €์žฅ, ์˜๊ตฌ ๋ณด๊ด€ ์šฉ๋Ÿ‰: ๋ฌด์ œํ•œ | ์†๋„: ~200ms

๐Ÿ”ฅ Tier 1: Working Memory (์ž‘์—… ๋ฉ”๋ชจ๋ฆฌ)

ํ˜„์žฌ ์ง„ํ–‰ ์ค‘์ธ ๋Œ€ํ™”๋ฅผ ์œ„ํ•œ ์ดˆ๊ณ ์† ๋ฉ”๋ชจ๋ฆฌ์•ผ. Redis ๊ฐ™์€ ์ธ๋ฉ”๋ชจ๋ฆฌ DB๋ฅผ ์‚ฌ์šฉํ•ด์„œ ๋ฐ€๋ฆฌ์ดˆ ๋‹จ์œ„๋กœ ์ ‘๊ทผ ๊ฐ€๋Šฅํ•ด. ๐Ÿ’จ

ํŠน์ง•:
โ€ข ํ˜„์žฌ ์„ธ์…˜์˜ ์ตœ๊ทผ 10-100๊ฐœ ๋ฉ”์‹œ์ง€ ์ €์žฅ
โ€ข TTL(Time To Live) ์„ค์ •์œผ๋กœ ์ž๋™ ๋งŒ๋ฃŒ
โ€ข ํ† ํฐ ์นด์šดํŒ…์œผ๋กœ ์ปจํ…์ŠคํŠธ ์œˆ๋„์šฐ ๊ด€๋ฆฌ
โ€ข ๋Œ€ํ™” ํ๋ฆ„ ์œ ์ง€์— ์ตœ์ ํ™”

import redis
import json

class WorkingMemory:
    def __init__(self, session_id: str):
        self.redis_client = redis.Redis(host='localhost', port=6379, db=0)
        self.session_id = session_id
        self.key = f"session:{session_id}"
        
    def add_message(self, role: str, content: str):
        """๋ฉ”์‹œ์ง€ ์ถ”๊ฐ€ (์ตœ๋Œ€ 100๊ฐœ ์œ ์ง€)"""
        message = {
            'role': role,
            'content': content,
            'timestamp': datetime.now().isoformat()
        }
        
        self.redis_client.lpush(self.key, json.dumps(message))
        self.redis_client.ltrim(self.key, 0, 99)  # ์ตœ์‹  100๊ฐœ๋งŒ
        self.redis_client.expire(self.key, 7200)  # 2์‹œ๊ฐ„ ํ›„ ๋งŒ๋ฃŒ
    
    def get_recent_messages(self, count: int = 10) -> List[Dict]:
        """์ตœ๊ทผ ๋ฉ”์‹œ์ง€ ๊ฐ€์ ธ์˜ค๊ธฐ"""
        messages = self.redis_client.lrange(self.key, 0, count-1)
        return [json.loads(msg) for msg in messages]

๐ŸŒŠ Tier 2: Recent Memory (์ตœ๊ทผ ๋ฉ”๋ชจ๋ฆฌ)

์ง€๋‚œ ๋ฉฐ์น ๊ฐ„์˜ ๋Œ€ํ™”๋ฅผ ๋ฒกํ„ฐ ๊ฒ€์ƒ‰์œผ๋กœ ๋น ๋ฅด๊ฒŒ ์ฐพ์„ ์ˆ˜ ์žˆ๋Š” ๊ณ„์ธต์ด์•ผ. ์—ฌ๊ธฐ๊ฐ€ ์ง„์งœ ์žฅ๊ธฐ๊ธฐ์–ต์˜ ํ•ต์‹ฌ์ด์ง€! ๐ŸŽฏ

ํŠน์ง•:
โ€ข ๋ฒกํ„ฐ ์ž„๋ฒ ๋”ฉ์œผ๋กœ ์˜๋ฏธ ๊ธฐ๋ฐ˜ ๊ฒ€์ƒ‰
โ€ข ์ตœ๊ทผ 1์ฃผ์ผ~1๊ฐœ์›” ๋ฐ์ดํ„ฐ hot storage
โ€ข ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋ง (๋‚ ์งœ, ์ฃผ์ œ, ์ค‘์š”๋„)
โ€ข ์ž๋™ ์š”์•ฝ ๋ฐ ์••์ถ•

๐Ÿ’ก Pro Tip: ๋ฉ”๋ชจ๋ฆฌ ์••์ถ• ์ „๋žต ๋ชจ๋“  ๋Œ€ํ™”๋ฅผ ๋‹ค ์ €์žฅํ•˜๋ฉด ๋น„์šฉ์ด ์—„์ฒญ๋‚˜๊ฒŒ ๋Š˜์–ด๋‚˜. ๊ทธ๋ž˜์„œ ์ค‘์š”ํ•œ ์ •๋ณด๋งŒ ์ถ”์ถœํ•ด์„œ ์ €์žฅํ•˜๋Š” ๊ฒŒ ์ข‹์•„:

1. ์—”ํ‹ฐํ‹ฐ ์ถ”์ถœ: ์ด๋ฆ„, ๋‚ ์งœ, ์žฅ์†Œ, ์ค‘์š” ํ‚ค์›Œ๋“œ๋งŒ ์ €์žฅ
2. ๋Œ€ํ™” ์š”์•ฝ: GPT๋กœ ๊ธด ๋Œ€ํ™”๋ฅผ 3-4๋ฌธ์žฅ์œผ๋กœ ์š”์•ฝ
3. ์ค‘์š”๋„ ์ ์ˆ˜: ์‚ฌ์šฉ์ž๊ฐ€ ๊ฐ•์กฐํ•œ ๋‚ด์šฉ์— ๋†’์€ ์ ์ˆ˜ ๋ถ€์—ฌ
4. ์ค‘๋ณต ์ œ๊ฑฐ: ๋น„์Šทํ•œ ๋‚ด์šฉ์€ ํ•˜๋‚˜๋กœ ํ†ตํ•ฉ

๐Ÿ›๏ธ Tier 3: Long-Term Memory (์žฅ๊ธฐ ๋ฉ”๋ชจ๋ฆฌ)

๋ชจ๋“  ๋Œ€ํ™” ํžˆ์Šคํ† ๋ฆฌ๋ฅผ ์˜๊ตฌ ๋ณด๊ด€ํ•˜๋Š” ๊ณ„์ธต์ด์•ผ. ์ž์ฃผ ์ ‘๊ทผํ•˜์ง€๋Š” ์•Š์ง€๋งŒ, ํ•„์š”ํ•  ๋•Œ ๊บผ๋‚ด ์“ธ ์ˆ˜ ์žˆ์–ด.

ํŠน์ง•:
โ€ข PostgreSQL ๊ฐ™์€ ๊ด€๊ณ„ํ˜• DB ์‚ฌ์šฉ
โ€ข ์••์ถ• ๋ฐ ์•„์นด์ด๋น™
โ€ข ๋ถ„์„ ๋ฐ ํ†ต๊ณ„ ์ถ”์ถœ
โ€ข ๊ทœ์ • ์ค€์ˆ˜ ๋ฐ ๊ฐ์‚ฌ ๋กœ๊ทธ

-- PostgreSQL ์Šคํ‚ค๋งˆ ์˜ˆ์‹œ
CREATE TABLE long_term_memories (
    id UUID PRIMARY KEY,
    user_id VARCHAR(255) NOT NULL,
    session_id VARCHAR(255),
    content TEXT,
    summary TEXT,
    entities JSONB,
    embedding_id VARCHAR(255),
    importance_score FLOAT,
    created_at TIMESTAMP DEFAULT NOW(),
    accessed_at TIMESTAMP,
    access_count INTEGER DEFAULT 0,
    INDEX idx_user_created (user_id, created_at),
    INDEX idx_importance (importance_score DESC)
);

๐ŸŽช ์‹ค์ „ ์‹œ๋‚˜๋ฆฌ์˜ค: ๊ฐœ์ธ ๋น„์„œ AI ๋งŒ๋“ค๊ธฐ

์ด๋ก ๊ณผ ์ฝ”๋“œ๋งŒ ๋ณด๋ฉด ์žฌ๋ฏธ์—†์ง€? ์‹ค์ œ๋กœ ์–ด๋–ป๊ฒŒ ํ™œ์šฉ๋˜๋Š”์ง€ ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ ํ†ตํ•ด ์•Œ์•„๋ณด์ž! ๐ŸŽฌ ๊ฐœ์ธ ๋น„์„œ AI๋ฅผ ๋งŒ๋“ ๋‹ค๊ณ  ์ƒ๊ฐํ•ด๋ด.

๐Ÿ“… ์‹œ๋‚˜๋ฆฌ์˜ค 1: ์ผ์ • ๊ด€๋ฆฌ

์‚ฌ์šฉ์ž: "๋‹ค์Œ ์ฃผ ํ™”์š”์ผ ์˜คํ›„ 3์‹œ์— ์น˜๊ณผ ์˜ˆ์•ฝ ์ข€ ํ•ด์ค˜."
AI: "๋„ค, ๋‹ค์Œ ์ฃผ ํ™”์š”์ผ(12์›” 17์ผ) ์˜คํ›„ 3์‹œ์— ์น˜๊ณผ ์˜ˆ์•ฝ์„ ๊ธฐ๋กํ–ˆ์Šต๋‹ˆ๋‹ค."

[3์ผ ํ›„]

์‚ฌ์šฉ์ž: "์ด๋ฒˆ ์ฃผ์— ๋ญ ํ•  ์ผ ์žˆ์—ˆ์ง€?"
AI: "ํ™”์š”์ผ ์˜คํ›„ 3์‹œ์— ์น˜๊ณผ ์˜ˆ์•ฝ์ด ์žˆ์œผ์‹ญ๋‹ˆ๋‹ค. ๋ฏธ๋ฆฌ ์•Œ๋ฆผ์„ ์„ค์ •ํ•ด๋“œ๋ฆด๊นŒ์š”?"

์ด๊ฒŒ ๊ฐ€๋Šฅํ•˜๋ ค๋ฉด:
1. ์ฒซ ๋Œ€ํ™”์—์„œ "์น˜๊ณผ ์˜ˆ์•ฝ", "ํ™”์š”์ผ", "์˜คํ›„ 3์‹œ" ๊ฐ™์€ ์—”ํ‹ฐํ‹ฐ๋ฅผ ์ถ”์ถœํ•ด์„œ ๊ตฌ์กฐํ™”๋œ ํ˜•ํƒœ๋กœ ์ €์žฅ
2. ๋‚ ์งœ ์ •๋ณด๋ฅผ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ๋กœ ์ธ๋ฑ์‹ฑ
3. "์ด๋ฒˆ ์ฃผ" ๊ฐ™์€ ์‹œ๊ฐ„ ์ฟผ๋ฆฌ๋ฅผ ํ˜„์žฌ ๋‚ ์งœ ๊ธฐ์ค€์œผ๋กœ ํ•„ํ„ฐ๋ง
4. ๊ด€๋ จ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ๊ฒ€์ƒ‰ํ•ด์„œ ์ปจํ…์ŠคํŠธ์— ํฌํ•จ

class PersonalAssistantMemory:
    def process_appointment(self, user_input: str):
        # 1. ์—”ํ‹ฐํ‹ฐ ์ถ”์ถœ (NER ์‚ฌ์šฉ)
        entities = self.extract_entities(user_input)
        # {'type': '์น˜๊ณผ', 'date': '2024-12-17', 'time': '15:00'}
        
        # 2. ๊ตฌ์กฐํ™”๋œ ๋ฉ”๋ชจ๋ฆฌ ์ƒ์„ฑ
        memory = {
            'type': 'appointment',
            'raw_text': user_input,
            'entities': entities,
            'date': entities['date'],
            'importance': 0.9  # ์˜ˆ์•ฝ์€ ์ค‘์š”!
        }
        
        # 3. ๋‹ค์ค‘ ์ €์žฅ
        self.working_memory.add(memory)  # ์ฆ‰์‹œ ์ ‘๊ทผ์šฉ
        self.vector_store.add(memory)    # ๊ฒ€์ƒ‰์šฉ
        self.sql_db.insert(memory)       # ์˜๊ตฌ ๋ณด๊ด€
        
        # 4. ์•Œ๋ฆผ ์Šค์ผ€์ค„๋ง
        self.schedule_reminder(entities['date'], memory)
    
    def query_schedule(self, time_range: str):
        # "์ด๋ฒˆ ์ฃผ" โ†’ ๋‚ ์งœ ๋ฒ”์œ„๋กœ ๋ณ€ํ™˜
        start_date, end_date = self.parse_time_range(time_range)
        
        # ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ๋กœ ๊ฒ€์ƒ‰
        results = self.vector_store.query(
            query="์ผ์ • ์˜ˆ์•ฝ",
            where={
                "date": {"$gte": start_date, "$lte": end_date},
                "type": "appointment"
            }
        )
        
        return results

๐Ÿ• ์‹œ๋‚˜๋ฆฌ์˜ค 2: ์„ ํ˜ธ๋„ ํ•™์Šต

์‚ฌ์šฉ์ž: "์˜ค๋Š˜ ์ ์‹ฌ ๋ญ ๋จน์„๊นŒ?"
AI: "์ง€๋‚œ์ฃผ์— ํ”ผ์ž ๋“œ์‹œ๊ณ  ์‹ถ๋‹ค๊ณ  ํ•˜์…จ๋Š”๋ฐ, ์•„์ง ์•ˆ ๋“œ์…จ๋„ค์š”. ๊ทผ์ฒ˜ ๋ง›์žˆ๋Š” ํ”ผ์ž์ง‘ ์ถ”์ฒœํ•ด๋“œ๋ฆด๊นŒ์š”?"

์‚ฌ์šฉ์ž: "์˜ค ์ข‹์•„! ๊ทธ๋Ÿฐ๋ฐ ํŒŒ์ธ์• ํ”Œ ํ”ผ์ž๋Š” ๋นผ๊ณ ."
AI: "์•Œ๊ฒ ์Šต๋‹ˆ๋‹ค. ํŒŒ์ธ์• ํ”Œ ํ”ผ์ž๋Š” ์„ ํ˜ธํ•˜์ง€ ์•Š์œผ์‹ ๋‹ค๊ณ  ๊ธฐ์–ตํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค."

์ด๋Ÿฐ ๊ฐœ์ธํ™”๊ฐ€ ๊ฐ€๋Šฅํ•˜๋ ค๋ฉด:
1. ์„ ํ˜ธ๋„ ์ •๋ณด๋ฅผ ๋ณ„๋„ ์นดํ…Œ๊ณ ๋ฆฌ๋กœ ๋ถ„๋ฅ˜
2. ๊ธ์ •/๋ถ€์ • ๊ฐ์ • ๋ถ„์„
3. ์‹œ๊ฐ„์— ๋”ฐ๋ฅธ ์„ ํ˜ธ๋„ ๋ณ€ํ™” ์ถ”์ 
4. ์ปจํ…์ŠคํŠธ ๊ธฐ๋ฐ˜ ์ถ”์ฒœ

class PreferenceMemory:
    def __init__(self):
        self.preferences = {}  # ์‹ค์ œ๋กœ๋Š” DB
        
    def update_preference(self, item: str, sentiment: float):
        """์„ ํ˜ธ๋„ ์—…๋ฐ์ดํŠธ (-1.0 ~ 1.0)"""
        if item not in self.preferences:
            self.preferences[item] = {
                'score': sentiment,
                'count': 1,
                'last_updated': datetime.now()
            }
        else:
            # ์ด๋™ ํ‰๊ท ์œผ๋กœ ์—…๋ฐ์ดํŠธ
            old_score = self.preferences[item]['score']
            old_count = self.preferences[item]['count']
            
            new_score = (old_score * old_count + sentiment) / (old_count + 1)
            
            self.preferences[item].update({
                'score': new_score,
                'count': old_count + 1,
                'last_updated': datetime.now()
            })
    
    def get_recommendations(self, category: str, exclude: List[str] = None):
        """์„ ํ˜ธ๋„ ๊ธฐ๋ฐ˜ ์ถ”์ฒœ"""
        items = [
            (item, data['score']) 
            for item, data in self.preferences.items()
            if data['score'] > 0.5 and item not in (exclude or [])
        ]
        
        items.sort(key=lambda x: x[1], reverse=True)
        return items[:5]

โš ๏ธ ์ฃผ์˜์‚ฌํ•ญ๊ณผ ๋ฒ ์ŠคํŠธ ํ”„๋ž™ํ‹ฐ์Šค

์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์„ ๊ตฌํ˜„ํ•  ๋•Œ ๊ผญ ๊ณ ๋ คํ•ด์•ผ ํ•  ๊ฒƒ๋“ค์ด ์žˆ์–ด. ์‹ค์ˆ˜ํ•˜๋ฉด ํฐ์ผ ๋‚˜๋Š” ๊ฒƒ๋“ค์ด์ง€! ๐Ÿ˜ฐ

๐Ÿ”’ 1. ํ”„๋ผ์ด๋ฒ„์‹œ์™€ ๋ณด์•ˆ

์‚ฌ์šฉ์ž์˜ ๋ชจ๋“  ๋Œ€ํ™”๋ฅผ ์ €์žฅํ•œ๋‹ค๋Š” ๊ฑด ์—„์ฒญ๋‚œ ์ฑ…์ž„์ด ๋”ฐ๋ผ. ๊ฐœ์ธ์ •๋ณด ๋ณดํ˜ธ๋ฒ•, GDPR ๊ฐ™์€ ๊ทœ์ •์„ ๋ฐ˜๋“œ์‹œ ์ค€์ˆ˜ํ•ด์•ผ ํ•ด.

ํ•„์ˆ˜ ๊ตฌํ˜„ ์‚ฌํ•ญ:
โ€ข ์•”ํ˜ธํ™”: ์ €์žฅ ์‹œ AES-256 ์•”ํ˜ธํ™”, ์ „์†ก ์‹œ TLS/SSL
โ€ข ์ ‘๊ทผ ์ œ์–ด: ์‚ฌ์šฉ์ž๋ณ„ ๊ฒฉ๋ฆฌ, ๊ถŒํ•œ ๊ด€๋ฆฌ
โ€ข ๋ฐ์ดํ„ฐ ์‚ญ์ œ: "์žŠ์–ด์ค˜" ๊ธฐ๋Šฅ, GDPR์˜ '์žŠํž ๊ถŒ๋ฆฌ' ๊ตฌํ˜„
โ€ข ๊ฐ์‚ฌ ๋กœ๊ทธ: ๋ˆ„๊ฐ€ ์–ธ์ œ ์–ด๋–ค ๋ฐ์ดํ„ฐ์— ์ ‘๊ทผํ–ˆ๋Š”์ง€ ๊ธฐ๋ก
โ€ข ์ต๋ช…ํ™”: ๋ฏผ๊ฐ ์ •๋ณด๋Š” ๋งˆ์Šคํ‚น ๋˜๋Š” ํ† ํฐํ™”

from cryptography.fernet import Fernet

class SecureMemory:
    def __init__(self, encryption_key: bytes):
        self.cipher = Fernet(encryption_key)
    
    def encrypt_content(self, content: str) -> str:
        """๋‚ด์šฉ ์•”ํ˜ธํ™”"""
        return self.cipher.encrypt(content.encode()).decode()
    
    def decrypt_content(self, encrypted: str) -> str:
        """๋‚ด์šฉ ๋ณตํ˜ธํ™”"""
        return self.cipher.decrypt(encrypted.encode()).decode()
    
    def anonymize_pii(self, text: str) -> str:
        """๊ฐœ์ธ์‹๋ณ„์ •๋ณด ์ต๋ช…ํ™”"""
        import re
        # ์ด๋ฉ”์ผ ๋งˆ์Šคํ‚น
        text = re.sub(r'\b[\w.-]+@[\w.-]+\.\w+\b', '[EMAIL]', text)
        # ์ „ํ™”๋ฒˆํ˜ธ ๋งˆ์Šคํ‚น
        text = re.sub(r'\b\d{3}-\d{3,4}-\d{4}\b', '[PHONE]', text)
        # ์ฃผ๋ฏผ๋“ฑ๋ก๋ฒˆํ˜ธ ๋งˆ์Šคํ‚น
        text = re.sub(r'\b\d{6}-\d{7}\b', '[SSN]', text)
        return text

๐Ÿ’ฐ 2. ๋น„์šฉ ์ตœ์ ํ™”

์ž„๋ฒ ๋”ฉ API ํ˜ธ์ถœ, ๋ฒกํ„ฐ DB ์ €์žฅ, GPT ํ† ํฐ ์‚ฌ์šฉ... ๋‹ค ๋ˆ์ด์•ผ! ๐Ÿ’ธ ์ตœ์ ํ™” ์•ˆ ํ•˜๋ฉด ๋น„์šฉ์ด ํญ๋ฐœํ•  ์ˆ˜ ์žˆ์–ด.

๋น„์šฉ ์ ˆ๊ฐ ์ „๋žต:

ํ•ญ๋ชฉ ๋ฌธ์ œ ํ•ด๊ฒฐ์ฑ…
์ž„๋ฒ ๋”ฉ ๋ชจ๋“  ๋ฉ”์‹œ์ง€๋งˆ๋‹ค API ํ˜ธ์ถœ ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ, ๋กœ์ปฌ ๋ชจ๋ธ ์‚ฌ์šฉ
(Sentence-BERT)
๋ฒกํ„ฐ ์ €์žฅ ๊ณ ์ฐจ์› ๋ฒกํ„ฐ ์ €์žฅ ๋น„์šฉ ์ฐจ์› ์ถ•์†Œ(PCA), ์–‘์žํ™”
์ค‘์š”๋„ ๋‚ฎ์€ ๋ฉ”๋ชจ๋ฆฌ ์‚ญ์ œ
GPT ํ† ํฐ ๊ธด ์ปจํ…์ŠคํŠธ ์ „๋‹ฌ ์š”์•ฝ ์‚ฌ์šฉ, ๊ด€๋ จ์„ฑ ๋†’์€ ๊ฒƒ๋งŒ
์„ ๋ณ„, ํ† ํฐ ์นด์šดํŒ…
DB ์ฟผ๋ฆฌ ๋นˆ๋ฒˆํ•œ ๊ฒ€์ƒ‰ ์บ์‹ฑ(Redis), ์ธ๋ฑ์‹ฑ
์ฟผ๋ฆฌ ์ตœ์ ํ™”
class CostOptimizedMemory:
    def __init__(self):
        self.cache = {}  # ๊ฐ„๋‹จํ•œ ์บ์‹œ
        self.batch_queue = []
        self.batch_size = 10
    
    def add_to_batch(self, text: str):
        """๋ฐฐ์น˜ ํ์— ์ถ”๊ฐ€"""
        self.batch_queue.append(text)
        
        if len(self.batch_queue) >= self.batch_size:
            self.process_batch()
    
    def process_batch(self):
        """๋ฐฐ์น˜๋กœ ํ•œ ๋ฒˆ์— ์ฒ˜๋ฆฌ (๋น„์šฉ ์ ˆ๊ฐ)"""
        if not self.batch_queue:
            return
        
        # ํ•œ ๋ฒˆ์˜ API ํ˜ธ์ถœ๋กœ ์—ฌ๋Ÿฌ ํ…์ŠคํŠธ ์ž„๋ฒ ๋”ฉ
        embeddings = openai.Embedding.create(
            model="text-embedding-ada-002",
            input=self.batch_queue
        )
        
        for i, text in enumerate(self.batch_queue):
            embedding = embeddings['data'][i]['embedding']
            self.store_embedding(text, embedding)
        
        self.batch_queue.clear()
    
    def get_cached_or_search(self, query: str):
        """์บ์‹œ ๋จผ์ € ํ™•์ธ"""
        cache_key = hash(query)
        
        if cache_key in self.cache:
            return self.cache[cache_key]
        
        result = self.search(query)
        self.cache[cache_key] = result
        return result

โšก 3. ์„ฑ๋Šฅ ์ตœ์ ํ™”

์‚ฌ์šฉ์ž๋Š” ๊ธฐ๋‹ค๋ฆฌ์ง€ ์•Š์•„. ์‘๋‹ต ์†๋„๊ฐ€ ๋А๋ฆฌ๋ฉด ๋ฐ”๋กœ ์ดํƒˆํ•ด๋ฒ„๋ ค. ๐Ÿƒโ€โ™‚๏ธ๐Ÿ’จ

์„ฑ๋Šฅ ํ–ฅ์ƒ ๊ธฐ๋ฒ•:
โ€ข ๋น„๋™๊ธฐ ์ฒ˜๋ฆฌ: ๋ฉ”๋ชจ๋ฆฌ ์ €์žฅ์€ ๋ฐฑ๊ทธ๋ผ์šด๋“œ์—์„œ
โ€ข ๋ณ‘๋ ฌ ๊ฒ€์ƒ‰: ์—ฌ๋Ÿฌ ์†Œ์Šค๋ฅผ ๋™์‹œ์— ์ฟผ๋ฆฌ
โ€ข ์ธ๋ฑ์‹ฑ: ์ž์ฃผ ๊ฒ€์ƒ‰ํ•˜๋Š” ํ•„๋“œ์— ์ธ๋ฑ์Šค
โ€ข ์บ์‹ฑ: ์ตœ๊ทผ ๊ฒ€์ƒ‰ ๊ฒฐ๊ณผ ์บ์‹ฑ
โ€ข ํ”„๋ฆฌํŽ˜์นญ: ์‚ฌ์šฉ์ž ํŒจํ„ด ๋ถ„์„ํ•ด์„œ ๋ฏธ๋ฆฌ ๋กœ๋“œ

import asyncio
from concurrent.futures import ThreadPoolExecutor

class FastMemory:
    def __init__(self):
        self.executor = ThreadPoolExecutor(max_workers=5)
    
    async def parallel_search(self, query: str):
        """์—ฌ๋Ÿฌ ์†Œ์Šค๋ฅผ ๋ณ‘๋ ฌ๋กœ ๊ฒ€์ƒ‰"""
        tasks = [
            self.search_working_memory(query),
            self.search_vector_db(query),
            self.search_sql_db(query)
        ]
        
        results = await asyncio.gather(*tasks)
        return self.merge_results(results)
    
    async def search_working_memory(self, query: str):
        """Redis์—์„œ ๊ฒ€์ƒ‰ (๊ฐ€์žฅ ๋น ๋ฆ„)"""
        loop = asyncio.get_event_loop()
        return await loop.run_in_executor(
            self.executor, 
            self._sync_search_redis, 
            query
        )
    
    async def search_vector_db(self, query: str):
        """๋ฒกํ„ฐ DB ๊ฒ€์ƒ‰ (์ค‘๊ฐ„ ์†๋„)"""
        loop = asyncio.get_event_loop()
        return await loop.run_in_executor(
            self.executor, 
            self._sync_search_vector, 
            query
        )
    
    def merge_results(self, results: List[List[Dict]]) -> List[Dict]:
        """๊ฒฐ๊ณผ ๋ณ‘ํ•ฉ ๋ฐ ์ค‘๋ณต ์ œ๊ฑฐ"""
        merged = []
        seen_ids = set()
        
        for result_list in results:
            for item in result_list:
                if item['id'] not in seen_ids:
                    merged.append(item)
                    seen_ids.add(item['id'])
        
        # ๊ด€๋ จ์„ฑ ์ ์ˆ˜๋กœ ์ •๋ ฌ
        merged.sort(key=lambda x: x.get('score', 0), reverse=True)
        return merged[:10]

๐Ÿงช 4. ํ…Œ์ŠคํŠธ์™€ ํ‰๊ฐ€

์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์ด ์ œ๋Œ€๋กœ ์ž‘๋™ํ•˜๋Š”์ง€ ์–ด๋–ป๊ฒŒ ์•Œ ์ˆ˜ ์žˆ์„๊นŒ? ์ •๋Ÿ‰์  ํ‰๊ฐ€๊ฐ€ ํ•„์š”ํ•ด! ๐Ÿ“Š

๐ŸŽฏ ํ‰๊ฐ€ ์ง€ํ‘œ 1. Recall (์žฌํ˜„์œจ): ๊ด€๋ จ ์žˆ๋Š” ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์–ผ๋งˆ๋‚˜ ์ž˜ ์ฐพ์•„๋‚ด๋Š”๊ฐ€?
2. Precision (์ •๋ฐ€๋„): ์ฐพ์•„๋‚ธ ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ์‹ค์ œ๋กœ ๊ด€๋ จ ์žˆ๋Š”๊ฐ€?
3. Latency (์ง€์—ฐ์‹œ๊ฐ„): ๊ฒ€์ƒ‰์— ๊ฑธ๋ฆฌ๋Š” ์‹œ๊ฐ„
4. Relevance Score (๊ด€๋ จ์„ฑ ์ ์ˆ˜): ์‚ฌ์šฉ์ž๊ฐ€ ๋งŒ์กฑํ•˜๋Š”๊ฐ€?
5. Memory Utilization (๋ฉ”๋ชจ๋ฆฌ ํ™œ์šฉ๋„): ์ €์žฅ๋œ ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ์‹ค์ œ๋กœ ์‚ฌ์šฉ๋˜๋Š”๊ฐ€?
class MemoryEvaluator:
    def evaluate_recall(self, query: str, expected_ids: List[str], 
                       retrieved_ids: List[str]) -> float:
        """์žฌํ˜„์œจ ๊ณ„์‚ฐ"""
        relevant_retrieved = set(expected_ids) & set(retrieved_ids)
        return len(relevant_retrieved) / len(expected_ids) if expected_ids else 0
    
    def evaluate_precision(self, query: str, expected_ids: List[str], 
                          retrieved_ids: List[str]) -> float:
        """์ •๋ฐ€๋„ ๊ณ„์‚ฐ"""
        relevant_retrieved = set(expected_ids) & set(retrieved_ids)
        return len(relevant_retrieved) / len(retrieved_ids) if retrieved_ids else 0
    
    def run_benchmark(self, test_cases: List[Dict]):
        """๋ฒค์น˜๋งˆํฌ ์‹คํ–‰"""
        results = {
            'recall': [],
            'precision': [],
            'latency': []
        }
        
        for case in test_cases:
            start_time = time.time()
            retrieved = self.memory.search(case['query'])
            latency = time.time() - start_time
            
            retrieved_ids = [r['id'] for r in retrieved]
            
            recall = self.evaluate_recall(
                case['query'], 
                case['expected_ids'], 
                retrieved_ids
            )
            precision = self.evaluate_precision(
                case['query'], 
                case['expected_ids'], 
                retrieved_ids
            )
            
            results['recall'].append(recall)
            results['precision'].append(precision)
            results['latency'].append(latency)
        
        return {
            'avg_recall': sum(results['recall']) / len(results['recall']),
            'avg_precision': sum(results['precision']) / len(results['precision']),
            'avg_latency': sum(results['latency']) / len(results['latency'])
        }

๐Ÿš€ ์ตœ์‹  ํŠธ๋ Œ๋“œ์™€ ๋ฏธ๋ž˜ ๋ฐฉํ–ฅ

AI ์žฅ๊ธฐ๊ธฐ์–ต ๋ถ„์•ผ๋Š” ์ง€๊ธˆ๋„ ๋น ๋ฅด๊ฒŒ ๋ฐœ์ „ํ•˜๊ณ  ์žˆ์–ด. ์ตœ์‹  ํŠธ๋ Œ๋“œ๋ฅผ ์•Œ์•„๋‘๋ฉด ๋” ๋‚˜์€ ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ์ง€! ๐ŸŒŸ

๐Ÿงฌ 1. ์—ํ”ผ์†Œ๋”• ๋ฉ”๋ชจ๋ฆฌ (Episodic Memory)

์ธ๊ฐ„์˜ ์ผํ™” ๊ธฐ์–ต์ฒ˜๋Ÿผ, AI๋„ ํŠน์ • ์‚ฌ๊ฑด์ด๋‚˜ ๊ฒฝํ—˜์„ ์‹œ๊ฐ„ ์ˆœ์„œ๋Œ€๋กœ ๊ธฐ์–ตํ•˜๋Š” ๊ฑฐ์•ผ. ๋‹จ์ˆœํžˆ ์ •๋ณด๋ฅผ ์ €์žฅํ•˜๋Š” ๊ฒŒ ์•„๋‹ˆ๋ผ, "์–ธ์ œ, ์–ด๋””์„œ, ๋ฌด์Šจ ์ผ์ด ์žˆ์—ˆ๋Š”์ง€"๋ฅผ ๋งฅ๋ฝ๊ณผ ํ•จ๊ป˜ ์ €์žฅํ•ด.

์˜ˆ๋ฅผ ๋“ค์–ด, "์ง€๋‚œ๋ฒˆ์— ์žฌ๋Šฅ๋„ท์—์„œ ํ”„๋กœ๊ทธ๋ž˜๋ฐ ๊ฐ•์˜ ๋“ค์—ˆ๋˜ ๊ฑฐ ๊ธฐ์–ต๋‚˜?"๋ผ๊ณ  ๋ฌผ์œผ๋ฉด, ๋‹จ์ˆœํžˆ "ํ”„๋กœ๊ทธ๋ž˜๋ฐ"์ด๋ผ๋Š” ํ‚ค์›Œ๋“œ๋งŒ ์ฐพ๋Š” ๊ฒŒ ์•„๋‹ˆ๋ผ, ๊ทธ๋•Œ์˜ ์ „์ฒด ์ƒํ™ฉ์„ ์žฌ๊ตฌ์„ฑํ•  ์ˆ˜ ์žˆ์–ด. ๐ŸŽฌ

๐Ÿ”„ 2. ์ง€์†์  ํ•™์Šต (Continual Learning)

๊ธฐ์กด ์ง€์‹์„ ์žŠ์ง€ ์•Š์œผ๋ฉด์„œ ์ƒˆ๋กœ์šด ์ •๋ณด๋ฅผ ๊ณ„์† ํ•™์Šตํ•˜๋Š” ๊ฑฐ์•ผ. ์ด๊ฒŒ ์ง„์งœ ์–ด๋ ค์šด ๋ฌธ์ œ์ธ๋ฐ, "Catastrophic Forgetting"(ํŒŒ๊ตญ์  ๋ง๊ฐ)์ด๋ผ๋Š” ํ˜„์ƒ ๋•Œ๋ฌธ์ด์•ผ.

ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•:
โ€ข Elastic Weight Consolidation (EWC): ์ค‘์š”ํ•œ ๊ฐ€์ค‘์น˜๋Š” ๋œ ๋ณ€๊ฒฝ
โ€ข Progressive Neural Networks: ์ƒˆ๋กœ์šด ํƒœ์Šคํฌ๋งˆ๋‹ค ๋„คํŠธ์›Œํฌ ํ™•์žฅ
โ€ข Memory Replay: ๊ณผ๊ฑฐ ๋ฐ์ดํ„ฐ๋ฅผ ์ฃผ๊ธฐ์ ์œผ๋กœ ์žฌํ•™์Šต

๐Ÿง  3. ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ๋ฉ”๋ชจ๋ฆฌ

ํ…์ŠคํŠธ๋งŒ์ด ์•„๋‹ˆ๋ผ ์ด๋ฏธ์ง€, ์Œ์„ฑ, ๋น„๋””์˜ค๊นŒ์ง€ ํ†ตํ•ฉํ•ด์„œ ๊ธฐ์–ตํ•˜๋Š” ๊ฑฐ์•ผ. "์ €๋ฒˆ์— ๋ณด์—ฌ์ค€ ๊ทธ ์‚ฌ์ง„ ๊ธฐ์–ต๋‚˜?"๋ผ๊ณ  ๋ฌผ์œผ๋ฉด ์ฐพ์•„์ค„ ์ˆ˜ ์žˆ์–ด์•ผ์ง€! ๐Ÿ“ธ

CLIP, DALL-E ๊ฐ™์€ ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ๋ชจ๋ธ์„ ํ™œ์šฉํ•˜๋ฉด ๊ฐ€๋Šฅํ•ด. ํ…์ŠคํŠธ์™€ ์ด๋ฏธ์ง€๋ฅผ ๊ฐ™์€ ์ž„๋ฒ ๋”ฉ ๊ณต๊ฐ„์— ๋งคํ•‘ํ•˜๋Š” ๊ฑฐ์ง€.

๐Ÿค 4. ํŽ˜๋”๋ ˆ์ดํ‹ฐ๋“œ ๋ฉ”๋ชจ๋ฆฌ (Federated Memory)

์—ฌ๋Ÿฌ ์‚ฌ์šฉ์ž์˜ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ํ”„๋ผ์ด๋ฒ„์‹œ๋ฅผ ๋ณดํ˜ธํ•˜๋ฉด์„œ ๊ณต์œ ํ•˜๋Š” ๊ฐœ๋…์ด์•ผ. ์˜ˆ๋ฅผ ๋“ค์–ด, "๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์€ ์ด ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ–ˆ์ง€?"๋ผ๋Š” ์งˆ๋ฌธ์— ๋‹ตํ•  ์ˆ˜ ์žˆ์–ด.

๊ฐœ์ธ ์ •๋ณด๋Š” ๋กœ์ปฌ์— ์ €์žฅํ•˜๊ณ , ์ผ๋ฐ˜ํ™”๋œ ํŒจํ„ด๋งŒ ์ค‘์•™ ์„œ๋ฒ„์™€ ๊ณต์œ ํ•˜๋Š” ๋ฐฉ์‹์ด์•ผ. ๐Ÿ”

โšก 5. ์‹ค์‹œ๊ฐ„ ๋ฉ”๋ชจ๋ฆฌ ์—…๋ฐ์ดํŠธ

๋Œ€ํ™” ์ค‘์— ์‹ค์‹œ๊ฐ„์œผ๋กœ ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์—…๋ฐ์ดํŠธํ•˜๊ณ , ์ฆ‰์‹œ ๋ฐ˜์˜ํ•˜๋Š” ๊ฑฐ์•ผ. ๊ธฐ์กด์—๋Š” ๋Œ€ํ™”๊ฐ€ ๋๋‚œ ํ›„์— ๋ฐฐ์น˜๋กœ ์ฒ˜๋ฆฌํ–ˆ๋Š”๋ฐ, ์ด์ œ๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐฉ์‹์œผ๋กœ ์ฒ˜๋ฆฌํ•ด.

WebSocket์ด๋‚˜ Server-Sent Events๋ฅผ ์‚ฌ์šฉํ•ด์„œ ๊ตฌํ˜„ํ•  ์ˆ˜ ์žˆ์–ด.

๐ŸŽ“ ์‹ค์ „ ํ”„๋กœ์ ํŠธ: ์™„์ „ํ•œ ๋ฉ”๋ชจ๋ฆฌ ์‹œ์Šคํ…œ ๊ตฌ์ถ•

์ž, ์ด์ œ ๋ฐฐ์šด ๊ฑธ ๋‹ค ํ•ฉ์ณ์„œ ์™„์ „ํ•œ ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค์–ด๋ณด์ž! ๐Ÿ—๏ธ ์‹ค์ œ๋กœ ๋ฐฐํฌ ๊ฐ€๋Šฅํ•œ ์ˆ˜์ค€์˜ ์ฝ”๋“œ์•ผ.

๐Ÿ“ฆ ์ „์ฒด ์•„ํ‚คํ…์ฒ˜

memory_system/
โ”œโ”€โ”€ core/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ”œโ”€โ”€ memory_manager.py      # ๋ฉ”์ธ ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ์ž
โ”‚   โ”œโ”€โ”€ embeddings.py          # ์ž„๋ฒ ๋”ฉ ์ฒ˜๋ฆฌ
โ”‚   โ””โ”€โ”€ retrieval.py           # ๊ฒ€์ƒ‰ ๋กœ์ง
โ”œโ”€โ”€ storage/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ”œโ”€โ”€ redis_store.py         # Working Memory
โ”‚   โ”œโ”€โ”€ vector_store.py        # Recent Memory
โ”‚   โ””โ”€โ”€ sql_store.py           # Long-Term Memory
โ”œโ”€โ”€ utils/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ”œโ”€โ”€ encryption.py          # ์•”ํ˜ธํ™”
โ”‚   โ”œโ”€โ”€ preprocessing.py       # ์ „์ฒ˜๋ฆฌ
โ”‚   โ””โ”€โ”€ evaluation.py          # ํ‰๊ฐ€
โ”œโ”€โ”€ api/
โ”‚   โ”œโ”€โ”€ __init__.py
โ”‚   โ””โ”€โ”€ endpoints.py           # FastAPI ์—”๋“œํฌ์ธํŠธ
โ”œโ”€โ”€ config.py                  # ์„ค์ •
โ””โ”€โ”€ main.py                    # ์ง„์ž…์ 

๐Ÿ”ง ํ•ต์‹ฌ ๊ตฌํ˜„: MemoryManager

from typing import List, Dict, Optional
from datetime import datetime, timedelta
import asyncio

class MemoryManager:
    """ํ†ตํ•ฉ ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ ์‹œ์Šคํ…œ"""
    
    def __init__(self, user_id: str, config: Dict):
        self.user_id = user_id
        self.config = config
        
        # 3-Tier ์ €์žฅ์†Œ ์ดˆ๊ธฐํ™”
        self.working_memory = RedisStore(config['redis'])
        self.vector_memory = VectorStore(config['vector_db'])
        self.longterm_memory = SQLStore(config['postgres'])
        
        # ์œ ํ‹ธ๋ฆฌํ‹ฐ
        self.embedder = EmbeddingService(config['embedding'])
        self.encryptor = EncryptionService(config['encryption_key'])
        
    async def add_memory(self, content: str, metadata: Dict = None) -> str:
        """์ƒˆ๋กœ์šด ๋ฉ”๋ชจ๋ฆฌ ์ถ”๊ฐ€ (๋น„๋™๊ธฐ)"""
        try:
            # 1. ์ „์ฒ˜๋ฆฌ
            processed = self.preprocess(content)
            
            # 2. ๋ฏผ๊ฐ์ •๋ณด ์ต๋ช…ํ™”
            anonymized = self.encryptor.anonymize_pii(processed)
            
            # 3. ์ž„๋ฒ ๋”ฉ ์ƒ์„ฑ
            embedding = await self.embedder.get_embedding_async(anonymized)
            
            # 4. ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ๋ณด๊ฐ•
            enriched_metadata = self.enrich_metadata(content, metadata)
            
            # 5. 3-Tier์— ๋ณ‘๋ ฌ ์ €์žฅ
            memory_id = self.generate_id()
            
            tasks = [
                self.working_memory.add_async(memory_id, content),
                self.vector_memory.add_async(
                    memory_id, embedding, anonymized, enriched_metadata
                ),
                self.longterm_memory.add_async(
                    memory_id, content, enriched_metadata
                )
            ]
            
            await asyncio.gather(*tasks)
            
            return memory_id
            
        except Exception as e:
            logger.error(f"Failed to add memory: {e}")
            raise
    
    async def search_memory(self, query: str, 
                           filters: Dict = None,
                           top_k: int = 5) -> List[Dict]:
        """๋ฉ”๋ชจ๋ฆฌ ๊ฒ€์ƒ‰ (ํ•˜์ด๋ธŒ๋ฆฌ๋“œ)"""
        
        # 1. ์ฟผ๋ฆฌ ์ž„๋ฒ ๋”ฉ
        query_embedding = await self.embedder.get_embedding_async(query)
        
        # 2. 3-Tier ๋ณ‘๋ ฌ ๊ฒ€์ƒ‰
        working_results = await self.working_memory.search_async(query)
        vector_results = await self.vector_memory.search_async(
            query_embedding, filters, top_k * 2
        )
        longterm_results = await self.longterm_memory.search_async(
            query, filters
        )
        
        # 3. ๊ฒฐ๊ณผ ๋ณ‘ํ•ฉ ๋ฐ ๋ฆฌ๋žญํ‚น
        merged = self.merge_and_rerank(
            working_results, 
            vector_results, 
            longterm_results,
            query
        )
        
        # 4. ์ ‘๊ทผ ํšŸ์ˆ˜ ์—…๋ฐ์ดํŠธ (๋น„๋™๊ธฐ)
        asyncio.create_task(self.update_access_counts(merged))
        
        return merged[:top_k]
    
    def merge_and_rerank(self, *result_lists, query: str) -> List[Dict]:
        """๊ฒฐ๊ณผ ๋ณ‘ํ•ฉ ๋ฐ ๋ฆฌ๋žญํ‚น"""
        # ์ค‘๋ณต ์ œ๊ฑฐ
        seen = set()
        merged = []
        
        for results in result_lists:
            for item in results:
                if item['id'] not in seen:
                    merged.append(item)
                    seen.add(item['id'])
        
        # ๋ฆฌ๋žญํ‚น ์ ์ˆ˜ ๊ณ„์‚ฐ
        for item in merged:
            score = self.calculate_relevance_score(item, query)
            item['final_score'] = score
        
        # ์ •๋ ฌ
        merged.sort(key=lambda x: x['final_score'], reverse=True)
        return merged
    
    def calculate_relevance_score(self, item: Dict, query: str) -> float:
        """๊ด€๋ จ์„ฑ ์ ์ˆ˜ ๊ณ„์‚ฐ"""
        # ์—ฌ๋Ÿฌ ์š”์†Œ๋ฅผ ๊ณ ๋ คํ•œ ์ ์ˆ˜
        vector_score = item.get('similarity', 0.0)
        recency_score = self.calculate_recency_score(item['timestamp'])
        importance_score = item.get('importance', 0.5)
        access_score = min(item.get('access_count', 0) / 10, 1.0)
        
        # ๊ฐ€์ค‘ ํ‰๊ท 
        final_score = (
            vector_score * 0.4 +
            recency_score * 0.3 +
            importance_score * 0.2 +
            access_score * 0.1
        )
        
        return final_score
    
    def calculate_recency_score(self, timestamp: str) -> float:
        """์‹œ๊ฐ„ ๊ธฐ๋ฐ˜ ์ ์ˆ˜ (์ตœ๊ทผ์ผ์ˆ˜๋ก ๋†’์Œ)"""
        dt = datetime.fromisoformat(timestamp)
        age_days = (datetime.now() - dt).days
        
        # ์ง€์ˆ˜ ๊ฐ์‡ 
        return math.exp(-age_days / 30)  # 30์ผ ๋ฐ˜๊ฐ๊ธฐ
    
    async def forget_memory(self, memory_id: str):
        """๋ฉ”๋ชจ๋ฆฌ ์‚ญ์ œ (GDPR ์ค€์ˆ˜)"""
        tasks = [
            self.working_memory.delete_async(memory_id),
            self.vector_memory.delete_async(memory_id),
            self.longterm_memory.delete_async(memory_id)
        ]
        
        await asyncio.gather(*tasks)
        
        # ๊ฐ์‚ฌ ๋กœ๊ทธ
        await self.log_deletion(memory_id)
    
    async def compress_old_memories(self, days_threshold: int = 30):
        """์˜ค๋ž˜๋œ ๋ฉ”๋ชจ๋ฆฌ ์••์ถ•"""
        cutoff_date = datetime.now() - timedelta(days=days_threshold)
        
        # ์˜ค๋ž˜๋œ ๋ฉ”๋ชจ๋ฆฌ ์กฐํšŒ
        old_memories = await self.longterm_memory.get_before_date(cutoff_date)
        
        for memory in old_memories:
            # GPT๋กœ ์š”์•ฝ
            summary = await self.summarize_memory(memory['content'])
            
            # ์š”์•ฝ๋ณธ์œผ๋กœ ๊ต์ฒด
            await self.longterm_memory.update_async(
                memory['id'],
                {'content': summary, 'compressed': True}
            )
            
            # ๋ฒกํ„ฐ DB์—์„œ๋Š” ์‚ญ์ œ (๋น„์šฉ ์ ˆ๊ฐ)
            await self.vector_memory.delete_async(memory['id'])

๐ŸŒ FastAPI ์—”๋“œํฌ์ธํŠธ

from fastapi import FastAPI, HTTPException, Depends
from pydantic import BaseModel

app = FastAPI(title="AI Memory System")

class MemoryInput(BaseModel):
    content: str
    metadata: Optional[Dict] = None

class SearchQuery(BaseModel):
    query: str
    filters: Optional[Dict] = None
    top_k: int = 5

@app.post("/memory/add")
async def add_memory(
    input: MemoryInput,
    user_id: str = Depends(get_current_user)
):
    """๋ฉ”๋ชจ๋ฆฌ ์ถ”๊ฐ€"""
    manager = get_memory_manager(user_id)
    
    try:
        memory_id = await manager.add_memory(
            input.content, 
            input.metadata
        )
        return {"memory_id": memory_id, "status": "success"}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

@app.post("/memory/search")
async def search_memory(
    query: SearchQuery,
    user_id: str = Depends(get_current_user)
):
    """๋ฉ”๋ชจ๋ฆฌ ๊ฒ€์ƒ‰"""
    manager = get_memory_manager(user_id)
    
    results = await manager.search_memory(
        query.query,
        query.filters,
        query.top_k
    )
    
    return {"results": results, "count": len(results)}

@app.delete("/memory/{memory_id}")
async def delete_memory(
    memory_id: str,
    user_id: str = Depends(get_current_user)
):
    """๋ฉ”๋ชจ๋ฆฌ ์‚ญ์ œ"""
    manager = get_memory_manager(user_id)
    
    await manager.forget_memory(memory_id)
    return {"status": "deleted"}

@app.get("/memory/stats")
async def get_stats(user_id: str = Depends(get_current_user)):
    """๋ฉ”๋ชจ๋ฆฌ ํ†ต๊ณ„"""
    manager = get_memory_manager(user_id)
    
    stats = await manager.get_statistics()
    return stats

๐ŸŽฏ ๋งˆ๋ฌด๋ฆฌ: ํ•ต์‹ฌ ์š”์•ฝ

์™€, ์—ฌ๊ธฐ๊นŒ์ง€ ์™”๋‹ค๋‹ˆ ๋Œ€๋‹จํ•ด! ๐ŸŽ‰ ๊ธด ์—ฌ์ •์ด์—ˆ์ง€? ํ•ต์‹ฌ ๋‚ด์šฉ์„ ์ •๋ฆฌํ•ด๋ณผ๊ฒŒ.

โœ… ํ•ต์‹ฌ ํฌ์ธํŠธ 1. 3-Tier ์•„ํ‚คํ…์ฒ˜: Working Memory (Redis) + Recent Memory (Vector DB) + Long-Term Memory (SQL)

2. ์ž„๋ฒ ๋”ฉ์ด ํ•ต์‹ฌ: ์˜๋ฏธ ๊ธฐ๋ฐ˜ ๊ฒ€์ƒ‰์„ ์œ„ํ•ด ํ…์ŠคํŠธ๋ฅผ ๋ฒกํ„ฐ๋กœ ๋ณ€ํ™˜

3. ํ•˜์ด๋ธŒ๋ฆฌ๋“œ ๊ฒ€์ƒ‰: ๋ฒกํ„ฐ ์œ ์‚ฌ๋„ + ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ํ•„ํ„ฐ + ์‹œ๊ฐ„ ๊ฐ€์ค‘์น˜

4. ํ”„๋ผ์ด๋ฒ„์‹œ ์ตœ์šฐ์„ : ์•”ํ˜ธํ™”, ์ต๋ช…ํ™”, ์ ‘๊ทผ ์ œ์–ด, ์‚ญ์ œ ๊ธฐ๋Šฅ

5. ๋น„์šฉ ์ตœ์ ํ™”: ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ, ์บ์‹ฑ, ์••์ถ•, ๋กœ์ปฌ ๋ชจ๋ธ ํ™œ์šฉ

6. ์„ฑ๋Šฅ ์ค‘์š”: ๋น„๋™๊ธฐ ์ฒ˜๋ฆฌ, ๋ณ‘๋ ฌ ๊ฒ€์ƒ‰, ์ธ๋ฑ์‹ฑ

7. ์ง€์†์  ๊ฐœ์„ : ํ‰๊ฐ€ ์ง€ํ‘œ ๋ชจ๋‹ˆํ„ฐ๋ง, A/B ํ…Œ์ŠคํŠธ, ์‚ฌ์šฉ์ž ํ”ผ๋“œ๋ฐฑ

AI ์žฅ๊ธฐ๊ธฐ์–ต ์‹œ์Šคํ…œ์€ ๋‹จ์ˆœํžˆ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•˜๋Š” ๊ฒŒ ์•„๋‹ˆ์•ผ. ์‚ฌ์šฉ์ž๋ฅผ ์ดํ•ดํ•˜๊ณ , ๋งฅ๋ฝ์„ ํŒŒ์•…ํ•˜๊ณ , ๊ฐœ์ธํ™”๋œ ๊ฒฝํ—˜์„ ์ œ๊ณตํ•˜๋Š” ํ•ต์‹ฌ ๊ธฐ์ˆ ์ด์ง€. ๐Ÿง 

์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ๋„ ์ด๋Ÿฐ ๊ธฐ์ˆ ์„ ํ™œ์šฉํ•œ AI ์„œ๋น„์Šค๋“ค์ด ์ ์  ๋Š˜์–ด๋‚˜๊ณ  ์žˆ์–ด. ๊ฐœ๋ฐœ์ž๋“ค์ด ์ž์‹ ์˜ AI ํ”„๋กœ์ ํŠธ์— ์žฅ๊ธฐ๊ธฐ์–ต ๊ธฐ๋Šฅ์„ ์ถ”๊ฐ€ํ•˜๋ฉด์„œ, ๋” ๋˜‘๋˜‘ํ•˜๊ณ  ์œ ์šฉํ•œ ์„œ๋น„์Šค๋“ค์ด ํƒ„์ƒํ•˜๊ณ  ์žˆ์ง€.

๋„ˆ๋„ ์ด์ œ ์ง์ ‘ ๊ตฌํ˜„ํ•ด๋ณผ ์ˆ˜ ์žˆ์„ ๊ฑฐ์•ผ! ์ฒ˜์Œ์—๋Š” ๊ฐ„๋‹จํ•œ ๋ฒกํ„ฐ DB ์—ฐ๋™๋ถ€ํ„ฐ ์‹œ์ž‘ํ•ด์„œ, ์ ์ฐจ ๋ณต์žกํ•œ ๊ธฐ๋Šฅ๋“ค์„ ์ถ”๊ฐ€ํ•ด๋‚˜๊ฐ€๋ฉด ๋ผ. ์‹คํŒจ๋ฅผ ๋‘๋ ค์›Œํ•˜์ง€ ๋ง๊ณ , ๊ณ„์† ์‹คํ—˜ํ•˜๊ณ  ๊ฐœ์„ ํ•ด๋‚˜๊ฐ€๋Š” ๊ฒŒ ์ค‘์š”ํ•ด. ๐Ÿ’ช

๊ธฐ์–ตํ•ด, AI์˜ ์ง„์งœ ํž˜์€ ๋‹จ์ˆœํžˆ ๋งŽ์€ ๋ฐ์ดํ„ฐ๋ฅผ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ฒŒ ์•„๋‹ˆ๋ผ, ์ ์ ˆํ•œ ์ˆœ๊ฐ„์— ์ ์ ˆํ•œ ์ •๋ณด๋ฅผ ๊ธฐ์–ตํ•ด๋‚ด๋Š” ๋Šฅ๋ ฅ์— ์žˆ์–ด. ๋งˆ์น˜ ์ข‹์€ ์นœ๊ตฌ์ฒ˜๋Ÿผ ๋ง์ด์•ผ. ๐Ÿ˜Š

์ž, ์ด์ œ ๋‚˜๊ฐ€์„œ ๋ฉ‹์ง„ AI ๋ฉ”๋ชจ๋ฆฌ ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค์–ด๋ด! ํ™”์ดํŒ…! ๐Ÿš€โœจ

๐Ÿ’ก ๋” ๊นŠ์ด ๋ฐฐ์šฐ๊ณ  ์‹ถ๋‹ค๋ฉด? ์žฌ๋Šฅ๋„ท์—์„œ AI ๊ฐœ๋ฐœ ์ „๋ฌธ๊ฐ€๋“ค์˜ ๊ฐ•์˜๋ฅผ ๋“ค์–ด๋ณด์„ธ์š”!
์‹ค์ „ ํ”„๋กœ์ ํŠธ ๊ฒฝํ—˜์„ ๊ณต์œ ํ•˜๋Š” ๊ฐœ๋ฐœ์ž๋“ค๊ณผ ํ•จ๊ป˜ ์„ฑ์žฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๐ŸŒฑ

๋Œ“๊ธ€ ์ž‘์„ฑ

์ด ๊ธ€์— ๋Œ€ํ•œ ์—ฌ๋Ÿฌ๋ถ„์˜ ์ƒ๊ฐ์„ ๋“ค๋ ค์ฃผ์„ธ์š”

๋Œ“๊ธ€ 0
๋Œ“๊ธ€์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ์ค‘์ž…๋‹ˆ๋‹ค.

์•„์ง ๋Œ“๊ธ€์ด ์—†์Šต๋‹ˆ๋‹ค.