์ฝ˜ํ…์ธ  ๋Œ€ํ‘œ ์ด๋ฏธ์ง€ - ๐Ÿค– Transformers Pipeline์œผ๋กœ ํ…์ŠคํŠธ ์š”์•ฝ ์™„์ „ ์ •๋ณต

๐Ÿค– Transformers Pipeline์œผ๋กœ ํ…์ŠคํŠธ ์š”์•ฝ ์™„์ „ ์ •๋ณต

AI๊ฐ€ ๊ธด ๊ธ€์„ ์ˆœ์‹๊ฐ„์— ์š”์•ฝํ•ด์ฃผ๋Š” ๋งˆ๋ฒ• ๊ฐ™์€ ๊ธฐ์ˆ ! ๐ŸŽฏ

๐Ÿ“š ํ…์ŠคํŠธ ์š”์•ฝ, ์™œ ์ด๋ ‡๊ฒŒ ํ•ซํ•œ ๊ธฐ์ˆ ์ผ๊นŒ?

์•ˆ๋…•! ์˜ค๋Š˜์€ ์ •๋ง ์‹ค์šฉ์ ์ด๊ณ  ์žฌ๋ฏธ์žˆ๋Š” ์ฃผ์ œ๋ฅผ ๊ฐ€์ง€๊ณ  ์™”์–ด. ๋ฐ”๋กœ Transformers Pipeline์„ ํ™œ์šฉํ•œ ํ…์ŠคํŠธ ์š”์•ฝ์ด์•ผ. ๐Ÿ˜Š

์š”์ฆ˜ ์ •๋ณด์˜ ํ™์ˆ˜ ์‹œ๋Œ€์— ์‚ด๊ณ  ์žˆ์ž–์•„? ๋‰ด์Šค ๊ธฐ์‚ฌ, ๋…ผ๋ฌธ, ๋ณด๊ณ ์„œ, ์‹ฌ์ง€์–ด SNS ๊ธ€๊นŒ์ง€... ์ฝ์–ด์•ผ ํ•  ํ…์ŠคํŠธ๊ฐ€ ๋„ˆ๋ฌด ๋งŽ์•„์„œ ๋จธ๋ฆฌ๊ฐ€ ์•„ํ”Œ ์ง€๊ฒฝ์ด์ง€. ์ด๋Ÿด ๋•Œ AI๊ฐ€ ์ž๋™์œผ๋กœ ํ•ต์‹ฌ๋งŒ ์™์™ ๋ฝ‘์•„์„œ ์š”์•ฝํ•ด์ค€๋‹ค๋ฉด ์–ผ๋งˆ๋‚˜ ์ข‹์„๊นŒ? ๐ŸŽ

๋ฐ”๋กœ ์ด๊ฒŒ ํ…์ŠคํŠธ ์š”์•ฝ(Text Summarization) ๊ธฐ์ˆ ์ด์•ผ. ๊ทธ๋ฆฌ๊ณ  Hugging Face์˜ Transformers ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋Š” ์ด ์ž‘์—…์„ ์ •๋ง ์‰ฝ๊ฒŒ ๋งŒ๋“ค์–ด์ค˜. ๋ณต์žกํ•œ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ์„ ๋ช‡ ์ค„์˜ ์ฝ”๋“œ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค๋‹ˆ, ์ •๋ง ๋†€๋ž์ง€ ์•Š์•„?

๐Ÿ’ก ์‹ค์ œ ํ™œ์šฉ ์‚ฌ๋ก€ โ€ข ๋‰ด์Šค ๊ธฐ์‚ฌ ์ž๋™ ์š”์•ฝ ์„œ๋น„์Šค
โ€ข ํ•™์ˆ  ๋…ผ๋ฌธ ์ดˆ๋ก ์ƒ์„ฑ
โ€ข ๊ณ ๊ฐ ๋ฆฌ๋ทฐ ํ•ต์‹ฌ ๋‚ด์šฉ ์ถ”์ถœ
โ€ข ํšŒ์˜๋ก ์š”์•ฝ ์ž๋™ํ™”
โ€ข ๋ฒ•๋ฅ  ๋ฌธ์„œ ํ•ต์‹ฌ ์กฐํ•ญ ์ •๋ฆฌ
โ€ข SNS ๊ธด ๊ธ€ ์š”์•ฝ๋ด‡

ํŠนํžˆ ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ AI/๋จธ์‹ ๋Ÿฌ๋‹ ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•˜๊ฑฐ๋‚˜ ๋ฐฐ์šฐ๋ ค๋Š” ์‚ฌ๋žŒ๋“ค์—๊ฒŒ ์ด ๊ธฐ์ˆ ์€ ์ •๋ง ์œ ์šฉํ•ด. ์‹ค์ œ ํ”„๋กœ์ ํŠธ์— ๋ฐ”๋กœ ์ ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ์‹ค์šฉ์ ์ธ ์Šคํ‚ฌ์ด๊ฑฐ๋“ ! ๐Ÿš€

ํ…์ŠคํŠธ ์š”์•ฝ ํ”„๋กœ์„ธ์Šค ์›๋ณธ ํ…์ŠคํŠธ AI ๋ชจ๋ธ ์š”์•ฝ๋œ ํ…์ŠคํŠธ ๊ธด ๋ฌธ์„œ๋ฅผ ํ•ต์‹ฌ ๋‚ด์šฉ๋งŒ ์ถ”์ถœํ•˜์—ฌ ์งง๊ฒŒ ์••์ถ•

๐ŸŽฏ Transformers Pipeline์ด๋ž€ ๋ฌด์—‡์ธ๊ฐ€?

๋ณธ๊ฒฉ์ ์œผ๋กœ ๋“ค์–ด๊ฐ€๊ธฐ ์ „์—, Transformers Pipeline์ด ๋ญ”์ง€๋ถ€ํ„ฐ ์•Œ์•„๋ณด์ž. ๐Ÿ˜Ž

Hugging Face Transformers๋Š” ์ตœ์‹  ์ž์—ฐ์–ด์ฒ˜๋ฆฌ(NLP) ๋ชจ๋ธ๋“ค์„ ์‰ฝ๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ๋Š” ์˜คํ”ˆ์†Œ์Šค ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์•ผ. ๊ทธ๋ฆฌ๊ณ  Pipeline์€ ์ด ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์—์„œ ์ œ๊ณตํ•˜๋Š” ๊ฐ€์žฅ ๊ฐ„ํŽธํ•œ ์ธํ„ฐํŽ˜์ด์Šค์ง€.

์ƒ๊ฐํ•ด๋ด. ์ผ๋ฐ˜์ ์œผ๋กœ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜๋ ค๋ฉด:

1๋‹จ๊ณ„: ๋ชจ๋ธ ์•„ํ‚คํ…์ฒ˜ ์ •์˜ ๐Ÿ—๏ธ
2๋‹จ๊ณ„: ์‚ฌ์ „ํ•™์Šต๋œ ๊ฐ€์ค‘์น˜ ๋กœ๋“œ ๐Ÿ“ฆ
3๋‹จ๊ณ„: ํ† ํฌ๋‚˜์ด์ € ์„ค์ • โœ‚๏ธ
4๋‹จ๊ณ„: ์ž…๋ ฅ ๋ฐ์ดํ„ฐ ์ „์ฒ˜๋ฆฌ ๐Ÿ”ง
5๋‹จ๊ณ„: ๋ชจ๋ธ ์ถ”๋ก  ์‹คํ–‰ ๐Ÿš€
6๋‹จ๊ณ„: ์ถœ๋ ฅ ๋ฐ์ดํ„ฐ ํ›„์ฒ˜๋ฆฌ ๐Ÿ“Š

์ด ๋ชจ๋“  ๊ณผ์ •์ด ํ•„์š”ํ•ด. ๋ณต์žกํ•˜์ง€? ๐Ÿ˜ต

ํ•˜์ง€๋งŒ Pipeline์„ ์‚ฌ์šฉํ•˜๋ฉด:

from transformers import pipeline

summarizer = pipeline("summarization")
result = summarizer("๊ธด ํ…์ŠคํŠธ...")
print(result)

๋! ์ด๊ฒŒ ์ „๋ถ€์•ผ. ๐Ÿ˜ฒ

Pipeline์ด ์œ„์˜ ๋ชจ๋“  ๋ณต์žกํ•œ ๊ณผ์ •์„ ๋‚ด๋ถ€์ ์œผ๋กœ ์ž๋™์œผ๋กœ ์ฒ˜๋ฆฌํ•ด์ค˜. ๋งˆ์น˜ ์ž๋™์ฐจ๋ฅผ ์šด์ „ํ•  ๋•Œ ์—”์ง„์˜ ๋ณต์žกํ•œ ์ž‘๋™ ์›๋ฆฌ๋ฅผ ๋ชฐ๋ผ๋„ ๋˜๋Š” ๊ฒƒ์ฒ˜๋Ÿผ ๋ง์ด์•ผ. ๐Ÿš—

๐ŸŽ“ Pipeline์˜ ์žฅ์  1. ๊ฐ„ํŽธ์„ฑ: ๋ช‡ ์ค„์˜ ์ฝ”๋“œ๋กœ ์ตœ์‹  AI ๋ชจ๋ธ ์‚ฌ์šฉ
2. ์ผ๊ด€์„ฑ: ๋ชจ๋“  ์ž‘์—…์— ๋™์ผํ•œ ์ธํ„ฐํŽ˜์ด์Šค ์ œ๊ณต
3. ์ตœ์ ํ™”: ์ž๋™์œผ๋กœ ์ตœ์ ์˜ ์„ค์ • ์ ์šฉ
4. ์œ ์—ฐ์„ฑ: ํ•„์š”์‹œ ์„ธ๋ถ€ ์„ค์ • ์ปค์Šคํ„ฐ๋งˆ์ด์ง• ๊ฐ€๋Šฅ
5. ๋‹ค์–‘์„ฑ: 20๊ฐœ ์ด์ƒ์˜ NLP ์ž‘์—… ์ง€์›

๐Ÿ” ํ…์ŠคํŠธ ์š”์•ฝ์˜ ๋‘ ๊ฐ€์ง€ ๋ฐฉ์‹

ํ…์ŠคํŠธ ์š”์•ฝ์—๋Š” ํฌ๊ฒŒ ๋‘ ๊ฐ€์ง€ ์ ‘๊ทผ ๋ฐฉ์‹์ด ์žˆ์–ด. ๊ฐ๊ฐ์˜ ํŠน์ง•์„ ์ดํ•ดํ•˜๋ฉด ์ƒํ™ฉ์— ๋งž๊ฒŒ ์„ ํƒํ•  ์ˆ˜ ์žˆ์ง€! ๐Ÿ“–

๐Ÿ“ ์ถ”์ถœ์  ์š”์•ฝ (Extractive)

์›๋ฆฌ: ์›๋ณธ ํ…์ŠคํŠธ์—์„œ ์ค‘์š”ํ•œ ๋ฌธ์žฅ๋“ค์„ ๊ทธ๋Œ€๋กœ ๋ฝ‘์•„๋‚ด๋Š” ๋ฐฉ์‹

ํŠน์ง•:
โ€ข ์›๋ฌธ์˜ ๋ฌธ์žฅ์„ ๊ทธ๋Œ€๋กœ ์‚ฌ์šฉ
โ€ข ๋ฌธ๋ฒ•์ ์œผ๋กœ ์•ˆ์ „ํ•จ
โ€ข ์‚ฌ์‹ค ์™œ๊ณก ์œ„ํ—˜ ๋‚ฎ์Œ
โ€ข ์ž์—ฐ์Šค๋Ÿฝ์ง€ ์•Š์„ ์ˆ˜ ์žˆ์Œ

๋น„์œ : ํ˜•๊ด‘ํŽœ์œผ๋กœ ์ค‘์š”ํ•œ ๋ถ€๋ถ„๋งŒ ํ‘œ์‹œํ•˜๋Š” ๊ฒƒ ๐Ÿ–๏ธ

โœจ ์ƒ์„ฑ์  ์š”์•ฝ (Abstractive)

์›๋ฆฌ: ๋‚ด์šฉ์„ ์ดํ•ดํ•˜๊ณ  ์ƒˆ๋กœ์šด ๋ฌธ์žฅ์œผ๋กœ ์žฌ๊ตฌ์„ฑํ•˜๋Š” ๋ฐฉ์‹

ํŠน์ง•:
โ€ข ์ƒˆ๋กœ์šด ๋ฌธ์žฅ ์ƒ์„ฑ
โ€ข ๋” ์ž์—ฐ์Šค๋Ÿฌ์šด ์š”์•ฝ
โ€ข ์˜์—ญ๊ณผ ์žฌ๊ตฌ์„ฑ ๊ฐ€๋Šฅ
โ€ข ๊ณ ๊ธ‰ ๋ชจ๋ธ ํ•„์š”

๋น„์œ : ์ฑ…์„ ์ฝ๊ณ  ์ž๊ธฐ ๋ง๋กœ ์„ค๋ช…ํ•˜๋Š” ๊ฒƒ ๐Ÿ’ฌ

Transformers Pipeline์˜ "summarization" ์ž‘์—…์€ ์ฃผ๋กœ ์ƒ์„ฑ์  ์š”์•ฝ์„ ์ˆ˜ํ–‰ํ•ด. BART, T5, Pegasus ๊ฐ™์€ ์ตœ์‹  ๋ชจ๋ธ๋“ค์ด ํ…์ŠคํŠธ๋ฅผ ์ดํ•ดํ•˜๊ณ  ์ƒˆ๋กญ๊ฒŒ ์žฌ๊ตฌ์„ฑํ•˜๋Š” ๋ฐฉ์‹์ด์ง€. ๐Ÿค–

์ด๊ฒŒ ์™œ ์ค‘์š”ํ•˜๋ƒ๋ฉด, ์ƒ์„ฑ์  ์š”์•ฝ์ด ํ›จ์”ฌ ๋” ์ž์—ฐ์Šค๋Ÿฝ๊ณ  ์ฝ๊ธฐ ์ข‹์€ ๊ฒฐ๊ณผ๋ฅผ ๋งŒ๋“ค์–ด๋‚ด๊ฑฐ๋“ . ๋งˆ์น˜ ์‚ฌ๋žŒ์ด ์š”์•ฝํ•œ ๊ฒƒ์ฒ˜๋Ÿผ ๋ง์ด์•ผ!

๐Ÿ“Š ๋น„๊ต ์˜ˆ์‹œ ์›๋ฌธ: "์ธ๊ณต์ง€๋Šฅ ๊ธฐ์ˆ ์˜ ๋ฐœ์ „์œผ๋กœ ์ž์—ฐ์–ด์ฒ˜๋ฆฌ ๋ถ„์•ผ์—์„œ ๋†€๋ผ์šด ์„ฑ๊ณผ๊ฐ€ ๋‚˜ํƒ€๋‚˜๊ณ  ์žˆ๋‹ค. ํŠนํžˆ Transformer ์•„ํ‚คํ…์ฒ˜์˜ ๋“ฑ์žฅ ์ดํ›„ ๊ธฐ๊ณ„๋ฒˆ์—ญ, ํ…์ŠคํŠธ ์ƒ์„ฑ, ์งˆ์˜์‘๋‹ต ๋“ฑ ๋‹ค์–‘ํ•œ ์ž‘์—…์—์„œ ์ธ๊ฐ„ ์ˆ˜์ค€์˜ ์„ฑ๋Šฅ์„ ๋ณด์ด๊ณ  ์žˆ๋‹ค."

์ถ”์ถœ์  ์š”์•ฝ: "์ธ๊ณต์ง€๋Šฅ ๊ธฐ์ˆ ์˜ ๋ฐœ์ „์œผ๋กœ ์ž์—ฐ์–ด์ฒ˜๋ฆฌ ๋ถ„์•ผ์—์„œ ๋†€๋ผ์šด ์„ฑ๊ณผ๊ฐ€ ๋‚˜ํƒ€๋‚˜๊ณ  ์žˆ๋‹ค."

์ƒ์„ฑ์  ์š”์•ฝ: "AI ๋ฐœ์ „์œผ๋กœ NLP ๋ถ„์•ผ๊ฐ€ ํฌ๊ฒŒ ์„ฑ์žฅํ–ˆ์œผ๋ฉฐ, Transformer ๊ธฐ์ˆ ์ด ํ•ต์‹ฌ ์—ญํ• ์„ ํ•˜๊ณ  ์žˆ๋‹ค."

๐Ÿ› ๏ธ ํ™˜๊ฒฝ ์„ค์ • ๋ฐ ์„ค์น˜

์ž, ์ด์ œ ์‹ค์ „์œผ๋กœ ๋“ค์–ด๊ฐ€๋ณผ๊นŒ? ๋จผ์ € ํ•„์š”ํ•œ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋“ค์„ ์„ค์น˜ํ•ด์•ผ ํ•ด. ๊ฑฑ์ • ๋งˆ, ์–ด๋ ต์ง€ ์•Š์•„! ๐Ÿ˜Š

Step 1: Python ํ™˜๊ฒฝ ํ™•์ธ

Python 3.7 ์ด์ƒ์ด ํ•„์š”ํ•ด. ํ„ฐ๋ฏธ๋„์—์„œ ํ™•์ธํ•ด๋ณด์ž:
python --version
# ๋˜๋Š”
python3 --version
Step 2: ๊ฐ€์ƒํ™˜๊ฒฝ ์ƒ์„ฑ (์„ ํƒ์‚ฌํ•ญ์ด์ง€๋งŒ ๊ฐ•๋ ฅ ์ถ”์ฒœ!)

ํ”„๋กœ์ ํŠธ๋ณ„๋กœ ๋…๋ฆฝ์ ์ธ ํ™˜๊ฒฝ์„ ๋งŒ๋“ค๋ฉด ํŒจํ‚ค์ง€ ์ถฉ๋Œ์„ ๋ฐฉ์ง€ํ•  ์ˆ˜ ์žˆ์–ด:
# venv ์‚ฌ์šฉ
python -m venv summarization_env

# ํ™œ์„ฑํ™” (Windows)
summarization_env\Scripts\activate

# ํ™œ์„ฑํ™” (Mac/Linux)
source summarization_env/bin/activate
Step 3: Transformers ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์„ค์น˜

์ด๊ฒŒ ํ•ต์‹ฌ์ด์•ผ! ๐ŸŽฏ
pip install transformers
Step 4: PyTorch ๋˜๋Š” TensorFlow ์„ค์น˜

Transformers๋Š” ๋ฐฑ์—”๋“œ๋กœ PyTorch๋‚˜ TensorFlow๊ฐ€ ํ•„์š”ํ•ด. ๋Œ€๋ถ€๋ถ„ PyTorch๋ฅผ ์‚ฌ์šฉํ•˜์ง€:
# PyTorch (๊ถŒ์žฅ)
pip install torch

# ๋˜๋Š” TensorFlow
pip install tensorflow
โš ๏ธ ์ฃผ์˜์‚ฌํ•ญ GPU๋ฅผ ์‚ฌ์šฉํ•˜๋ ค๋ฉด CUDA ๋ฒ„์ „์— ๋งž๋Š” PyTorch๋ฅผ ์„ค์น˜ํ•ด์•ผ ํ•ด. PyTorch ๊ณต์‹ ์›น์‚ฌ์ดํŠธ(pytorch.org)์—์„œ ์‹œ์Šคํ…œ์— ๋งž๋Š” ์„ค์น˜ ๋ช…๋ น์–ด๋ฅผ ํ™•์ธํ•˜์ž!

CPU๋งŒ ์‚ฌ์šฉํ•œ๋‹ค๋ฉด ์œ„ ๋ช…๋ น์–ด๋กœ ์ถฉ๋ถ„ํ•ด. ์†๋„๋Š” ๋А๋ฆฌ์ง€๋งŒ ํ•™์Šต์šฉ์œผ๋กœ๋Š” ๋ฌธ์ œ์—†์–ด. ๐Ÿ’ป
Step 5: ์ถ”๊ฐ€ ์œ ํ‹ธ๋ฆฌํ‹ฐ ์„ค์น˜

๋” ํŽธ๋ฆฌํ•œ ์‚ฌ์šฉ์„ ์œ„ํ•ด:
# sentencepiece: ์ผ๋ถ€ ๋ชจ๋ธ์˜ ํ† ํฌ๋‚˜์ด์ €์— ํ•„์š”
pip install sentencepiece

# datasets: Hugging Face ๋ฐ์ดํ„ฐ์…‹ ์‚ฌ์šฉ์‹œ
pip install datasets

# accelerate: ์ถ”๋ก  ์†๋„ ํ–ฅ์ƒ
pip install accelerate
๐Ÿ’ก ํ•œ ๋ฒˆ์— ์„ค์น˜ํ•˜๊ธฐ requirements.txt ํŒŒ์ผ์„ ๋งŒ๋“ค์–ด์„œ ํ•œ ๋ฒˆ์— ์„ค์น˜ํ•  ์ˆ˜๋„ ์žˆ์–ด:
# requirements.txt ๋‚ด์šฉ
transformers>=4.30.0
torch>=2.0.0
sentencepiece>=0.1.99
datasets>=2.12.0
accelerate>=0.20.0

# ์„ค์น˜
pip install -r requirements.txt

์„ค์น˜๊ฐ€ ์™„๋ฃŒ๋˜๋ฉด ๊ฐ„๋‹จํ•œ ํ…Œ์ŠคํŠธ๋ฅผ ํ•ด๋ณด์ž:

python -c "import transformers; print(transformers.__version__)"

๋ฒ„์ „ ๋ฒˆํ˜ธ๊ฐ€ ์ถœ๋ ฅ๋˜๋ฉด ์„ฑ๊ณต! ๐ŸŽ‰

์„ค์น˜ ํ”„๋กœ์„ธ์Šค ํ๋ฆ„๋„ Python ํ™˜๊ฒฝ ํ™•์ธ Transformers ์„ค์น˜ PyTorch ์„ค์น˜ TensorFlow ์„ค์น˜ ์„ค์น˜ ์™„๋ฃŒ! 1 2 3

๐Ÿš€ ์ฒซ ๋ฒˆ์งธ ํ…์ŠคํŠธ ์š”์•ฝ ์‹คํ–‰ํ•˜๊ธฐ

๋“œ๋””์–ด ์‹ค์ „์ด์•ผ! ๊ฐ€์žฅ ๊ธฐ๋ณธ์ ์ธ ํ…์ŠคํŠธ ์š”์•ฝ๋ถ€ํ„ฐ ์‹œ์ž‘ํ•ด๋ณด์ž. ์ •๋ง ๊ฐ„๋‹จํ•ด์„œ ๋†€๋ž„ ๊ฑฐ์•ผ! ๐Ÿ˜

๊ฐ€์žฅ ๊ธฐ๋ณธ์ ์ธ ์˜ˆ์ œ
from transformers import pipeline

# ์š”์•ฝ ํŒŒ์ดํ”„๋ผ์ธ ์ƒ์„ฑ
summarizer = pipeline("summarization")

# ์š”์•ฝํ•  ํ…์ŠคํŠธ
text = """
์ธ๊ณต์ง€๋Šฅ ๊ธฐ์ˆ ์€ ์ตœ๊ทผ ๋ช‡ ๋…„๊ฐ„ ๊ธ‰๊ฒฉํ•œ ๋ฐœ์ „์„ ์ด๋ฃจ์—ˆ์Šต๋‹ˆ๋‹ค. 
ํŠนํžˆ ์ž์—ฐ์–ด์ฒ˜๋ฆฌ ๋ถ„์•ผ์—์„œ๋Š” Transformer ์•„ํ‚คํ…์ฒ˜์˜ ๋“ฑ์žฅ์œผ๋กœ 
ํ˜์‹ ์ ์ธ ๋ณ€ํ™”๊ฐ€ ์ผ์–ด๋‚ฌ์Šต๋‹ˆ๋‹ค. BERT, GPT, T5 ๋“ฑ์˜ ๋ชจ๋ธ๋“ค์€ 
๋‹ค์–‘ํ•œ ์–ธ์–ด ์ž‘์—…์—์„œ ์ธ๊ฐ„ ์ˆ˜์ค€์˜ ์„ฑ๋Šฅ์„ ๋ณด์—ฌ์ฃผ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. 
์ด๋Ÿฌํ•œ ๊ธฐ์ˆ ๋“ค์€ ๋ฒˆ์—ญ, ์š”์•ฝ, ์งˆ์˜์‘๋‹ต, ๊ฐ์„ฑ๋ถ„์„ ๋“ฑ 
์‹ค์ƒํ™œ์˜ ๋งŽ์€ ์˜์—ญ์— ์ ์šฉ๋˜๊ณ  ์žˆ์œผ๋ฉฐ, ์•ž์œผ๋กœ๋„ 
๋”์šฑ ๋ฐœ์ „ํ•  ๊ฒƒ์œผ๋กœ ๊ธฐ๋Œ€๋ฉ๋‹ˆ๋‹ค.
"""

# ์š”์•ฝ ์‹คํ–‰
summary = summarizer(text, max_length=50, min_length=10, do_sample=False)

# ๊ฒฐ๊ณผ ์ถœ๋ ฅ
print(summary[0]['summary_text'])

์ด๊ฒŒ ๋์ด์•ผ! ๐Ÿ˜ฒ ์ •๋ง ๊ฐ„๋‹จํ•˜์ง€?

์ฝ”๋“œ๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ์ฒ˜์Œ์—๋Š” ๋ชจ๋ธ์„ ๋‹ค์šด๋กœ๋“œํ•˜๋Š” ์‹œ๊ฐ„์ด ์ข€ ๊ฑธ๋ ค. ํ•˜์ง€๋งŒ ํ•œ ๋ฒˆ ๋‹ค์šด๋กœ๋“œํ•˜๋ฉด ๋กœ์ปฌ์— ์บ์‹œ๋˜์–ด์„œ ๋‹ค์Œ๋ถ€ํ„ฐ๋Š” ๋น ๋ฅด๊ฒŒ ์‹คํ–‰๋ผ. ๐Ÿ“ฆ

๐ŸŽฏ ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค๋ช… max_length: ์š”์•ฝ๋ฌธ์˜ ์ตœ๋Œ€ ํ† ํฐ ์ˆ˜ (๊ธฐ๋ณธ๊ฐ’: 130)
min_length: ์š”์•ฝ๋ฌธ์˜ ์ตœ์†Œ ํ† ํฐ ์ˆ˜ (๊ธฐ๋ณธ๊ฐ’: 30)
do_sample: ์ƒ˜ํ”Œ๋ง ์‚ฌ์šฉ ์—ฌ๋ถ€ (False๋ฉด greedy decoding)
length_penalty: ๊ธธ์ด ํŽ˜๋„ํ‹ฐ (์–‘์ˆ˜๋ฉด ๊ธด ๋ฌธ์žฅ ์„ ํ˜ธ)
num_beams: Beam search์˜ ๋น” ๊ฐœ์ˆ˜ (๋†’์„์ˆ˜๋ก ํ’ˆ์งˆ ํ–ฅ์ƒ)
๋” ์„ธ๋ฐ€ํ•œ ์ œ์–ด๊ฐ€ ํ•„์š”ํ•˜๋‹ค๋ฉด?
from transformers import pipeline

# ํŠน์ • ๋ชจ๋ธ ์ง€์ •
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",  # ์˜์–ด ์š”์•ฝ์— ๊ฐ•๋ ฅํ•œ ๋ชจ๋ธ
    device=0  # GPU ์‚ฌ์šฉ (CPU๋Š” -1)
)

text = """
The field of artificial intelligence has witnessed remarkable 
progress in recent years. Natural language processing, in particular, 
has been revolutionized by the introduction of Transformer architectures. 
Models like BERT, GPT, and T5 have achieved human-level performance 
across various language tasks. These technologies are being applied 
in real-world scenarios including translation, summarization, 
question answering, and sentiment analysis.
"""

# ๊ณ ๊ธ‰ ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค์ •
summary = summarizer(
    text,
    max_length=60,
    min_length=20,
    do_sample=False,
    num_beams=4,  # Beam search ์‚ฌ์šฉ
    length_penalty=2.0,  # ๋” ๊ธด ์š”์•ฝ ์„ ํ˜ธ
    early_stopping=True
)

print(summary[0]['summary_text'])

์œ„ ์ฝ”๋“œ์—์„œ๋Š” facebook/bart-large-cnn ๋ชจ๋ธ์„ ๋ช…์‹œ์ ์œผ๋กœ ์ง€์ •ํ–ˆ์–ด. ์ด ๋ชจ๋ธ์€ CNN/DailyMail ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ํ•™์Šต๋˜์–ด ๋‰ด์Šค ๊ธฐ์‚ฌ ์š”์•ฝ์— ํŠนํžˆ ๊ฐ•๋ ฅํ•ด. ๐Ÿ“ฐ

โš ๏ธ ์˜์–ด vs ํ•œ๊ตญ์–ด ๋Œ€๋ถ€๋ถ„์˜ ์‚ฌ์ „ํ•™์Šต๋œ ์š”์•ฝ ๋ชจ๋ธ์€ ์˜์–ด์— ์ตœ์ ํ™”๋˜์–ด ์žˆ์–ด. ํ•œ๊ตญ์–ด ํ…์ŠคํŠธ๋ฅผ ์š”์•ฝํ•˜๋ ค๋ฉด:

1. ๋‹ค๊ตญ์–ด ๋ชจ๋ธ ์‚ฌ์šฉ (์˜ˆ: mT5, mBART)
2. ํ•œ๊ตญ์–ด๋กœ ํŒŒ์ธํŠœ๋‹๋œ ๋ชจ๋ธ ์‚ฌ์šฉ
3. ์ง์ ‘ ํ•œ๊ตญ์–ด ๋ฐ์ดํ„ฐ๋กœ ํŒŒ์ธํŠœ๋‹

์ด ๋ถ€๋ถ„์€ ๋’ค์—์„œ ๋” ์ž์„ธํžˆ ๋‹ค๋ฃฐ๊ฒŒ! ๐Ÿ‡ฐ๐Ÿ‡ท

๐ŸŽจ ๋‹ค์–‘ํ•œ ์š”์•ฝ ๋ชจ๋ธ ์‚ดํŽด๋ณด๊ธฐ

Hugging Face Model Hub์—๋Š” ์ˆ˜๋งŽ์€ ์š”์•ฝ ๋ชจ๋ธ์ด ์žˆ์–ด. ๊ฐ ๋ชจ๋ธ๋งˆ๋‹ค ํŠน์ง•์ด ๋‹ค๋ฅด๋‹ˆ๊นŒ, ์ƒํ™ฉ์— ๋งž๊ฒŒ ์„ ํƒํ•˜๋Š” ๊ฒŒ ์ค‘์š”ํ•ด! ๐ŸŽฏ

๋ชจ๋ธ ํŠน์ง• ์žฅ์  ์šฉ๋„
BART Seq2Seq ๊ตฌ์กฐ ๋‰ด์Šค ์š”์•ฝ์— ๊ฐ•๋ ฅ ์ผ๋ฐ˜ ํ…์ŠคํŠธ ์š”์•ฝ
T5 Text-to-Text ๋‹ค๋ชฉ์  ํ™œ์šฉ ๊ฐ€๋Šฅ ๋ฒ”์šฉ ์š”์•ฝ
Pegasus ์š”์•ฝ ํŠนํ™” ํ•™์Šต ์š”์•ฝ ํ’ˆ์งˆ ์šฐ์ˆ˜ ๊ณ ํ’ˆ์งˆ ์š”์•ฝ
mT5 ๋‹ค๊ตญ์–ด ์ง€์› 100๊ฐœ ์ด์ƒ ์–ธ์–ด ํ•œ๊ตญ์–ด ํฌํ•จ ๋‹ค๊ตญ์–ด
LED ๊ธด ๋ฌธ์„œ ์ฒ˜๋ฆฌ 16K ํ† ํฐ ์ฒ˜๋ฆฌ ๋…ผ๋ฌธ, ๋ณด๊ณ ์„œ
1. BART (Bidirectional and Auto-Regressive Transformers)

BART๋Š” Facebook AI์—์„œ ๊ฐœ๋ฐœํ•œ ๋ชจ๋ธ๋กœ, ๋…ธ์ด์ฆˆ ์ œ๊ฑฐ ์ž๋™์ธ์ฝ”๋” ๋ฐฉ์‹์œผ๋กœ ํ•™์Šต๋ผ. ํŠนํžˆ facebook/bart-large-cnn ๋ฒ„์ „์€ CNN/DailyMail ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ํŒŒ์ธํŠœ๋‹๋˜์–ด ๋‰ด์Šค ์š”์•ฝ์— ํƒ์›”ํ•ด. ๐Ÿ“ฐ

from transformers import pipeline

# BART ๋ชจ๋ธ ์‚ฌ์šฉ
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

article = """
Scientists have discovered a new species of deep-sea fish 
in the Mariana Trench. The fish, which has been named 
Pseudoliparis swirei, was found at a depth of 8,000 meters, 
making it one of the deepest-living fish ever recorded. 
The discovery was made during a research expedition 
using advanced underwater cameras and collection devices.
"""

result = summarizer(article, max_length=40, min_length=10)
print(result[0]['summary_text'])
2. T5 (Text-to-Text Transfer Transformer)

T5๋Š” Google์—์„œ ๊ฐœ๋ฐœํ•œ ๋ชจ๋ธ๋กœ, ๋ชจ๋“  NLP ์ž‘์—…์„ ํ…์ŠคํŠธ-ํˆฌ-ํ…์ŠคํŠธ ํ˜•์‹์œผ๋กœ ํ†ต์ผํ–ˆ์–ด. ์š”์•ฝ ์ž‘์—…๋„ "summarize: [ํ…์ŠคํŠธ]" ํ˜•์‹์œผ๋กœ ์ž…๋ ฅํ•˜๋ฉด ๋ผ. ๐Ÿ”„

from transformers import pipeline

# T5 ๋ชจ๋ธ ์‚ฌ์šฉ
summarizer = pipeline("summarization", model="t5-base")

text = "summarize: " + """
Climate change is causing significant impacts on global ecosystems. 
Rising temperatures are leading to melting ice caps, rising sea levels, 
and more frequent extreme weather events. Scientists warn that 
immediate action is needed to reduce greenhouse gas emissions 
and mitigate the worst effects of climate change.
"""

result = summarizer(text, max_length=50)
print(result[0]['summary_text'])
3. Pegasus (Pre-training with Extracted Gap-sentences)

Pegasus๋Š” Google Research์—์„œ ์š”์•ฝ ์ž‘์—…์„ ์œ„ํ•ด ํŠน๋ณ„ํžˆ ์„ค๊ณ„ํ•œ ๋ชจ๋ธ์ด์•ผ. Gap Sentence Generation์ด๋ผ๋Š” ๋…ํŠนํ•œ ์‚ฌ์ „ํ•™์Šต ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•ด์„œ ์š”์•ฝ ํ’ˆ์งˆ์ด ๋งค์šฐ ์šฐ์ˆ˜ํ•ด. โญ

from transformers import pipeline

# Pegasus ๋ชจ๋ธ ์‚ฌ์šฉ
summarizer = pipeline("summarization", model="google/pegasus-xsum")

text = """
A new study published in Nature reveals that regular exercise 
can significantly improve cognitive function in older adults. 
Researchers followed 500 participants over five years and found 
that those who exercised at least three times per week showed 
better memory, attention, and problem-solving skills compared 
to sedentary individuals.
"""

result = summarizer(text, max_length=60, min_length=20)
print(result[0]['summary_text'])
๐Ÿ’ก ๋ชจ๋ธ ์„ ํƒ ๊ฐ€์ด๋“œ ๋‰ด์Šค ๊ธฐ์‚ฌ ์š”์•ฝ: facebook/bart-large-cnn
์ผ๋ฐ˜ ๋ฌธ์„œ ์š”์•ฝ: t5-base ๋˜๋Š” t5-large
๊ณ ํ’ˆ์งˆ ์š”์•ฝ: google/pegasus-xsum
๊ธด ๋ฌธ์„œ ์š”์•ฝ: allenai/led-base-16384
๋‹ค๊ตญ์–ด ์š”์•ฝ: google/mt5-base
ํ•œ๊ตญ์–ด ์š”์•ฝ: eenzeenee/t5-base-korean-summarization
4. ํ•œ๊ตญ์–ด ์š”์•ฝ ๋ชจ๋ธ ์‚ฌ์šฉํ•˜๊ธฐ

ํ•œ๊ตญ์–ด ํ…์ŠคํŠธ๋ฅผ ์š”์•ฝํ•˜๋ ค๋ฉด ํ•œ๊ตญ์–ด๋กœ ํ•™์Šต๋œ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•ด์•ผ ํ•ด. ๋‹คํ–‰ํžˆ Hugging Face์—๋Š” ํ•œ๊ตญ์–ด ์š”์•ฝ ๋ชจ๋ธ๋“ค์ด ์žˆ์–ด! ๐Ÿ‡ฐ๐Ÿ‡ท

from transformers import pipeline

# ํ•œ๊ตญ์–ด ์š”์•ฝ ๋ชจ๋ธ
summarizer = pipeline(
    "summarization",
    model="eenzeenee/t5-base-korean-summarization"
)

korean_text = """
์ธ๊ณต์ง€๋Šฅ ๊ธฐ์ˆ ์˜ ๋ฐœ์ „์œผ๋กœ ๋‹ค์–‘ํ•œ ์‚ฐ์—… ๋ถ„์•ผ์—์„œ ํ˜์‹ ์ด ์ผ์–ด๋‚˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
ํŠนํžˆ ์ž์—ฐ์–ด์ฒ˜๋ฆฌ ๊ธฐ์ˆ ์€ ๊ณ ๊ฐ ์„œ๋น„์Šค, ์ฝ˜ํ…์ธ  ์ƒ์„ฑ, ๋ฐ์ดํ„ฐ ๋ถ„์„ ๋“ฑ
์—ฌ๋Ÿฌ ์˜์—ญ์—์„œ ํ™œ์šฉ๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๊ธฐ์—…๋“ค์€ AI ๊ธฐ์ˆ ์„ ๋„์ž…ํ•˜์—ฌ
์—…๋ฌด ํšจ์œจ์„ฑ์„ ๋†’์ด๊ณ  ์ƒˆ๋กœ์šด ๋น„์ฆˆ๋‹ˆ์Šค ๊ธฐํšŒ๋ฅผ ์ฐฝ์ถœํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
์ „๋ฌธ๊ฐ€๋“ค์€ ์•ž์œผ๋กœ AI ๊ธฐ์ˆ ์ด ๋”์šฑ ๋ฐœ์ „ํ•˜์—ฌ ์šฐ๋ฆฌ ์ƒํ™œ ์ „๋ฐ˜์—
ํฐ ์˜ํ–ฅ์„ ๋ฏธ์น  ๊ฒƒ์œผ๋กœ ์ „๋งํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
"""

result = summarizer(korean_text, max_length=100, min_length=20)
print(result[0]['summary_text'])

ํ•œ๊ตญ์–ด ๋ชจ๋ธ์€ ์˜์–ด ๋ชจ๋ธ๋ณด๋‹ค ์„ ํƒ์ง€๊ฐ€ ์ ์ง€๋งŒ, ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ๊ฐ€์ง„ ์ „๋ฌธ๊ฐ€๋“ค์ด ์ปค์Šคํ…€ ๋ชจ๋ธ์„ ๋งŒ๋“ค์–ด์ฃผ๊ธฐ๋„ ํ•ด. ํŠน์ • ๋„๋ฉ”์ธ(๋ฒ•๋ฅ , ์˜๋ฃŒ, ๊ธˆ์œต ๋“ฑ)์— ํŠนํ™”๋œ ์š”์•ฝ ๋ชจ๋ธ์ด ํ•„์š”ํ•˜๋‹ค๋ฉด ์ „๋ฌธ๊ฐ€์˜ ๋„์›€์„ ๋ฐ›๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ! ๐Ÿค

์ฃผ์š” ์š”์•ฝ ๋ชจ๋ธ ๋น„๊ต BART Facebook AI ๋‰ด์Šค ์š”์•ฝ ํŠนํ™” CNN/DailyMail ํ•™์Šต ๋ฐ์ดํ„ฐ T5 Google Research ๋ฒ”์šฉ Text-to-Text ๋‹ค๋ชฉ์  ํ™œ์šฉ ๊ฐ€๋Šฅ Pegasus Google Research ์š”์•ฝ ์ „์šฉ ์„ค๊ณ„ ์ตœ๊ณ  ํ’ˆ์งˆ ์š”์•ฝ ์„ฑ๋Šฅ mT5 Google ๋‹ค๊ตญ์–ด ์ง€์› 100+ ์–ธ์–ด ํ•œ๊ตญ์–ด ํฌํ•จ ์„ฑ๋Šฅ ๋น„๊ต ์ง€ํ‘œ BART: 44.2 T5: 42.5 Pegasus: 45.8 ๋น ๋ฆ„ ๋ณดํ†ต ๋А๋ฆผ ROUGE ์ ์ˆ˜ ์ถ”๋ก  ์†๋„

โš™๏ธ ๊ณ ๊ธ‰ ํŒŒ๋ผ๋ฏธํ„ฐ์™€ ์ตœ์ ํ™” ๊ธฐ๋ฒ•

๊ธฐ๋ณธ ์‚ฌ์šฉ๋ฒ•์„ ์ตํ˜”๋‹ค๋ฉด, ์ด์ œ ๋” ์„ธ๋ฐ€ํ•œ ์ œ์–ด๋ฅผ ์œ„ํ•œ ๊ณ ๊ธ‰ ๊ธฐ๋ฒ•๋“ค์„ ์•Œ์•„๋ณด์ž! ์ด ๋ถ€๋ถ„์„ ์ž˜ ํ™œ์šฉํ•˜๋ฉด ์š”์•ฝ ํ’ˆ์งˆ์„ ํฌ๊ฒŒ ํ–ฅ์ƒ์‹œํ‚ฌ ์ˆ˜ ์žˆ์–ด. ๐ŸŽฏ

1. Beam Search vs Sampling

ํ…์ŠคํŠธ ์ƒ์„ฑ์—๋Š” ํฌ๊ฒŒ ๋‘ ๊ฐ€์ง€ ๋””์ฝ”๋”ฉ ์ „๋žต์ด ์žˆ์–ด:

๐Ÿ” Beam Search (๊ฒฐ์ •๋ก ์ ) โ€ข ์—ฌ๋Ÿฌ ํ›„๋ณด๋ฅผ ๋™์‹œ์— ํƒ์ƒ‰
โ€ข ๊ฐ€์žฅ ํ™•๋ฅ  ๋†’์€ ์‹œํ€€์Šค ์„ ํƒ
โ€ข ์•ˆ์ •์ ์ด๊ณ  ์ผ๊ด€๋œ ๊ฒฐ๊ณผ
โ€ข do_sample=False, num_beams > 1

๐ŸŽฒ Sampling (ํ™•๋ฅ ๋ก ์ )

โ€ข ํ™•๋ฅ  ๋ถ„ํฌ์—์„œ ์ƒ˜ํ”Œ๋ง
โ€ข ๋‹ค์–‘ํ•˜๊ณ  ์ฐฝ์˜์ ์ธ ๊ฒฐ๊ณผ
โ€ข ๋งค๋ฒˆ ๋‹ค๋ฅธ ์ถœ๋ ฅ ๊ฐ€๋Šฅ
โ€ข do_sample=True
from transformers import pipeline

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

text = """
Your long text here...
"""

# Beam Search ์‚ฌ์šฉ (์•ˆ์ •์ )
result_beam = summarizer(
    text,
    max_length=60,
    min_length=20,
    do_sample=False,
    num_beams=4,  # 4๊ฐœ์˜ ๋น” ์‚ฌ์šฉ
    early_stopping=True
)

# Sampling ์‚ฌ์šฉ (๋‹ค์–‘์„ฑ)
result_sample = summarizer(
    text,
    max_length=60,
    min_length=20,
    do_sample=True,
    top_k=50,  # ์ƒ์œ„ 50๊ฐœ ํ† ํฐ์—์„œ ์ƒ˜ํ”Œ๋ง
    top_p=0.95,  # ๋ˆ„์  ํ™•๋ฅ  95%๊นŒ์ง€์˜ ํ† ํฐ ๊ณ ๋ ค
    temperature=0.8  # ๋‚ฎ์„์ˆ˜๋ก ๋ณด์ˆ˜์ 
)

print("Beam Search:", result_beam[0]['summary_text'])
print("Sampling:", result_sample[0]['summary_text'])
๐Ÿ’ก ํŒŒ๋ผ๋ฏธํ„ฐ ์„ ํƒ ๊ฐ€์ด๋“œ num_beams: 2-5๊ฐ€ ์ ๋‹น (๋†’์„์ˆ˜๋ก ํ’ˆ์งˆโ†‘, ์†๋„โ†“)
temperature: 0.7-1.0 ์ถ”์ฒœ (๋‚ฎ์œผ๋ฉด ๋ณด์ˆ˜์ , ๋†’์œผ๋ฉด ์ฐฝ์˜์ )
top_k: 40-50 ์ •๋„๊ฐ€ ์ ๋‹น
top_p: 0.9-0.95 ์‚ฌ์ด๊ฐ€ ์ข‹์Œ
length_penalty: 1.0์ด ๊ธฐ๋ณธ, 2.0์ด๋ฉด ๊ธด ๋ฌธ์žฅ ์„ ํ˜ธ
2. ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ๋กœ ์†๋„ ํ–ฅ์ƒ

์—ฌ๋Ÿฌ ํ…์ŠคํŠธ๋ฅผ ํ•œ ๋ฒˆ์— ์ฒ˜๋ฆฌํ•˜๋ฉด ํ›จ์”ฌ ํšจ์œจ์ ์ด์•ผ! ๐Ÿš€

from transformers import pipeline

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

# ์—ฌ๋Ÿฌ ํ…์ŠคํŠธ๋ฅผ ๋ฆฌ์ŠคํŠธ๋กœ ์ค€๋น„
texts = [
    "First article text...",
    "Second article text...",
    "Third article text..."
]

# ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ
results = summarizer(
    texts,
    max_length=60,
    min_length=20,
    batch_size=8  # ํ•œ ๋ฒˆ์— 8๊ฐœ์”ฉ ์ฒ˜๋ฆฌ
)

# ๊ฒฐ๊ณผ ์ถœ๋ ฅ
for i, result in enumerate(results):
    print(f"Summary {i+1}: {result['summary_text']}\n")
3. GPU ๊ฐ€์† ํ™œ์šฉ

GPU๊ฐ€ ์žˆ๋‹ค๋ฉด ๋ฐ˜๋“œ์‹œ ํ™œ์šฉํ•˜์ž! ์†๋„๊ฐ€ 10๋ฐฐ ์ด์ƒ ๋นจ๋ผ์ ธ. โšก

from transformers import pipeline
import torch

# GPU ์‚ฌ์šฉ ๊ฐ€๋Šฅ ์—ฌ๋ถ€ ํ™•์ธ
device = 0 if torch.cuda.is_available() else -1
print(f"Using device: {'GPU' if device == 0 else 'CPU'}")

# GPU ์‚ฌ์šฉ ์„ค์ •
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device=device
)

# ์ถ”๋ก  ์‹คํ–‰
text = "Your text here..."
result = summarizer(text, max_length=60)
print(result[0]['summary_text'])
โš ๏ธ ๋ฉ”๋ชจ๋ฆฌ ๊ด€๋ฆฌ ํฐ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•  ๋•Œ๋Š” ๋ฉ”๋ชจ๋ฆฌ ๋ถ€์กฑ ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์–ด. ์ด๋Ÿด ๋•Œ๋Š”:

1. ๋ฐฐ์น˜ ํฌ๊ธฐ ์ค„์ด๊ธฐ
2. ๋” ์ž‘์€ ๋ชจ๋ธ ์‚ฌ์šฉ (base ๋Œ€์‹  small)
3. ํ…์ŠคํŠธ ๊ธธ์ด ์ œํ•œ
4. 8bit ์–‘์žํ™” ์‚ฌ์šฉ (๋‚˜์ค‘์— ์„ค๋ช…!)
4. ๊ธด ๋ฌธ์„œ ์ฒ˜๋ฆฌ ์ „๋žต

๋Œ€๋ถ€๋ถ„์˜ ๋ชจ๋ธ์€ 512-1024 ํ† ํฐ ์ •๋„๋งŒ ์ฒ˜๋ฆฌํ•  ์ˆ˜ ์žˆ์–ด. ๊ธด ๋ฌธ์„œ๋Š” ์–ด๋–ป๊ฒŒ ํ• ๊นŒ? ๐Ÿค”

from transformers import pipeline

def summarize_long_text(text, max_chunk_length=1000):
    """๊ธด ํ…์ŠคํŠธ๋ฅผ ์ฒญํฌ๋กœ ๋‚˜๋ˆ ์„œ ์š”์•ฝ"""
    summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
    
    # ํ…์ŠคํŠธ๋ฅผ ๋ฌธ์žฅ ๋‹จ์œ„๋กœ ๋ถ„๋ฆฌ
    sentences = text.split('. ')
    
    # ์ฒญํฌ๋กœ ๋‚˜๋ˆ„๊ธฐ
    chunks = []
    current_chunk = ""
    
    for sentence in sentences:
        if len(current_chunk) + len(sentence) < max_chunk_length:
            current_chunk += sentence + ". "
        else:
            if current_chunk:
                chunks.append(current_chunk)
            current_chunk = sentence + ". "
    
    if current_chunk:
        chunks.append(current_chunk)
    
    # ๊ฐ ์ฒญํฌ ์š”์•ฝ
    summaries = []
    for chunk in chunks:
        result = summarizer(chunk, max_length=100, min_length=30)
        summaries.append(result[0]['summary_text'])
    
    # ์š”์•ฝ๋“ค์„ ํ•ฉ์ณ์„œ ์ตœ์ข… ์š”์•ฝ
    combined = " ".join(summaries)
    
    if len(combined) > 500:  # ์—ฌ์ „ํžˆ ๊ธธ๋ฉด ํ•œ ๋ฒˆ ๋” ์š”์•ฝ
        final_result = summarizer(combined, max_length=150, min_length=50)
        return final_result[0]['summary_text']
    
    return combined

# ์‚ฌ์šฉ ์˜ˆ์‹œ
long_document = """
Very long document text that exceeds the model's maximum length...
[์—ฌ๋Ÿฌ ํŽ˜์ด์ง€ ๋ถ„๋Ÿ‰์˜ ํ…์ŠคํŠธ]
"""

summary = summarize_long_text(long_document)
print(summary)

๋˜๋Š” ๊ธด ๋ฌธ์„œ ์ „์šฉ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•  ์ˆ˜๋„ ์žˆ์–ด:

from transformers import pipeline

# LED (Longformer Encoder-Decoder) - 16K ํ† ํฐ ์ฒ˜๋ฆฌ ๊ฐ€๋Šฅ
summarizer = pipeline(
    "summarization",
    model="allenai/led-base-16384"
)

long_text = """
[๋งค์šฐ ๊ธด ๋ฌธ์„œ...]
"""

result = summarizer(long_text, max_length=200, min_length=50)
print(result[0]['summary_text'])
5. ์š”์•ฝ ํ’ˆ์งˆ ํ‰๊ฐ€

์š”์•ฝ์ด ์ž˜ ๋˜์—ˆ๋Š”์ง€ ์–ด๋–ป๊ฒŒ ์•Œ ์ˆ˜ ์žˆ์„๊นŒ? ROUGE ๋ฉ”ํŠธ๋ฆญ์„ ์‚ฌ์šฉํ•ด๋ณด์ž! ๐Ÿ“Š

from transformers import pipeline
from datasets import load_metric

# ROUGE ๋ฉ”ํŠธ๋ฆญ ๋กœ๋“œ
rouge = load_metric("rouge")

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

# ์›๋ณธ ํ…์ŠคํŠธ์™€ ์ฐธ์กฐ ์š”์•ฝ
text = "Your article text..."
reference_summary = "Human-written summary..."

# ๋ชจ๋ธ ์š”์•ฝ
model_summary = summarizer(text, max_length=60)[0]['summary_text']

# ROUGE ์ ์ˆ˜ ๊ณ„์‚ฐ
scores = rouge.compute(
    predictions=[model_summary],
    references=[reference_summary]
)

print("ROUGE-1:", scores['rouge1'].mid.fmeasure)
print("ROUGE-2:", scores['rouge2'].mid.fmeasure)
print("ROUGE-L:", scores['rougeL'].mid.fmeasure)
๐Ÿ“ˆ ROUGE ์ ์ˆ˜ ํ•ด์„ ROUGE-1: ๋‹จ์–ด ๋‹จ์œ„ ๊ฒน์นจ (0.3-0.5๋ฉด ์–‘ํ˜ธ)
ROUGE-2: 2-gram ๊ฒน์นจ (0.15-0.25๋ฉด ์–‘ํ˜ธ)
ROUGE-L: ์ตœ์žฅ ๊ณตํ†ต ๋ถ€๋ถ„์ˆ˜์—ด (0.25-0.4๋ฉด ์–‘ํ˜ธ)

์ ์ˆ˜๊ฐ€ ๋†’์„์ˆ˜๋ก ์ฐธ์กฐ ์š”์•ฝ๊ณผ ์œ ์‚ฌํ•ด. ํ•˜์ง€๋งŒ ์ ์ˆ˜๊ฐ€ ๋‚ฎ๋‹ค๊ณ  ๋‚˜์œ ๊ฑด ์•„๋‹ˆ์•ผ. ๋‹ค๋ฅธ ํ‘œํ˜„์œผ๋กœ ๊ฐ™์€ ์˜๋ฏธ๋ฅผ ์ „๋‹ฌํ•  ์ˆ˜ ์žˆ๊ฑฐ๋“ ! ๐ŸŽฏ

๐Ÿ› ๏ธ ์‹ค์ „ ํ”„๋กœ์ ํŠธ: ๋‰ด์Šค ์š”์•ฝ ์‹œ์Šคํ…œ ๋งŒ๋“ค๊ธฐ

์ด์ œ ๋ฐฐ์šด ๋‚ด์šฉ์„ ์ข…ํ•ฉํ•ด์„œ ์‹ค์ œ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋‰ด์Šค ์š”์•ฝ ์‹œ์Šคํ…œ์„ ๋งŒ๋“ค์–ด๋ณด์ž! ์ด๊ฑด ํฌํŠธํด๋ฆฌ์˜ค์—๋„ ๋„ฃ์„ ์ˆ˜ ์žˆ๋Š” ๋ฉ‹์ง„ ํ”„๋กœ์ ํŠธ์•ผ. ๐ŸŒŸ

ํ”„๋กœ์ ํŠธ ๊ตฌ์กฐ
news_summarizer/
โ”‚
โ”œโ”€โ”€ summarizer.py          # ํ•ต์‹ฌ ์š”์•ฝ ๋กœ์ง
โ”œโ”€โ”€ web_scraper.py         # ๋‰ด์Šค ํฌ๋กค๋ง
โ”œโ”€โ”€ app.py                 # Flask ์›น ์•ฑ
โ”œโ”€โ”€ requirements.txt       # ์˜์กด์„ฑ ํŒจํ‚ค์ง€
โ””โ”€โ”€ templates/
    โ””โ”€โ”€ index.html         # ์›น ์ธํ„ฐํŽ˜์ด์Šค
1. summarizer.py - ํ•ต์‹ฌ ์š”์•ฝ ์—”์ง„
from transformers import pipeline
import torch

class NewsSummarizer:
    def __init__(self, model_name="facebook/bart-large-cnn"):
        """๋‰ด์Šค ์š”์•ฝ๊ธฐ ์ดˆ๊ธฐํ™”"""
        device = 0 if torch.cuda.is_available() else -1
        self.summarizer = pipeline(
            "summarization",
            model=model_name,
            device=device
        )
        
    def summarize(self, text, max_length=130, min_length=30):
        """ํ…์ŠคํŠธ ์š”์•ฝ ์‹คํ–‰"""
        try:
            # ํ…์ŠคํŠธ ๊ธธ์ด ํ™•์ธ
            if len(text.split()) < 50:
                return "ํ…์ŠคํŠธ๊ฐ€ ๋„ˆ๋ฌด ์งง์•„์„œ ์š”์•ฝํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค."
            
            # ์š”์•ฝ ์‹คํ–‰
            result = self.summarizer(
                text,
                max_length=max_length,
                min_length=min_length,
                do_sample=False,
                num_beams=4,
                length_penalty=2.0,
                early_stopping=True
            )
            
            return result[0]['summary_text']
            
        except Exception as e:
            return f"์š”์•ฝ ์ค‘ ์˜ค๋ฅ˜ ๋ฐœ์ƒ: {str(e)}"
    
    def summarize_batch(self, texts, max_length=130, min_length=30):
        """์—ฌ๋Ÿฌ ํ…์ŠคํŠธ ๋ฐฐ์น˜ ์š”์•ฝ"""
        try:
            results = self.summarizer(
                texts,
                max_length=max_length,
                min_length=min_length,
                batch_size=4,
                do_sample=False,
                num_beams=4
            )
            
            return [r['summary_text'] for r in results]
            
        except Exception as e:
            return [f"์˜ค๋ฅ˜: {str(e)}"] * len(texts)

# ์‚ฌ์šฉ ์˜ˆ์‹œ
if __name__ == "__main__":
    summarizer = NewsSummarizer()
    
    article = """
    Artificial intelligence is transforming industries worldwide.
    Companies are investing billions in AI research and development.
    Machine learning models are becoming more sophisticated,
    enabling new applications in healthcare, finance, and transportation.
    Experts predict AI will continue to reshape the global economy
    in the coming decades.
    """
    
    summary = summarizer.summarize(article)
    print("์š”์•ฝ:", summary)
2. web_scraper.py - ๋‰ด์Šค ํฌ๋กค๋ง
import requests
from bs4 import BeautifulSoup
from newspaper import Article

class NewsScaper:
    def __init__(self):
        self.headers = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
        }
    
    def fetch_article(self, url):
        """URL์—์„œ ๊ธฐ์‚ฌ ๋‚ด์šฉ ์ถ”์ถœ"""
        try:
            article = Article(url)
            article.download()
            article.parse()
            
            return {
                'title': article.title,
                'text': article.text,
                'authors': article.authors,
                'publish_date': article.publish_date,
                'url': url
            }
            
        except Exception as e:
            return {'error': str(e)}
    
    def fetch_multiple_articles(self, urls):
        """์—ฌ๋Ÿฌ URL์—์„œ ๊ธฐ์‚ฌ ์ถ”์ถœ"""
        articles = []
        for url in urls:
            article = self.fetch_article(url)
            if 'error' not in article:
                articles.append(article)
        return articles

# ์‚ฌ์šฉ ์˜ˆ์‹œ
if __name__ == "__main__":
    scraper = NewsScaper()
    
    url = "https://example.com/news/article"
    article = scraper.fetch_article(url)
    
    if 'error' not in article:
        print("์ œ๋ชฉ:", article['title'])
        print("๋‚ด์šฉ ๊ธธ์ด:", len(article['text']))
3. app.py - Flask ์›น ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜
from flask import Flask, render_template, request, jsonify
from summarizer import NewsSummarizer
from web_scraper import NewsScaper

app = Flask(__name__)

# ์ „์—ญ ๊ฐ์ฒด ์ดˆ๊ธฐํ™”
summarizer = NewsSummarizer()
scraper = NewsScaper()

@app.route('/')
def index():
    """๋ฉ”์ธ ํŽ˜์ด์ง€"""
    return render_template('index.html')

@app.route('/summarize', methods=['POST'])
def summarize():
    """ํ…์ŠคํŠธ ์š”์•ฝ API"""
    data = request.json
    text = data.get('text', '')
    max_length = data.get('max_length', 130)
    min_length = data.get('min_length', 30)
    
    if not text:
        return jsonify({'error': 'ํ…์ŠคํŠธ๋ฅผ ์ž…๋ ฅํ•ด์ฃผ์„ธ์š”'}), 400
    
    summary = summarizer.summarize(text, max_length, min_length)
    
    return jsonify({
        'summary': summary,
        'original_length': len(text.split()),
        'summary_length': len(summary.split())
    })

@app.route('/summarize_url', methods=['POST'])
def summarize_url():
    """URL์—์„œ ๊ธฐ์‚ฌ ์ถ”์ถœ ๋ฐ ์š”์•ฝ"""
    data = request.json
    url = data.get('url', '')
    
    if not url:
        return jsonify({'error': 'URL์„ ์ž…๋ ฅํ•ด์ฃผ์„ธ์š”'}), 400
    
    # ๊ธฐ์‚ฌ ํฌ๋กค๋ง
    article = scraper.fetch_article(url)
    
    if 'error' in article:
        return jsonify({'error': article['error']}), 400
    
    # ์š”์•ฝ ์‹คํ–‰
    summary = summarizer.summarize(article['text'])
    
    return jsonify({
        'title': article['title'],
        'summary': summary,
        'url': url,
        'original_length': len(article['text'].split()),
        'summary_length': len(summary.split())
    })

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=5000)
4. requirements.txt
transformers>=4.30.0
torch>=2.0.0
flask>=2.3.0
newspaper3k>=0.2.8
beautifulsoup4>=4.12.0
requests>=2.31.0
sentencepiece>=0.1.99
5. ์‹คํ–‰ ๋ฐฉ๋ฒ•
# ํŒจํ‚ค์ง€ ์„ค์น˜
pip install -r requirements.txt

# ์„œ๋ฒ„ ์‹คํ–‰
python app.py

# ๋ธŒ๋ผ์šฐ์ €์—์„œ ์ ‘์†
# http://localhost:5000
๐Ÿš€ ๋ฐฐํฌ ํŒ 1. Heroku ๋ฐฐํฌ: ๋ฌด๋ฃŒ ํ‹ฐ์–ด๋กœ ์‹œ์ž‘ ๊ฐ€๋Šฅ
2. AWS Lambda: ์„œ๋ฒ„๋ฆฌ์Šค๋กœ ๋น„์šฉ ์ ˆ๊ฐ
3. Docker: ์ปจํ…Œ์ด๋„ˆํ™”๋กœ ์ด์‹์„ฑ ํ–ฅ์ƒ
4. Streamlit: ๋น ๋ฅธ ํ”„๋กœํ† ํƒ€์ž… ์ œ์ž‘

๋ชจ๋ธ ํฌ๊ธฐ๊ฐ€ ํฌ๋ฉด ๋ฐฐํฌ๊ฐ€ ์–ด๋ ค์šธ ์ˆ˜ ์žˆ์–ด. ์ด๋Ÿด ๋•Œ๋Š” Hugging Face Inference API๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ! ๐ŸŽฏ
๐Ÿ’ผ ์‹ค์ „ ํ™œ์šฉ ์•„์ด๋””์–ด 1. ๋‰ด์Šค ํ๋ ˆ์ด์…˜ ์„œ๋น„์Šค: ์—ฌ๋Ÿฌ ๋‰ด์Šค๋ฅผ ์ž๋™ ์š”์•ฝ
2. ์—ฐ๊ตฌ ๋…ผ๋ฌธ ์š”์•ฝ: arXiv ๋…ผ๋ฌธ ์ž๋™ ์š”์•ฝ
3. ํšŒ์˜๋ก ์š”์•ฝ: ๊ธด ํšŒ์˜ ๋‚ด์šฉ ํ•ต์‹ฌ ์ •๋ฆฌ
4. ๊ณ ๊ฐ ๋ฆฌ๋ทฐ ๋ถ„์„: ์ˆ˜์ฒœ ๊ฐœ ๋ฆฌ๋ทฐ ์š”์•ฝ
5. ๋ฒ•๋ฅ  ๋ฌธ์„œ ์š”์•ฝ: ๊ณ„์•ฝ์„œ ํ•ต์‹ฌ ์กฐํ•ญ ์ถ”์ถœ
6. SNS ๋ชจ๋‹ˆํ„ฐ๋ง: ๋ธŒ๋žœ๋“œ ์–ธ๊ธ‰ ์š”์•ฝ

์ด๋Ÿฐ ํ”„๋กœ์ ํŠธ๋“ค์€ ์‹ค์ œ ๋น„์ฆˆ๋‹ˆ์Šค ๊ฐ€์น˜๊ฐ€ ์žˆ์–ด์„œ, ํ”„๋ฆฌ๋žœ์„œ ํ”Œ๋žซํผ์ด๋‚˜ ์žฌ๋Šฅ ๊ณต์œ  ์‚ฌ์ดํŠธ์—์„œ ์ˆ˜์ตํ™”ํ•  ์ˆ˜ ์žˆ์–ด! ๐Ÿ’ฐ
๋‰ด์Šค ์š”์•ฝ ์‹œ์Šคํ…œ ์•„ํ‚คํ…์ฒ˜ ์‚ฌ์šฉ์ž ์›น ๋ธŒ๋ผ์šฐ์ € Flask ์„œ๋ฒ„ API ์—”๋“œํฌ์ธํŠธ ์›น ์Šคํฌ๋ž˜ํผ ๋‰ด์Šค ํฌ๋กค๋ง ์š”์•ฝ ์—”์ง„ Transformers AI ๋ชจ๋ธ BART/T5/Pegasus ์ฒ˜๋ฆฌ ํ๋ฆ„ URL ์ž…๋ ฅ ๊ธฐ์‚ฌ ์ถ”์ถœ ์ „์ฒ˜๋ฆฌ AI ์š”์•ฝ ๊ฒฐ๊ณผ ๋ฐ˜ํ™˜ ๊ฒฐ๊ณผ ํ‘œ์‹œ ์š”์•ฝ๋ฌธ ์ถœ๋ ฅ ์‹ค์‹œ๊ฐ„ ๋‰ด์Šค ์š”์•ฝ ์„œ๋น„์Šค ๊ตฌ์กฐ

๐ŸŽ“ ๋ชจ๋ธ ํŒŒ์ธํŠœ๋‹: ๋‚˜๋งŒ์˜ ์š”์•ฝ ๋ชจ๋ธ ๋งŒ๋“ค๊ธฐ

์‚ฌ์ „ํ•™์Šต๋œ ๋ชจ๋ธ๋„ ์ข‹์ง€๋งŒ, ํŠน์ • ๋„๋ฉ”์ธ์ด๋‚˜ ์Šคํƒ€์ผ์— ๋งž์ถ˜ ์ปค์Šคํ…€ ๋ชจ๋ธ์ด ํ•„์š”ํ•  ๋•Œ๊ฐ€ ์žˆ์–ด. ์ด๋Ÿด ๋•Œ๋Š” ํŒŒ์ธํŠœ๋‹์„ ํ•ด์•ผ ํ•ด! ๐ŸŽฏ

์˜ˆ๋ฅผ ๋“ค์–ด, ์˜๋ฃŒ ๋…ผ๋ฌธ ์š”์•ฝ, ๋ฒ•๋ฅ  ๋ฌธ์„œ ์š”์•ฝ, ๋˜๋Š” ํŠน์ • ํšŒ์‚ฌ์˜ ๋ณด๊ณ ์„œ ์Šคํƒ€์ผ์— ๋งž์ถ˜ ์š”์•ฝ์ด ํ•„์š”ํ•˜๋‹ค๋ฉด ํŒŒ์ธํŠœ๋‹์ด ๋‹ต์ด์•ผ.

1. ๋ฐ์ดํ„ฐ์…‹ ์ค€๋น„

ํŒŒ์ธํŠœ๋‹์„ ์œ„ํ•ด์„œ๋Š” (์›๋ณธ ํ…์ŠคํŠธ, ์š”์•ฝ๋ฌธ) ์Œ์œผ๋กœ ์ด๋ฃจ์–ด์ง„ ๋ฐ์ดํ„ฐ๊ฐ€ ํ•„์š”ํ•ด:

import pandas as pd
from datasets import Dataset

# ๋ฐ์ดํ„ฐ ์ค€๋น„ (์˜ˆ์‹œ)
data = {
    'article': [
        "Long article text 1...",
        "Long article text 2...",
        "Long article text 3..."
    ],
    'summary': [
        "Summary 1...",
        "Summary 2...",
        "Summary 3..."
    ]
}

df = pd.DataFrame(data)

# Hugging Face Dataset์œผ๋กœ ๋ณ€ํ™˜
dataset = Dataset.from_pandas(df)

# ํ•™์Šต/๊ฒ€์ฆ ๋ถ„๋ฆฌ
dataset = dataset.train_test_split(test_size=0.1)

print(f"ํ•™์Šต ๋ฐ์ดํ„ฐ: {len(dataset['train'])}๊ฐœ")
print(f"๊ฒ€์ฆ ๋ฐ์ดํ„ฐ: {len(dataset['test'])}๊ฐœ")
๐Ÿ“Š ๋ฐ์ดํ„ฐ์…‹ ์†Œ์Šค ๊ณต๊ฐœ ๋ฐ์ดํ„ฐ์…‹:
โ€ข CNN/DailyMail (๋‰ด์Šค)
โ€ข XSum (BBC ๋‰ด์Šค)
โ€ข arXiv (ํ•™์ˆ  ๋…ผ๋ฌธ)
โ€ข PubMed (์˜๋ฃŒ ๋…ผ๋ฌธ)
โ€ข MultiNews (๋‹ค์ค‘ ๋ฌธ์„œ)

์ง์ ‘ ์ˆ˜์ง‘:
โ€ข ์›น ํฌ๋กค๋ง
โ€ข ์‚ฌ๋‚ด ๋ฌธ์„œ
โ€ข ํฌ๋ผ์šฐ๋“œ์†Œ์‹ฑ

์ตœ์†Œ 1,000๊ฐœ ์ด์ƒ์˜ ์ƒ˜ํ”Œ์ด ์žˆ์œผ๋ฉด ์ข‹์€ ๊ฒฐ๊ณผ๋ฅผ ์–ป์„ ์ˆ˜ ์žˆ์–ด! ๐ŸŽฏ
2. ํ† ํฌ๋‚˜์ด์ €์™€ ์ „์ฒ˜๋ฆฌ
from transformers import AutoTokenizer

# ํ† ํฌ๋‚˜์ด์ € ๋กœ๋“œ
model_name = "facebook/bart-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)

def preprocess_function(examples):
    """๋ฐ์ดํ„ฐ ์ „์ฒ˜๋ฆฌ ํ•จ์ˆ˜"""
    # ์ž…๋ ฅ ํ…์ŠคํŠธ ํ† ํฐํ™”
    inputs = tokenizer(
        examples['article'],
        max_length=1024,
        truncation=True,
        padding='max_length'
    )
    
    # ํƒ€๊ฒŸ ์š”์•ฝ๋ฌธ ํ† ํฐํ™”
    targets = tokenizer(
        examples['summary'],
        max_length=128,
        truncation=True,
        padding='max_length'
    )
    
    inputs['labels'] = targets['input_ids']
    return inputs

# ๋ฐ์ดํ„ฐ์…‹์— ์ „์ฒ˜๋ฆฌ ์ ์šฉ
tokenized_dataset = dataset.map(
    preprocess_function,
    batched=True,
    remove_columns=['article', 'summary']
)

print("์ „์ฒ˜๋ฆฌ ์™„๋ฃŒ!")
3. ๋ชจ๋ธ ํ•™์Šต ์„ค์ •
from transformers import (
    AutoModelForSeq2SeqLM,
    Seq2SeqTrainingArguments,
    Seq2SeqTrainer,
    DataCollatorForSeq2Seq
)

# ๋ชจ๋ธ ๋กœ๋“œ
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

# ํ•™์Šต ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค์ •
training_args = Seq2SeqTrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    num_train_epochs=3,
    weight_decay=0.01,
    save_total_limit=3,
    predict_with_generate=True,
    fp16=True,  # Mixed precision (GPU ํ•„์š”)
    push_to_hub=False,
    logging_dir='./logs',
    logging_steps=100,
    save_steps=500
)

# ๋ฐ์ดํ„ฐ ์ฝœ๋ ˆ์ดํ„ฐ
data_collator = DataCollatorForSeq2Seq(
    tokenizer=tokenizer,
    model=model
)

# Trainer ์ดˆ๊ธฐํ™”
trainer = Seq2SeqTrainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset['train'],
    eval_dataset=tokenized_dataset['test'],
    tokenizer=tokenizer,
    data_collator=data_collator
)

print("ํ•™์Šต ์ค€๋น„ ์™„๋ฃŒ!")
4. ํ•™์Šต ์‹คํ–‰
# ํ•™์Šต ์‹œ์ž‘
print("ํ•™์Šต ์‹œ์ž‘...")
trainer.train()

# ๋ชจ๋ธ ์ €์žฅ
trainer.save_model("./my_summarizer")
tokenizer.save_pretrained("./my_summarizer")

print("ํ•™์Šต ์™„๋ฃŒ! ๋ชจ๋ธ์ด ์ €์žฅ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.")
5. ํŒŒ์ธํŠœ๋‹๋œ ๋ชจ๋ธ ์‚ฌ์šฉ
from transformers import pipeline

# ์ปค์Šคํ…€ ๋ชจ๋ธ ๋กœ๋“œ
my_summarizer = pipeline(
    "summarization",
    model="./my_summarizer",
    tokenizer="./my_summarizer"
)

# ํ…Œ์ŠคํŠธ
text = "Your domain-specific text..."
result = my_summarizer(text, max_length=100, min_length=30)
print(result[0]['summary_text'])
โš ๏ธ ํ•™์Šต ์‹œ ์ฃผ์˜์‚ฌํ•ญ 1. ๋ฉ”๋ชจ๋ฆฌ ๋ถ€์กฑ: ๋ฐฐ์น˜ ํฌ๊ธฐ๋ฅผ ์ค„์ด๊ฑฐ๋‚˜ gradient accumulation ์‚ฌ์šฉ
2. ๊ณผ์ ํ•ฉ: Early stopping๊ณผ ์ •๊ทœํ™” ์ ์šฉ
3. ํ•™์Šต ์‹œ๊ฐ„: GPU ์‚ฌ์šฉ ๊ถŒ์žฅ (CPU๋Š” ๋งค์šฐ ๋А๋ฆผ)
4. ๋ฐ์ดํ„ฐ ํ’ˆ์งˆ: ๋…ธ์ด์ฆˆ ์ œ๊ฑฐ์™€ ์ „์ฒ˜๋ฆฌ๊ฐ€ ์ค‘์š”
5. ํ‰๊ฐ€: ROUGE ์ ์ˆ˜๋กœ ์ •๊ธฐ์ ์œผ๋กœ ์„ฑ๋Šฅ ์ฒดํฌ

ํด๋ผ์šฐ๋“œ GPU (Colab, AWS, Azure)๋ฅผ ํ™œ์šฉํ•˜๋ฉด ๋น„์šฉ ํšจ์œจ์ ์œผ๋กœ ํ•™์Šตํ•  ์ˆ˜ ์žˆ์–ด! โ˜๏ธ
๐Ÿ’ก ํŒŒ์ธํŠœ๋‹ ์ตœ์ ํ™” ํŒ 1. Learning Rate Scheduling: ํ•™์Šต๋ฅ ์„ ์ ์ง„์ ์œผ๋กœ ๊ฐ์†Œ
2. Gradient Accumulation: ์ž‘์€ ๋ฐฐ์น˜๋กœ ํฐ ๋ฐฐ์น˜ ํšจ๊ณผ
3. Mixed Precision: FP16์œผ๋กœ ๋ฉ”๋ชจ๋ฆฌ ์ ˆ์•ฝ
4. LoRA/QLoRA: ํŒŒ๋ผ๋ฏธํ„ฐ ํšจ์œจ์  ํŒŒ์ธํŠœ๋‹
5. Checkpoint Averaging: ์—ฌ๋Ÿฌ ์ฒดํฌํฌ์ธํŠธ ํ‰๊ท 

์ด๋Ÿฐ ๊ณ ๊ธ‰ ๊ธฐ๋ฒ•๋“ค์€ ์„ฑ๋Šฅ์„ ํฌ๊ฒŒ ํ–ฅ์ƒ์‹œํ‚ฌ ์ˆ˜ ์žˆ์–ด! ๐Ÿš€

โšก ์„ฑ๋Šฅ ์ตœ์ ํ™”์™€ ๋ฐฐํฌ ์ „๋žต

๋ชจ๋ธ์„ ๋งŒ๋“ค์—ˆ๋‹ค๋ฉด ์ด์ œ ์‹ค์ œ ์„œ๋น„์Šค์— ๋ฐฐํฌํ•ด์•ผ ํ•ด. ํ•˜์ง€๋งŒ Transformer ๋ชจ๋ธ์€ ํฌ๊ณ  ๋А๋ ค์„œ ์ตœ์ ํ™”๊ฐ€ ํ•„์ˆ˜์•ผ! ๐Ÿ’จ

1. ๋ชจ๋ธ ์–‘์žํ™” (Quantization)

๋ชจ๋ธ ํฌ๊ธฐ๋ฅผ ์ค„์ด๊ณ  ์ถ”๋ก  ์†๋„๋ฅผ ๋†’์ด๋Š” ๊ฐ€์žฅ ํšจ๊ณผ์ ์ธ ๋ฐฉ๋ฒ•์ด์•ผ:

from transformers import pipeline
import torch

# 8bit ์–‘์žํ™” ์‚ฌ์šฉ
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device_map="auto",
    load_in_8bit=True  # 8bit ์–‘์žํ™”
)

text = "Your text here..."
result = summarizer(text, max_length=60)
print(result[0]['summary_text'])
๐Ÿ“Š ์–‘์žํ™” ํšจ๊ณผ FP32 (๊ธฐ๋ณธ):
โ€ข ๋ชจ๋ธ ํฌ๊ธฐ: 100%
โ€ข ์ถ”๋ก  ์†๋„: ๊ธฐ์ค€
โ€ข ์ •ํ™•๋„: 100%

FP16 (Half Precision):
โ€ข ๋ชจ๋ธ ํฌ๊ธฐ: 50%
โ€ข ์ถ”๋ก  ์†๋„: 2๋ฐฐ ๋น ๋ฆ„
โ€ข ์ •ํ™•๋„: 99.9%

INT8 (8bit):
โ€ข ๋ชจ๋ธ ํฌ๊ธฐ: 25%
โ€ข ์ถ”๋ก  ์†๋„: 3-4๋ฐฐ ๋น ๋ฆ„
โ€ข ์ •ํ™•๋„: 98-99%

๋Œ€๋ถ€๋ถ„์˜ ๊ฒฝ์šฐ INT8๋กœ๋„ ์ถฉ๋ถ„ํ•œ ์„ฑ๋Šฅ์„ ์œ ์ง€ํ•ด! ๐ŸŽฏ
2. ONNX ๋ณ€ํ™˜

ONNX๋Š” ๋‹ค์–‘ํ•œ ํ”Œ๋žซํผ์—์„œ ๋น ๋ฅด๊ฒŒ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋Š” ํ‘œ์ค€ ํฌ๋งท์ด์•ผ:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from optimum.onnxruntime import ORTModelForSeq2SeqLM

# ์›๋ณธ ๋ชจ๋ธ ๋กœ๋“œ
model_name = "facebook/bart-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# ONNX๋กœ ๋ณ€ํ™˜ ๋ฐ ์ €์žฅ
ort_model = ORTModelForSeq2SeqLM.from_pretrained(
    model_name,
    export=True
)
ort_model.save_pretrained("./bart_onnx")
tokenizer.save_pretrained("./bart_onnx")

# ONNX ๋ชจ๋ธ ์‚ฌ์šฉ
from transformers import pipeline

summarizer = pipeline(
    "summarization",
    model=ort_model,
    tokenizer=tokenizer
)

# ์ถ”๋ก  (๋” ๋น ๋ฆ„!)
text = "Your text..."
result = summarizer(text)
print(result[0]['summary_text'])
3. ๋ชจ๋ธ ์ฆ๋ฅ˜ (Distillation)

ํฐ ๋ชจ๋ธ์˜ ์ง€์‹์„ ์ž‘์€ ๋ชจ๋ธ๋กœ ์ „๋‹ฌํ•˜๋Š” ๊ธฐ๋ฒ•์ด์•ผ:

from transformers import pipeline

# ์ฆ๋ฅ˜๋œ ๋ชจ๋ธ ์‚ฌ์šฉ (ํ›จ์”ฌ ์ž‘๊ณ  ๋น ๋ฆ„)
summarizer = pipeline(
    "summarization",
    model="sshleifer/distilbart-cnn-12-6"  # BART์˜ ์ฆ๋ฅ˜ ๋ฒ„์ „
)

text = "Your text..."
result = summarizer(text, max_length=60)
print(result[0]['summary_text'])

# ์„ฑ๋Šฅ ๋น„๊ต
# bart-large-cnn: 400M ํŒŒ๋ผ๋ฏธํ„ฐ, ๋А๋ฆผ
# distilbart: 300M ํŒŒ๋ผ๋ฏธํ„ฐ, 40% ๋น ๋ฆ„, ์„ฑ๋Šฅ 95% ์œ ์ง€
4. ์บ์‹ฑ ์ „๋žต

๊ฐ™์€ ํ…์ŠคํŠธ๋ฅผ ๋ฐ˜๋ณต ์š”์•ฝํ•˜์ง€ ์•Š๋„๋ก ์บ์‹ฑ์„ ๊ตฌํ˜„ํ•˜์ž:

import hashlib
import json
from functools import lru_cache

class CachedSummarizer:
    def __init__(self, model_name="facebook/bart-large-cnn"):
        from transformers import pipeline
        self.summarizer = pipeline("summarization", model=model_name)
        self.cache = {}
    
    def _get_cache_key(self, text, max_length, min_length):
        """ํ…์ŠคํŠธ ํ•ด์‹œ๋กœ ์บ์‹œ ํ‚ค ์ƒ์„ฑ"""
        content = f"{text}_{max_length}_{min_length}"
        return hashlib.md5(content.encode()).hexdigest()
    
    def summarize(self, text, max_length=130, min_length=30):
        """์บ์‹œ๋ฅผ ํ™œ์šฉํ•œ ์š”์•ฝ"""
        cache_key = self._get_cache_key(text, max_length, min_length)
        
        # ์บ์‹œ ํ™•์ธ
        if cache_key in self.cache:
            print("์บ์‹œ์—์„œ ๊ฐ€์ ธ์˜ด!")
            return self.cache[cache_key]
        
        # ์ƒˆ๋กœ ์š”์•ฝ
        print("์ƒˆ๋กœ ์š”์•ฝ ์ค‘...")
        result = self.summarizer(
            text,
            max_length=max_length,
            min_length=min_length
        )
        summary = result[0]['summary_text']
        
        # ์บ์‹œ ์ €์žฅ
        self.cache[cache_key] = summary
        return summary

# ์‚ฌ์šฉ
summarizer = CachedSummarizer()

text = "Same text..."
summary1 = summarizer.summarize(text)  # ์ƒˆ๋กœ ์š”์•ฝ
summary2 = summarizer.summarize(text)  # ์บ์‹œ์—์„œ ๊ฐ€์ ธ์˜ด (๋น ๋ฆ„!)
5. ๋น„๋™๊ธฐ ์ฒ˜๋ฆฌ

์—ฌ๋Ÿฌ ์š”์ฒญ์„ ๋™์‹œ์— ์ฒ˜๋ฆฌํ•˜๋ ค๋ฉด ๋น„๋™๊ธฐ ๋ฐฉ์‹์ด ํ•„์š”ํ•ด:

import asyncio
from concurrent.futures import ThreadPoolExecutor
from transformers import pipeline

class AsyncSummarizer:
    def __init__(self, model_name="facebook/bart-large-cnn"):
        self.summarizer = pipeline("summarization", model=model_name)
        self.executor = ThreadPoolExecutor(max_workers=4)
    
    async def summarize_async(self, text, max_length=130, min_length=30):
        """๋น„๋™๊ธฐ ์š”์•ฝ"""
        loop = asyncio.get_event_loop()
        result = await loop.run_in_executor(
            self.executor,
            lambda: self.summarizer(
                text,
                max_length=max_length,
                min_length=min_length
            )
        )
        return result[0]['summary_text']
    
    async def summarize_batch_async(self, texts):
        """์—ฌ๋Ÿฌ ํ…์ŠคํŠธ ๋™์‹œ ์ฒ˜๋ฆฌ"""
        tasks = [self.summarize_async(text) for text in texts]
        return await asyncio.gather(*tasks)

# ์‚ฌ์šฉ
async def main():
    summarizer = AsyncSummarizer()
    
    texts = [
        "Text 1...",
        "Text 2...",
        "Text 3..."
    ]
    
    # ๋™์‹œ์— ์ฒ˜๋ฆฌ
    summaries = await summarizer.summarize_batch_async(texts)
    
    for i, summary in enumerate(summaries):
        print(f"Summary {i+1}: {summary}")

# ์‹คํ–‰
asyncio.run(main())
๐Ÿš€ ๋ฐฐํฌ ํ”Œ๋žซํผ ์„ ํƒ ๊ฐ€์ด๋“œ Hugging Face Inference API:
โ€ข ์žฅ์ : ์„ค์ • ๋ถˆํ•„์š”, ์ž๋™ ์Šค์ผ€์ผ๋ง
โ€ข ๋‹จ์ : ๋น„์šฉ, API ์ œํ•œ
โ€ข ์ถ”์ฒœ: ํ”„๋กœํ† ํƒ€์ž…, ์†Œ๊ทœ๋ชจ ์„œ๋น„์Šค

AWS SageMaker:
โ€ข ์žฅ์ : ๊ฐ•๋ ฅํ•œ ์ธํ”„๋ผ, ๋ชจ๋‹ˆํ„ฐ๋ง
โ€ข ๋‹จ์ : ๋ณต์žกํ•œ ์„ค์ •, ๋น„์šฉ
โ€ข ์ถ”์ฒœ: ๋Œ€๊ทœ๋ชจ ์—”ํ„ฐํ”„๋ผ์ด์ฆˆ

Docker + Kubernetes:
โ€ข ์žฅ์ : ์œ ์—ฐ์„ฑ, ํ™•์žฅ์„ฑ
โ€ข ๋‹จ์ : ๊ด€๋ฆฌ ๋ณต์žก๋„
โ€ข ์ถ”์ฒœ: ์ค‘๋Œ€๊ทœ๋ชจ ์„œ๋น„์Šค

Streamlit/Gradio:
โ€ข ์žฅ์ : ๋น ๋ฅธ ํ”„๋กœํ† ํƒ€์ž…
โ€ข ๋‹จ์ : ์ œํ•œ์  ์ปค์Šคํ„ฐ๋งˆ์ด์ง•
โ€ข ์ถ”์ฒœ: ๋ฐ๋ชจ, ๋‚ด๋ถ€ ๋„๊ตฌ
๐Ÿ’ฐ ๋น„์šฉ ์ตœ์ ํ™” ์ „๋žต 1. ๋ชจ๋ธ ํฌ๊ธฐ ์ค„์ด๊ธฐ: Distillation, Pruning
2. ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ: ์—ฌ๋Ÿฌ ์š”์ฒญ ๋ฌถ์–ด์„œ ์ฒ˜๋ฆฌ
3. ์บ์‹ฑ: ์ค‘๋ณต ์š”์ฒญ ๋ฐฉ์ง€
4. ์˜คํ† ์Šค์ผ€์ผ๋ง: ํŠธ๋ž˜ํ”ฝ์— ๋”ฐ๋ผ ์ž๋™ ์กฐ์ ˆ
5. Spot ์ธ์Šคํ„ด์Šค: AWS/GCP ์ €๋ ดํ•œ ์ธ์Šคํ„ด์Šค ํ™œ์šฉ
6. Edge ๋ฐฐํฌ: ํด๋ผ์ด์–ธํŠธ ์ธก์—์„œ ์‹คํ–‰ (๋ชจ๋ฐ”์ผ ์•ฑ ๋“ฑ)

์ž‘์€ ๋ชจ๋ธ(distilbart)๋กœ ์‹œ์ž‘ํ•ด์„œ ํ•„์š”์‹œ ํฐ ๋ชจ๋ธ๋กœ ์—…๊ทธ๋ ˆ์ด๋“œํ•˜๋Š” ์ „๋žต์ด ์ข‹์•„! ๐Ÿ“ˆ

๐Ÿ› ๋ฌธ์ œ ํ•ด๊ฒฐ๊ณผ ๋””๋ฒ„๊น…

์‹ค์ „์—์„œ๋Š” ํ•ญ์ƒ ์˜ˆ์ƒ์น˜ ๋ชปํ•œ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ด. ์ž์ฃผ ๋งŒ๋‚˜๋Š” ์ด์Šˆ๋“ค๊ณผ ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•์„ ์•Œ์•„๋ณด์ž! ๐Ÿ”ง

๋ฌธ์ œ 1: ๋ฉ”๋ชจ๋ฆฌ ๋ถ€์กฑ (Out of Memory)
์ฆ์ƒ: "CUDA out of memory" ๋˜๋Š” "RuntimeError: out of memory"

์›์ธ: ๋ชจ๋ธ์ด๋‚˜ ๋ฐฐ์น˜ ํฌ๊ธฐ๊ฐ€ GPU/RAM ์šฉ๋Ÿ‰์„ ์ดˆ๊ณผ
# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 1: ๋ฐฐ์น˜ ํฌ๊ธฐ ์ค„์ด๊ธฐ
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device=0
)

# ํ•œ ๋ฒˆ์— ํ•˜๋‚˜์”ฉ ์ฒ˜๋ฆฌ
for text in texts:
    result = summarizer(text, max_length=60)
    print(result[0]['summary_text'])

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 2: ๋” ์ž‘์€ ๋ชจ๋ธ ์‚ฌ์šฉ
summarizer = pipeline(
    "summarization",
    model="facebook/bart-base"  # large ๋Œ€์‹  base
)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 3: 8bit ์–‘์žํ™”
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device_map="auto",
    load_in_8bit=True
)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 4: CPU ์‚ฌ์šฉ
summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device=-1  # CPU ์‚ฌ์šฉ
)
๋ฌธ์ œ 2: ์š”์•ฝ ํ’ˆ์งˆ์ด ๋‚ฎ์Œ
์ฆ์ƒ: ์š”์•ฝ์ด ์ด์ƒํ•˜๊ฑฐ๋‚˜ ์›๋ณธ๊ณผ ๊ด€๋ จ ์—†๋Š” ๋‚ด์šฉ

์›์ธ: ๋ถ€์ ์ ˆํ•œ ํŒŒ๋ผ๋ฏธํ„ฐ, ๋ชจ๋ธ ์„ ํƒ ์˜ค๋ฅ˜, ๋ฐ์ดํ„ฐ ํ’ˆ์งˆ ๋ฌธ์ œ
# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 1: ํŒŒ๋ผ๋ฏธํ„ฐ ์กฐ์ •
result = summarizer(
    text,
    max_length=130,  # ๋„ˆ๋ฌด ์งง์ง€ ์•Š๊ฒŒ
    min_length=30,   # ์ตœ์†Œ ๊ธธ์ด ๋ณด์žฅ
    do_sample=False,  # Greedy decoding
    num_beams=4,     # Beam search ์‚ฌ์šฉ
    length_penalty=2.0,  # ๊ธด ๋ฌธ์žฅ ์„ ํ˜ธ
    no_repeat_ngram_size=3,  # ๋ฐ˜๋ณต ๋ฐฉ์ง€
    early_stopping=True
)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 2: ๋‹ค๋ฅธ ๋ชจ๋ธ ์‹œ๋„
models_to_try = [
    "facebook/bart-large-cnn",
    "google/pegasus-xsum",
    "t5-base",
    "sshleifer/distilbart-cnn-12-6"
]

for model_name in models_to_try:
    summarizer = pipeline("summarization", model=model_name)
    result = summarizer(text, max_length=60)
    print(f"{model_name}: {result[0]['summary_text']}\n")

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 3: ํ…์ŠคํŠธ ์ „์ฒ˜๋ฆฌ
def preprocess_text(text):
    """ํ…์ŠคํŠธ ์ •์ œ"""
    # ๋ถˆํ•„์š”ํ•œ ๊ณต๋ฐฑ ์ œ๊ฑฐ
    text = " ".join(text.split())
    # ํŠน์ˆ˜๋ฌธ์ž ์ •๋ฆฌ
    text = text.replace('\n', ' ')
    # ๋„ˆ๋ฌด ์งง์€ ํ…์ŠคํŠธ ํ•„ํ„ฐ๋ง
    if len(text.split()) < 50:
        return None
    return text

cleaned_text = preprocess_text(text)
if cleaned_text:
    result = summarizer(cleaned_text)
    print(result[0]['summary_text'])
๋ฌธ์ œ 3: ์ถ”๋ก  ์†๋„๊ฐ€ ๋„ˆ๋ฌด ๋А๋ฆผ
์ฆ์ƒ: ํ•œ ๋ฒˆ ์š”์•ฝํ•˜๋Š”๋ฐ 10์ดˆ ์ด์ƒ ์†Œ์š”

์›์ธ: CPU ์‚ฌ์šฉ, ํฐ ๋ชจ๋ธ, ์ตœ์ ํ™” ๋ถ€์กฑ
# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 1: GPU ์‚ฌ์šฉ
import torch
device = 0 if torch.cuda.is_available() else -1
summarizer = pipeline("summarization", device=device)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 2: ๋ชจ๋ธ ์ตœ์ ํ™”
from optimum.onnxruntime import ORTModelForSeq2SeqLM

ort_model = ORTModelForSeq2SeqLM.from_pretrained(
    "facebook/bart-base",
    export=True
)
summarizer = pipeline("summarization", model=ort_model)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 3: ๋ฐฐ์น˜ ์ฒ˜๋ฆฌ
texts = ["text1", "text2", "text3"]
results = summarizer(texts, batch_size=4)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 4: ๋” ์ž‘์€ ๋ชจ๋ธ
summarizer = pipeline(
    "summarization",
    model="sshleifer/distilbart-cnn-6-6"  # ๊ฐ€์žฅ ์ž‘์€ ๋ฒ„์ „
)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 5: ํ…์ŠคํŠธ ๊ธธ์ด ์ œํ•œ
def truncate_text(text, max_words=500):
    words = text.split()
    if len(words) > max_words:
        return " ".join(words[:max_words])
    return text

short_text = truncate_text(long_text)
result = summarizer(short_text)
๋ฌธ์ œ 4: ํ•œ๊ตญ์–ด ์š”์•ฝ์ด ์•ˆ ๋จ
์ฆ์ƒ: ํ•œ๊ตญ์–ด ํ…์ŠคํŠธ ์ž…๋ ฅ ์‹œ ์ด์ƒํ•œ ๊ฒฐ๊ณผ ๋˜๋Š” ์—๋Ÿฌ

์›์ธ: ์˜์–ด ์ „์šฉ ๋ชจ๋ธ ์‚ฌ์šฉ
# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 1: ๋‹ค๊ตญ์–ด ๋ชจ๋ธ ์‚ฌ์šฉ
from transformers import pipeline

summarizer = pipeline(
    "summarization",
    model="google/mt5-base"  # ๋‹ค๊ตญ์–ด ์ง€์›
)

korean_text = """
ํ•œ๊ตญ์–ด๋กœ ์ž‘์„ฑ๋œ ๊ธด ํ…์ŠคํŠธ...
"""

result = summarizer(korean_text, max_length=100)
print(result[0]['summary_text'])

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 2: ํ•œ๊ตญ์–ด ํŠนํ™” ๋ชจ๋ธ
summarizer = pipeline(
    "summarization",
    model="eenzeenee/t5-base-korean-summarization"
)

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 3: ๋ฒˆ์—ญ ํ›„ ์š”์•ฝ (์šฐํšŒ ๋ฐฉ๋ฒ•)
from transformers import pipeline

translator = pipeline("translation", model="Helsinki-NLP/opus-mt-ko-en")
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
back_translator = pipeline("translation", model="Helsinki-NLP/opus-mt-en-ko")

# ํ•œ๊ตญ์–ด -> ์˜์–ด -> ์š”์•ฝ -> ํ•œ๊ตญ์–ด
korean_text = "ํ•œ๊ตญ์–ด ํ…์ŠคํŠธ..."
english_text = translator(korean_text)[0]['translation_text']
summary_english = summarizer(english_text)[0]['summary_text']
summary_korean = back_translator(summary_english)[0]['translation_text']

print(summary_korean)
๋ฌธ์ œ 5: ๋ชจ๋ธ ๋‹ค์šด๋กœ๋“œ ์‹คํŒจ
์ฆ์ƒ: "Connection error" ๋˜๋Š” ๋‹ค์šด๋กœ๋“œ ์ค‘๋‹จ

์›์ธ: ๋„คํŠธ์›Œํฌ ๋ฌธ์ œ, Hugging Face Hub ์ ‘๊ทผ ์ œํ•œ
# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 1: ์žฌ์‹œ๋„ ๋กœ์ง
from transformers import pipeline
import time

def load_model_with_retry(model_name, max_retries=3):
    for i in range(max_retries):
        try:
            summarizer = pipeline("summarization", model=model_name)
            return summarizer
        except Exception as e:
            print(f"์‹œ๋„ {i+1} ์‹คํŒจ: {e}")
            if i < max_retries - 1:
                time.sleep(5)
            else:
                raise

summarizer = load_model_with_retry("facebook/bart-large-cnn")

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 2: ๋กœ์ปฌ ์บ์‹œ ์‚ฌ์šฉ
from transformers import pipeline
import os

# ์บ์‹œ ๋””๋ ‰ํ† ๋ฆฌ ์„ค์ •
os.environ['TRANSFORMERS_CACHE'] = './model_cache'

summarizer = pipeline("summarization", model="facebook/bart-large-cnn")

# ํ•ด๊ฒฐ ๋ฐฉ๋ฒ• 3: ์ˆ˜๋™ ๋‹ค์šด๋กœ๋“œ
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "facebook/bart-large-cnn"

# ์ˆ˜๋™์œผ๋กœ ๋‹ค์šด๋กœ๋“œ ๋ฐ ์ €์žฅ
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

tokenizer.save_pretrained("./local_model")
model.save_pretrained("./local_model")

# ๋กœ์ปฌ์—์„œ ๋กœ๋“œ
summarizer = pipeline("summarization", model="./local_model")
๐Ÿ” ๋””๋ฒ„๊น… ์ฒดํฌ๋ฆฌ์ŠคํŠธ โœ… Python ๋ฒ„์ „ ํ™•์ธ (3.7 ์ด์ƒ)
โœ… Transformers ๋ฒ„์ „ ํ™•์ธ (์ตœ์‹  ๋ฒ„์ „ ๊ถŒ์žฅ)
โœ… PyTorch/TensorFlow ์„ค์น˜ ํ™•์ธ
โœ… GPU ๋“œ๋ผ์ด๋ฒ„ ๋ฐ CUDA ๋ฒ„์ „ ํ™•์ธ
โœ… ์ถฉ๋ถ„ํ•œ ๋””์Šคํฌ ๊ณต๊ฐ„ (๋ชจ๋ธ ์บ์‹œ์šฉ)
โœ… ๋„คํŠธ์›Œํฌ ์—ฐ๊ฒฐ ์ƒํƒœ
โœ… ์ž…๋ ฅ ํ…์ŠคํŠธ ์ธ์ฝ”๋”ฉ (UTF-8)
โœ… ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰ ๋ชจ๋‹ˆํ„ฐ๋ง

๋ฌธ์ œ๊ฐ€ ๊ณ„์†๋˜๋ฉด GitHub Issues๋‚˜ Hugging Face ํฌ๋Ÿผ์—์„œ ๋„์›€์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์–ด! ๐Ÿค

๐ŸŒŸ ์‹ค์ „ ํ™œ์šฉ ์‚ฌ๋ก€์™€ ํŒ

์ด๋ก ๊ณผ ๊ธฐ์ˆ ์„ ๋ฐฐ์› ์œผ๋‹ˆ, ์ด์ œ ์‹ค์ œ๋กœ ์–ด๋–ป๊ฒŒ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ๋Š”์ง€ ๊ตฌ์ฒด์ ์ธ ์‚ฌ๋ก€๋ฅผ ์‚ดํŽด๋ณด์ž! ๐Ÿ’ผ

์‚ฌ๋ก€ 1: ๋‰ด์Šค ํ๋ ˆ์ด์…˜ ์„œ๋น„์Šค
from transformers import pipeline
import feedparser
from datetime import datetime

class NewsCurator:
    def __init__(self):
        self.summarizer = pipeline(
            "summarization",
            model="facebook/bart-large-cnn"
        )
    
    def fetch_rss_feeds(self, rss_urls):
        """RSS ํ”ผ๋“œ์—์„œ ๋‰ด์Šค ๊ฐ€์ ธ์˜ค๊ธฐ"""
        articles = []
        for url in rss_urls:
            feed = feedparser.parse(url)
            for entry in feed.entries[:10]:  # ์ตœ์‹  10๊ฐœ
                articles.append({
                    'title': entry.title,
                    'content': entry.summary,
                    'link': entry.link,
                    'published': entry.published
                })
        return articles
    
    def create_digest(self, articles):
        """๋‰ด์Šค ๋‹ค์ด์ œ์ŠคํŠธ ์ƒ์„ฑ"""
        digest = []
        for article in articles:
            try:
                summary = self.summarizer(
                    article['content'],
                    max_length=60,
                    min_length=20
                )[0]['summary_text']
                
                digest.append({
                    'title': article['title'],
                    'summary': summary,
                    'link': article['link']
                })
            except:
                continue
        
        return digest
    
    def format_digest(self, digest):
        """์ฝ๊ธฐ ์ข‹์€ ํ˜•์‹์œผ๋กœ ํฌ๋งท"""
        output = f"๐Ÿ“ฐ ์˜ค๋Š˜์˜ ๋‰ด์Šค ๋‹ค์ด์ œ์ŠคํŠธ ({datetime.now().strftime('%Y-%m-%d')})\n\n"
        
        for i, item in enumerate(digest, 1):
            output += f"{i}. {item['title']}\n"
            output += f"   {item['summary']}\n"
            output += f"   ๐Ÿ”— {item['link']}\n\n"
        
        return output

# ์‚ฌ์šฉ
curator = NewsCurator()

rss_feeds = [
    "http://rss.cnn.com/rss/cnn_topstories.rss",
    "http://feeds.bbci.co.uk/news/rss.xml"
]

articles = curator.fetch_rss_feeds(rss_feeds)
digest = curator.create_digest(articles)
formatted = curator.format_digest(digest)

print(formatted)
์‚ฌ๋ก€ 2: ํšŒ์˜๋ก ์ž๋™ ์š”์•ฝ
from transformers import pipeline
import re

class MeetingMinutesSummarizer:
    def __init__(self):
        self.summarizer = pipeline(
            "summarization",
            model="facebook/bart-large-cnn"
        )
    
    def extract_sections(self, transcript):
        """ํšŒ์˜๋ก์„ ์„น์…˜๋ณ„๋กœ ๋ถ„๋ฆฌ"""
        sections = {
            'agenda': [],
            'discussion': [],
            'decisions': [],
            'action_items': []
        }
        
        # ํ‚ค์›Œ๋“œ ๊ธฐ๋ฐ˜ ์„น์…˜ ๋ถ„๋ฆฌ
        current_section = 'discussion'
        
        for line in transcript.split('\n'):
            line = line.strip()
            if not line:
                continue
            
            if 'agenda' in line.lower():
                current_section = 'agenda'
            elif 'decision' in line.lower():
                current_section = 'decisions'
            elif 'action' in line.lower() or 'todo' in line.lower():
                current_section = 'action_items'
            else:
                sections[current_section].append(line)
        
        return sections
    
    def summarize_meeting(self, transcript):
        """ํšŒ์˜ ์ „์ฒด ์š”์•ฝ"""
        sections = self.extract_sections(transcript)
        
        summary = "๐Ÿ“‹ ํšŒ์˜ ์š”์•ฝ\n\n"
        
        # ์ „์ฒด ๋…ผ์˜ ์š”์•ฝ
        if sections['discussion']:
            discussion_text = " ".join(sections['discussion'])
            if len(discussion_text.split()) > 50:
                discussion_summary = self.summarizer(
                    discussion_text,
                    max_length=100,
                    min_length=30
                )[0]['summary_text']
                summary += f"๐Ÿ’ฌ ์ฃผ์š” ๋…ผ์˜:\n{discussion_summary}\n\n"
        
        # ๊ฒฐ์ •์‚ฌํ•ญ
        if sections['decisions']:
            summary += "โœ… ๊ฒฐ์ •์‚ฌํ•ญ:\n"
            for decision in sections['decisions']:
                summary += f"  โ€ข {decision}\n"
            summary += "\n"
        
        # ์•ก์…˜ ์•„์ดํ…œ
        if sections['action_items']:
            summary += "๐Ÿ“Œ ์•ก์…˜ ์•„์ดํ…œ:\n"
            for item in sections['action_items']:
                summary += f"  โ€ข {item}\n"
        
        return summary

# ์‚ฌ์šฉ
summarizer = MeetingMinutesSummarizer()

transcript = """
Meeting Transcript - Product Planning
Agenda: Discuss Q4 product roadmap

Discussion:
John: We need to prioritize the mobile app development.
Sarah: I agree, but we should also focus on improving the API.
Mike: The customer feedback suggests we need better analytics.
John: Let's allocate more resources to the mobile team.

Decisions:
- Increase mobile team by 2 developers
- Launch new analytics dashboard by end of Q4

Action Items:
- John: Hire 2 mobile developers by next month
- Sarah: Prepare API improvement proposal
- Mike: Design analytics dashboard mockups
"""

summary = summarizer.summarize_meeting(transcript)
print(summary)
์‚ฌ๋ก€ 3: ํ•™์ˆ  ๋…ผ๋ฌธ ์š”์•ฝ
from transformers import pipeline
import arxiv

class PaperSummarizer:
    def __init__(self):
        self.summarizer = pipeline(
            "summarization",
            model="allenai/led-base-16384"  # ๊ธด ๋ฌธ์„œ์šฉ
        )
    
    def fetch_paper(self, arxiv_id):
        """arXiv์—์„œ ๋…ผ๋ฌธ ๊ฐ€์ ธ์˜ค๊ธฐ"""
        search = arxiv.Search(id_list=[arxiv_id])
        paper = next(search.results())
        
        return {
            'title': paper.title,
            'authors': [author.name for author in paper.authors],
            'abstract': paper.summary,
            'pdf_url': paper.pdf_url
        }
    
    def summarize_abstract(self, abstract):
        """์ดˆ๋ก ์š”์•ฝ"""
        if len(abstract.split()) < 100:
            return abstract
        
        summary = self.summarizer(
            abstract,
            max_length=80,
            min_length=30
        )[0]['summary_text']
        
        return summary
    
    def create_paper_summary(self, arxiv_id):
        """๋…ผ๋ฌธ ์š”์•ฝ ์ƒ์„ฑ"""
        paper = self.fetch_paper(arxiv_id)
        summary = self.summarize_abstract(paper['abstract'])
        
        output = f"๐Ÿ“„ {paper['title']}\n\n"
        output += f"๐Ÿ‘ฅ ์ €์ž: {', '.join(paper['authors'][:3])}"
        if len(paper['authors']) > 3:
            output += f" ์™ธ {len(paper['authors'])-3}๋ช…"
        output += f"\n\n๐Ÿ“ ์š”์•ฝ:\n{summary}\n\n"
        output += f"๐Ÿ”— PDF: {paper['pdf_url']}"
        
        return output

# ์‚ฌ์šฉ
summarizer = PaperSummarizer()

# Attention Is All You Need ๋…ผ๋ฌธ
paper_summary = summarizer.create_paper_summary("1706.03762")
print(paper_summary)
๐Ÿ’ก ๋„๋ฉ”์ธ๋ณ„ ์ตœ์ ํ™” ํŒ ๋‰ด์Šค: facebook/bart-large-cnn, ์งง๊ณ  ๊ฐ„๊ฒฐํ•œ ์š”์•ฝ
ํ•™์ˆ : allenai/led-base-16384, ๊ธฐ์ˆ  ์šฉ์–ด ๋ณด์กด
๋ฒ•๋ฅ : ์ปค์Šคํ…€ ํŒŒ์ธํŠœ๋‹, ์ •ํ™•์„ฑ ์ตœ์šฐ์„ 
์˜๋ฃŒ: BioBERT ๊ธฐ๋ฐ˜ ๋ชจ๋ธ, ์ „๋ฌธ ์šฉ์–ด ์ฒ˜๋ฆฌ
๋น„์ฆˆ๋‹ˆ์Šค: ์•ก์…˜ ์•„์ดํ…œ ์ถ”์ถœ, ๊ตฌ์กฐํ™”๋œ ์š”์•ฝ
SNS: ๋น ๋ฅธ ์ถ”๋ก , ์งง์€ ์š”์•ฝ, ๊ฐ์„ฑ ๋ณด์กด

๊ฐ ๋„๋ฉ”์ธ๋งˆ๋‹ค ํŠน์„ฑ์ด ๋‹ค๋ฅด๋‹ˆ ์ ์ ˆํ•œ ๋ชจ๋ธ๊ณผ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์„ ํƒํ•˜๋Š” ๊ฒŒ ์ค‘์š”ํ•ด! ๐ŸŽฏ
์‚ฌ๋ก€ 4: ๊ณ ๊ฐ ๋ฆฌ๋ทฐ ์š”์•ฝ
from transformers import pipeline
from collections import Counter
import re

class ReviewSummarizer:
    def __init__(self):
        self.summarizer = pipeline(
            "summarization",
            model="facebook/bart-large-cnn"
        )
    
    def aggregate_reviews(self, reviews):
        """์—ฌ๋Ÿฌ ๋ฆฌ๋ทฐ๋ฅผ ํ•˜๋‚˜์˜ ํ…์ŠคํŠธ๋กœ ํ•ฉ์น˜๊ธฐ"""
        # ๊ธ์ •/๋ถ€์ • ๋ถ„๋ฆฌ
        positive = []
        negative = []
        
        for review in reviews:
            if review['rating'] >= 4:
                positive.append(review['text'])
            else:
                negative.append(review['text'])
        
        return {
            'positive': " ".join(positive),
            'negative': " ".join(negative)
        }
    
    def extract_keywords(self, text):
        """์ฃผ์š” ํ‚ค์›Œ๋“œ ์ถ”์ถœ"""
        words = re.findall(r'\b\w+\b', text.lower())
        # ๋ถˆ์šฉ์–ด ์ œ๊ฑฐ (๊ฐ„๋‹จํ•œ ์˜ˆ์‹œ)
        stopwords = {'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are'}
        words = [w for w in words if w not in stopwords and len(w) > 3]
        
        return Counter(words).most_common(10)
    
    def create_review_summary(self, reviews):
        """๋ฆฌ๋ทฐ ์ข…ํ•ฉ ์š”์•ฝ"""
        aggregated = self.aggregate_reviews(reviews)
        
        summary = "โญ ๊ณ ๊ฐ ๋ฆฌ๋ทฐ ์š”์•ฝ\n\n"
        
        # ๊ธ์ •์  ๋ฆฌ๋ทฐ ์š”์•ฝ
        if aggregated['positive']:
            pos_summary = self.summarizer(
                aggregated['positive'][:1000],  # ๊ธธ์ด ์ œํ•œ
                max_length=80,
                min_length=20
            )[0]['summary_text']
            summary += f"๐Ÿ‘ ๊ธ์ •์  ์˜๊ฒฌ:\n{pos_summary}\n\n"
        
        # ๋ถ€์ •์  ๋ฆฌ๋ทฐ ์š”์•ฝ
        if aggregated['negative']:
            neg_summary = self.summarizer(
                aggregated['negative'][:1000],
                max_length=80,
                min_length=20
            )[0]['summary_text']
            summary += f"๐Ÿ‘Ž ๊ฐœ์„  ํ•„์š” ์‚ฌํ•ญ:\n{neg_summary}\n\n"
        
        # ์ฃผ์š” ํ‚ค์›Œ๋“œ
        all_text = aggregated['positive'] + " " + aggregated['negative']
        keywords = self.extract_keywords(all_text)
        summary += "๐Ÿ”‘ ์ฃผ์š” ํ‚ค์›Œ๋“œ:\n"
        summary += ", ".join([f"{word}({count})" for word, count in keywords[:5]])
        
        return summary

# ์‚ฌ์šฉ
summarizer = ReviewSummarizer()

reviews = [
    {'rating': 5, 'text': 'Great product! Fast delivery and excellent quality.'},
    {'rating': 4, 'text': 'Good value for money. Works as expected.'},
    {'rating': 2, 'text': 'Disappointed with the build quality. Too expensive.'},
    {'rating': 5, 'text': 'Amazing! Exceeded my expectations.'},
    {'rating': 3, 'text': 'Average product. Nothing special but does the job.'}
]

summary = summarizer.create_review_summary(reviews)
print(summary)

์ด๋Ÿฐ ์‹ค์ „ ์˜ˆ์ œ๋“ค์„ ํฌํŠธํด๋ฆฌ์˜ค์— ์ถ”๊ฐ€ํ•˜๋ฉด, ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ AI ๊ฐœ๋ฐœ ์žฌ๋Šฅ์„ ํŒ๋งคํ•  ๋•Œ ํฐ ๋„์›€์ด ๋  ๊ฑฐ์•ผ! ์‹ค์ œ ๋น„์ฆˆ๋‹ˆ์Šค ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๋Š” ๋Šฅ๋ ฅ์„ ๋ณด์—ฌ์ค„ ์ˆ˜ ์žˆ๊ฑฐ๋“ . ๐Ÿš€

๐ŸŽฏ ๋งˆ๋ฌด๋ฆฌํ•˜๋ฉฐ

์™€! ์—ฌ๊ธฐ๊นŒ์ง€ ์ •๋ง ๊ธด ์—ฌ์ •์ด์—ˆ์–ด. Transformers Pipeline์„ ํ™œ์šฉํ•œ ํ…์ŠคํŠธ ์š”์•ฝ์— ๋Œ€ํ•ด ์ •๋ง ๋งŽ์€ ๊ฑธ ๋ฐฐ์› ์ง€? ๐Ÿ˜Š

๊ฐ„๋‹จํ•œ Pipeline ์‚ฌ์šฉ๋ฒ•๋ถ€ํ„ฐ ์‹œ์ž‘ํ•ด์„œ, ๋‹ค์–‘ํ•œ ๋ชจ๋ธ ๋น„๊ต, ๊ณ ๊ธ‰ ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค์ •, ์‹ค์ „ ํ”„๋กœ์ ํŠธ ๊ตฌํ˜„, ์ตœ์ ํ™” ๊ธฐ๋ฒ•, ๊ทธ๋ฆฌ๊ณ  ์‹ค์ œ ๋น„์ฆˆ๋‹ˆ์Šค ํ™œ์šฉ ์‚ฌ๋ก€๊นŒ์ง€ ๋‹ค๋ค˜์–ด. ์ด ์ •๋„๋ฉด ์‹ค๋ฌด์—์„œ ๋ฐ”๋กœ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ์ˆ˜์ค€์ด์•ผ! ๐Ÿ’ช

๐ŸŽ“ ํ•ต์‹ฌ ์š”์•ฝ 1. Pipeline์˜ ๊ฐ•๋ ฅํ•จ: ๋ช‡ ์ค„์˜ ์ฝ”๋“œ๋กœ ์ตœ์‹  AI ๋ชจ๋ธ ํ™œ์šฉ
2. ๋ชจ๋ธ ์„ ํƒ์˜ ์ค‘์š”์„ฑ: ์ž‘์—…์— ๋งž๋Š” ์ ์ ˆํ•œ ๋ชจ๋ธ ์„ ํƒ
3. ํŒŒ๋ผ๋ฏธํ„ฐ ํŠœ๋‹: ํ’ˆ์งˆ๊ณผ ์†๋„์˜ ๊ท ํ˜• ์ฐพ๊ธฐ
4. ์ตœ์ ํ™” ํ•„์ˆ˜: ์‹ค์„œ๋น„์Šค๋ฅผ ์œ„ํ•œ ์„ฑ๋Šฅ ๊ฐœ์„ 
5. ๋„๋ฉ”์ธ ํŠนํ™”: ๊ฐ ๋ถ„์•ผ์— ๋งž๋Š” ์ปค์Šคํ„ฐ๋งˆ์ด์ง•
6. ์ง€์†์  ํ•™์Šต: ์ƒˆ๋กœ์šด ๋ชจ๋ธ๊ณผ ๊ธฐ๋ฒ• ๊ณ„์† ํƒ๊ตฌ
๐Ÿš€ ๋‹ค์Œ ๋‹จ๊ณ„ ์ถ”์ฒœ ์ดˆ๋ณด์ž:
โ€ข ๋‹ค์–‘ํ•œ ๋ชจ๋ธ ์‹คํ—˜ํ•ด๋ณด๊ธฐ
โ€ข ๊ฐ„๋‹จํ•œ ์›น ์•ฑ ๋งŒ๋“ค์–ด๋ณด๊ธฐ
โ€ข ๊ณต๊ฐœ ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ์—ฐ์Šต

์ค‘๊ธ‰์ž:
โ€ข ์ปค์Šคํ…€ ๋ฐ์ดํ„ฐ๋กœ ํŒŒ์ธํŠœ๋‹
โ€ข ํ”„๋กœ๋•์…˜ ๋ฐฐํฌ ๊ฒฝํ—˜
โ€ข ์„ฑ๋Šฅ ์ตœ์ ํ™” ์‹ฌํ™”

๊ณ ๊ธ‰์ž:
โ€ข ์ƒˆ๋กœ์šด ์•„ํ‚คํ…์ฒ˜ ์—ฐ๊ตฌ
โ€ข ๋Œ€๊ทœ๋ชจ ์‹œ์Šคํ…œ ๊ตฌ์ถ•
โ€ข ์˜คํ”ˆ์†Œ์Šค ๊ธฐ์—ฌ

์–ด๋–ค ๋ ˆ๋ฒจ์ด๋“  ๊ณ„์† ์‹คํ—˜ํ•˜๊ณ  ๋ฐฐ์šฐ๋Š” ๊ฒŒ ์ค‘์š”ํ•ด! ๐ŸŽฏ

AI ๊ธฐ์ˆ ์€ ๋น ๋ฅด๊ฒŒ ๋ฐœ์ „ํ•˜๊ณ  ์žˆ์–ด. ์˜ค๋Š˜ ๋ฐฐ์šด ๋‚ด์šฉ๋„ ๋ช‡ ๋‹ฌ ํ›„๋ฉด ๋” ์ข‹์€ ๋ฐฉ๋ฒ•์ด ๋‚˜์˜ฌ ์ˆ˜ ์žˆ์ง€. ํ•˜์ง€๋งŒ ๊ธฐ๋ณธ ์›๋ฆฌ์™€ ์ ‘๊ทผ ๋ฐฉ์‹์€ ๋ณ€ํ•˜์ง€ ์•Š์•„. ์ด ๊ธ€์—์„œ ๋ฐฐ์šด ๊ฐœ๋…๋“ค์„ ๋ฐ”ํƒ•์œผ๋กœ ๊ณ„์† ์ƒˆ๋กœ์šด ๊ธฐ์ˆ ์„ ์ตํ˜€๋‚˜๊ฐ€๋ฉด ๋ผ! ๐Ÿ“š

ํ…์ŠคํŠธ ์š”์•ฝ์€ ์ •๋ง ์‹ค์šฉ์ ์ธ ๊ธฐ์ˆ ์ด์•ผ. ๋‰ด์Šค, ์—ฐ๊ตฌ, ๋น„์ฆˆ๋‹ˆ์Šค, ๊ต์œก ๋“ฑ ๊ฑฐ์˜ ๋ชจ๋“  ๋ถ„์•ผ์—์„œ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ๊ฑฐ๋“ . ์—ฌ๋Ÿฌ๋ถ„๋„ ์ด ๊ธฐ์ˆ ์„ ํ™œ์šฉํ•ด์„œ ๋ฉ‹์ง„ ํ”„๋กœ์ ํŠธ๋ฅผ ๋งŒ๋“ค์–ด๋ณด๊ธธ ๋ฐ”๋ผ! ๐Ÿ’ก

ํ˜น์‹œ ๋” ๊นŠ์ด ๋ฐฐ์šฐ๊ณ  ์‹ถ๊ฑฐ๋‚˜, ํ”„๋กœ์ ํŠธ์— ๋„์›€์ด ํ•„์š”ํ•˜๋‹ค๋ฉด AI/๋จธ์‹ ๋Ÿฌ๋‹ ์ „๋ฌธ๊ฐ€๋“ค์ด ๋งŽ์ด ํ™œ๋™ํ•˜๋Š” ์ปค๋ฎค๋‹ˆํ‹ฐ๋‚˜ ํ”Œ๋žซํผ์„ ํ™œ์šฉํ•ด๋ด. ํ•จ๊ป˜ ๋ฐฐ์šฐ๊ณ  ์„ฑ์žฅํ•˜๋Š” ๊ฒŒ ๊ฐ€์žฅ ๋น ๋ฅธ ๊ธธ์ด์•ผ! ๐Ÿค

์ž, ์ด์ œ ์—ฌ๋Ÿฌ๋ถ„ ์ฐจ๋ก€์•ผ! ์˜ค๋Š˜ ๋ฐฐ์šด ๋‚ด์šฉ์„ ๋ฐ”ํƒ•์œผ๋กœ ์ง์ ‘ ์ฝ”๋“œ๋ฅผ ์ž‘์„ฑํ•ด๋ณด๊ณ , ์‹คํ—˜ํ•ด๋ณด๊ณ , ๋‚˜๋งŒ์˜ ํ”„๋กœ์ ํŠธ๋ฅผ ๋งŒ๋“ค์–ด๋ด. ์‹คํŒจ๋ฅผ ๋‘๋ ค์›Œํ•˜์ง€ ๋ง๊ณ , ๊ณ„์† ์‹œ๋„ํ•˜๋‹ค ๋ณด๋ฉด ์–ด๋А์ƒˆ ์ „๋ฌธ๊ฐ€๊ฐ€ ๋˜์–ด ์žˆ์„ ๊ฑฐ์•ผ! ๐ŸŒŸ

ํ–‰์šด์„ ๋นŒ์–ด! ํ™”์ดํŒ…! ๐Ÿš€โœจ

Happy Coding! Transformers๋กœ ํ…์ŠคํŠธ ์š”์•ฝ ๋งˆ์Šคํ„ฐํ•˜๊ธฐ ๐Ÿค– ๐Ÿ“ ๐Ÿš€ ๊ณ„์† ๋ฐฐ์šฐ๊ณ , ์‹คํ—˜ํ•˜๊ณ , ์„ฑ์žฅํ•˜์„ธ์š”!
๋Œ“๊ธ€ ์ž‘์„ฑ

์ด ๊ธ€์— ๋Œ€ํ•œ ์—ฌ๋Ÿฌ๋ถ„์˜ ์ƒ๊ฐ์„ ๋“ค๋ ค์ฃผ์„ธ์š”

๋Œ“๊ธ€ 0
๋Œ“๊ธ€์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ์ค‘์ž…๋‹ˆ๋‹ค.

์•„์ง ๋Œ“๊ธ€์ด ์—†์Šต๋‹ˆ๋‹ค.