์ฝ˜ํ…์ธ  ๋Œ€ํ‘œ ์ด๋ฏธ์ง€ - ๐Ÿš€ GPU ์ปจํ…Œ์ด๋„ˆ๋กœ AI ํ•™์Šตํ™˜๊ฒฝ ๊ตฌ์ถ•ํ•˜๊ธฐ: ์ดˆ๋ณด์ž๋„ ๋”ฐ๋ผํ•  ์ˆ˜ ์žˆ๋Š” ์™„๋ฒฝ ๊ฐ€์ด๋“œ

๐Ÿš€ GPU ์ปจํ…Œ์ด๋„ˆ๋กœ AI ํ•™์Šตํ™˜๊ฒฝ ๊ตฌ์ถ•ํ•˜๊ธฐ: ์ดˆ๋ณด์ž๋„ ๋”ฐ๋ผํ•  ์ˆ˜ ์žˆ๋Š” ์™„๋ฒฝ ๊ฐ€์ด๋“œ

Docker์™€ Kubernetes๋กœ ๋งŒ๋“œ๋Š” ๋‚˜๋งŒ์˜ AI ์—ฐ๊ตฌ์‹ค ๐ŸŽฏ

๐ŸŽฌ ์‹œ์ž‘ํ•˜๊ธฐ ์ „์—: ์™œ GPU ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ํ•„์š”ํ• ๊นŒ?

์•ˆ๋…•, ์นœ๊ตฌ! ๐Ÿ˜Š ์˜ค๋Š˜์€ AI ๊ฐœ๋ฐœ์ž๋ผ๋ฉด ๋ˆ„๊ตฌ๋‚˜ ํ•œ ๋ฒˆ์ฏค ๊ณ ๋ฏผํ•˜๊ฒŒ ๋˜๋Š” GPU ์ปจํ…Œ์ด๋„ˆ ํ™˜๊ฒฝ ๊ตฌ์ถ•์— ๋Œ€ํ•ด ์ด์•ผ๊ธฐํ•ด๋ณด๋ ค๊ณ  ํ•ด. ์š”์ฆ˜ ChatGPT, Stable Diffusion ๊ฐ™์€ AI ๋ชจ๋ธ๋“ค์ด ๋Œ€์„ธ์ž–์•„? ์ด๋Ÿฐ ๋ชจ๋ธ๋“ค์„ ํ•™์Šต์‹œํ‚ค๊ฑฐ๋‚˜ ํŒŒ์ธํŠœ๋‹ํ•˜๋ ค๋ฉด ์—„์ฒญ๋‚œ ์ปดํ“จํŒ… ํŒŒ์›Œ๊ฐ€ ํ•„์š”ํ•œ๋ฐ, ๋ฐ”๋กœ ์—ฌ๊ธฐ์„œ GPU๊ฐ€ ๋“ฑ์žฅํ•˜์ง€! ๐ŸŽฎ

๊ทผ๋ฐ ๋ฌธ์ œ๊ฐ€ ๋ญ๋ƒ๋ฉด, GPU ํ™˜๊ฒฝ ์„ค์ •์ด ์ง„์งœ ๊ณจ์น˜ ์•„ํ”„๋‹ค๋Š” ๊ฑฐ์•ผ. CUDA ๋ฒ„์ „ ๋งž์ถ”๊ณ , cuDNN ์„ค์น˜ํ•˜๊ณ , PyTorch๋‚˜ TensorFlow ๋ฒ„์ „ ํ˜ธํ™˜์„ฑ ์ฒดํฌํ•˜๊ณ ... ํ•œ ๋ฒˆ ์„ธํŒ…ํ•ด๋†“์œผ๋ฉด "๋‚ด ์ปดํ“จํ„ฐ์—์„œ๋Š” ๋˜๋Š”๋ฐ?"๋ผ๋Š” ๋ง์„ ๋‹ฌ๊ณ  ์‚ด๊ฒŒ ๋ผ. ๐Ÿ˜…

๊ทธ๋ž˜์„œ ๋“ฑ์žฅํ•œ ๊ฒŒ ๋ฐ”๋กœ ์ปจํ…Œ์ด๋„ˆ ๊ธฐ์ˆ ์ด์•ผ! Docker๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ํ™˜๊ฒฝ์„ ํ†ต์งธ๋กœ ํŒจํ‚ค์ง•ํ•  ์ˆ˜ ์žˆ์–ด์„œ, ์–ด๋””์„œ๋“  ๋˜‘๊ฐ™์ด ์ž‘๋™ํ•˜๋Š” AI ํ•™์Šต ํ™˜๊ฒฝ์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ๊ฑฐ๋“ . ๋งˆ์น˜ ๋ ˆ๊ณ  ๋ธ”๋ก์ฒ˜๋Ÿผ ์กฐ๋ฆฝํ•˜๊ณ  ๋ถ„ํ•ดํ•˜๊ณ , ํ•„์š”ํ•  ๋•Œ๋งˆ๋‹ค ๊บผ๋‚ด ์“ธ ์ˆ˜ ์žˆ๋Š” ๊ฑฐ์ง€! ๐Ÿงฑ

๐Ÿ’ก ์žฌ๋Šฅ๋„ท TIP: AI ๊ฐœ๋ฐœ์ด๋‚˜ GPU ํ™˜๊ฒฝ ๊ตฌ์ถ•์ด ์ฒ˜์Œ์ด๋ผ๋ฉด, ์žฌ๋Šฅ๋„ท์—์„œ ํด๋ผ์šฐ๋“œ/DevOps ์ „๋ฌธ๊ฐ€๋“ค์˜ ๋„์›€์„ ๋ฐ›์•„๋ณด๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ. ์‹ค์ „ ๊ฒฝํ—˜์ด ํ’๋ถ€ํ•œ ๊ฐœ๋ฐœ์ž๋“ค์ด 1:1๋กœ ์ปจ์„คํŒ…ํ•ด์ฃผ๋‹ˆ๊นŒ ์‹œํ–‰์ฐฉ์˜ค๋ฅผ ํฌ๊ฒŒ ์ค„์ผ ์ˆ˜ ์žˆ์–ด! ๐ŸŽ“
GPU ์ปจํ…Œ์ด๋„ˆ ์•„ํ‚คํ…์ฒ˜ Host OS Ubuntu / CentOS NVIDIA Driver Docker Engine Container Runtime NVIDIA Container Toolkit Docker / containerd GPU Resource Manager AI Containers PyTorch Container TensorFlow Container Jupyter Notebook GPU 0 RTX 4090 24GB VRAM GPU 1 RTX 4090 24GB VRAM GPU 2 RTX 4090 24GB VRAM

๐Ÿ”ง ๊ธฐ๋ณธ ๊ฐœ๋…: GPU ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ๋ญ”๋ฐ?

์ž, ๋ณธ๊ฒฉ์ ์œผ๋กœ ๋“ค์–ด๊ฐ€๊ธฐ ์ „์— ๊ธฐ๋ณธ ๊ฐœ๋…๋ถ€ํ„ฐ ํ™•์‹คํžˆ ์žก๊ณ  ๊ฐ€์ž! ๐ŸŽฏ

๐Ÿ“ฆ ์ปจํ…Œ์ด๋„ˆ๋ž€?

์ปจํ…Œ์ด๋„ˆ๋Š” ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜๊ณผ ๊ทธ ์‹คํ–‰์— ํ•„์š”ํ•œ ๋ชจ๋“  ๊ฒƒ(๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ, ์˜์กด์„ฑ, ์„ค์ • ํŒŒ์ผ ๋“ฑ)์„ ํ•˜๋‚˜์˜ ํŒจํ‚ค์ง€๋กœ ๋ฌถ์€ ๊ฑฐ์•ผ. ๊ฐ€์ƒ๋จธ์‹ (VM)๊ณผ ๋น„์Šทํ•˜์ง€๋งŒ ํ›จ์”ฌ ๊ฐ€๋ณ๊ณ  ๋น ๋ฅด์ง€! VM์€ ์ „์ฒด OS๋ฅผ ํฌํ•จํ•˜๋Š” ๋ฐ˜๋ฉด, ์ปจํ…Œ์ด๋„ˆ๋Š” ํ˜ธ์ŠคํŠธ OS์˜ ์ปค๋„์„ ๊ณต์œ ํ•˜๋ฉด์„œ ๊ฒฉ๋ฆฌ๋œ ํ™˜๊ฒฝ์„ ์ œ๊ณตํ•ด. ๐Ÿƒโ€โ™‚๏ธ

๐ŸŽญ ๋น„์œ ๋กœ ์ดํ•ดํ•˜๊ธฐ: ๊ฐ€์ƒ๋จธ์‹ (VM): ์ง‘ ์ „์ฒด๋ฅผ ํ†ต์งธ๋กœ ๋ณต์‚ฌํ•ด์„œ ์˜ฎ๊ธฐ๋Š” ๊ฒƒ. ๋ฌด๊ฒ๊ณ  ์‹œ๊ฐ„์ด ์˜ค๋ž˜ ๊ฑธ๋ฆผ.
์ปจํ…Œ์ด๋„ˆ: ํ•„์š”ํ•œ ์ง๋งŒ ๊ฐ€๋ฐฉ์— ์‹ธ์„œ ์˜ฎ๊ธฐ๋Š” ๊ฒƒ. ๊ฐ€๋ณ๊ณ  ๋น ๋ฆ„!
GPU ์ปจํ…Œ์ด๋„ˆ: ๊ฑฐ๊ธฐ์— ๊ณ ์„ฑ๋Šฅ ๊ฒŒ์ž„๊ธฐ(GPU)๊นŒ์ง€ ์ฑ™๊ฒจ๊ฐ€๋Š” ๊ฒƒ! ๐ŸŽฎ

๐ŸŽฎ GPU๋ฅผ ์ปจํ…Œ์ด๋„ˆ์—์„œ ์‚ฌ์šฉํ•˜๋ ค๋ฉด?

์ผ๋ฐ˜ ์ปจํ…Œ์ด๋„ˆ๋Š” CPU๋งŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์–ด. GPU๋ฅผ ์‚ฌ์šฉํ•˜๋ ค๋ฉด ํŠน๋ณ„ํ•œ ์„ค์ •์ด ํ•„์š”ํ•œ๋ฐ, ๋ฐ”๋กœ NVIDIA Container Toolkit์ด์•ผ! ์ด๊ฒŒ ์žˆ์–ด์•ผ ์ปจํ…Œ์ด๋„ˆ ์•ˆ์—์„œ GPU๋ฅผ ์ธ์‹ํ•˜๊ณ  ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ฑฐ๋“ . ๋งˆ์น˜ USB ๋“œ๋ผ์ด๋ฒ„ ๊ฐ™์€ ์—ญํ• ์ด๋ผ๊ณ  ๋ณด๋ฉด ๋ผ. ๐Ÿ”Œ

ํ•ต์‹ฌ ๊ตฌ์„ฑ ์š”์†Œ๋ฅผ ์ •๋ฆฌํ•˜๋ฉด ์ด๋ ‡๊ฒŒ ๋ผ:

๊ตฌ์„ฑ ์š”์†Œ ์—ญํ•  ํ•„์ˆ˜ ์—ฌ๋ถ€
NVIDIA Driver ํ˜ธ์ŠคํŠธ OS์—์„œ GPU ํ•˜๋“œ์›จ์–ด ์ œ์–ด โœ… ํ•„์ˆ˜
Docker Engine ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ ํ™˜๊ฒฝ ์ œ๊ณต โœ… ํ•„์ˆ˜
NVIDIA Container Toolkit ์ปจํ…Œ์ด๋„ˆ์— GPU ์ ‘๊ทผ ๊ถŒํ•œ ๋ถ€์—ฌ โœ… ํ•„์ˆ˜
CUDA GPU ํ”„๋กœ๊ทธ๋ž˜๋ฐ ํ”Œ๋žซํผ (์ปจํ…Œ์ด๋„ˆ ๋‚ด๋ถ€) โœ… ํ•„์ˆ˜
cuDNN ๋”ฅ๋Ÿฌ๋‹ ๊ฐ€์† ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ โญ ๊ถŒ์žฅ
Kubernetes ๋‹ค์ค‘ ์ปจํ…Œ์ด๋„ˆ ์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜ ๐Ÿ”„ ์„ ํƒ

๐Ÿ› ๏ธ ํ™˜๊ฒฝ ๊ตฌ์ถ• Step 1: ํ˜ธ์ŠคํŠธ ์‹œ์Šคํ…œ ์ค€๋น„ํ•˜๊ธฐ

์ž, ์ด์ œ ์‹ค์ „์ด์•ผ! ๋จผ์ € ํ˜ธ์ŠคํŠธ ์‹œ์Šคํ…œ(์‹ค์ œ ๋ฌผ๋ฆฌ ์„œ๋ฒ„๋‚˜ ์›Œํฌ์Šคํ…Œ์ด์…˜)์„ ์ค€๋น„ํ•ด์•ผ ํ•ด. ๋‚˜๋Š” Ubuntu 22.04 LTS ๊ธฐ์ค€์œผ๋กœ ์„ค๋ช…ํ• ๊ฒŒ. CentOS๋‚˜ ๋‹ค๋ฅธ ๋ฐฐํฌํŒ๋„ ๋น„์Šทํ•˜๋‹ˆ๊นŒ ๊ฑฑ์ • ๋งˆ! ๐Ÿ˜‰

๐Ÿ“‹ ์‹œ์Šคํ…œ ์š”๊ตฌ์‚ฌํ•ญ ์ฒดํฌ

โœ… ์ตœ์†Œ ์š”๊ตฌ์‚ฌํ•ญ: โ€ข GPU: NVIDIA GPU (CUDA Compute Capability 3.5 ์ด์ƒ)
โ€ข OS: Ubuntu 18.04/20.04/22.04, CentOS 7/8, RHEL 7/8
โ€ข RAM: ์ตœ์†Œ 16GB (32GB ์ด์ƒ ๊ถŒ์žฅ)
โ€ข Storage: SSD 100GB ์ด์ƒ (์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ์ €์žฅ์šฉ)
โ€ข Network: ์ธํ„ฐ๋„ท ์—ฐ๊ฒฐ (์ด๋ฏธ์ง€ ๋‹ค์šด๋กœ๋“œ์šฉ)

๐ŸŽฏ Step 1-1: NVIDIA Driver ์„ค์น˜

๊ฐ€์žฅ ๋จผ์ € ํ•ด์•ผ ํ•  ์ผ์€ NVIDIA ๋“œ๋ผ์ด๋ฒ„ ์„ค์น˜์•ผ. ์ด๊ฒŒ ์—†์œผ๋ฉด GPU ์ž์ฒด๋ฅผ ์ธ์‹ ๋ชป ํ•ด! ๐Ÿ˜ฑ

# ์‹œ์Šคํ…œ ์—…๋ฐ์ดํŠธ
sudo apt update && sudo apt upgrade -y

# ๊ธฐ์กด NVIDIA ๋“œ๋ผ์ด๋ฒ„ ์ œ๊ฑฐ (๊นจ๋—ํ•œ ์„ค์น˜๋ฅผ ์œ„ํ•ด)
sudo apt remove --purge nvidia-* -y
sudo apt autoremove -y

# ์ถ”์ฒœ ๋“œ๋ผ์ด๋ฒ„ ํ™•์ธ
ubuntu-drivers devices

# ์ถ”์ฒœ ๋“œ๋ผ์ด๋ฒ„ ์ž๋™ ์„ค์น˜
sudo ubuntu-drivers autoinstall

# ๋˜๋Š” ํŠน์ • ๋ฒ„์ „ ์„ค์น˜ (์˜ˆ: 535 ๋ฒ„์ „)
sudo apt install nvidia-driver-535 -y

# ์žฌ๋ถ€ํŒ…
sudo reboot

์žฌ๋ถ€ํŒ… ํ›„ ๋“œ๋ผ์ด๋ฒ„๊ฐ€ ์ œ๋Œ€๋กœ ์„ค์น˜๋๋Š”์ง€ ํ™•์ธํ•ด๋ณด์ž:

# GPU ์ •๋ณด ํ™•์ธ
nvidia-smi

์ด ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ์ด๋Ÿฐ ํ™”๋ฉด์ด ๋‚˜์™€์•ผ ์ •์ƒ์ด์•ผ:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03   Driver Version: 535.129.03   CUDA Version: 12.2   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  N/A |
| 30%   45C    P8    25W / 450W |    512MiB / 24564MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

๐Ÿณ Step 1-2: Docker ์„ค์น˜

์ด์ œ Docker๋ฅผ ์„ค์น˜ํ•  ์ฐจ๋ก€์•ผ! Docker๋Š” ์ปจํ…Œ์ด๋„ˆ๋ฅผ ์‹คํ–‰ํ•˜๋Š” ํ•ต์‹ฌ ์—”์ง„์ด์ง€. ๐Ÿšข

# Docker ๊ณต์‹ GPG ํ‚ค ์ถ”๊ฐ€
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Docker ์ €์žฅ์†Œ ์ถ”๊ฐ€
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Docker ์„ค์น˜
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

# Docker ์„œ๋น„์Šค ์‹œ์ž‘ ๋ฐ ์ž๋™ ์‹œ์ž‘ ์„ค์ •
sudo systemctl start docker
sudo systemctl enable docker

# ํ˜„์žฌ ์‚ฌ์šฉ์ž๋ฅผ docker ๊ทธ๋ฃน์— ์ถ”๊ฐ€ (sudo ์—†์ด docker ๋ช…๋ น ์‚ฌ์šฉ)
sudo usermod -aG docker $USER

# ๋ณ€๊ฒฝ์‚ฌํ•ญ ์ ์šฉ์„ ์œ„ํ•ด ๋กœ๊ทธ์•„์›ƒ ํ›„ ์žฌ๋กœ๊ทธ์ธ ๋˜๋Š”
newgrp docker

# Docker ์„ค์น˜ ํ™•์ธ
docker --version
docker run hello-world
๐Ÿ’ก Pro Tip: docker run hello-world๊ฐ€ ์„ฑ๊ณต์ ์œผ๋กœ ์‹คํ–‰๋˜๋ฉด Docker๊ฐ€ ์ œ๋Œ€๋กœ ์„ค์น˜๋œ ๊ฑฐ์•ผ! ์ด ๋ช…๋ น์–ด๋Š” ๊ฐ„๋‹จํ•œ ํ…Œ์ŠคํŠธ ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋‹ค์šด๋กœ๋“œํ•˜๊ณ  ์‹คํ–‰ํ•ด์„œ "Hello from Docker!" ๋ฉ”์‹œ์ง€๋ฅผ ์ถœ๋ ฅํ•ด์ค˜. ๐ŸŽ‰

๐Ÿ”Œ Step 1-3: NVIDIA Container Toolkit ์„ค์น˜

์ž, ์ด์ œ ๊ฐ€์žฅ ์ค‘์š”ํ•œ ๋ถ€๋ถ„์ด์•ผ! NVIDIA Container Toolkit์„ ์„ค์น˜ํ•ด์„œ Docker ์ปจํ…Œ์ด๋„ˆ๊ฐ€ GPU๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ฒŒ ๋งŒ๋“ค์–ด๋ณด์ž. ๐ŸŽฏ

# NVIDIA Container Toolkit ์ €์žฅ์†Œ ์„ค์ •
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# ํŒจํ‚ค์ง€ ์„ค์น˜
sudo apt update
sudo apt install -y nvidia-container-toolkit

# Docker ๋ฐ๋ชฌ ์„ค์ • ์—…๋ฐ์ดํŠธ
sudo nvidia-ctk runtime configure --runtime=docker

# Docker ์žฌ์‹œ์ž‘
sudo systemctl restart docker

# ์„ค์น˜ ํ™•์ธ
docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

๋งˆ์ง€๋ง‰ ๋ช…๋ น์–ด๊ฐ€ ์„ฑ๊ณต์ ์œผ๋กœ ์‹คํ–‰๋˜๊ณ  nvidia-smi ์ถœ๋ ฅ์ด ๋‚˜์˜ค๋ฉด ์™„๋ฒฝํ•˜๊ฒŒ ์„ค์ •๋œ ๊ฑฐ์•ผ! ๐ŸŽŠ ์ด์ œ ์ปจํ…Œ์ด๋„ˆ ์•ˆ์—์„œ๋„ GPU๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์–ด!

๐ŸŽจ ํ™˜๊ฒฝ ๊ตฌ์ถ• Step 2: AI ํ•™์Šต์šฉ Docker ์ด๋ฏธ์ง€ ๋งŒ๋“ค๊ธฐ

ํ˜ธ์ŠคํŠธ ์‹œ์Šคํ…œ ์ค€๋น„๊ฐ€ ๋๋‚ฌ์œผ๋‹ˆ, ์ด์ œ ์‹ค์ œ๋กœ AI ํ•™์Šต์— ์‚ฌ์šฉํ•  Docker ์ด๋ฏธ์ง€๋ฅผ ๋งŒ๋“ค์–ด๋ณด์ž! ์—ฌ๊ธฐ์„œ๋Š” PyTorch ๊ธฐ๋ฐ˜ ์ด๋ฏธ์ง€๋ฅผ ์˜ˆ์‹œ๋กœ ๋“ค๊ฒŒ. TensorFlow๋„ ๋น„์Šทํ•œ ๋ฐฉ์‹์œผ๋กœ ๋งŒ๋“ค ์ˆ˜ ์žˆ์–ด! ๐ŸŽจ

๐Ÿ“ Dockerfile ์ž‘์„ฑํ•˜๊ธฐ

Dockerfile์€ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€๋ฅผ ๋งŒ๋“œ๋Š” ๋ ˆ์‹œํ”ผ์•ผ. ์–ด๋–ค ๋ฒ ์ด์Šค ์ด๋ฏธ์ง€๋ฅผ ์‚ฌ์šฉํ• ์ง€, ์–ด๋–ค ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ• ์ง€ ๋“ฑ์„ ์ •์˜ํ•˜์ง€. ์‹ค์ „์—์„œ ๋ฐ”๋กœ ์“ธ ์ˆ˜ ์žˆ๋Š” Dockerfile์„ ๋งŒ๋“ค์–ด๋ณผ๊ฒŒ! ๐Ÿ’ช

# Dockerfile
FROM nvidia/cuda:12.2.0-cudnn8-devel-ubuntu22.04

# ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
ENV CUDA_HOME=/usr/local/cuda
ENV PATH=${CUDA_HOME}/bin:${PATH}
ENV LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}

# ๊ธฐ๋ณธ ํŒจํ‚ค์ง€ ์„ค์น˜
RUN apt-get update && apt-get install -y \
    python3.10 \
    python3-pip \
    python3-dev \
    git \
    wget \
    curl \
    vim \
    tmux \
    htop \
    libgl1-mesa-glx \
    libglib2.0-0 \
    && rm -rf /var/lib/apt/lists/*

# Python ์‹ฌ๋ณผ๋ฆญ ๋งํฌ ์ƒ์„ฑ
RUN ln -s /usr/bin/python3 /usr/bin/python

# pip ์—…๊ทธ๋ ˆ์ด๋“œ
RUN pip install --no-cache-dir --upgrade pip setuptools wheel

# PyTorch ๋ฐ ์ฃผ์š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์„ค์น˜
RUN pip install --no-cache-dir \
    torch==2.1.0 \
    torchvision==0.16.0 \
    torchaudio==2.1.0 \
    --index-url https://download.pytorch.org/whl/cu121

# ๋ฐ์ดํ„ฐ ์‚ฌ์ด์–ธ์Šค ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ
RUN pip install --no-cache-dir \
    numpy==1.24.3 \
    pandas==2.0.3 \
    scikit-learn==1.3.0 \
    matplotlib==3.7.2 \
    seaborn==0.12.2 \
    jupyter==1.0.0 \
    jupyterlab==4.0.5 \
    ipywidgets==8.1.0

# ๋”ฅ๋Ÿฌ๋‹ ์œ ํ‹ธ๋ฆฌํ‹ฐ
RUN pip install --no-cache-dir \
    transformers==4.33.0 \
    datasets==2.14.5 \
    accelerate==0.23.0 \
    tensorboard==2.14.0 \
    wandb==0.15.10 \
    opencv-python==4.8.0.76 \
    albumentations==1.3.1 \
    timm==0.9.7

# ์ž‘์—… ๋””๋ ‰ํ† ๋ฆฌ ์„ค์ •
WORKDIR /workspace

# Jupyter ์„ค์ •
RUN jupyter notebook --generate-config && \
    echo "c.NotebookApp.ip = '0.0.0.0'" >> ~/.jupyter/jupyter_notebook_config.py && \
    echo "c.NotebookApp.open_browser = False" >> ~/.jupyter/jupyter_notebook_config.py && \
    echo "c.NotebookApp.token = ''" >> ~/.jupyter/jupyter_notebook_config.py && \
    echo "c.NotebookApp.password = ''" >> ~/.jupyter/jupyter_notebook_config.py

# ํฌํŠธ ๋…ธ์ถœ
EXPOSE 8888 6006

# ์‹œ์ž‘ ๋ช…๋ น
CMD ["/bin/bash"]
๐ŸŽฏ Dockerfile ์ฃผ์š” ๊ตฌ์„ฑ ์„ค๋ช…: 1. ๋ฒ ์ด์Šค ์ด๋ฏธ์ง€: NVIDIA ๊ณต์‹ CUDA ์ด๋ฏธ์ง€ ์‚ฌ์šฉ (CUDA 12.2 + cuDNN 8)
2. Python ํ™˜๊ฒฝ: Python 3.10 ๋ฐ ํ•„์ˆ˜ ๊ฐœ๋ฐœ ๋„๊ตฌ
3. PyTorch: CUDA 12.1 ํ˜ธํ™˜ ๋ฒ„์ „ ์„ค์น˜
4. ML/DL ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ: ์‹ค๋ฌด์—์„œ ์ž์ฃผ ์“ฐ๋Š” ํŒจํ‚ค์ง€๋“ค
5. Jupyter: ์›น ๊ธฐ๋ฐ˜ ๊ฐœ๋ฐœ ํ™˜๊ฒฝ (ํฌํŠธ 8888)
6. TensorBoard: ํ•™์Šต ๋ชจ๋‹ˆํ„ฐ๋ง ๋„๊ตฌ (ํฌํŠธ 6006)

๐Ÿ—๏ธ ์ด๋ฏธ์ง€ ๋นŒ๋“œํ•˜๊ธฐ

Dockerfile์„ ์ž‘์„ฑํ–ˆ์œผ๋ฉด ์ด์ œ ์‹ค์ œ ์ด๋ฏธ์ง€๋ฅผ ๋นŒ๋“œํ•ด๋ณด์ž! ์ฒ˜์Œ ๋นŒ๋“œํ•  ๋•Œ๋Š” ์‹œ๊ฐ„์ด ์ข€ ๊ฑธ๋ฆด ์ˆ˜ ์žˆ์–ด. ์ปคํ”ผ ํ•œ ์ž” ํ•˜๋ฉด์„œ ๊ธฐ๋‹ค๋ ค~ โ˜•

# ์ด๋ฏธ์ง€ ๋นŒ๋“œ (ํ˜„์žฌ ๋””๋ ‰ํ† ๋ฆฌ์— Dockerfile์ด ์žˆ๋‹ค๊ณ  ๊ฐ€์ •)
docker build -t ai-pytorch:latest .

# ๋นŒ๋“œ ๊ณผ์ •์—์„œ ์บ์‹œ ์‚ฌ์šฉ ์•ˆ ํ•˜๊ณ  ์ฒ˜์Œ๋ถ€ํ„ฐ ๋‹ค์‹œ ๋นŒ๋“œ
docker build --no-cache -t ai-pytorch:latest .

# ๋นŒ๋“œ๋œ ์ด๋ฏธ์ง€ ํ™•์ธ
docker images

# ์ด๋ฏธ์ง€ ์ƒ์„ธ ์ •๋ณด ํ™•์ธ
docker inspect ai-pytorch:latest

๐Ÿš€ ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ํ•˜๊ธฐ

์ด๋ฏธ์ง€๊ฐ€ ์ค€๋น„๋์œผ๋‹ˆ ์ด์ œ ์ปจํ…Œ์ด๋„ˆ๋ฅผ ์‹คํ–‰ํ•ด๋ณด์ž! GPU๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋„์šฐ๋Š” ๋ฐฉ๋ฒ•์€ ์—ฌ๋Ÿฌ ๊ฐ€์ง€๊ฐ€ ์žˆ์–ด. ์ƒํ™ฉ์— ๋งž๊ฒŒ ๊ณจ๋ผ ์“ฐ๋ฉด ๋ผ! ๐ŸŽฎ

# ๊ธฐ๋ณธ ์‹คํ–‰ (๋ชจ๋“  GPU ์‚ฌ์šฉ, ์ธํ„ฐ๋ž™ํ‹ฐ๋ธŒ ๋ชจ๋“œ)
docker run --gpus all -it --rm \
    --name ai-workspace \
    -v $(pwd)/workspace:/workspace \
    -p 8888:8888 -p 6006:6006 \
    ai-pytorch:latest

# ํŠน์ • GPU๋งŒ ์‚ฌ์šฉ (GPU 0๋ฒˆ๋งŒ)
docker run --gpus '"device=0"' -it --rm \
    --name ai-workspace \
    -v $(pwd)/workspace:/workspace \
    -p 8888:8888 -p 6006:6006 \
    ai-pytorch:latest

# ์—ฌ๋Ÿฌ GPU ์„ ํƒ (GPU 0, 2๋ฒˆ)
docker run --gpus '"device=0,2"' -it --rm \
    --name ai-workspace \
    -v $(pwd)/workspace:/workspace \
    -p 8888:8888 -p 6006:6006 \
    ai-pytorch:latest

# ๋ฐฑ๊ทธ๋ผ์šด๋“œ ์‹คํ–‰ + Jupyter ์ž๋™ ์‹œ์ž‘
docker run --gpus all -d \
    --name ai-workspace \
    -v $(pwd)/workspace:/workspace \
    -p 8888:8888 -p 6006:6006 \
    ai-pytorch:latest \
    jupyter lab --allow-root

# ์‹คํ–‰ ์ค‘์ธ ์ปจํ…Œ์ด๋„ˆ์— ์ ‘์†
docker exec -it ai-workspace /bin/bash
๐Ÿ’ก ์˜ต์…˜ ์„ค๋ช…: --gpus all: ๋ชจ๋“  GPU๋ฅผ ์ปจํ…Œ์ด๋„ˆ์—์„œ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•จ
-it: ์ธํ„ฐ๋ž™ํ‹ฐ๋ธŒ ํ„ฐ๋ฏธ๋„ ๋ชจ๋“œ
--rm: ์ปจํ…Œ์ด๋„ˆ ์ข…๋ฃŒ ์‹œ ์ž๋™ ์‚ญ์ œ
-v: ํ˜ธ์ŠคํŠธ ๋””๋ ‰ํ† ๋ฆฌ๋ฅผ ์ปจํ…Œ์ด๋„ˆ์— ๋งˆ์šดํŠธ (๋ฐ์ดํ„ฐ ์˜์†์„ฑ)
-p: ํฌํŠธ ํฌ์›Œ๋”ฉ (ํ˜ธ์ŠคํŠธ:์ปจํ…Œ์ด๋„ˆ)
-d: ๋ฐฑ๊ทธ๋ผ์šด๋“œ ์‹คํ–‰ (detached mode)

๐Ÿงช GPU ์ž‘๋™ ํ…Œ์ŠคํŠธ

์ปจํ…Œ์ด๋„ˆ๊ฐ€ ์‹คํ–‰๋์œผ๋ฉด GPU๊ฐ€ ์ œ๋Œ€๋กœ ์ธ์‹๋˜๋Š”์ง€ ํ…Œ์ŠคํŠธํ•ด๋ณด์ž! Python ์ฝ”๋“œ๋กœ ๊ฐ„๋‹จํžˆ ํ™•์ธํ•  ์ˆ˜ ์žˆ์–ด. ๐Ÿ”ฌ

# ์ปจํ…Œ์ด๋„ˆ ์•ˆ์—์„œ Python ์‹คํ–‰
python3

# Python ์ธํ„ฐํ”„๋ฆฌํ„ฐ์—์„œ ์‹คํ–‰
import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"CUDA version: {torch.version.cuda}")
print(f"Number of GPUs: {torch.cuda.device_count()}")

# GPU ์ •๋ณด ์ถœ๋ ฅ
for i in range(torch.cuda.device_count()):
    print(f"GPU {i}: {torch.cuda.get_device_name(i)}")
    print(f"  Memory: {torch.cuda.get_device_properties(i).total_memory / 1024**3:.2f} GB")

# ๊ฐ„๋‹จํ•œ ํ…์„œ ์—ฐ์‚ฐ ํ…Œ์ŠคํŠธ
x = torch.rand(1000, 1000).cuda()
y = torch.rand(1000, 1000).cuda()
z = torch.matmul(x, y)
print(f"Matrix multiplication successful on GPU!")

์ด ์ฝ”๋“œ๊ฐ€ ์—๋Ÿฌ ์—†์ด ์‹คํ–‰๋˜๊ณ  GPU ์ •๋ณด๊ฐ€ ์ถœ๋ ฅ๋˜๋ฉด ์„ฑ๊ณต์ด์•ผ! ๐ŸŽ‰ ์ด์ œ ๋ณธ๊ฒฉ์ ์œผ๋กœ AI ๋ชจ๋ธ์„ ํ•™์Šต์‹œํ‚ฌ ์ค€๋น„๊ฐ€ ๋œ ๊ฑฐ์ง€!

๐ŸŽฏ ํ™˜๊ฒฝ ๊ตฌ์ถ• Step 3: Docker Compose๋กœ ๋ฉ€ํ‹ฐ ์ปจํ…Œ์ด๋„ˆ ๊ด€๋ฆฌ

์‹ค์ œ ํ”„๋กœ์ ํŠธ์—์„œ๋Š” ์—ฌ๋Ÿฌ ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋™์‹œ์— ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ์•„. ์˜ˆ๋ฅผ ๋“ค์–ด ํ•™์Šต์šฉ ์ปจํ…Œ์ด๋„ˆ, ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค, ๋ชจ๋‹ˆํ„ฐ๋ง ๋„๊ตฌ ๋“ฑ์„ ํ•จ๊ป˜ ๋„์›Œ์•ผ ํ•  ๋•Œ๊ฐ€ ์žˆ์ง€. ์ด๋Ÿด ๋•Œ Docker Compose๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ์—„์ฒญ ํŽธํ•ด! ๐Ÿ“ฆ

๐Ÿ“„ docker-compose.yml ์ž‘์„ฑ

Docker Compose๋Š” YAML ํŒŒ์ผ๋กœ ์—ฌ๋Ÿฌ ์ปจํ…Œ์ด๋„ˆ์˜ ์„ค์ •์„ ํ•œ ๋ฒˆ์— ์ •์˜ํ•  ์ˆ˜ ์žˆ์–ด. ์‹ค๋ฌด์—์„œ ๋ฐ”๋กœ ์“ธ ์ˆ˜ ์žˆ๋Š” ๊ตฌ์„ฑ์„ ๋งŒ๋“ค์–ด๋ณผ๊ฒŒ! ๐Ÿ› ๏ธ

# docker-compose.yml
version: '3.8'

services:
  # PyTorch ํ•™์Šต ํ™˜๊ฒฝ
  pytorch-gpu:
    image: ai-pytorch:latest
    container_name: pytorch-workspace
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    volumes:
      - ./workspace:/workspace
      - ./datasets:/datasets
      - ./models:/models
    ports:
      - "8888:8888"
      - "6006:6006"
    shm_size: '16gb'
    stdin_open: true
    tty: true
    command: jupyter lab --allow-root
    networks:
      - ai-network

  # TensorFlow ํ•™์Šต ํ™˜๊ฒฝ
  tensorflow-gpu:
    image: tensorflow/tensorflow:2.13.0-gpu-jupyter
    container_name: tensorflow-workspace
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    volumes:
      - ./workspace-tf:/workspace
      - ./datasets:/datasets
      - ./models:/models
    ports:
      - "8889:8888"
      - "6007:6006"
    shm_size: '16gb'
    networks:
      - ai-network

  # MLflow ์‹คํ—˜ ์ถ”์ 
  mlflow:
    image: ghcr.io/mlflow/mlflow:v2.7.1
    container_name: mlflow-server
    ports:
      - "5000:5000"
    volumes:
      - ./mlflow:/mlflow
    command: mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri sqlite:///mlflow/mlflow.db --default-artifact-root /mlflow/artifacts
    networks:
      - ai-network

  # PostgreSQL (์‹คํ—˜ ๊ฒฐ๊ณผ ์ €์žฅ์šฉ)
  postgres:
    image: postgres:15
    container_name: ai-postgres
    environment:
      - POSTGRES_USER=aiuser
      - POSTGRES_PASSWORD=aipassword
      - POSTGRES_DB=experiments
    volumes:
      - postgres-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    networks:
      - ai-network

  # Redis (์บ์‹ฑ ๋ฐ ํ)
  redis:
    image: redis:7-alpine
    container_name: ai-redis
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - ai-network

volumes:
  postgres-data:
  redis-data:

networks:
  ai-network:
    driver: bridge
๐ŸŽฏ ๊ตฌ์„ฑ ์š”์†Œ ์„ค๋ช…: pytorch-gpu: PyTorch ๊ธฐ๋ฐ˜ ํ•™์Šต ํ™˜๊ฒฝ (Jupyter ํฌํŠธ 8888)
tensorflow-gpu: TensorFlow ํ•™์Šต ํ™˜๊ฒฝ (Jupyter ํฌํŠธ 8889)
mlflow: ์‹คํ—˜ ์ถ”์  ๋ฐ ๋ชจ๋ธ ๊ด€๋ฆฌ (ํฌํŠธ 5000)
postgres: ์‹คํ—˜ ๊ฒฐ๊ณผ ์ €์žฅ์šฉ ๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค
redis: ์บ์‹ฑ ๋ฐ ์ž‘์—… ํ ๊ด€๋ฆฌ

shm_size: ๊ณต์œ  ๋ฉ”๋ชจ๋ฆฌ ํฌ๊ธฐ (DataLoader ๋ฉ€ํ‹ฐํ”„๋กœ์„ธ์‹ฑ์— ํ•„์š”)

๐Ÿš€ Docker Compose ์‹คํ–‰

# ๋ชจ๋“  ์„œ๋น„์Šค ์‹œ์ž‘
docker-compose up -d

# ํŠน์ • ์„œ๋น„์Šค๋งŒ ์‹œ์ž‘
docker-compose up -d pytorch-gpu mlflow

# ๋กœ๊ทธ ํ™•์ธ
docker-compose logs -f pytorch-gpu

# ์„œ๋น„์Šค ์ƒํƒœ ํ™•์ธ
docker-compose ps

# ํŠน์ • ์ปจํ…Œ์ด๋„ˆ์— ์ ‘์†
docker-compose exec pytorch-gpu /bin/bash

# ๋ชจ๋“  ์„œ๋น„์Šค ์ค‘์ง€
docker-compose down

# ๋ณผ๋ฅจ๊นŒ์ง€ ์‚ญ์ œํ•˜๋ฉฐ ์ค‘์ง€
docker-compose down -v

๐Ÿ“Š ์‹ค์ „ ํ™œ์šฉ ์˜ˆ์‹œ

์ด์ œ ์‹ค์ œ๋กœ ์–ด๋–ป๊ฒŒ ์‚ฌ์šฉํ•˜๋Š”์ง€ ์˜ˆ์‹œ๋ฅผ ๋ณด์—ฌ์ค„๊ฒŒ! ๊ฐ„๋‹จํ•œ ์ด๋ฏธ์ง€ ๋ถ„๋ฅ˜ ๋ชจ๋ธ์„ ํ•™์Šต์‹œ์ผœ๋ณด์ž. ๐Ÿ–ผ๏ธ

# train.py
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms, models
from torch.utils.data import DataLoader
import mlflow
import mlflow.pytorch

# MLflow ์„ค์ •
mlflow.set_tracking_uri("http://mlflow:5000")
mlflow.set_experiment("image-classification")

# ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ
BATCH_SIZE = 64
EPOCHS = 10
LEARNING_RATE = 0.001

# ๋ฐ์ดํ„ฐ ์ „์ฒ˜๋ฆฌ
transform = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], 
                       std=[0.229, 0.224, 0.225])
])

# ๋ฐ์ดํ„ฐ์…‹ ๋กœ๋“œ
train_dataset = datasets.ImageFolder('/datasets/train', transform=transform)
train_loader = DataLoader(train_dataset, batch_size=BATCH_SIZE, 
                         shuffle=True, num_workers=4, pin_memory=True)

# ๋ชจ๋ธ ์ •์˜
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = models.resnet50(pretrained=True)
model.fc = nn.Linear(model.fc.in_features, len(train_dataset.classes))
model = model.to(device)

# ์†์‹ค ํ•จ์ˆ˜ ๋ฐ ์˜ตํ‹ฐ๋งˆ์ด์ €
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=LEARNING_RATE)

# MLflow ์‹คํ—˜ ์‹œ์ž‘
with mlflow.start_run():
    # ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ ๋กœ๊น…
    mlflow.log_params({
        "batch_size": BATCH_SIZE,
        "epochs": EPOCHS,
        "learning_rate": LEARNING_RATE,
        "model": "resnet50"
    })
    
    # ํ•™์Šต ๋ฃจํ”„
    for epoch in range(EPOCHS):
        model.train()
        running_loss = 0.0
        correct = 0
        total = 0
        
        for i, (inputs, labels) in enumerate(train_loader):
            inputs, labels = inputs.to(device), labels.to(device)
            
            optimizer.zero_grad()
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
            
            running_loss += loss.item()
            _, predicted = outputs.max(1)
            total += labels.size(0)
            correct += predicted.eq(labels).sum().item()
            
            if i % 10 == 0:
                print(f'Epoch [{epoch+1}/{EPOCHS}], Step [{i}/{len(train_loader)}], '
                      f'Loss: {loss.item():.4f}, Acc: {100.*correct/total:.2f}%')
        
        # ์—ํญ๋ณ„ ๋ฉ”ํŠธ๋ฆญ ๋กœ๊น…
        epoch_loss = running_loss / len(train_loader)
        epoch_acc = 100. * correct / total
        mlflow.log_metrics({
            "train_loss": epoch_loss,
            "train_accuracy": epoch_acc
        }, step=epoch)
        
        print(f'Epoch [{epoch+1}/{EPOCHS}] completed - Loss: {epoch_loss:.4f}, Acc: {epoch_acc:.2f}%')
    
    # ๋ชจ๋ธ ์ €์žฅ
    mlflow.pytorch.log_model(model, "model")
    torch.save(model.state_dict(), '/models/resnet50_final.pth')
    
print("Training completed! ๐ŸŽ‰")

์ด ์ฝ”๋“œ๋ฅผ ์ปจํ…Œ์ด๋„ˆ ์•ˆ์—์„œ ์‹คํ–‰ํ•˜๋ฉด:

# ์ปจํ…Œ์ด๋„ˆ ์ ‘์†
docker-compose exec pytorch-gpu /bin/bash

# ํ•™์Šต ์‹คํ–‰
cd /workspace
python train.py

# ๋˜๋Š” ๋ฐฑ๊ทธ๋ผ์šด๋“œ ์‹คํ–‰
nohup python train.py > train.log 2>&1 &

ํ•™์Šต์ด ์ง„ํ–‰๋˜๋Š” ๋™์•ˆ ๋ธŒ๋ผ์šฐ์ €์—์„œ http://localhost:5000์œผ๋กœ MLflow UI์— ์ ‘์†ํ•˜๋ฉด ์‹ค์‹œ๊ฐ„์œผ๋กœ ํ•™์Šต ๊ณผ์ •์„ ๋ชจ๋‹ˆํ„ฐ๋งํ•  ์ˆ˜ ์žˆ์–ด! ๐Ÿ“ˆ

โ˜ธ๏ธ ํ™˜๊ฒฝ ๊ตฌ์ถ• Step 4: Kubernetes๋กœ ์Šค์ผ€์ผ ์•„์›ƒํ•˜๊ธฐ

์—ฌ๋Ÿฌ ๋Œ€์˜ ์„œ๋ฒ„์—์„œ ๋Œ€๊ทœ๋ชจ ํ•™์Šต์„ ๋Œ๋ ค์•ผ ํ•œ๋‹ค๋ฉด? ๋ฐ”๋กœ Kubernetes์˜ ์˜์—ญ์ด์•ผ! Docker Compose๋Š” ๋‹จ์ผ ํ˜ธ์ŠคํŠธ์—์„œ๋งŒ ์ž‘๋™ํ•˜์ง€๋งŒ, Kubernetes๋Š” ์—ฌ๋Ÿฌ ์„œ๋ฒ„๋ฅผ ํด๋Ÿฌ์Šคํ„ฐ๋กœ ๋ฌถ์–ด์„œ ๊ด€๋ฆฌํ•  ์ˆ˜ ์žˆ๊ฑฐ๋“ . ๐Ÿš€

โš ๏ธ ์ฃผ์˜: Kubernetes๋Š” ํ•™์Šต ๊ณก์„ ์ด ๊ฐ€ํŒŒ๋ฅธ ํŽธ์ด์•ผ. ์†Œ๊ทœ๋ชจ ํ”„๋กœ์ ํŠธ๋ผ๋ฉด Docker Compose๋กœ๋„ ์ถฉ๋ถ„ํ•˜๋‹ˆ๊นŒ, ์ •๋ง ํ•„์š”ํ•œ ๊ฒฝ์šฐ์—๋งŒ ๋„์ž…ํ•˜๋Š” ๊ฑธ ์ถ”์ฒœํ•ด! ๋งŒ์•ฝ Kubernetes ๊ตฌ์ถ•์ด ๋ง‰๋ง‰ํ•˜๋‹ค๋ฉด ์žฌ๋Šฅ๋„ท์—์„œ DevOps ์ „๋ฌธ๊ฐ€์˜ ์ปจ์„คํŒ…์„ ๋ฐ›์•„๋ณด๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ. ๐ŸŽ“

๐ŸŽฏ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์„ฑ

๋จผ์ € Kubernetes ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๊ตฌ์ถ•ํ•ด์•ผ ํ•ด. ์—ฌ๊ธฐ์„œ๋Š” kubeadm์„ ์‚ฌ์šฉํ•œ ์˜จํ”„๋ ˆ๋ฏธ์Šค ๊ตฌ์ถ• ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ• ๊ฒŒ. ํด๋ผ์šฐ๋“œ(GKE, EKS, AKS)๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค๋ฉด ๋” ๊ฐ„๋‹จํ•˜์ง€! โ˜๏ธ

# ๋ชจ๋“  ๋…ธ๋“œ์—์„œ ์‹คํ–‰ (๋งˆ์Šคํ„ฐ + ์›Œ์ปค)

# 1. ์Šค์™‘ ๋น„ํ™œ์„ฑํ™”
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# 2. ์ปค๋„ ๋ชจ๋“ˆ ๋กœ๋“œ
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# 3. ์‹œ์Šคํ…œ ์„ค์ •
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

# 4. containerd ์„ค์น˜
sudo apt-get update
sudo apt-get install -y containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd

# 5. Kubernetes ํŒจํ‚ค์ง€ ์„ค์น˜
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# 6. ๋งˆ์Šคํ„ฐ ๋…ธ๋“œ ์ดˆ๊ธฐํ™” (๋งˆ์Šคํ„ฐ ๋…ธ๋“œ์—์„œ๋งŒ)
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

# 7. kubectl ์„ค์ • (๋งˆ์Šคํ„ฐ ๋…ธ๋“œ์—์„œ๋งŒ)
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# 8. CNI ํ”Œ๋Ÿฌ๊ทธ์ธ ์„ค์น˜ (Flannel)
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

# 9. ์›Œ์ปค ๋…ธ๋“œ ์กฐ์ธ (์›Œ์ปค ๋…ธ๋“œ์—์„œ)
# kubeadm init ์ถœ๋ ฅ์— ๋‚˜์˜จ join ๋ช…๋ น์–ด ์‹คํ–‰
sudo kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

๐ŸŽฎ NVIDIA GPU Operator ์„ค์น˜

Kubernetes์—์„œ GPU๋ฅผ ์‚ฌ์šฉํ•˜๋ ค๋ฉด NVIDIA GPU Operator๋ฅผ ์„ค์น˜ํ•ด์•ผ ํ•ด. ์ด๊ฒŒ ๊ฐ ๋…ธ๋“œ์— ํ•„์š”ํ•œ ๋“œ๋ผ์ด๋ฒ„์™€ ํ”Œ๋Ÿฌ๊ทธ์ธ์„ ์ž๋™์œผ๋กœ ์„ค์น˜ํ•ด์ค˜! ๐Ÿ”Œ

# Helm ์„ค์น˜
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# NVIDIA GPU Operator ์ €์žฅ์†Œ ์ถ”๊ฐ€
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

# GPU Operator ์„ค์น˜
helm install --wait --generate-name \
    -n gpu-operator --create-namespace \
    nvidia/gpu-operator

# ์„ค์น˜ ํ™•์ธ
kubectl get pods -n gpu-operator
kubectl get nodes -o json | jq '.items[].status.allocatable'

๐Ÿ“ GPU ํ•™์Šต Job ๋ฐฐํฌ

์ด์ œ ์‹ค์ œ๋กœ GPU๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ํ•™์Šต Job์„ Kubernetes์— ๋ฐฐํฌํ•ด๋ณด์ž! ๐Ÿš€

# pytorch-training-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: pytorch-training-job
  namespace: ai-training
spec:
  parallelism: 1
  completions: 1
  backoffLimit: 3
  template:
    metadata:
      labels:
        app: pytorch-training
    spec:
      restartPolicy: OnFailure
      containers:
      - name: pytorch-trainer
        image: ai-pytorch:latest
        command: ["python", "/workspace/train.py"]
        resources:
          limits:
            nvidia.com/gpu: 1  # GPU 1๊ฐœ ์š”์ฒญ
            memory: "32Gi"
            cpu: "8"
          requests:
            nvidia.com/gpu: 1
            memory: "16Gi"
            cpu: "4"
        volumeMounts:
        - name: workspace
          mountPath: /workspace
        - name: datasets
          mountPath: /datasets
        - name: models
          mountPath: /models
        env:
        - name: NVIDIA_VISIBLE_DEVICES
          value: "all"
        - name: NVIDIA_DRIVER_CAPABILITIES
          value: "compute,utility"
      volumes:
      - name: workspace
        persistentVolumeClaim:
          claimName: workspace-pvc
      - name: datasets
        persistentVolumeClaim:
          claimName: datasets-pvc
      - name: models
        persistentVolumeClaim:
          claimName: models-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: workspace-pvc
  namespace: ai-training
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: nfs-client
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: datasets-pvc
  namespace: ai-training
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 500Gi
  storageClassName: nfs-client
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: models-pvc
  namespace: ai-training
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 200Gi
  storageClassName: nfs-client
# Job ๋ฐฐํฌ
kubectl create namespace ai-training
kubectl apply -f pytorch-training-job.yaml

# Job ์ƒํƒœ ํ™•์ธ
kubectl get jobs -n ai-training
kubectl get pods -n ai-training

# ๋กœ๊ทธ ํ™•์ธ
kubectl logs -f <pod-name> -n ai-training

# Job ์‚ญ์ œ
kubectl delete job pytorch-training-job -n ai-training

๐Ÿ”„ ๋ถ„์‚ฐ ํ•™์Šต ๊ตฌ์„ฑ

์—ฌ๋Ÿฌ GPU๋ฅผ ์‚ฌ์šฉํ•œ ๋ถ„์‚ฐ ํ•™์Šต๋„ ๊ฐ€๋Šฅํ•ด! PyTorch DDP(Distributed Data Parallel)๋ฅผ ์‚ฌ์šฉํ•œ ์˜ˆ์‹œ๋ฅผ ๋ณด์—ฌ์ค„๊ฒŒ. ๐Ÿ’ช

# distributed-training-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: distributed-training
  namespace: ai-training
spec:
  parallelism: 4  # 4๊ฐœ์˜ Pod ๋™์‹œ ์‹คํ–‰
  completions: 4
  template:
    metadata:
      labels:
        app: distributed-training
    spec:
      restartPolicy: OnFailure
      containers:
      - name: pytorch-ddp
        image: ai-pytorch:latest
        command:
        - "python"
        - "-m"
        - "torch.distributed.launch"
        - "--nproc_per_node=1"
        - "--nnodes=4"
        - "--node_rank=$(POD_INDEX)"
        - "--master_addr=distributed-training-master"
        - "--master_port=29500"
        - "/workspace/train_distributed.py"
        resources:
          limits:
            nvidia.com/gpu: 1
            memory: "32Gi"
            cpu: "8"
        env:
        - name: POD_INDEX
          valueFrom:
            fieldRef:
              fieldPath: metadata.annotations['batch.kubernetes.io/job-completion-index']
        volumeMounts:
        - name: workspace
          mountPath: /workspace
        - name: datasets
          mountPath: /datasets
      volumes:
      - name: workspace
        persistentVolumeClaim:
          claimName: workspace-pvc
      - name: datasets
        persistentVolumeClaim:
          claimName: datasets-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: distributed-training-master
  namespace: ai-training
spec:
  clusterIP: None
  selector:
    app: distributed-training
  ports:
  - port: 29500
    targetPort: 29500
Kubernetes GPU ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์กฐ Kubernetes Cluster Master Node Control Plane API Server Scheduler Controller Manager etcd GPU Operator Device Plugin Driver Manager Toolkit Manager Worker Nodes Worker Node 1 GPU Pod 1 PyTorch Training GPU Pod 2 TensorFlow Inference Worker Node 2 GPU Pod 3 Distributed Training GPU Pod 4 Model Fine-tuning Worker Node 3 GPU Pod 5 Jupyter Notebook GPU Pod 6 Data Processing Worker Node 4 GPU Pod 7 Model Evaluation GPU Pod 8 Hyperparameter Tuning Shared Storage (NFS/Ceph) Datasets, Models, Checkpoints Persistent Volume Claims

๐ŸŽจ ์‹ค์ „ ํ™œ์šฉ: ๋‹ค์–‘ํ•œ ์‹œ๋‚˜๋ฆฌ์˜ค๋ณ„ ๊ตฌ์„ฑ

์ด๋ก ์€ ์ถฉ๋ถ„ํžˆ ๋ฐฐ์› ์œผ๋‹ˆ, ์ด์ œ ์‹ค์ œ ์ƒํ™ฉ์—์„œ ์–ด๋–ป๊ฒŒ ํ™œ์šฉํ•˜๋Š”์ง€ ์•Œ์•„๋ณด์ž! ์—ฌ๋Ÿฌ ์‹œ๋‚˜๋ฆฌ์˜ค๋ณ„๋กœ ์ตœ์ ์˜ ๊ตฌ์„ฑ์„ ์†Œ๊ฐœํ• ๊ฒŒ. ๐Ÿš€

๐ŸŽฏ ์‹œ๋‚˜๋ฆฌ์˜ค 1: ๊ฐœ์ธ ์—ฐ๊ตฌ์ž / ์†Œ๊ทœ๋ชจ ํŒ€

๐Ÿ“Œ ์š”๊ตฌ์‚ฌํ•ญ: โ€ข GPU 1~2๊ฐœ
โ€ข ๊ฐ„๋‹จํ•œ ๋ชจ๋ธ ์‹คํ—˜ ๋ฐ ํ”„๋กœํ† ํƒ€์ดํ•‘
โ€ข ๋น ๋ฅธ ํ™˜๊ฒฝ ์ „ํ™˜ ํ•„์š”

๐Ÿ’ก ์ถ”์ฒœ ๊ตฌ์„ฑ:
โ€ข Docker + Docker Compose
โ€ข Jupyter Notebook ๊ธฐ๋ฐ˜ ๊ฐœ๋ฐœ
โ€ข ๋กœ์ปฌ ์Šคํ† ๋ฆฌ์ง€ ์‚ฌ์šฉ
โ€ข MLflow๋กœ ์‹คํ—˜ ์ถ”์ 
# ๊ฐ„๋‹จํ•œ docker-compose.yml
version: '3.8'
services:
  jupyter:
    image: ai-pytorch:latest
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - ./work:/workspace
    ports:
      - "8888:8888"
    command: jupyter lab --allow-root
    
  mlflow:
    image: ghcr.io/mlflow/mlflow:v2.7.1
    ports:
      - "5000:5000"
    volumes:
      - ./mlflow:/mlflow
    command: mlflow server --host 0.0.0.0 --backend-store-uri sqlite:///mlflow/mlflow.db

๐Ÿข ์‹œ๋‚˜๋ฆฌ์˜ค 2: ์ค‘์†Œ ๊ธฐ์—… / ์Šคํƒ€ํŠธ์—…

๐Ÿ“Œ ์š”๊ตฌ์‚ฌํ•ญ: โ€ข GPU 4~8๊ฐœ
โ€ข ์—ฌ๋Ÿฌ ํŒ€์›์ด ๋™์‹œ ์‚ฌ์šฉ
โ€ข ์‹คํ—˜ ๊ด€๋ฆฌ ๋ฐ ํ˜‘์—… ํ•„์š”
โ€ข ๋น„์šฉ ํšจ์œจ์ ์ธ ๋ฆฌ์†Œ์Šค ํ™œ์šฉ

๐Ÿ’ก ์ถ”์ฒœ ๊ตฌ์„ฑ:
โ€ข Docker + Kubernetes (๋‹จ์ผ ํด๋Ÿฌ์Šคํ„ฐ)
โ€ข JupyterHub๋กœ ๋ฉ€ํ‹ฐ ์œ ์ € ์ง€์›
โ€ข NFS ๊ณต์œ  ์Šคํ† ๋ฆฌ์ง€
โ€ข GPU ๋ฆฌ์†Œ์Šค ์ฟผํ„ฐ ์„ค์ •
โ€ข MLflow + PostgreSQL
# jupyterhub-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: jupyterhub-config
  namespace: ai-platform
data:
  jupyterhub_config.py: |
    c.JupyterHub.authenticator_class = 'nativeauthenticator.NativeAuthenticator'
    c.Spawner.default_url = '/lab'
    c.Spawner.cpu_limit = 4
    c.Spawner.mem_limit = '16G'
    c.Spawner.extra_resource_limits = {"nvidia.com/gpu": "1"}
    c.Spawner.image = 'ai-pytorch:latest'
    c.Spawner.volume_mounts = [
        {
            'name': 'shared-data',
            'mountPath': '/shared'
        }
    ]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jupyterhub
  namespace: ai-platform
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jupyterhub
  template:
    metadata:
      labels:
        app: jupyterhub
    spec:
      containers:
      - name: jupyterhub
        image: jupyterhub/k8s-hub:3.0.0
        ports:
        - containerPort: 8000
        volumeMounts:
        - name: config
          mountPath: /etc/jupyterhub
        - name: shared-data
          mountPath: /shared
      volumes:
      - name: config
        configMap:
          name: jupyterhub-config
      - name: shared-data
        persistentVolumeClaim:
          claimName: shared-data-pvc

๐Ÿญ ์‹œ๋‚˜๋ฆฌ์˜ค 3: ๋Œ€๊ธฐ์—… / ์—ฐ๊ตฌ์†Œ

๐Ÿ“Œ ์š”๊ตฌ์‚ฌํ•ญ: โ€ข GPU ์ˆ˜์‹ญ~์ˆ˜๋ฐฑ ๊ฐœ
โ€ข ๋Œ€๊ทœ๋ชจ ๋ถ„์‚ฐ ํ•™์Šต
โ€ข ๋†’์€ ๊ฐ€์šฉ์„ฑ ๋ฐ ์•ˆ์ •์„ฑ
โ€ข ์„ธ๋ฐ€ํ•œ ๋ฆฌ์†Œ์Šค ๊ด€๋ฆฌ ๋ฐ ๋ชจ๋‹ˆํ„ฐ๋ง
โ€ข ๋ณด์•ˆ ๋ฐ ๊ถŒํ•œ ๊ด€๋ฆฌ

๐Ÿ’ก ์ถ”์ฒœ ๊ตฌ์„ฑ:
โ€ข Kubernetes ๋ฉ€ํ‹ฐ ํด๋Ÿฌ์Šคํ„ฐ
โ€ข Kubeflow๋กœ ML ํŒŒ์ดํ”„๋ผ์ธ ๊ตฌ์ถ•
โ€ข Distributed Storage (Ceph/GlusterFS)
โ€ข Prometheus + Grafana ๋ชจ๋‹ˆํ„ฐ๋ง
โ€ข RBAC ๊ธฐ๋ฐ˜ ๊ถŒํ•œ ๊ด€๋ฆฌ
โ€ข CI/CD ํŒŒ์ดํ”„๋ผ์ธ ํ†ตํ•ฉ
# kubeflow-pipeline-example.yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: distributed-training-pipeline-
  namespace: kubeflow
spec:
  entrypoint: training-pipeline
  arguments:
    parameters:
    - name: learning-rate
      value: "0.001"
    - name: batch-size
      value: "64"
    - name: num-gpus
      value: "8"
  
  templates:
  - name: training-pipeline
    steps:
    - - name: data-preprocessing
        template: preprocess-data
    - - name: distributed-training
        template: train-model
        arguments:
          parameters:
          - name: learning-rate
            value: "{{workflow.parameters.learning-rate}}"
          - name: batch-size
            value: "{{workflow.parameters.batch-size}}"
          - name: num-gpus
            value: "{{workflow.parameters.num-gpus}}"
    - - name: model-evaluation
        template: evaluate-model
    - - name: model-deployment
        template: deploy-model
  
  - name: preprocess-data
    container:
      image: ai-pytorch:latest
      command: [python]
      args: ["/workspace/preprocess.py"]
      resources:
        limits:
          nvidia.com/gpu: 1
          memory: 32Gi
  
  - name: train-model
    inputs:
      parameters:
      - name: learning-rate
      - name: batch-size
      - name: num-gpus
    resource:
      action: create
      manifest: |
        apiVersion: "kubeflow.org/v1"
        kind: PyTorchJob
        metadata:
          name: distributed-training
          namespace: kubeflow
        spec:
          pytorchReplicaSpecs:
            Master:
              replicas: 1
              restartPolicy: OnFailure
              template:
                spec:
                  containers:
                  - name: pytorch
                    image: ai-pytorch:latest
                    command:
                    - python
                    - -m
                    - torch.distributed.launch
                    - --nproc_per_node={{inputs.parameters.num-gpus}}
                    - /workspace/train_distributed.py
                    - --lr={{inputs.parameters.learning-rate}}
                    - --batch-size={{inputs.parameters.batch-size}}
                    resources:
                      limits:
                        nvidia.com/gpu: {{inputs.parameters.num-gpus}}
                        memory: 128Gi
            Worker:
              replicas: 7
              restartPolicy: OnFailure
              template:
                spec:
                  containers:
                  - name: pytorch
                    image: ai-pytorch:latest
                    command:
                    - python
                    - -m
                    - torch.distributed.launch
                    - --nproc_per_node={{inputs.parameters.num-gpus}}
                    - /workspace/train_distributed.py
                    - --lr={{inputs.parameters.learning-rate}}
                    - --batch-size={{inputs.parameters.batch-size}}
                    resources:
                      limits:
                        nvidia.com/gpu: {{inputs.parameters.num-gpus}}
                        memory: 128Gi
  
  - name: evaluate-model
    container:
      image: ai-pytorch:latest
      command: [python]
      args: ["/workspace/evaluate.py"]
      resources:
        limits:
          nvidia.com/gpu: 1
  
  - name: deploy-model
    container:
      image: ai-pytorch:latest
      command: [python]
      args: ["/workspace/deploy.py"]

๐Ÿ” ๋ชจ๋‹ˆํ„ฐ๋ง ๋ฐ ์ตœ์ ํ™”

GPU ํ™˜๊ฒฝ์„ ๊ตฌ์ถ•ํ–ˆ๋‹ค๋ฉด, ์ด์ œ ์ œ๋Œ€๋กœ ํ™œ์šฉํ•˜๊ณ  ์žˆ๋Š”์ง€ ๋ชจ๋‹ˆํ„ฐ๋งํ•˜๋Š” ๊ฒŒ ์ค‘์š”ํ•ด! ๋น„์‹ผ GPU๋ฅผ ๋†€๋ฆฌ๋ฉด ์•ˆ ๋˜์ž–์•„? ๐Ÿ˜…

๐Ÿ“Š GPU ์‚ฌ์šฉ๋ฅ  ๋ชจ๋‹ˆํ„ฐ๋ง

Prometheus์™€ Grafana๋ฅผ ์‚ฌ์šฉํ•ด์„œ GPU ๋ฉ”ํŠธ๋ฆญ์„ ์‹ค์‹œ๊ฐ„์œผ๋กœ ๋ชจ๋‹ˆํ„ฐ๋งํ•  ์ˆ˜ ์žˆ์–ด. NVIDIA์—์„œ ์ œ๊ณตํ•˜๋Š” DCGM Exporter๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ๋ผ! ๐Ÿ“ˆ

# dcgm-exporter ์„ค์น˜ (Kubernetes)
helm repo add gpu-helm-charts https://nvidia.github.io/dcgm-exporter/helm-charts
helm repo update
helm install dcgm-exporter gpu-helm-charts/dcgm-exporter \
    --namespace monitoring \
    --create-namespace

# Prometheus ์„ค์ •์— ์ถ”๊ฐ€
# prometheus-config.yaml
scrape_configs:
  - job_name: 'dcgm-exporter'
    static_configs:
    - targets: ['dcgm-exporter.monitoring.svc:9400']

Grafana ๋Œ€์‹œ๋ณด๋“œ์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋Š” ์ฃผ์š” ๋ฉ”ํŠธ๋ฆญ:

๋ฉ”ํŠธ๋ฆญ ์„ค๋ช… ์ตœ์  ๋ฒ”์œ„
GPU Utilization GPU ์ฝ”์–ด ์‚ฌ์šฉ๋ฅ  80% ์ด์ƒ
Memory Utilization GPU ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋ฅ  70-90%
Temperature GPU ์˜จ๋„ 60-80ยฐC
Power Usage ์ „๋ ฅ ์†Œ๋น„๋Ÿ‰ TDP์˜ 80-95%
SM Clock ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฉ€ํ‹ฐํ”„๋กœ์„ธ์„œ ํด๋Ÿญ ์ตœ๋Œ€ ํด๋Ÿญ ์œ ์ง€
Memory Clock ๋ฉ”๋ชจ๋ฆฌ ํด๋Ÿญ ์ตœ๋Œ€ ํด๋Ÿญ ์œ ์ง€

โšก ์„ฑ๋Šฅ ์ตœ์ ํ™” ํŒ

๐Ÿ’ก GPU ํ™œ์šฉ๋ฅ ์„ ๋†’์ด๋Š” ๋ฐฉ๋ฒ•: 1. ๋ฐฐ์น˜ ํฌ๊ธฐ ์ตœ์ ํ™”
GPU ๋ฉ”๋ชจ๋ฆฌ๋ฅผ ์ตœ๋Œ€ํ•œ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฐ์น˜ ํฌ๊ธฐ๋ฅผ ์ฐพ์•„์•ผ ํ•ด. ๋„ˆ๋ฌด ์ž‘์œผ๋ฉด GPU๊ฐ€ ๋†€๊ณ , ๋„ˆ๋ฌด ํฌ๋ฉด OOM(Out of Memory) ์—๋Ÿฌ๊ฐ€ ๋‚˜์ง€.

2. Mixed Precision Training
FP16(๋ฐ˜์ •๋ฐ€๋„)์„ ์‚ฌ์šฉํ•˜๋ฉด ๋ฉ”๋ชจ๋ฆฌ ์‚ฌ์šฉ๋Ÿ‰์„ ์ค„์ด๊ณ  ํ•™์Šต ์†๋„๋ฅผ ๋†’์ผ ์ˆ˜ ์žˆ์–ด. PyTorch์˜ ๊ฒฝ์šฐ torch.cuda.amp๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ๋ผ!

3. DataLoader ์ตœ์ ํ™”
num_workers๋ฅผ ์ ์ ˆํžˆ ์„ค์ •ํ•ด์„œ ๋ฐ์ดํ„ฐ ๋กœ๋”ฉ์ด ๋ณ‘๋ชฉ์ด ๋˜์ง€ ์•Š๋„๋ก ํ•ด์•ผ ํ•ด. ๋ณดํ†ต CPU ์ฝ”์–ด ์ˆ˜์˜ ์ ˆ๋ฐ˜ ์ •๋„๊ฐ€ ์ ๋‹นํ•ด.

4. Gradient Accumulation
๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ๋ถ€์กฑํ•  ๋•Œ ์—ฌ๋Ÿฌ ์Šคํ…์— ๊ฑธ์ณ ๊ทธ๋ž˜๋””์–ธํŠธ๋ฅผ ๋ˆ„์ ํ•˜๋ฉด ํฐ ๋ฐฐ์น˜ ํฌ๊ธฐ ํšจ๊ณผ๋ฅผ ๋‚ผ ์ˆ˜ ์žˆ์–ด.

5. Pinned Memory
pin_memory=True ์˜ต์…˜์„ ์‚ฌ์šฉํ•˜๋ฉด CPU-GPU ๊ฐ„ ๋ฐ์ดํ„ฐ ์ „์†ก ์†๋„๊ฐ€ ๋นจ๋ผ์ ธ.
# ์ตœ์ ํ™”๋œ ํ•™์Šต ์ฝ”๋“œ ์˜ˆ์‹œ
import torch
from torch.cuda.amp import autocast, GradScaler

# Mixed Precision Training ์„ค์ •
scaler = GradScaler()

# DataLoader ์ตœ์ ํ™”
train_loader = DataLoader(
    dataset,
    batch_size=64,
    num_workers=8,  # CPU ์ฝ”์–ด ์ˆ˜์— ๋งž๊ฒŒ ์กฐ์ •
    pin_memory=True,  # ๋น ๋ฅธ GPU ์ „์†ก
    persistent_workers=True  # ์›Œ์ปค ์žฌ์‚ฌ์šฉ
)

# Gradient Accumulation ์„ค์ •
accumulation_steps = 4

model.train()
optimizer.zero_grad()

for i, (inputs, labels) in enumerate(train_loader):
    inputs, labels = inputs.cuda(), labels.cuda()
    
    # Mixed Precision Forward Pass
    with autocast():
        outputs = model(inputs)
        loss = criterion(outputs, labels) / accumulation_steps
    
    # Backward Pass
    scaler.scale(loss).backward()
    
    # Gradient Accumulation
    if (i + 1) % accumulation_steps == 0:
        scaler.step(optimizer)
        scaler.update()
        optimizer.zero_grad()

๐Ÿ› ํŠธ๋Ÿฌ๋ธ”์ŠˆํŒ… ๊ฐ€์ด๋“œ

โš ๏ธ ์ž์ฃผ ๋ฐœ์ƒํ•˜๋Š” ๋ฌธ์ œ์™€ ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•: ๋ฌธ์ œ 1: "CUDA out of memory" ์—๋Ÿฌ
์›์ธ: ๋ฐฐ์น˜ ํฌ๊ธฐ๊ฐ€ ๋„ˆ๋ฌด ํฌ๊ฑฐ๋‚˜ ๋ชจ๋ธ์ด ๋„ˆ๋ฌด ํผ
ํ•ด๊ฒฐ: ๋ฐฐ์น˜ ํฌ๊ธฐ ์ค„์ด๊ธฐ, Gradient Checkpointing ์‚ฌ์šฉ, Mixed Precision Training ์ ์šฉ

๋ฌธ์ œ 2: GPU ์‚ฌ์šฉ๋ฅ ์ด ๋‚ฎ์Œ (30% ์ดํ•˜)
์›์ธ: ๋ฐ์ดํ„ฐ ๋กœ๋”ฉ์ด ๋ณ‘๋ชฉ, ๋ฐฐ์น˜ ํฌ๊ธฐ๊ฐ€ ๋„ˆ๋ฌด ์ž‘์Œ
ํ•ด๊ฒฐ: num_workers ์ฆ๊ฐ€, ๋ฐฐ์น˜ ํฌ๊ธฐ ์ฆ๊ฐ€, ๋ฐ์ดํ„ฐ ์ „์ฒ˜๋ฆฌ ์ตœ์ ํ™”

๋ฌธ์ œ 3: "RuntimeError: CUDA error: device-side assert triggered"
์›์ธ: ์ž˜๋ชป๋œ ๋ ˆ์ด๋ธ” ์ธ๋ฑ์Šค, ํ…์„œ ํฌ๊ธฐ ๋ถˆ์ผ์น˜
ํ•ด๊ฒฐ: ๋ ˆ์ด๋ธ” ๋ฒ”์œ„ ํ™•์ธ, ํ…์„œ shape ๋””๋ฒ„๊น…

๋ฌธ์ œ 4: ์ปจํ…Œ์ด๋„ˆ์—์„œ GPU ์ธ์‹ ์•ˆ ๋จ
์›์ธ: NVIDIA Container Toolkit ๋ฏธ์„ค์น˜, Docker ์„ค์ • ์˜ค๋ฅ˜
ํ•ด๊ฒฐ: nvidia-ctk runtime configure ์žฌ์‹คํ–‰, Docker ์žฌ์‹œ์ž‘

๋ฌธ์ œ 5: ๋ฉ€ํ‹ฐ GPU ํ•™์Šต ์‹œ ์†๋„๊ฐ€ ์„ ํ˜•์ ์œผ๋กœ ์ฆ๊ฐ€ํ•˜์ง€ ์•Š์Œ
์›์ธ: ํ†ต์‹  ์˜ค๋ฒ„ํ—ค๋“œ, ๋ฐฐ์น˜ ํฌ๊ธฐ ๋ถ€์กฑ
ํ•ด๊ฒฐ: ๋ฐฐ์น˜ ํฌ๊ธฐ ์ฆ๊ฐ€, NCCL ์ตœ์ ํ™”, ๋ชจ๋ธ ๋ณ‘๋ ฌํ™” ๊ณ ๋ ค

๐Ÿ” ๋ณด์•ˆ ๋ฐ ๋ฒ ์ŠคํŠธ ํ”„๋ž™ํ‹ฐ์Šค

GPU ํ™˜๊ฒฝ์€ ๋น„์šฉ์ด ๋งŽ์ด ๋“ค๊ธฐ ๋•Œ๋ฌธ์— ๋ณด์•ˆ๊ณผ ๋ฆฌ์†Œ์Šค ๊ด€๋ฆฌ๊ฐ€ ์ •๋ง ์ค‘์š”ํ•ด! ์‹ค์ˆ˜๋กœ ๋ˆ„๊ตฐ๊ฐ€ ๋ฌดํ•œ ๋ฃจํ”„๋ฅผ ๋Œ๋ฆฌ๊ฑฐ๋‚˜ ํ•˜๋ฉด... ๐Ÿ˜ฑ

๐Ÿ›ก๏ธ ๋ณด์•ˆ ์ฒดํฌ๋ฆฌ์ŠคํŠธ

โœ… ํ•„์ˆ˜ ๋ณด์•ˆ ์„ค์ •: 1. ๋„คํŠธ์›Œํฌ ๊ฒฉ๋ฆฌ
โ€ข ๋ฐฉํ™”๋ฒฝ ์„ค์ •์œผ๋กœ ํ•„์š”ํ•œ ํฌํŠธ๋งŒ ๊ฐœ๋ฐฉ
โ€ข VPN ๋˜๋Š” Private Network ์‚ฌ์šฉ
โ€ข Kubernetes NetworkPolicy๋กœ Pod ๊ฐ„ ํ†ต์‹  ์ œํ•œ

2. ์ธ์ฆ ๋ฐ ๊ถŒํ•œ ๊ด€๋ฆฌ
โ€ข Jupyter ๋น„๋ฐ€๋ฒˆํ˜ธ ์„ค์ • ํ•„์ˆ˜
โ€ข RBAC(Role-Based Access Control) ๊ตฌ์„ฑ
โ€ข SSH ํ‚ค ๊ธฐ๋ฐ˜ ์ธ์ฆ ์‚ฌ์šฉ
โ€ข ์ •๊ธฐ์ ์ธ ๋น„๋ฐ€๋ฒˆํ˜ธ ๋ณ€๊ฒฝ

3. ๋ฆฌ์†Œ์Šค ์ฟผํ„ฐ
โ€ข Kubernetes ResourceQuota๋กœ ๋„ค์ž„์ŠคํŽ˜์ด์Šค๋ณ„ ์ œํ•œ
โ€ข LimitRange๋กœ Pod๋ณ„ ์ตœ๋Œ€ ๋ฆฌ์†Œ์Šค ์„ค์ •
โ€ข GPU ์‚ฌ์šฉ ์‹œ๊ฐ„ ๋ชจ๋‹ˆํ„ฐ๋ง ๋ฐ ์•Œ๋ฆผ

4. ์ปจํ…Œ์ด๋„ˆ ๋ณด์•ˆ
โ€ข ์ตœ์†Œ ๊ถŒํ•œ ์›์น™ (non-root ์‚ฌ์šฉ์ž)
โ€ข ์ด๋ฏธ์ง€ ์ทจ์•ฝ์  ์Šค์บ” (Trivy, Clair)
โ€ข ์‹ ๋ขฐํ•  ์ˆ˜ ์žˆ๋Š” ๋ฒ ์ด์Šค ์ด๋ฏธ์ง€ ์‚ฌ์šฉ
โ€ข ์ •๊ธฐ์ ์ธ ์ด๋ฏธ์ง€ ์—…๋ฐ์ดํŠธ

5. ๋ฐ์ดํ„ฐ ๋ณดํ˜ธ
โ€ข ์ค‘์š” ๋ฐ์ดํ„ฐ ์•”ํ˜ธํ™”
โ€ข ์ •๊ธฐ์ ์ธ ๋ฐฑ์—…
โ€ข ์ ‘๊ทผ ๋กœ๊ทธ ๊ธฐ๋ก ๋ฐ ๊ฐ์‚ฌ
# Kubernetes ResourceQuota ์˜ˆ์‹œ
apiVersion: v1
kind: ResourceQuota
metadata:
  name: gpu-quota
  namespace: ai-team-a
spec:
  hard:
    requests.nvidia.com/gpu: "4"  # ์ตœ๋Œ€ 4๊ฐœ GPU
    limits.nvidia.com/gpu: "4"
    requests.memory: "128Gi"
    limits.memory: "256Gi"
    requests.cpu: "32"
    limits.cpu: "64"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: gpu-limit-range
  namespace: ai-team-a
spec:
  limits:
  - max:
      nvidia.com/gpu: "2"  # Pod๋‹น ์ตœ๋Œ€ 2๊ฐœ GPU
      memory: "64Gi"
      cpu: "16"
    min:
      memory: "1Gi"
      cpu: "1"
    type: Container

๐Ÿ“‹ ์šด์˜ ๋ฒ ์ŠคํŠธ ํ”„๋ž™ํ‹ฐ์Šค

๐Ÿ’ก ํšจ์œจ์ ์ธ GPU ํ™˜๊ฒฝ ์šด์˜ ํŒ: 1. ์ด๋ฏธ์ง€ ๊ด€๋ฆฌ
โ€ข ๊ณตํ†ต ๋ฒ ์ด์Šค ์ด๋ฏธ์ง€ ๋งŒ๋“ค์–ด์„œ ์žฌ์‚ฌ์šฉ
โ€ข ๋ฒ„์ „ ํƒœ๊ทธ ๋ช…ํ™•ํžˆ ๊ด€๋ฆฌ (latest ํƒœ๊ทธ ์ง€์–‘)
โ€ข Private Registry ์šด์˜ (Harbor, GitLab Registry)
โ€ข ๋ถˆํ•„์š”ํ•œ ์ด๋ฏธ์ง€ ์ •๊ธฐ์ ์œผ๋กœ ์ •๋ฆฌ

2. ์‹คํ—˜ ๊ด€๋ฆฌ
โ€ข MLflow, Weights & Biases ๋“ฑ์œผ๋กœ ์‹คํ—˜ ์ถ”์ 
โ€ข Git์œผ๋กœ ์ฝ”๋“œ ๋ฒ„์ „ ๊ด€๋ฆฌ
โ€ข DVC๋กœ ๋ฐ์ดํ„ฐ์…‹ ๋ฒ„์ „ ๊ด€๋ฆฌ
โ€ข ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ ํ™˜๊ฒฝ ๊ตฌ์„ฑ (requirements.txt, environment.yml)

3. ๋น„์šฉ ์ตœ์ ํ™”
โ€ข Spot Instance ํ™œ์šฉ (ํด๋ผ์šฐ๋“œ)
โ€ข ์ž๋™ ์Šค์ผ€์ผ๋ง ์„ค์ •
โ€ข ๋ฏธ์‚ฌ์šฉ ๋ฆฌ์†Œ์Šค ์ž๋™ ์ •๋ฆฌ
โ€ข GPU ๊ณต์œ  (MIG, Time-slicing)

4. ํ˜‘์—… ํ™˜๊ฒฝ
โ€ข ๊ณต์œ  ๋ฐ์ดํ„ฐ์…‹ ๋””๋ ‰ํ† ๋ฆฌ ๊ตฌ์„ฑ
โ€ข ๋ชจ๋ธ ์ฒดํฌํฌ์ธํŠธ ์ค‘์•™ ์ €์žฅ์†Œ
โ€ข ๋ฌธ์„œํ™” ๋ฐ ์œ„ํ‚ค ์šด์˜
โ€ข ์ •๊ธฐ์ ์ธ ํŒ€ ๋ฆฌ๋ทฐ ๋ฐ ์ง€์‹ ๊ณต์œ 

5. ์žฅ์•  ๋Œ€์‘
โ€ข ์ž๋™ ์žฌ์‹œ์ž‘ ์ •์ฑ… ์„ค์ •
โ€ข ์ฒดํฌํฌ์ธํŠธ ์ •๊ธฐ ์ €์žฅ
โ€ข ์•Œ๋ฆผ ์‹œ์Šคํ…œ ๊ตฌ์ถ• (Slack, Email)
โ€ข ์žฅ์•  ๋Œ€์‘ ๋งค๋‰ด์–ผ ์ž‘์„ฑ

๐ŸŽ“ ํ•™์Šต ๋ฆฌ์†Œ์Šค ๋ฐ ์ปค๋ฎค๋‹ˆํ‹ฐ

GPU ์ปจํ…Œ์ด๋„ˆ ํ™˜๊ฒฝ ๊ตฌ์ถ•์€ ๊ณ„์† ๋ฐœ์ „ํ•˜๋Š” ๋ถ„์•ผ์•ผ. ์ตœ์‹  ์ •๋ณด๋ฅผ ๋”ฐ๋ผ๊ฐ€๊ณ  ๋ฌธ์ œ ํ•ด๊ฒฐ์„ ์œ„ํ•ด ์ข‹์€ ๋ฆฌ์†Œ์Šค๋“ค์„ ์†Œ๊ฐœํ• ๊ฒŒ! ๐Ÿ“š

๐Ÿ“– ๊ณต์‹ ๋ฌธ์„œ

๐Ÿ”— ํ•„์ˆ˜ ์ฐธ๊ณ  ์ž๋ฃŒ: NVIDIA ๊ด€๋ จ
โ€ข NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/
โ€ข CUDA Documentation: https://docs.nvidia.com/cuda/
โ€ข NVIDIA GPU Operator: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/

Docker ๊ด€๋ จ
โ€ข Docker Documentation: https://docs.docker.com/
โ€ข Docker Compose: https://docs.docker.com/compose/

Kubernetes ๊ด€๋ จ
โ€ข Kubernetes Documentation: https://kubernetes.io/docs/
โ€ข Kubeflow: https://www.kubeflow.org/docs/
โ€ข KubeVirt: https://kubevirt.io/

ML/DL ํ”„๋ ˆ์ž„์›Œํฌ
โ€ข PyTorch: https://pytorch.org/docs/
โ€ข TensorFlow: https://www.tensorflow.org/guide
โ€ข Hugging Face: https://huggingface.co/docs

๐Ÿ‘ฅ ์ปค๋ฎค๋‹ˆํ‹ฐ ๋ฐ ํฌ๋Ÿผ

๋ง‰ํžˆ๋Š” ๋ถ€๋ถ„์ด ์žˆ์„ ๋•Œ ๋„์›€์„ ๋ฐ›์„ ์ˆ˜ ์žˆ๋Š” ๊ณณ๋“ค์ด์•ผ:

ํ”Œ๋žซํผ ํŠน์ง• ์ถ”์ฒœ ๋Œ€์ƒ
Stack Overflow ๊ธฐ์ˆ ์  ์งˆ๋ฌธ ๋ฐ ๋‹ต๋ณ€ ๊ตฌ์ฒด์ ์ธ ์—๋Ÿฌ ํ•ด๊ฒฐ
GitHub Issues ์˜คํ”ˆ์†Œ์Šค ํ”„๋กœ์ ํŠธ ์ด์Šˆ ๋ฒ„๊ทธ ๋ฆฌํฌํŠธ, ๊ธฐ๋Šฅ ์š”์ฒญ
Reddit (r/MachineLearning) ์ตœ์‹  ํŠธ๋ Œ๋“œ ๋ฐ ๋…ผ์˜ ์—…๊ณ„ ๋™ํ–ฅ ํŒŒ์•…
Discord (AI/ML ์„œ๋ฒ„๋“ค) ์‹ค์‹œ๊ฐ„ ์ฑ„ํŒ… ๋ฐ ํ˜‘์—… ๋น ๋ฅธ ํ”ผ๋“œ๋ฐฑ
NVIDIA Developer Forums NVIDIA ์ œํ’ˆ ๊ด€๋ จ GPU ํŠนํ™” ๋ฌธ์ œ
Kubernetes Slack K8s ์ปค๋ฎค๋‹ˆํ‹ฐ ์˜ค์ผ€์ŠคํŠธ๋ ˆ์ด์…˜ ๊ด€๋ จ
๐Ÿ’ก ์žฌ๋Šฅ๋„ท ํ™œ์šฉํ•˜๊ธฐ: GPU ํ™˜๊ฒฝ ๊ตฌ์ถ•์ด๋‚˜ AI ๊ฐœ๋ฐœ์ด ์ฒ˜์Œ์ด๋ผ๋ฉด ํ˜ผ์ž ๊ณ ๋ฏผํ•˜์ง€ ๋ง๊ณ  ์ „๋ฌธ๊ฐ€์˜ ๋„์›€์„ ๋ฐ›๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ! ์žฌ๋Šฅ๋„ท(https://www.jaenung.net)์—์„œ๋Š” ํด๋ผ์šฐ๋“œ/DevOps ์ „๋ฌธ๊ฐ€๋“ค์ด 1:1 ์ปจ์„คํŒ…, ํ™˜๊ฒฝ ๊ตฌ์ถ• ๋Œ€ํ–‰, ํŠธ๋Ÿฌ๋ธ”์ŠˆํŒ… ๋“ฑ ๋‹ค์–‘ํ•œ ์„œ๋น„์Šค๋ฅผ ์ œ๊ณตํ•˜๊ณ  ์žˆ์–ด. ์‹œํ–‰์ฐฉ์˜ค๋ฅผ ์ค„์ด๊ณ  ๋น ๋ฅด๊ฒŒ ํ”„๋กœ์ ํŠธ๋ฅผ ์ง„ํ–‰ํ•˜๊ณ  ์‹ถ๋‹ค๋ฉด ํ•œ๋ฒˆ ์ฐพ์•„๋ณด๋Š” ๊ฑธ ์ถ”์ฒœํ•ด! ๐Ÿš€

๐ŸŽฏ ๋งˆ๋ฌด๋ฆฌํ•˜๋ฉฐ

์ž, ์—ฌ๊ธฐ๊นŒ์ง€ GPU ์ปจํ…Œ์ด๋„ˆ ํ™˜๊ฒฝ ๊ตฌ์ถ•์— ๋Œ€ํ•ด ์•Œ์•„๋ดค์–ด! ์ฒ˜์Œ์—๋Š” ๋ณต์žกํ•ด ๋ณด์ผ ์ˆ˜ ์žˆ์ง€๋งŒ, ํ•œ ๋ฒˆ ์ œ๋Œ€๋กœ ๊ตฌ์ถ•ํ•ด๋‘๋ฉด ์ •๋ง ํŽธํ•˜๊ฒŒ AI ๊ฐœ๋ฐœ์„ ํ•  ์ˆ˜ ์žˆ์–ด. ๐ŸŽ‰

ํ•ต์‹ฌ ๋‚ด์šฉ์„ ๋‹ค์‹œ ํ•œ๋ฒˆ ์ •๋ฆฌํ•ด๋ณผ๊ฒŒ:

๐ŸŽฏ ํ•ต์‹ฌ ์š”์•ฝ: 1. ๊ธฐ๋ณธ ๊ตฌ์„ฑ
โ€ข NVIDIA Driver + Docker + NVIDIA Container Toolkit
โ€ข ์ด 3๊ฐ€์ง€๋งŒ ์žˆ์œผ๋ฉด ๊ธฐ๋ณธ์ ์ธ GPU ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ ๊ฐ€๋Šฅ

2. ๊ฐœ๋ฐœ ํ™˜๊ฒฝ
โ€ข Dockerfile๋กœ ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ ํ™˜๊ฒฝ ๊ตฌ์„ฑ
โ€ข Docker Compose๋กœ ๋ฉ€ํ‹ฐ ์ปจํ…Œ์ด๋„ˆ ๊ด€๋ฆฌ
โ€ข Jupyter Notebook์œผ๋กœ ์ธํ„ฐ๋ž™ํ‹ฐ๋ธŒ ๊ฐœ๋ฐœ

3. ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ
โ€ข Kubernetes๋กœ ๋Œ€๊ทœ๋ชจ ํด๋Ÿฌ์Šคํ„ฐ ๊ด€๋ฆฌ
โ€ข GPU Operator๋กœ ์ž๋™ํ™”๋œ GPU ๊ด€๋ฆฌ
โ€ข MLflow, Kubeflow๋กœ ML ํŒŒ์ดํ”„๋ผ์ธ ๊ตฌ์ถ•

4. ์ตœ์ ํ™”
โ€ข Mixed Precision Training์œผ๋กœ ์†๋„ ํ–ฅ์ƒ
โ€ข ์ ์ ˆํ•œ ๋ฐฐ์น˜ ํฌ๊ธฐ์™€ DataLoader ์„ค์ •
โ€ข ๋ชจ๋‹ˆํ„ฐ๋ง์œผ๋กœ ๋ฆฌ์†Œ์Šค ํ™œ์šฉ๋ฅ  ์ถ”์ 

5. ๋ณด์•ˆ ๋ฐ ์šด์˜
โ€ข ๋ฆฌ์†Œ์Šค ์ฟผํ„ฐ๋กœ ๊ณต์ •ํ•œ ๋ถ„๋ฐฐ
โ€ข ์ •๊ธฐ์ ์ธ ๋ฐฑ์—… ๋ฐ ์—…๋ฐ์ดํŠธ
โ€ข ๋ฌธ์„œํ™” ๋ฐ ํŒ€ ํ˜‘์—… ์ฒด๊ณ„ ๊ตฌ์ถ•

GPU ํ™˜๊ฒฝ ๊ตฌ์ถ•์€ ํ•œ ๋ฒˆ์— ์™„๋ฒฝํ•˜๊ฒŒ ํ•˜๊ธฐ ์–ด๋ ค์›Œ. ์ฒ˜์Œ์—๋Š” ๊ฐ„๋‹จํ•˜๊ฒŒ ์‹œ์ž‘ํ•ด์„œ ํ•„์š”์— ๋”ฐ๋ผ ์ ์ง„์ ์œผ๋กœ ํ™•์žฅํ•˜๋Š” ๊ฒŒ ์ข‹์•„. Docker๋งŒ์œผ๋กœ ์‹œ์ž‘ํ–ˆ๋‹ค๊ฐ€ ํŒ€์ด ์ปค์ง€๋ฉด Kubernetes๋ฅผ ๋„์ž…ํ•˜๊ณ , ์‹คํ—˜์ด ๋งŽ์•„์ง€๋ฉด MLflow๋ฅผ ์ถ”๊ฐ€ํ•˜๋Š” ์‹์œผ๋กœ ๋ง์ด์•ผ! ๐Ÿ“ˆ

๊ทธ๋ฆฌ๊ณ  ๊ฐ€์žฅ ์ค‘์š”ํ•œ ๊ฑด, ์‹ค์ œ๋กœ ํ•ด๋ณด๋Š” ๊ฒƒ์ด์•ผ! ์ด ๊ธ€์„ ์ฝ๋Š” ๊ฒƒ๋งŒ์œผ๋กœ๋Š” ๋ถ€์กฑํ•ด. ์ง์ ‘ ์„œ๋ฒ„์— ์ ‘์†ํ•ด์„œ ๋ช…๋ น์–ด๋ฅผ ์ž…๋ ฅํ•˜๊ณ , ์—๋Ÿฌ๋ฅผ ๋งŒ๋‚˜๊ณ , ํ•ด๊ฒฐํ•˜๋ฉด์„œ ๋ฐฐ์šฐ๋Š” ๊ฑฐ์ง€. ์ฒ˜์Œ์—๋Š” ์‹ค์ˆ˜๋„ ๋งŽ์ด ํ•˜๊ณ  ์‹œ๊ฐ„๋„ ์˜ค๋ž˜ ๊ฑธ๋ฆฌ๊ฒ ์ง€๋งŒ, ๊ทธ๊ฒŒ ๋‹ค ๊ฒฝํ—˜์ด ๋˜๋Š” ๊ฑฐ์•ผ! ๐Ÿ’ช

ํ˜น์‹œ ๋ง‰ํžˆ๋Š” ๋ถ€๋ถ„์ด ์žˆ๊ฑฐ๋‚˜ ๋„์›€์ด ํ•„์š”ํ•˜๋ฉด ์ปค๋ฎค๋‹ˆํ‹ฐ์— ์งˆ๋ฌธํ•˜๊ฑฐ๋‚˜, ์žฌ๋Šฅ๋„ท ๊ฐ™์€ ํ”Œ๋žซํผ์—์„œ ์ „๋ฌธ๊ฐ€์˜ ๋„์›€์„ ๋ฐ›๋Š” ๊ฒƒ๋„ ์ข‹์€ ๋ฐฉ๋ฒ•์ด์•ผ. ํ˜ผ์ž ๋™๋™๋Œ€๋ฉด์„œ ์ผ์ฃผ์ผ ๊ฑธ๋ฆด ์ผ์„ ์ „๋ฌธ๊ฐ€์™€ ํ•จ๊ป˜ํ•˜๋ฉด ํ•˜๋ฃจ ๋งŒ์— ํ•ด๊ฒฐํ•  ์ˆ˜๋„ ์žˆ๊ฑฐ๋“ ! ๐Ÿ˜Š

์ž, ์ด์ œ ๋‹น์‹ ์˜ GPU ์ปจํ…Œ์ด๋„ˆ ํ™˜๊ฒฝ์„ ๊ตฌ์ถ•ํ•  ์ฐจ๋ก€์•ผ! ํ™”์ดํŒ…! ๐Ÿš€โœจ

๐ŸŽ‰ Happy GPU Computing! ๐ŸŽ‰

์ด ๊ธ€์ด ๋„์›€์ด ๋˜์—ˆ๋‹ค๋ฉด, ๋‹น์‹ ์˜ AI ํ”„๋กœ์ ํŠธ๊ฐ€ ์„ฑ๊ณตํ•˜๊ธฐ๋ฅผ ์‘์›ํ•ด!
๊ถ๊ธˆํ•œ ์ ์ด ์žˆ๋‹ค๋ฉด ์–ธ์ œ๋“  ์ปค๋ฎค๋‹ˆํ‹ฐ์— ์งˆ๋ฌธํ•˜๊ฑฐ๋‚˜
์žฌ๋Šฅ๋„ท์—์„œ ์ „๋ฌธ๊ฐ€์˜ ๋„์›€์„ ๋ฐ›์•„๋ณด์„ธ์š”! ๐Ÿ’ก

๋Œ“๊ธ€ ์ž‘์„ฑ

์ด ๊ธ€์— ๋Œ€ํ•œ ์—ฌ๋Ÿฌ๋ถ„์˜ ์ƒ๊ฐ์„ ๋“ค๋ ค์ฃผ์„ธ์š”

๋Œ“๊ธ€ 0
๋Œ“๊ธ€์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ์ค‘์ž…๋‹ˆ๋‹ค.

์•„์ง ๋Œ“๊ธ€์ด ์—†์Šต๋‹ˆ๋‹ค.