์ฝ˜ํ…์ธ  ๋Œ€ํ‘œ ์ด๋ฏธ์ง€ - ๐ŸŽฎ Unity ML-Agents๋กœ ๋˜‘๋˜‘ํ•œ AI ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ ๋งŒ๋“ค๊ธฐ: ์ดˆ๋ณด์ž๋„ ๋”ฐ๋ผํ•  ์ˆ˜ ์žˆ๋Š” ๊ฐ•ํ™”ํ•™์Šต ๊ฐ€์ด๋“œ ๐Ÿค–

๐ŸŽฎ Unity ML-Agents๋กœ ๋˜‘๋˜‘ํ•œ AI ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ ๋งŒ๋“ค๊ธฐ: ์ดˆ๋ณด์ž๋„ ๋”ฐ๋ผํ•  ์ˆ˜ ์žˆ๋Š” ๊ฐ•ํ™”ํ•™์Šต ๊ฐ€์ด๋“œ ๐Ÿค–

์•ˆ๋…•ํ•˜์„ธ์š”, ๊ฒŒ์ž„ ๊ฐœ๋ฐœ์— ๊ด€์‹ฌ ์žˆ๋Š” ์—ฌ๋Ÿฌ๋ถ„! ์˜ค๋Š˜์€ Unity ML-Agents๋ฅผ ํ™œ์šฉํ•ด์„œ ์ธ๊ณต์ง€๋Šฅ ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ๋ฅผ ํ›ˆ๋ จ์‹œํ‚ค๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•ด ์•Œ์•„๋ณผ๊ฒŒ์š”. ๐Ÿ˜Ž "์–ด? AI๋ผ๋‹ˆ? ๋„ˆ๋ฌด ์–ด๋ ค์šด ๊ฑฐ ์•„๋ƒ?" ๋ผ๊ณ  ์ƒ๊ฐํ•˜์‹œ๋Š” ๋ถ„๋“ค! ๊ฑฑ์ • ๋…ธ๋…ธ~ ์ด ๊ธ€์—์„œ๋Š” ์ตœ๋Œ€ํ•œ ์‰ฝ๊ณ  ์žฌ๋ฐŒ๊ฒŒ ์„ค๋ช…ํ•ด๋“œ๋ฆด๊ฒŒ์š”! ๋งˆ์น˜ ์นดํ†ก์œผ๋กœ ์ˆ˜๋‹ค ๋–จ๋“ฏ์ด ใ…‹ใ…‹ใ…‹

๊ฒŒ์ž„ ์† AI ์บ๋ฆญํ„ฐ๊ฐ€ ํ”Œ๋ ˆ์ด์–ด์ฒ˜๋Ÿผ ํ•™์Šตํ•˜๊ณ  ์ง„ํ™”ํ•˜๋Š” ๋ชจ์Šต์„ ์ง์ ‘ ๋งŒ๋“ค์–ด๋ณผ ์ˆ˜ ์žˆ๋‹ค๋ฉด ์–ผ๋งˆ๋‚˜ ์‹ ๊ธฐํ• ๊นŒ์š”? ๋ฐ”๋กœ ๊ทธ ๋งˆ๋ฒ• ๊ฐ™์€ ๊ฒฝํ—˜์„ Unity ML-Agents๊ฐ€ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•ด์ค๋‹ˆ๋‹ค!

๐ŸŽฏ ๐ŸŽฒ ๐ŸŽฎ ๐Ÿค– ๐ŸŽฏ

๐Ÿ“š ๋ชฉ์ฐจ

  1. Unity ML-Agents๋ž€ ๋ฌด์—‡์ธ๊ฐ€์š”?
  2. ๊ฐ•ํ™”ํ•™์Šต์˜ ๊ธฐ๋ณธ ๊ฐœ๋… ์ดํ•ดํ•˜๊ธฐ
  3. ๊ฐœ๋ฐœ ํ™˜๊ฒฝ ์„ค์ •ํ•˜๊ธฐ
  4. ์ฒซ ๋ฒˆ์งธ ML-Agents ํ”„๋กœ์ ํŠธ ๋งŒ๋“ค๊ธฐ
  5. AI ์—์ด์ „ํŠธ ํ›ˆ๋ จ์‹œํ‚ค๊ธฐ
  6. ํ›ˆ๋ จ ๊ฒฐ๊ณผ ๋ถ„์„ ๋ฐ ์ตœ์ ํ™”
  7. ์‹ค์ „ ํ”„๋กœ์ ํŠธ: AI ํ”Œ๋žซํฌ๋จธ ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ ๋งŒ๋“ค๊ธฐ
  8. ์ž์ฃผ ๋ฌป๋Š” ์งˆ๋ฌธ๊ณผ ๋ฌธ์ œ ํ•ด๊ฒฐ
๐ŸŽฎ ๐Ÿง  ๐ŸŽฒ ๐Ÿค– ๐ŸŽฎ

1. Unity ML-Agents๋ž€ ๋ฌด์—‡์ธ๊ฐ€์š”? ๐Ÿค”

Unity ML-Agents(Machine Learning Agents)๋Š” Unity์—์„œ ๊ฐœ๋ฐœํ•œ ์˜คํ”ˆ์†Œ์Šค ํ”„๋กœ์ ํŠธ๋กœ, ๊ฒŒ์ž„ ๊ฐœ๋ฐœ์ž๋“ค์ด ๊ฐ•ํ™”ํ•™์Šต์„ ํ†ตํ•ด ์ง€๋Šฅ์ ์ธ AI ์บ๋ฆญํ„ฐ๋ฅผ ๋งŒ๋“ค ์ˆ˜ ์žˆ๊ฒŒ ๋„์™€์ฃผ๋Š” ํˆดํ‚ท์ด์—์š”. ์‰ฝ๊ฒŒ ๋งํ•ด์„œ, ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ๊ฐ€ ์Šค์Šค๋กœ ํ•™์Šตํ•˜๊ณ  ๋ฐœ์ „ํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ๋Š” ๋งˆ๋ฒ• ๊ฐ™์€ ๋„๊ตฌ๋ผ๊ณ  ์ƒ๊ฐํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค! โœจ

ML-Agents์˜ ์ฃผ์š” ํŠน์ง• ๐ŸŒŸ

1. ๊ฐ•ํ™”ํ•™์Šต(Reinforcement Learning)์„ ๊ฒŒ์ž„ ํ™˜๊ฒฝ์— ์‰ฝ๊ฒŒ ์ ์šฉ ๊ฐ€๋Šฅ

2. ๋ชจ๋ฐฉํ•™์Šต(Imitation Learning)์œผ๋กœ ์‚ฌ๋žŒ์˜ ํ”Œ๋ ˆ์ด ์Šคํƒ€์ผ ํ•™์Šต ๊ฐ€๋Šฅ

3. ์ปค๋ฆฌํ˜๋Ÿผ ํ•™์Šต(Curriculum Learning)์œผ๋กœ ๋‚œ์ด๋„๋ฅผ ์ ์ง„์ ์œผ๋กœ ๋†’์ด๋ฉฐ ํ›ˆ๋ จ

4. ๋ฉ€ํ‹ฐ์—์ด์ „ํŠธ ํ™˜๊ฒฝ์—์„œ ํ˜‘๋ ฅ ๋˜๋Š” ๊ฒฝ์Ÿ ํ•™์Šต ์ง€์›

5. Python API๋ฅผ ํ†ตํ•œ ๊ณ ๊ธ‰ ํ›ˆ๋ จ ๊ธฐ๋Šฅ ์ œ๊ณต

"๊ทผ๋ฐ ์ด๊ฑฐ ์‹ค์ œ๋กœ ์–ด๋””์— ์“ฐ์ด๋Š” ๊ฑฐ์˜ˆ์š”?" ๋ผ๋Š” ์งˆ๋ฌธ์ด ๋“ค๋ฆฌ๋Š” ๊ฒƒ ๊ฐ™๋„ค์š”! ใ…‹ใ…‹ใ…‹ ์‹ค์ œ๋กœ ML-Agents๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๊ณณ์— ํ™œ์šฉ๋ฉ๋‹ˆ๋‹ค:

๊ฒŒ์ž„ AI NPC ํ–‰๋™ ํŒจํ„ด ์  ์บ๋ฆญํ„ฐ ์ „๋žต ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ๋กœ๋ด‡ ์ œ์–ด ์ž์œจ์ฃผํ–‰ ํ•™์Šต ์—ฐ๊ตฌ ๋ฐ ๊ต์œก AI ์•Œ๊ณ ๋ฆฌ์ฆ˜ ํ…Œ์ŠคํŠธ ํ•™์Šต ํ™˜๊ฒฝ ๊ตฌ์ถ• ML-Agents ํ™œ์šฉ ๋ถ„์•ผ

์žฌ๋Šฅ๋„ท์—์„œ๋„ ๊ฒŒ์ž„ ๊ฐœ๋ฐœ ๊ด€๋ จ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•˜๋Š” ๋ถ„๋“ค์ด ML-Agents๋ฅผ ํ™œ์šฉํ•œ ํ”„๋กœ์ ํŠธ๋ฅผ ๋งŽ์ด ์„ ๋ณด์ด๊ณ  ์žˆ์–ด์š”. ํŠนํžˆ ์ธ๋”” ๊ฒŒ์ž„ ๊ฐœ๋ฐœ์ž๋“ค์—๊ฒŒ ํฐ ๋„์›€์ด ๋˜๋Š” ๊ธฐ์ˆ ์ด๋ž๋‹ˆ๋‹ค! ๐Ÿ‘

2. ๊ฐ•ํ™”ํ•™์Šต์˜ ๊ธฐ๋ณธ ๊ฐœ๋… ์ดํ•ดํ•˜๊ธฐ ๐Ÿง 

ML-Agents๋ฅผ ์ œ๋Œ€๋กœ ํ™œ์šฉํ•˜๋ ค๋ฉด ๊ฐ•ํ™”ํ•™์Šต์˜ ๊ธฐ๋ณธ ๊ฐœ๋…์„ ์•Œ์•„์•ผ ํ•ด์š”. ๊ทผ๋ฐ ๋„ˆ๋ฌด ์–ด๋ ต๊ฒŒ ์ƒ๊ฐํ•˜์ง€ ๋งˆ์„ธ์š”! ๊ฐ•ํ™”ํ•™์Šต์€ ๊ทธ๋ƒฅ "์‹œํ–‰์ฐฉ์˜ค๋ฅผ ํ†ตํ•ด ๋ฐฐ์šฐ๋Š” ๋ฐฉ์‹"์ด๋ผ๊ณ  ์ƒ๊ฐํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.

๊ฐ•ํ™”ํ•™์Šต์€ ๋งˆ์น˜ ์•„์ด๊ฐ€ ๊ฑธ์Œ๋งˆ๋ฅผ ๋ฐฐ์šฐ๋Š” ๊ฒƒ๊ณผ ๋น„์Šทํ•ด์š”. ์ฒ˜์Œ์—๋Š” ๋„˜์–ด์ง€๊ณ  ์‹คํŒจํ•˜์ง€๋งŒ, ๊ณ„์† ์‹œ๋„ํ•˜๋ฉด์„œ ์–ด๋–ค ํ–‰๋™์ด ์ข‹์€ ๊ฒฐ๊ณผ(๋ณด์ƒ)๋ฅผ ๊ฐ€์ ธ์˜ค๋Š”์ง€ ํ•™์Šตํ•˜๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

๊ฐ•ํ™”ํ•™์Šต์˜ ํ•ต์‹ฌ ์š”์†Œ ๐Ÿ”‘

์—์ด์ „ํŠธ ํ™˜๊ฒฝ ํ–‰๋™ (Action) ์ƒํƒœ (State) / ๋ณด์ƒ (Reward) ๊ฐ•ํ™”ํ•™์Šต์˜ ๊ธฐ๋ณธ ๊ตฌ์กฐ ํ•™์Šตํ•˜๋Š” ์ฃผ์ฒด (๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ) ์—์ด์ „ํŠธ๊ฐ€ ์ƒํ˜ธ์ž‘์šฉํ•˜๋Š” ์„ธ๊ณ„ (๊ฒŒ์ž„ ์›”๋“œ)

๊ฐ•ํ™”ํ•™์Šต์—์„œ๋Š” ์œ„ ๋‹ค์ด์–ด๊ทธ๋žจ์ฒ˜๋Ÿผ ์—์ด์ „ํŠธ(Agent)๊ฐ€ ํ™˜๊ฒฝ(Environment)๊ณผ ์ƒํ˜ธ์ž‘์šฉํ•˜๋ฉด์„œ ํ•™์Šตํ•ด์š”. ์—์ด์ „ํŠธ๋Š” ํŠน์ • ํ–‰๋™(Action)์„ ์ทจํ•˜๊ณ , ํ™˜๊ฒฝ์€ ์ƒˆ๋กœ์šด ์ƒํƒœ(State)์™€ ๋ณด์ƒ(Reward)์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

์˜ˆ๋ฅผ ๋“ค์–ด๋ณผ๊นŒ์š”? ๐ŸŽฎ

  • ๐Ÿค– ์—์ด์ „ํŠธ: ํ”Œ๋žซํฌ๋จธ ๊ฒŒ์ž„์˜ ์ฃผ์ธ๊ณต ์บ๋ฆญํ„ฐ
  • ๐ŸŒ ํ™˜๊ฒฝ: ๊ฒŒ์ž„ ๋ ˆ๋ฒจ๊ณผ ์žฅ์• ๋ฌผ๋“ค
  • ๐Ÿƒโ€โ™‚๏ธ ํ–‰๋™: ์ ํ”„, ๋‹ฌ๋ฆฌ๊ธฐ, ๊ณต๊ฒฉ ๋“ฑ
  • ๐Ÿ“Š ์ƒํƒœ: ์บ๋ฆญํ„ฐ์˜ ์œ„์น˜, ์ฃผ๋ณ€ ์ ์˜ ์œ„์น˜, ์ฒด๋ ฅ ๋“ฑ
  • ๐ŸŽ ๋ณด์ƒ: ์ฝ”์ธ ํš๋“(+์ ์ˆ˜), ์  ์ฒ˜์น˜(+์ ์ˆ˜), ์ฃฝ์Œ(-์ ์ˆ˜)

์ด๋Ÿฐ ๊ตฌ์กฐ์—์„œ AI๋Š” ์ตœ๋Œ€ํ•œ ๋งŽ์€ ๋ณด์ƒ์„ ์–ป๋Š” ๋ฐฉํ–ฅ์œผ๋กœ ํ–‰๋™ ํŒจํ„ด์„ ํ•™์Šตํ•˜๊ฒŒ ๋ฉ๋‹ˆ๋‹ค. ์ฒ˜์Œ์—๋Š” ๋žœ๋คํ•˜๊ฒŒ ํ–‰๋™ํ•˜๋‹ค๊ฐ€ ์ ์  ์–ด๋–ค ํ–‰๋™์ด ์ข‹์€ ๊ฒฐ๊ณผ๋ฅผ ๊ฐ€์ ธ์˜ค๋Š”์ง€ ํŒŒ์•…ํ•˜๊ฒŒ ๋˜์ฃ . ์ด๊ฒŒ ๋ฐ”๋กœ ๊ฐ•ํ™”ํ•™์Šต์˜ ๋งˆ๋ฒ•์ด์—์š”! โœจ

3. ๊ฐœ๋ฐœ ํ™˜๊ฒฝ ์„ค์ •ํ•˜๊ธฐ ๐Ÿ› ๏ธ

์ž, ์ด์ œ ์‹ค์ œ๋กœ ML-Agents๋ฅผ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•œ ํ™˜๊ฒฝ์„ ์„ค์ •ํ•ด๋ณผ๊นŒ์š”? ๊ฑฑ์ • ๋งˆ์„ธ์š”, ์ƒ๊ฐ๋ณด๋‹ค ์–ด๋ ต์ง€ ์•Š์•„์š”! ใ…‹ใ…‹ใ…‹

ํ•„์š”ํ•œ ๋„๊ตฌ๋“ค ๐Ÿงฐ

  1. Unity Hub & Unity Editor (2020.3 LTS ์ด์ƒ ๋ฒ„์ „ ๊ถŒ์žฅ)
  2. Python (3.6 ~ 3.9 ๋ฒ„์ „)
  3. ML-Agents ํŒจํ‚ค์ง€
  4. PyTorch (ML-Agents์˜ ๋ฐฑ์—”๋“œ๋กœ ์‚ฌ์šฉ)

์„ค์น˜ ๊ณผ์ • ๋‹จ๊ณ„๋ณ„ ๊ฐ€์ด๋“œ ๐Ÿ“

1๋‹จ๊ณ„: Unity ์„ค์น˜ํ•˜๊ธฐ

Unity Hub๋ฅผ ๋จผ์ € ์„ค์น˜ํ•˜๊ณ , ๊ทธ ๋‹ค์Œ์— Unity Editor๋ฅผ ์„ค์น˜ํ•ด์ฃผ์„ธ์š”. Unity Hub๋Š” ์—ฌ๋Ÿฌ ๋ฒ„์ „์˜ Unity๋ฅผ ๊ด€๋ฆฌํ•˜๊ธฐ ์‰ฝ๊ฒŒ ํ•ด์ฃผ๋Š” ํ”„๋กœ๊ทธ๋žจ์ด์—์š”.

Unity Hub ๋‹ค์šด๋กœ๋“œ: https://unity.com/download

2๋‹จ๊ณ„: Python ์„ค์น˜ํ•˜๊ธฐ

Python์„ ์„ค์น˜ํ•  ๋•Œ๋Š” "Add Python to PATH" ์˜ต์…˜์„ ๊ผญ ์ฒดํฌํ•ด์ฃผ์„ธ์š”! ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ๋ช…๋ น ํ”„๋กฌํ”„ํŠธ์—์„œ Python์„ ๋ฐ”๋กœ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ์–ด์š”.

Python ๋‹ค์šด๋กœ๋“œ: https://www.python.org/downloads/

3๋‹จ๊ณ„: ML-Agents ์„ค์น˜ํ•˜๊ธฐ

์ด์ œ ๋ช…๋ น ํ”„๋กฌํ”„ํŠธ(์œˆ๋„์šฐ) ๋˜๋Š” ํ„ฐ๋ฏธ๋„(๋งฅ)์„ ์—ด๊ณ  ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์ž…๋ ฅํ•ด์ฃผ์„ธ์š”:

pip install mlagents

์ด ๋ช…๋ น์–ด๋กœ ML-Agents Python ํŒจํ‚ค์ง€๊ฐ€ ์„ค์น˜๋ฉ๋‹ˆ๋‹ค. ์ด ํŒจํ‚ค์ง€๋Š” Unity ๋‚ด์˜ ์—์ด์ „ํŠธ๋ฅผ ํ›ˆ๋ จ์‹œํ‚ค๋Š” ๋ฐ ํ•„์š”ํ•œ ๋„๊ตฌ๋“ค์„ ์ œ๊ณตํ•ด์š”.

4๋‹จ๊ณ„: Unity์— ML-Agents ํŒจํ‚ค์ง€ ์ถ”๊ฐ€ํ•˜๊ธฐ

Unity ํ”„๋กœ์ ํŠธ๋ฅผ ์—ด๊ณ  Window > Package Manager๋ฅผ ์„ ํƒํ•ฉ๋‹ˆ๋‹ค. "+" ๋ฒ„ํŠผ์„ ํด๋ฆญํ•˜๊ณ  "Add package from git URL..."์„ ์„ ํƒํ•œ ํ›„ ๋‹ค์Œ URL์„ ์ž…๋ ฅํ•ด์ฃผ์„ธ์š”:

com.unity.ml-agents

๐Ÿ’ก ๊ฟ€ํŒ! Unity ๋ฒ„์ „๊ณผ ML-Agents ๋ฒ„์ „์ด ํ˜ธํ™˜๋˜๋Š”์ง€ ํ™•์ธํ•˜๋Š” ๊ฒƒ์ด ์ค‘์š”ํ•ด์š”! ์ตœ์‹  ๋ฒ„์ „์ด ํ•ญ์ƒ ์ตœ์„ ์€ ์•„๋‹ ์ˆ˜ ์žˆ์œผ๋‹ˆ, ๊ณต์‹ ๋ฌธ์„œ์—์„œ ๊ถŒ์žฅํ•˜๋Š” ๋ฒ„์ „์„ ํ™•์ธํ•ด๋ณด์„ธ์š”.

์ด๋ ‡๊ฒŒ ์„ค์ •์ด ์™„๋ฃŒ๋˜๋ฉด ML-Agents๋ฅผ ์‚ฌ์šฉํ•  ์ค€๋น„๊ฐ€ ๋๋‚ฌ์–ด์š”! ์ง์ง์ง~ ๐Ÿ‘

4. ์ฒซ ๋ฒˆ์งธ ML-Agents ํ”„๋กœ์ ํŠธ ๋งŒ๋“ค๊ธฐ ๐Ÿš€

์ด๋ก ์€ ์ด์ œ ์ถฉ๋ถ„ํžˆ ์•Œ์•˜์œผ๋‹ˆ ์ง์ ‘ ์†์œผ๋กœ ๋งŒ๋“ค์–ด๋ณผ ์ฐจ๋ก€์˜ˆ์š”! ๊ฐ€์žฅ ๊ธฐ๋ณธ์ ์ธ ์˜ˆ์ œ์ธ "๊ณต ๊ตด๋ฆฌ๊ธฐ(Ball Rolling)" ๊ฒŒ์ž„์„ ๋งŒ๋“ค์–ด๋ณผ๊ฒŒ์š”. ์ด ๊ฒŒ์ž„์—์„œ AI๋Š” ๋ชฉํ‘œ ์ง€์ ๊นŒ์ง€ ๊ณต์„ ๊ตด๋ฆฌ๋Š” ๋ฐฉ๋ฒ•์„ ํ•™์Šตํ•˜๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

ํ”„๋กœ์ ํŠธ ์„ค์ •ํ•˜๊ธฐ

  1. Unity Hub์—์„œ ์ƒˆ 3D ํ”„๋กœ์ ํŠธ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
  2. Package Manager์—์„œ ML-Agents ํŒจํ‚ค์ง€๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
  3. ์ƒˆ๋กœ์šด ์”ฌ(Scene)์„ ๋งŒ๋“ค๊ณ  ์ €์žฅํ•ฉ๋‹ˆ๋‹ค.

๊ฒŒ์ž„ ํ™˜๊ฒฝ ๋งŒ๋“ค๊ธฐ

์ด์ œ ๊ฐ„๋‹จํ•œ ๊ฒŒ์ž„ ํ™˜๊ฒฝ์„ ๋งŒ๋“ค์–ด๋ณผ๊ฒŒ์š”:

  1. ๋ฐ”๋‹ฅ(Plane)์„ ์ƒ์„ฑํ•˜๊ณ  ํฌ๊ธฐ๋ฅผ ์กฐ์ ˆํ•ฉ๋‹ˆ๋‹ค.
  2. ๊ณต(Sphere)์„ ์ƒ์„ฑํ•˜๊ณ  Rigidbody ์ปดํฌ๋„ŒํŠธ๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
  3. ๋ชฉํ‘œ ์ง€์ (Target)์œผ๋กœ ์‚ฌ์šฉํ•  ํ๋ธŒ๋ฅผ ์ƒ์„ฑํ•˜๊ณ  ๋‹ค๋ฅธ ์ƒ‰์ƒ์œผ๋กœ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
  4. ๋ฒฝ(Wall)์„ ๋งŒ๋“ค์–ด ๊ณต์ด ๋–จ์–ด์ง€์ง€ ์•Š๋„๋ก ํ•ฉ๋‹ˆ๋‹ค.
Agent Goal Obstacle Ball Rolling ํ™˜๊ฒฝ ๊ตฌ์„ฑ

์—์ด์ „ํŠธ ์Šคํฌ๋ฆฝํŠธ ์ž‘์„ฑํ•˜๊ธฐ

์ด์ œ ML-Agents์˜ ํ•ต์‹ฌ์ธ ์—์ด์ „ํŠธ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์ž‘์„ฑํ•ด๋ณผ๊ฒŒ์š”. ์ƒˆ C# ์Šคํฌ๋ฆฝํŠธ๋ฅผ ๋งŒ๋“ค๊ณ  ์ด๋ฆ„์„ "RollerAgent"๋กœ ์ง€์ •ํ•ด์ฃผ์„ธ์š”.

using UnityEngine;
using Unity.MLAgents;
using Unity.MLAgents.Sensors;
using Unity.MLAgents.Actuators;

public class RollerAgent : Agent
{
    Rigidbody rBody;
    public Transform Target;
    public float forceMultiplier = 10;
    
    // ์ดˆ๊ธฐํ™” ์‹œ ํ˜ธ์ถœ
    public override void Initialize()
    {
        rBody = GetComponent<Rigidbody>();
    }
    
    // ์—ํ”ผ์†Œ๋“œ ์‹œ์ž‘ ์‹œ ํ˜ธ์ถœ
    public override void OnEpisodeBegin()
    {
        // ์—์ด์ „ํŠธ ์œ„์น˜ ์ดˆ๊ธฐํ™”
        this.rBody.angularVelocity = Vector3.zero;
        this.rBody.velocity = Vector3.zero;
        this.transform.localPosition = new Vector3(0, 0.5f, 0);
        
        // ๋ชฉํ‘œ๋ฌผ ์œ„์น˜ ๋žœ๋คํ•˜๊ฒŒ ์„ค์ •
        Target.localPosition = new Vector3(Random.value * 8 - 4, 0.5f, Random.value * 8 - 4);
    }
    
    // ๊ด€์ธก ์ •๋ณด ์ˆ˜์ง‘
    public override void CollectObservations(VectorSensor sensor)
    {
        // ๋ชฉํ‘œ๋ฌผ๊ณผ ์—์ด์ „ํŠธ์˜ ์œ„์น˜
        sensor.AddObservation(Target.localPosition);
        sensor.AddObservation(this.transform.localPosition);
        
        // ์—์ด์ „ํŠธ์˜ ์†๋„
        sensor.AddObservation(rBody.velocity.x);
        sensor.AddObservation(rBody.velocity.z);
    }
    
    // ํ–‰๋™ ์‹คํ–‰
    public override void OnActionReceived(ActionBuffers actionBuffers)
    {
        // ํ–‰๋™ ๊ฐ’ ๊ฐ€์ ธ์˜ค๊ธฐ
        Vector3 controlSignal = Vector3.zero;
        controlSignal.x = actionBuffers.ContinuousActions[0];
        controlSignal.z = actionBuffers.ContinuousActions[1];
        
        // ํž˜ ์ ์šฉํ•˜๊ธฐ
        rBody.AddForce(controlSignal * forceMultiplier);
        
        // ๋ณด์ƒ: ๋ชฉํ‘œ์— ๊ฐ€๊นŒ์›Œ์งˆ์ˆ˜๋ก ์ž‘์€ ์–‘์˜ ๋ณด์ƒ
        float distanceToTarget = Vector3.Distance(this.transform.localPosition, Target.localPosition);
        
        // ๋ชฉํ‘œ์— ๋„๋‹ฌํ–ˆ์„ ๋•Œ
        if (distanceToTarget < 1.42f)
        {
            SetReward(1.0f);
            EndEpisode();
        }
        
        // ํ”Œ๋žซํผ์—์„œ ๋–จ์–ด์กŒ์„ ๋•Œ
        if (this.transform.localPosition.y < 0)
        {
            EndEpisode();
        }
    }
    
    // ์ˆ˜๋™ ํ…Œ์ŠคํŠธ์šฉ (์„ ํƒ์‚ฌํ•ญ)
    public override void Heuristic(in ActionBuffers actionsOut)
    {
        var continuousActionsOut = actionsOut.ContinuousActions;
        continuousActionsOut[0] = Input.GetAxis("Horizontal");
        continuousActionsOut[1] = Input.GetAxis("Vertical");
    }
}

์ด ์Šคํฌ๋ฆฝํŠธ๋ฅผ ๊ณต ์˜ค๋ธŒ์ ํŠธ์— ์ถ”๊ฐ€ํ•˜๊ณ , Target ํ•„๋“œ์— ๋ชฉํ‘œ ํ๋ธŒ๋ฅผ ์—ฐ๊ฒฐํ•ด์ฃผ์„ธ์š”.

์ฝ”๋“œ ์„ค๋ช… ๐Ÿ“

Initialize(): ์—์ด์ „ํŠธ๊ฐ€ ์‹œ์ž‘๋  ๋•Œ ํ•„์š”ํ•œ ์ปดํฌ๋„ŒํŠธ๋ฅผ ๊ฐ€์ ธ์˜ต๋‹ˆ๋‹ค.

OnEpisodeBegin(): ๊ฐ ํ•™์Šต ์—ํ”ผ์†Œ๋“œ๊ฐ€ ์‹œ์ž‘๋  ๋•Œ ํ™˜๊ฒฝ์„ ์ดˆ๊ธฐํ™”ํ•ฉ๋‹ˆ๋‹ค.

CollectObservations(): ์—์ด์ „ํŠธ๊ฐ€ ํ™˜๊ฒฝ์„ ๊ด€์ฐฐํ•˜๋Š” ๋ฐ ํ•„์š”ํ•œ ์ •๋ณด๋ฅผ ์ˆ˜์ง‘ํ•ฉ๋‹ˆ๋‹ค.

OnActionReceived(): ์—์ด์ „ํŠธ๊ฐ€ ๊ฒฐ์ •ํ•œ ํ–‰๋™์„ ์‹คํ–‰ํ•˜๊ณ  ๋ณด์ƒ์„ ๊ณ„์‚ฐํ•ฉ๋‹ˆ๋‹ค.

Heuristic(): ๊ฐœ๋ฐœ์ž๊ฐ€ ์ง์ ‘ ์—์ด์ „ํŠธ๋ฅผ ์กฐ์ž‘ํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ๋Š” ํ•จ์ˆ˜์ž…๋‹ˆ๋‹ค.

Behavior Parameters ์„ค์ •ํ•˜๊ธฐ

์ด์ œ ์—์ด์ „ํŠธ์— Behavior Parameters ์ปดํฌ๋„ŒํŠธ๋ฅผ ์ถ”๊ฐ€ํ•˜๊ณ  ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์„ค์ •ํ•ด์ฃผ์„ธ์š”:

  • Behavior Name: RollerBall
  • Vector Observation > Space Size: 8
  • Actions > Continuous Actions Size: 2
  • Behavior Type: Default

์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ๊ธฐ๋ณธ์ ์ธ ํ™˜๊ฒฝ ์„ค์ •์ด ์™„๋ฃŒ๋ฉ๋‹ˆ๋‹ค! ๐ŸŽ‰

5. AI ์—์ด์ „ํŠธ ํ›ˆ๋ จ์‹œํ‚ค๊ธฐ ๐Ÿ‹๏ธโ€โ™‚๏ธ

์ด์ œ ์ง„์งœ ์žฌ๋ฏธ์žˆ๋Š” ๋ถ€๋ถ„์ด ์‹œ์ž‘๋ฉ๋‹ˆ๋‹ค! ์šฐ๋ฆฌ๊ฐ€ ๋งŒ๋“  ์—์ด์ „ํŠธ๋ฅผ ํ›ˆ๋ จ์‹œ์ผœ๋ณผ ์ฐจ๋ก€์˜ˆ์š”. ใ…‹ใ…‹ใ…‹

ํ›ˆ๋ จ ์„ค์ • ํŒŒ์ผ ๋งŒ๋“ค๊ธฐ

ML-Agents๋Š” YAML ํŒŒ์ผ์„ ์‚ฌ์šฉํ•ด ํ›ˆ๋ จ ์„ค์ •์„ ์ •์˜ํ•ฉ๋‹ˆ๋‹ค. ํ”„๋กœ์ ํŠธ ํด๋”์— "rollerball_config.yaml"์ด๋ผ๋Š” ํŒŒ์ผ์„ ๋งŒ๋“ค๊ณ  ๋‹ค์Œ ๋‚ด์šฉ์„ ์ž…๋ ฅํ•ด์ฃผ์„ธ์š”:

behaviors:
  RollerBall:
    trainer_type: ppo
    hyperparameters:
      batch_size: 64
      buffer_size: 12000
      learning_rate: 0.0003
      beta: 0.001
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
      learning_rate_schedule: linear
    network_settings:
      normalize: true
      hidden_units: 128
      num_layers: 2
      vis_encode_type: simple
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    max_steps: 500000
    time_horizon: 64
    summary_freq: 10000

์ฃผ์š” ํŒŒ๋ผ๋ฏธํ„ฐ ์„ค๋ช… ๐Ÿ”

trainer_type: ์‚ฌ์šฉํ•  ์•Œ๊ณ ๋ฆฌ์ฆ˜ (PPO๋Š” ๊ฐ€์žฅ ๋„๋ฆฌ ์‚ฌ์šฉ๋˜๋Š” ๊ฐ•ํ™”ํ•™์Šต ์•Œ๊ณ ๋ฆฌ์ฆ˜)

batch_size: ํ•œ ๋ฒˆ์— ์ฒ˜๋ฆฌํ•  ๊ฒฝํ—˜์˜ ์ˆ˜

learning_rate: ํ•™์Šต ์†๋„ (๋„ˆ๋ฌด ๋†’์œผ๋ฉด ๋ถˆ์•ˆ์ •, ๋„ˆ๋ฌด ๋‚ฎ์œผ๋ฉด ๋А๋ฆผ)

hidden_units & num_layers: ์‹ ๊ฒฝ๋ง์˜ ํฌ๊ธฐ์™€ ๊นŠ์ด

max_steps: ์ด ํ›ˆ๋ จ ์Šคํ… ์ˆ˜

ํ›ˆ๋ จ ์‹œ์ž‘ํ•˜๊ธฐ

์ด์ œ ๋ช…๋ น ํ”„๋กฌํ”„ํŠธ(๋˜๋Š” ํ„ฐ๋ฏธ๋„)๋ฅผ ์—ด๊ณ  ํ”„๋กœ์ ํŠธ ํด๋”๋กœ ์ด๋™ํ•œ ํ›„ ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•ด์ฃผ์„ธ์š”:

mlagents-learn rollerball_config.yaml --run-id=RollerBall1

๊ทธ๋ฆฌ๊ณ  Unity ์—๋””ํ„ฐ์—์„œ Play ๋ฒ„ํŠผ์„ ๋ˆŒ๋Ÿฌ์ฃผ์„ธ์š”. ๊ทธ๋Ÿฌ๋ฉด ํ›ˆ๋ จ์ด ์‹œ์ž‘๋ฉ๋‹ˆ๋‹ค! ๐Ÿš€

๋žœ๋ค ํ–‰๋™ ํŒจํ„ด ๋ฐœ๊ฒฌ ์ „๋žต ๊ฐœ๋ฐœ ์ตœ์ ํ™” 0% 25% 50% 75% 100% ๋ณด์ƒ ํ›ˆ๋ จ ์ง„ํ–‰๋„ AI ํ›ˆ๋ จ ๊ณผ์ •

ํ›ˆ๋ จ์ด ์ง„ํ–‰๋˜๋Š” ๋™์•ˆ ํ„ฐ๋ฏธ๋„์—์„œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ •๋ณด๋ฅผ ๋ณผ ์ˆ˜ ์žˆ์–ด์š”:

  • ๐Ÿ“ˆ ํ‰๊ท  ๋ณด์ƒ: ์—์ด์ „ํŠธ๊ฐ€ ๋ฐ›๋Š” ํ‰๊ท  ๋ณด์ƒ ๊ฐ’
  • โฑ๏ธ ์—ํ”ผ์†Œ๋“œ ๊ธธ์ด: ํ•œ ์—ํ”ผ์†Œ๋“œ๊ฐ€ ๋๋‚  ๋•Œ๊นŒ์ง€ ๊ฑธ๋ฆฌ๋Š” ํ‰๊ท  ์‹œ๊ฐ„
  • ๐Ÿง  ์ •์ฑ… ์†์‹ค: ํ•™์Šต ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ์•ˆ์ •์„ฑ ์ง€ํ‘œ
  • ๐Ÿ”„ ํ•™์Šต ์†๋„: ์ดˆ๋‹น ์ฒ˜๋ฆฌ๋˜๋Š” ํ™˜๊ฒฝ ์Šคํ… ์ˆ˜

๐Ÿ’ก ๊ฟ€ํŒ! ํ›ˆ๋ จ ์ค‘์—๋Š” ์—ฌ๋Ÿฌ ๊ฐœ์˜ ํ™˜๊ฒฝ์„ ๋™์‹œ์— ์‹คํ–‰ํ•˜๋ฉด ํ›ˆ๋ จ ์†๋„๋ฅผ ํฌ๊ฒŒ ๋†’์ผ ์ˆ˜ ์žˆ์–ด์š”! ์ด๋ฅผ ์œ„ํ•ด --num-envs ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

ํ›ˆ๋ จ์€ ์„ค์ •์— ๋”ฐ๋ผ ๋‹ค๋ฅด์ง€๋งŒ ๋ณดํ†ต 30๋ถ„~1์‹œ๊ฐ„ ์ •๋„ ๊ฑธ๋ฆฝ๋‹ˆ๋‹ค. ๊ทธ๋™์•ˆ โ˜• ํ•œ ์ž”์˜ ์ปคํ”ผ๋ฅผ ์ฆ๊ธฐ๋Š” ๊ฑด ์–ด๋–จ๊นŒ์š”? ใ…‹ใ…‹ใ…‹

6. ํ›ˆ๋ จ ๊ฒฐ๊ณผ ๋ถ„์„ ๋ฐ ์ตœ์ ํ™” ๐Ÿ“Š

ํ›ˆ๋ จ์ด ์™„๋ฃŒ๋˜๋ฉด results ํด๋”์— ํ•™์Šต๋œ ๋ชจ๋ธ๊ณผ ํ•จ๊ป˜ TensorBoard ๋กœ๊ทธ๊ฐ€ ์ €์žฅ๋ฉ๋‹ˆ๋‹ค. ์ด ๋กœ๊ทธ๋ฅผ ํ†ตํ•ด ํ›ˆ๋ จ ๊ณผ์ •์„ ์‹œ๊ฐ์ ์œผ๋กœ ๋ถ„์„ํ•  ์ˆ˜ ์žˆ์–ด์š”!

TensorBoard๋กœ ๊ฒฐ๊ณผ ๋ถ„์„ํ•˜๊ธฐ

๋‹ค์Œ ๋ช…๋ น์–ด๋กœ TensorBoard๋ฅผ ์‹คํ–‰ํ•ด๋ณด์„ธ์š”:

tensorboard --logdir=results

๊ทธ๋Ÿฌ๋ฉด ๋ธŒ๋ผ์šฐ์ €์—์„œ http://localhost:6006์œผ๋กœ ์ ‘์†ํ•ด ๋‹ค์–‘ํ•œ ์ง€ํ‘œ๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ฃผ์š” ์ง€ํ‘œ ์„ค๋ช… ๐Ÿ“ˆ

Environment/Cumulative Reward: ์—์ด์ „ํŠธ๊ฐ€ ๋ฐ›์€ ์ด ๋ณด์ƒ (๋†’์„์ˆ˜๋ก ์ข‹์Œ)

Environment/Episode Length: ์—ํ”ผ์†Œ๋“œ ํ‰๊ท  ๊ธธ์ด (๋ชฉํ‘œ์— ๋นจ๋ฆฌ ๋„๋‹ฌํ• ์ˆ˜๋ก ์งง์•„์ง)

Policy/Entropy: ์—์ด์ „ํŠธ ํ–‰๋™์˜ ๋ฌด์ž‘์œ„์„ฑ (ํ›ˆ๋ จ์ด ์ง„ํ–‰๋ ์ˆ˜๋ก ๊ฐ์†Œํ•ด์•ผ ํ•จ)

Policy/Learning Rate: ํ•™์Šต๋ฅ  (๋ณดํ†ต ์‹œ๊ฐ„์ด ์ง€๋‚จ์— ๋”ฐ๋ผ ๊ฐ์†Œ)

๋ชจ๋ธ ์„ฑ๋Šฅ ์ตœ์ ํ™”ํ•˜๊ธฐ

์ฒซ ๋ฒˆ์งธ ํ›ˆ๋ จ ๊ฒฐ๊ณผ๊ฐ€ ๋งŒ์กฑ์Šค๋Ÿฝ์ง€ ์•Š๋‹ค๋ฉด, ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋ฐฉ๋ฒ•์œผ๋กœ ์„ฑ๋Šฅ์„ ๊ฐœ์„ ํ•  ์ˆ˜ ์žˆ์–ด์š”:

  1. ๋ณด์ƒ ํ•จ์ˆ˜ ์กฐ์ •: ๋ณด์ƒ ๊ตฌ์กฐ๋ฅผ ๋” ์„ธ๋ฐ€ํ•˜๊ฒŒ ์„ค๊ณ„ํ•ด๋ณด์„ธ์š”. ์˜ˆ๋ฅผ ๋“ค์–ด, ๋ชฉํ‘œ์— ๊ฐ€๊นŒ์›Œ์งˆ์ˆ˜๋ก ๋” ๋งŽ์€ ๋ณด์ƒ์„ ์ฃผ๋Š” ๋ฐฉ์‹์œผ๋กœ ๋ณ€๊ฒฝํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  2. ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ ํŠœ๋‹: learning_rate, batch_size ๋“ฑ์˜ ๊ฐ’์„ ์กฐ์ •ํ•ด๋ณด์„ธ์š”.
  3. ๋„คํŠธ์›Œํฌ ๊ตฌ์กฐ ๋ณ€๊ฒฝ: ๋” ํฌ๊ฑฐ๋‚˜ ์ž‘์€ ์‹ ๊ฒฝ๋ง์„ ์‹œ๋„ํ•ด๋ณด์„ธ์š”.
  4. ๊ด€์ธก ์ •๋ณด ๊ฐœ์„ : ์—์ด์ „ํŠธ์—๊ฒŒ ๋” ์œ ์šฉํ•œ ์ •๋ณด๋ฅผ ์ œ๊ณตํ•ด๋ณด์„ธ์š”.

์˜ˆ๋ฅผ ๋“ค์–ด, ๋ณด์ƒ ํ•จ์ˆ˜๋ฅผ ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ˆ˜์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

// ๊ธฐ์กด ์ฝ”๋“œ
float distanceToTarget = Vector3.Distance(this.transform.localPosition, Target.localPosition);
if (distanceToTarget < 1.42f)
{
    SetReward(1.0f);
    EndEpisode();
}

// ๊ฐœ์„ ๋œ ์ฝ”๋“œ
float distanceToTarget = Vector3.Distance(this.transform.localPosition, Target.localPosition);
// ๊ฑฐ๋ฆฌ์— ๋ฐ˜๋น„๋ก€ํ•˜๋Š” ์ž‘์€ ๋ณด์ƒ ์ถ”๊ฐ€
SetReward(0.01f * (1.0f / (distanceToTarget + 0.1f)));
if (distanceToTarget < 1.42f)
{
    SetReward(1.0f);
    EndEpisode();
}

์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ์—์ด์ „ํŠธ๊ฐ€ ๋ชฉํ‘œ์— ๊ฐ€๊นŒ์›Œ์งˆ์ˆ˜๋ก ๋” ๋งŽ์€ ๋ณด์ƒ์„ ๋ฐ›๊ฒŒ ๋˜์–ด ํ•™์Šต ์†๋„๊ฐ€ ๋นจ๋ผ์งˆ ์ˆ˜ ์žˆ์–ด์š”!

7. ์‹ค์ „ ํ”„๋กœ์ ํŠธ: AI ํ”Œ๋žซํฌ๋จธ ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ ๋งŒ๋“ค๊ธฐ ๐ŸŽฎ

์ง€๊ธˆ๊นŒ์ง€ ๋ฐฐ์šด ๋‚ด์šฉ์„ ๋ฐ”ํƒ•์œผ๋กœ ์กฐ๊ธˆ ๋” ๋ณต์žกํ•œ ํ”„๋กœ์ ํŠธ๋ฅผ ๋งŒ๋“ค์–ด๋ณผ๊นŒ์š”? 2D ํ”Œ๋žซํฌ๋จธ ๊ฒŒ์ž„์˜ AI ์บ๋ฆญํ„ฐ๋ฅผ ํ›ˆ๋ จ์‹œ์ผœ ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค!

ํ”„๋กœ์ ํŠธ ์„ค์ •

  1. ์ƒˆ Unity 2D ํ”„๋กœ์ ํŠธ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
  2. ML-Agents ํŒจํ‚ค์ง€๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
  3. ๊ฐ„๋‹จํ•œ 2D ํ”Œ๋žซํผ ๋ ˆ๋ฒจ์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค (๋ฐ”๋‹ฅ, ํ”Œ๋žซํผ, ์žฅ์• ๋ฌผ, ๋ชฉํ‘œ ์ง€์  ๋“ฑ).

ํ”Œ๋ ˆ์ด์–ด ์บ๋ฆญํ„ฐ ์„ค์ •

๊ฐ„๋‹จํ•œ ์บ๋ฆญํ„ฐ๋ฅผ ๋งŒ๋“ค๊ณ  ๋‹ค์Œ ์ปดํฌ๋„ŒํŠธ๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค:

  • ๐ŸŽญ Sprite Renderer
  • ๐Ÿ“ฆ Box Collider 2D
  • ๐Ÿƒโ€โ™‚๏ธ Rigidbody 2D

AI ์—์ด์ „ํŠธ ์Šคํฌ๋ฆฝํŠธ ์ž‘์„ฑ

using UnityEngine;
using Unity.MLAgents;
using Unity.MLAgents.Sensors;
using Unity.MLAgents.Actuators;

public class PlatformerAgent : Agent
{
    [SerializeField] private Transform targetTransform;
    [SerializeField] private float jumpForce = 7f;
    [SerializeField] private float moveSpeed = 5f;
    
    private Rigidbody2D rb;
    private bool isGrounded;
    private float groundCheckRadius = 0.1f;
    [SerializeField] private LayerMask groundLayer;
    [SerializeField] private Transform groundCheck;
    
    public override void Initialize()
    {
        rb = GetComponent<Rigidbody2D>();
    }
    
    public override void OnEpisodeBegin()
    {
        // ์œ„์น˜ ์ดˆ๊ธฐํ™”
        transform.localPosition = new Vector3(-8f, 0f, 0f);
        rb.velocity = Vector2.zero;
    }
    
    public override void CollectObservations(VectorSensor sensor)
    {
        // ์—์ด์ „ํŠธ ์œ„์น˜ ๋ฐ ์†๋„
        sensor.AddObservation(transform.localPosition);
        sensor.AddObservation(rb.velocity);
        
        // ๋ชฉํ‘œ๊นŒ์ง€์˜ ์ƒ๋Œ€์  ์œ„์น˜
        sensor.AddObservation(targetTransform.localPosition - transform.localPosition);
        
        // ๋ฐ”๋‹ฅ์— ๋‹ฟ์•˜๋Š”์ง€ ์—ฌ๋ถ€
        sensor.AddObservation(IsGrounded());
    }
    
    public override void OnActionReceived(ActionBuffers actionBuffers)
    {
        // ํ–‰๋™ ํ•ด์„
        float moveX = actionBuffers.ContinuousActions[0];
        bool jump = actionBuffers.DiscreteActions[0] == 1;
        
        // ์ด๋™
        Vector2 moveForce = new Vector2(moveX * moveSpeed, 0);
        rb.AddForce(moveForce);
        
        // ์ ํ”„
        if (jump && IsGrounded())
        {
            rb.AddForce(Vector2.up * jumpForce, ForceMode2D.Impulse);
        }
        
        // ๋ณด์ƒ ๊ณ„์‚ฐ
        float distanceToTarget = Vector3.Distance(transform.localPosition, targetTransform.localPosition);
        
        // ์ž‘์€ ์‹œ๊ฐ„ ํŒจ๋„ํ‹ฐ (๋นจ๋ฆฌ ๋ชฉํ‘œ์— ๋„๋‹ฌํ•˜๋„๋ก ์œ ๋„)
        AddReward(-0.001f);
        
        // ๋ชฉํ‘œ์— ๋„๋‹ฌํ–ˆ์„ ๋•Œ
        if (distanceToTarget < 1.5f)
        {
            AddReward(1.0f);
            EndEpisode();
        }
        
        // ๋–จ์–ด์กŒ์„ ๋•Œ
        if (transform.localPosition.y < -5f)
        {
            AddReward(-1.0f);
            EndEpisode();
        }
    }
    
    private bool IsGrounded()
    {
        return Physics2D.OverlapCircle(groundCheck.position, groundCheckRadius, groundLayer);
    }
    
    // ์ˆ˜๋™ ํ…Œ์ŠคํŠธ์šฉ
    public override void Heuristic(in ActionBuffers actionsOut)
    {
        var continuousActionsOut = actionsOut.ContinuousActions;
        var discreteActionsOut = actionsOut.DiscreteActions;
        
        continuousActionsOut[0] = Input.GetAxis("Horizontal");
        discreteActionsOut[0] = Input.GetKey(KeyCode.Space) ? 1 : 0;
    }
}

์ด ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์บ๋ฆญํ„ฐ์— ์ถ”๊ฐ€ํ•˜๊ณ , Behavior Parameters ์ปดํฌ๋„ŒํŠธ๋„ ์ถ”๊ฐ€ํ•ด์ฃผ์„ธ์š”:

  • Behavior Name: PlatformerAgent
  • Vector Observation > Space Size: 8
  • Actions > Continuous Actions Size: 1
  • Actions > Discrete Branches Size: 1 (Branch 0 Size: 2)

์ด์ œ ํ›ˆ๋ จ ์„ค์ • ํŒŒ์ผ์„ ๋งŒ๋“ค๊ณ  ํ›ˆ๋ จ์„ ์‹œ์ž‘ํ•ด๋ณด์„ธ์š”! ๐Ÿš€

๐Ÿ”ฅ ๋„์ „ ๊ณผ์ œ! ๐Ÿ”ฅ

์œ„ ์˜ˆ์ œ๋ฅผ ํ™•์žฅํ•ด์„œ ๋‹ค์Œ ๊ธฐ๋Šฅ์„ ์ถ”๊ฐ€ํ•ด๋ณด์„ธ์š”:

  1. ์  ์บ๋ฆญํ„ฐ ์ถ”๊ฐ€ํ•˜๊ธฐ (ํ”ผํ•ด์•ผ ํ•จ)
  2. ์•„์ดํ…œ ์ˆ˜์ง‘ ๊ธฐ๋Šฅ ์ถ”๊ฐ€ํ•˜๊ธฐ
  3. ์—ฌ๋Ÿฌ ๋ ˆ๋ฒจ ๋””์ž์ธํ•ด์„œ ์ปค๋ฆฌํ˜๋Ÿผ ํ•™์Šต ์ ์šฉํ•˜๊ธฐ

์ด๋Ÿฐ ๋„์ „์„ ํ†ตํ•ด ML-Agents์˜ ๋‹ค์–‘ํ•œ ๊ธฐ๋Šฅ์„ ์‹คํ—˜ํ•ด๋ณผ ์ˆ˜ ์žˆ์–ด์š”!

์žฌ๋Šฅ๋„ท์—์„œ๋Š” ์ด๋Ÿฐ AI ๊ฒŒ์ž„ ๊ฐœ๋ฐœ ๊ด€๋ จ ์žฌ๋Šฅ์„ ๊ณต์œ ํ•˜๊ณ  ๊ฑฐ๋ž˜ํ•  ์ˆ˜ ์žˆ์–ด์š”. ์—ฌ๋Ÿฌ๋ถ„์ด ๋งŒ๋“  AI ์บ๋ฆญํ„ฐ๋‚˜ ๊ฒŒ์ž„ ์‹œ์Šคํ…œ์„ ๋‹ค๋ฅธ ๊ฐœ๋ฐœ์ž๋“ค๊ณผ ๊ณต์œ ํ•ด๋ณด๋Š” ๊ฑด ์–ด๋–จ๊นŒ์š”? ๐Ÿค

8. ์ž์ฃผ ๋ฌป๋Š” ์งˆ๋ฌธ๊ณผ ๋ฌธ์ œ ํ•ด๊ฒฐ โ“

Q: ํ›ˆ๋ จ์ด ๋„ˆ๋ฌด ์˜ค๋ž˜ ๊ฑธ๋ ค์š”. ์–ด๋–ป๊ฒŒ ํ•˜๋ฉด ๋นจ๋ผ์งˆ๊นŒ์š”?

A: ๋‹ค์Œ ๋ฐฉ๋ฒ•์„ ์‹œ๋„ํ•ด๋ณด์„ธ์š”:

  1. ์—ฌ๋Ÿฌ ํ™˜๊ฒฝ์„ ๋™์‹œ์— ์‹คํ–‰ (--num-envs ํŒŒ๋ผ๋ฏธํ„ฐ ์‚ฌ์šฉ)
  2. ๋” ๊ฐ„๋‹จํ•œ ์‹ ๊ฒฝ๋ง ๊ตฌ์กฐ ์‚ฌ์šฉ
  3. ๊ด€์ธก ์ •๋ณด ์ค„์ด๊ธฐ
  4. GPU๊ฐ€ ์žˆ๋‹ค๋ฉด ํ™œ์šฉํ•˜๊ธฐ

Q: ์—์ด์ „ํŠธ๊ฐ€ ์ „ํ˜€ ํ•™์Šตํ•˜์ง€ ์•Š์•„์š”!

A: ๋‹ค์Œ ์‚ฌํ•ญ์„ ํ™•์ธํ•ด๋ณด์„ธ์š”:

  1. ๋ณด์ƒ ํ•จ์ˆ˜๊ฐ€ ์ œ๋Œ€๋กœ ์„ค๊ณ„๋˜์—ˆ๋Š”์ง€ ํ™•์ธ (๋„ˆ๋ฌด ํฌ๋ฐ•ํ•˜์ง€ ์•Š์€์ง€)
  2. ๊ด€์ธก ์ •๋ณด๊ฐ€ ์ถฉ๋ถ„ํ•œ์ง€ ํ™•์ธ
  3. ํ•™์Šต๋ฅ ์ด ์ ์ ˆํ•œ์ง€ ํ™•์ธ (๋„ˆ๋ฌด ๋†’๊ฑฐ๋‚˜ ๋‚ฎ์„ ์ˆ˜ ์žˆ์Œ)
  4. ํ™˜๊ฒฝ์ด ๋„ˆ๋ฌด ์–ด๋ ต์ง€ ์•Š์€์ง€ ํ™•์ธ (์ฒ˜์Œ์—๋Š” ๊ฐ„๋‹จํ•œ ๋ฒ„์ „์œผ๋กœ ์‹œ์ž‘)

Q: ํ›ˆ๋ จ๋œ ๋ชจ๋ธ์„ ๊ฒŒ์ž„์— ์–ด๋–ป๊ฒŒ ์ ์šฉํ•˜๋‚˜์š”?

A: ํ›ˆ๋ จ์ด ์™„๋ฃŒ๋˜๋ฉด results ํด๋”์— .onnx ํŒŒ์ผ์ด ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค. ์ด ํŒŒ์ผ์„ Unity ํ”„๋กœ์ ํŠธ๋กœ ๊ฐ€์ ธ์™€์„œ Behavior Parameters ์ปดํฌ๋„ŒํŠธ์˜ Model ํ•„๋“œ์— ์—ฐ๊ฒฐํ•˜๊ณ , Behavior Type์„ 'Inference Only'๋กœ ์„ค์ •ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.

Q: ์—ฌ๋Ÿฌ ์—์ด์ „ํŠธ๊ฐ€ ํ˜‘๋ ฅํ•˜๋„๋ก ๋งŒ๋“ค ์ˆ˜ ์žˆ๋‚˜์š”?

A: ๋„ค! ML-Agents๋Š” ๋ฉ€ํ‹ฐ์—์ด์ „ํŠธ ํ™˜๊ฒฝ์„ ์ง€์›ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๋Ÿฌ ์—์ด์ „ํŠธ๊ฐ€ ๊ฐ™์€ ์ •์ฑ…์„ ๊ณต์œ ํ•˜๊ฑฐ๋‚˜, ์„œ๋กœ ๋‹ค๋ฅธ ์ •์ฑ…์„ ๊ฐ€์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํ˜‘๋ ฅ ํ•™์Šต์„ ์œ„ํ•ด์„œ๋Š” ํŒ€ ๋ณด์ƒ์„ ์„ค๊ณ„ํ•˜๋Š” ๊ฒƒ์ด ์ค‘์š”ํ•ฉ๋‹ˆ๋‹ค.

๋งˆ๋ฌด๋ฆฌ: ์—ฌ๋Ÿฌ๋ถ„์˜ AI ๊ฒŒ์ž„ ๊ฐœ๋ฐœ ์—ฌ์ •์ด ์‹œ์ž‘๋์–ด์š”! ๐Ÿš€

์ง€๊ธˆ๊นŒ์ง€ Unity ML-Agents๋ฅผ ํ™œ์šฉํ•ด AI ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ๋ฅผ ํ›ˆ๋ จ์‹œํ‚ค๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•ด ์•Œ์•„๋ดค์–ด์š”. ์ฒ˜์Œ์—๋Š” ์–ด๋ ค์›Œ ๋ณด์ผ ์ˆ˜ ์žˆ์ง€๋งŒ, ํ•˜๋‚˜์”ฉ ๋”ฐ๋ผํ•˜๋‹ค ๋ณด๋ฉด ์ƒ๊ฐ๋ณด๋‹ค ์‰ฝ๊ฒŒ ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋‹ต๋‹ˆ๋‹ค! ใ…‹ใ…‹ใ…‹

ML-Agents๋Š” ๊ฒŒ์ž„ ๊ฐœ๋ฐœ์˜ ์ƒˆ๋กœ์šด ์ง€ํ‰์„ ์—ด์–ด์ฃผ๋Š” ๊ฐ•๋ ฅํ•œ ๋„๊ตฌ์ž…๋‹ˆ๋‹ค. ๋‹จ์ˆœํžˆ ๊ทœ์น™ ๊ธฐ๋ฐ˜ AI๋ฅผ ๋„˜์–ด, ์‹ค์ œ๋กœ ํ•™์Šตํ•˜๊ณ  ์ ์‘ํ•˜๋Š” ์ง€๋Šฅํ˜• ์บ๋ฆญํ„ฐ๋ฅผ ๋งŒ๋“ค ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ์ฃ !

์ด ๊ธฐ์ˆ ์„ ๋งˆ์Šคํ„ฐํ•˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๊ฒƒ๋“ค์ด ๊ฐ€๋Šฅํ•ด์ง‘๋‹ˆ๋‹ค:

  1. ๋” ์ž์—ฐ์Šค๋Ÿฝ๊ณ  ๋„์ „์ ์ธ NPC ํ–‰๋™ ํŒจํ„ด ๊ตฌํ˜„
  2. ํ”Œ๋ ˆ์ด์–ด ์Šคํƒ€์ผ์— ์ ์‘ํ•˜๋Š” ๋™์  ๋‚œ์ด๋„ ์กฐ์ ˆ
  3. ์ ˆ์ฐจ์  ์ฝ˜ํ…์ธ  ์ƒ์„ฑ๊ณผ ๊ฒฐํ•ฉํ•œ ๋ฌดํ•œํ•œ ๊ฒŒ์ž„ ๊ฒฝํ—˜
  4. ๊ฒŒ์ž„ ํ…Œ์ŠคํŠธ ์ž๋™ํ™” ๋ฐ ๋ฐธ๋Ÿฐ์‹ฑ ์ตœ์ ํ™”

์žฌ๋Šฅ๋„ท์—์„œ๋Š” ์ด๋Ÿฐ ์ตœ์‹  ๊ฒŒ์ž„ ๊ฐœ๋ฐœ ๊ธฐ์ˆ ์— ๊ด€ํ•œ ๋‹ค์–‘ํ•œ ์žฌ๋Šฅ์„ ์ฐพ์•„๋ณผ ์ˆ˜ ์žˆ์–ด์š”. ์—ฌ๋Ÿฌ๋ถ„์ด ML-Agents๋กœ ๋งŒ๋“  ๋…ํŠนํ•œ AI ์‹œ์Šคํ…œ์ด๋‚˜ ๊ฒŒ์ž„ ๋ฉ”์ปค๋‹ˆ์ฆ˜์„ ๊ณต์œ ํ•˜๊ฑฐ๋‚˜, ๋‹ค๋ฅธ ๊ฐœ๋ฐœ์ž์˜ ๋…ธํ•˜์šฐ๋ฅผ ๋ฐฐ์šธ ์ˆ˜ ์žˆ๋Š” ์ข‹์€ ํ”Œ๋žซํผ์ด์ฃ ! ๐Ÿ’ก

์—ฌ๋Ÿฌ๋ถ„์˜ ์ฒซ ๋ฒˆ์งธ AI ๊ฒŒ์ž„ ์บ๋ฆญํ„ฐ๋Š” ์–ด๋–ค ๋ชจ์Šต์ผ๊นŒ์š”? ์ƒ์ƒ๋ ฅ์„ ๋ฐœํœ˜ํ•ด์„œ ๋…ํŠนํ•œ ํ•™์Šต ํ™˜๊ฒฝ์„ ๋งŒ๋“ค์–ด๋ณด์„ธ์š”! ๊ทธ๋ฆฌ๊ณ  ๊ทธ ๊ณผ์ •์—์„œ ๊ถ๊ธˆํ•œ ์ ์ด ์žˆ๋‹ค๋ฉด ์–ธ์ œ๋“  ์งˆ๋ฌธํ•ด์ฃผ์„ธ์š”. ํ•จ๊ป˜ ์„ฑ์žฅํ•˜๋Š” ๊ฐœ๋ฐœ์ž ์ปค๋ฎค๋‹ˆํ‹ฐ๊ฐ€ ์—ฌ๋Ÿฌ๋ถ„์„ ๊ธฐ๋‹ค๋ฆฌ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค! ๐Ÿ™Œ

๊ทธ๋Ÿผ ์ด์ œ Unity๋ฅผ ์ผœ๊ณ  ML-Agents์˜ ์„ธ๊ณ„๋กœ ๋›ฐ์–ด๋“ค ์ค€๋น„๊ฐ€ ๋๋‚˜์š”? Let's go~! ๐Ÿš€

๋Œ“๊ธ€ ์ž‘์„ฑ

์ด ๊ธ€์— ๋Œ€ํ•œ ์—ฌ๋Ÿฌ๋ถ„์˜ ์ƒ๊ฐ์„ ๋“ค๋ ค์ฃผ์„ธ์š”

๋Œ“๊ธ€ 0
๋Œ“๊ธ€์„ ๋ถˆ๋Ÿฌ์˜ค๋Š” ์ค‘์ž…๋‹ˆ๋‹ค.

์•„์ง ๋Œ“๊ธ€์ด ์—†์Šต๋‹ˆ๋‹ค.