Idea: Neural Net Learning

[Kilarin]

Xaa:

someone could start an "Idea" thread, if they want. However, please don't be suprised if I completely and utterly ignore it and just do whatever the hell I want, instead.

Absolutely! That was assumed. Suggestions are fine, but people should understand that suggestions are JUST suggestions, not orders.

Since you are playing with "training" an AI, I have a dream!

Something I mentioned a long time ago on Siege Network that was simply too difficult to do under the DS engine:

------------------------(begin old siegenetwork post)
I would like to be able to script some of my characters behavior without having to modify the game. A very simple scripting language would be fantastic. Something where I could say:

if ((HEALTH<100%) and (MANA>50%) and (NOT-IN-COMBAT)) cast HEALING-SPELL;
if ((NO-FAMILIAR) and (MANA>50%) and (NOT-IN-COMBAT)) cast FAMILIAR-SPELL;
if ((NOT-ACTIVE("ACCURACY")) and (MANA>50%) and (NOT-IN-COMBAT)) cast ACCURACY;
if ((NO-AMMO("DRAGON-MINI-GUN")) ready "HEART BOW";

You get the idea. The important point would be that I could setup or modify a script in-game. If I get a new spell or develop a new strategy I could easily pop up a characters AI script and make a simple modification on the fly. Unfortunately, having played with writing compilers, I know that this "simple" scripting language would probably be VERY complicated to implement. If possible at all. Not to mention the fact that unless you develop a very simple gui interface, even at "simple" it will be too complicated for most of the game playing public. Alas, it's probably just a dream.
--------------------------(end old siegnetwork post)

Being a programmer, I LOVE the idea of being able to write programs to run the AI of my character. BUT, I realize that this is NOT something that would interest the VAST majority of players. ALSO, it has the distinct disadvantage of being easily transferable. That means once some generous person generates a really top notch AI script, and then puts it online for download, EVERYONE will have the same advantages.

SO, your "training" for fencing idea inspired me on another possibile implementation of this idea.
Assuming you are turning the AI into a primitive "Neural Net" learning system, why not let the character learn MORE than just fencing styles? (which was probably already something you were thinking of) What if the character was "trained" in virtually every behavior.

For example, instead of a predetermined autodrink number. The AI will start out never using autodrink. But every time the player clicks "drink", the AI remembers the circumstances. Specifically, was he in combat or not, and what was his health or mana level at the time. You could, of course, expand that with lots of other variables, (how many enemies left, etc) but those two would be adequate. The characters "Neural Net" starts building up a habit. The next time those conditions are met, the chance of the character autodrinking, all by themselves, goes up. Keep repeating the same instructions, and soon the character will be autodrinking all by themselves.

If the player cast "summon killer gremel" whenever the character is at full mana, not in combat, and doesn't have a summon, then the AI will start building up a habit based on that, and will eventually start doing it all by themselves. If later the player wants the character to start summoning a green drake instead, they will have to start overriding the character and building up a NEW habit. At first the character will try to summon a gremel every time. After a while, the character will summon a gremel sometimes, and a drake sometimes. Eventually they learn to always summon the drake.

To REALLY make this work, you would need some equivalent of a "reward" and "punish" button. Preferably tied to a "Actions in the last 10 seconds" list. The character stops in the middle of combat and cast "sense heading". NOT GOOD! It's too late for the player to override the behavior, so they pull up the "actions in the last 10 seconds" list and let the character know that THAT action was BAD. NAUGHTY NAUGHTY!!! The "Neural Net" reduces the chance that it will repeat that behavior by a little bit.

An alternative to the "actions in the last 10 seconds" list that may or may not work would be to have the character pop up a small warning that they are about to do something, (cast a spell or drink a potion or switch weapons, etc). The player has a second or two to CANCEL and override the action then, which accomplishes the same thing as the above, but would be more intuitive for most players to use. That short delay might cause problems though, you sometimes need the character to do drink RIGHT THEN, not two seconds AFTER they reach the proper conditions. And, of course, you couldn't use it for direct combat behaviors at all.

The cool thing about this "learning system" would be that, after working and working on training your character, you wouldn't JUST have a character that was more powerful, you would have a character with trained responses that were better than most human reaction times (depending on how you implement it). The characters wisdom, intelligence, and skill would be a reflection of the time, effort, and wisdom that went into training it.

The downside? This would be a NIGHTMARE project. HUGE, COMPLICATED, NASTY to program. But OH! It would be fun to play...

[Xaa]

kilarin said:

Since you are playing with "training" an AI, I have a dream!

This is kind of along the lines I was thinking of, yes. I like the idea of the character's 'experience' being what you have trained them to do, rather than an arbitrary (and somewhat imaginary) ranking of experience points. I don't know how to implement it in a game, though.

[Iarus]

Kilarin are you talking about the game Black and White from Lion Head Studio? Because it look like a pretty similar learning system. Teach your giant cow to cast lightning on the enemy villages, when he does, pet him. When that big dumb beast cast it on your village, you slap him. And you just hope he'll understand what is a good beavior

[Kilarin]

Iarus:

Kilarin are you talking about the game Black and White from Lion Head Studio?

Really? No, I've never played that game. The descriptions I heard of Black and White never really appealed to me, but I didn't remember reading anything about this kind of training system.

Blondin235:

it strikes me that the actions could be triggered in the same way as training is in Mageworld

I'm not certain that I understand exactly how the system you are describing would work.

I AM a programmer, but I've never actually worked on a Neural Net, so I'm probably out in left field as well. I HAVE read a lot about Neural Nets, they are pretty cool.

Perhaps I should explain, a Neural Net is usually a software construct, not hardware. It's just a program, but it works in a unique manner, it LEARNS. When the NN(Neural Net) does something RIGHT, you reward it. The reward actually consists of increasing the strength of the connections that fired off the wanted behavior. This increases the chance that the behavior will occur again. When the NN does something wrong, you punish it. the punishment is nothing more than weakening all of the connections that fired off the unwanted behavior. Over time, all of the connections are strengthened or weakened appropriately to make the correct behavior occur more often the not.

NN's learn very fast, they can learn in a few hours to do things that would require programmers months or years to program in detail by detail. But they DO have their flaws. They seem to reach peak ability very soon and can't continue to improve significantly. (At least last time I checked the state of the art, which was years and years ago)

For example, in one famous NN experiment they wrote a NN program. It had NO rules of grammar built into it. None, it didn't understand a single thing about how words or sentences were put together.

Then, they started feeding it books. The NN would "read" a word and make sounds. At first the sounds were entirely random. BUT, every time the sounds just happened to sound a LITTLE bit like the word the NN had just read, the connections that fired that sound would be "rewarded". When the sounds didn't match the word at all, the connections were "punished". In less than a day, the NN "Learned" how to read with somewhere around 80% accuracy.

Now that is an absolutely INCREDIBLE accomplishment. Programming that one detail at a time is the work of years. However, the downside was discovered almost immediately. 80% seemed to be the best the program could do. It couldn't get any better. It might take programmers years and years to put all of the rules of reading and speech into code, but once they've done it, thier programs can out perform the NN.

Hmm, sorry, I'm babbling. I LIKE talking about programming.

Anyway, point was, IF an attempt was made to design a system like this, I think a good approach would be to design a VERY primitive NN, with conditions prepped for the needs of the game. For example, you wouldn't need to concentrate on the state of the entire machine in order to determine when to drink a health potion. Pick a few important related variables and monitor those only, thus simplifying the process by quite a bit.

[Xaa]

kilarin said:

Really? No, I've never played that game.

You didn't miss much. Story was good, graphics good, music good. And yes, the game does have NN programming for the creatures you control, but it's quirky and limited. Overall, it would have been a great game if it weren't for the final stage of the game, which sucked so bad it wasn't even funny. You spend literally 24 to 36 hours of gameplay building your creature, only to have it get scragged down to nothing in about an hour of gameplay, with all your work wasted. It sucked.





One of the few games I used a cheat on just so I could get to see the ending. The end was a "decision" ending - you choose to sacrifice your creature and end the war now, or you don't and the war goes on forever (the game ends there no matter what you choose). I chose the former - very heart-rending in-game cut-scene, overall I give 5 stars for the story. But the last stage, "Land 5", sucked so bad it makes me say "DO NOT BUY THIS GAME."

And "Creature Isle", the sequel, just stank. Definitely not worth getting. I think there's a third that's either on it's way or already released, but I lost interest.

[delcar]

Black and White 2 is on the horizon. They're trying to talk it up as the end-all-be-all city builder (a la Pharaoh/Ceaser etc) AND an awesome RTS.

I myself gave up on the scripted campaign, and opted to raise my critter in skirmish mode; I see now that it was a good decision.

[myncknm]

Neural network? Hey, why not make it actually *think*? We could give it memory, logical reasoning, causal reasoning, the ability to apply stuff learned about one thing to a different thing, the ability to analyze; heck, we could make it sapient!

I so want to do that.

[artikia]

A good way to use NN is to make a RPG that is fight-based. What I mean by this is the fight actions are like a console game, you have to hit commands in sequence for higher attacks more powerful then a single attack. If used with a NN the more common combo for you to use might become what the char would naturally use when attacked. while this may seem only good for melee fighters it would work for any char-type. you would only need the mouse or arrows for movement and 4 other input keys (numpad or WASD would work well).

This would eliminate the exp cheats some players love and require that the player actually have to use their grey matter. However I still like the idea that you have to find new attacks and spells so maybe combine the combo command with your craft hunt idea u used in MW. The char still has to find a trainer, train by using lower lvl attacks, or some how acquire the new abilities before being able to use them. What I mean is that a player can't just ramdomly hit keys and find a super powerful attack at the beginning he has to work up to it.

Well I will think on this some more and try to think of more concrete explanations of what I am suggesting, but I remembered Xaa said soming like this way way way back. So I guess I'll wait for coments while I read some Xaa forum arcives.