Microsoft Used AI to Master Ms. Pac-Man

M

monkeymagick

[H]News
Joined
Jun 22, 2008
Messages
480
While I'm still trying to get to Donkey Kong's kill screen myself, researchers at Microsoft developed artificial intelligience using reinforcement learning to beat Ms. Pac-Man. The team from Maluuba, recently acquired by Microsoft this year specifically used the Atari 2600 version of Ms. Pac-Man and was able to achieve the max score of 999,990. And why not just plain ol' Pac-Man, the game was designed harder to be less predicatable than regular Pac-Man and to steal your quarters. The method the group called "Hybrid Reward Architecture," used agents to find the best solutions to different assigned tasks. Working in parallel each agent would be rewarded for completing the tasks and achieved better results when acting more egotistically, not unlike a good game of Battlefield.

Check out the video.

...significant achievement among AI researchers, who have been using various videogames to test their systems but have found Ms. Pac-Man among the most difficult to crack.

It may seem strange that it takes some of the most advanced AI research methods to beat something as seemingly simple as a 1980s Atari game. But Rahul Mehrotra, a program manager at Maluuba, said figuring out how to win these types of videogames is actually quite complex, because of the huge variety of situations you can encounter while playing the game.
 
the link goes to a blank page.
never mind, it's up now.
 
in order words they used Pavlovian conditioning.

I can't consider these primitive programs with singular purposes to be true ai. There is nothing intelligent about it. It still follows a set of rules laid out by it's programming.
 
M76 said:
in order words they used Pavlovian conditioning.

I can't consider these primitive programs with singular purposes to be true ai. There is nothing intelligent about it. It still follows a set of rules laid out by it's programming.
Click to expand...

It is, in fact, considered AI.

AI Caliber 1) Artificial Narrow Intelligence (ANI): Sometimes referred to as Weak AI, Artificial Narrow Intelligence is AI that specializes in one area. There’s AI that can beat the world chess champion in chess, but that’s the only thing it does. Ask it to figure out a better way to store data on a hard drive, and it’ll look at you blankly.

AI Caliber 2) Artificial General Intelligence (AGI): Sometimes referred to as Strong AI, or Human-Level AI, Artificial General Intelligence refers to a computer that is as smart as a human across the board—a machine that can perform any intellectual task that a human being can. Creating AGI is a much harder task than creating ANI, and we’re yet to do it. Professor Linda Gottfredson describes intelligence as “a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience.” AGI would be able to do all of those things as easily as you can.

AI Caliber 3) Artificial Superintelligence (ASI): Oxford philosopher and leading AI thinker Nick Bostrom defines superintelligence as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” Artificial Superintelligence ranges from a computer that’s just a little smarter than a human to one that’s trillions of times smarter—across the board. ASI is the reason the topic of AI is such a spicy meatball and why the words “immortality” and “extinction” will both appear in these posts multiple times.
Click to expand...

Source
 
theplaidfad said:
It is, in fact, considered AI.
Click to expand...
I know the official definition. I still disagree with it.
All we have today is type 1. Yet most people think of type 2 and type 3 when AI is mentioned in the mainstream.
 
Well, you're certainly entitled to your own opinion.

Any system given a set of programming and is then capable of learning to adapt and evolve itself is the definition of AI in my opinion.

What's really cool about the example in the OP is that the AI started to get more brazen with its strategy as it got positive results.
 
M76 said:
in order words they used Pavlovian conditioning.

I can't consider these primitive programs with singular purposes to be true ai. There is nothing intelligent about it. It still follows a set of rules laid out by it's programming.
Click to expand...

The term AI often gets thrown around willy-nilly in place of the more appropriate term, neural network. However, a neural network does have the capacity for complex learning via a very simple set of principals. All configurations of segments of the neural net are iterated through and judged based on the ultimate outcome being positively or negatively compared (power of will) to a hardcoded goal (prolonged existance). The fact that such an applied neural network can learn and then apply newly derived logic to anything is, by definition, a form of intelligence.

While it is a bleak and puny little intelligence, it's still relevant. A neural network gradually encodes what you might call intelligence based on the same beneficial responses to stimuli that created our intelligence. Obviously, the extreme limitations of transistors as a medium will limit the speed it takes to solve useful problems, but it also limits any natural manifestation of frivolous features such as accidents, distractions, secrets or hidden motives, trust, et cetera. On the other hand, complex neural net configurations such as cooperative or adversarial neural nets are slower (due to medium), but can learn complex thoughts such as how to quickly decide between efficiency or fidelity in making decisions based on circumstance (impulsive vs thoughtful). I consider this to be the first major step toward singularity as the logic can be configured to design it's own ideal hardware to hasten the network.

Besides, Pavlovian conditioning would only work on something intelligent. ;)
 
I don't think this 2600 game had ever been beaten to that level by a human before is because it is a piece of shit and no one card enough to waste that much time on it!
But, a human has beaten Mrs. Pacman arcade. But not just that. The human never lost even one life. Never missed a single piece of fruit, and ate every ghost with every power pellet taken!
A.I. will one day be able to do this as well, but it is just deeper and deeper pattern learning.
 
funkydmunky said:
I don't think this 2600 game had ever been beaten to that level by a human before is because it is a piece of shit and no one card enough to waste that much time on it!
But, a human has beaten Mrs. Pacman arcade. But not just that. The human never lost even one life. Never missed a single piece of fruit, and ate every ghost with every power pellet taken!
A.I. will one day be able to do this as well, but it is just deeper and deeper pattern learning.
Click to expand...
yeah that is what I was thinking when I saw it "ewwww Atari 2600 version...."
 
velusip said:
Besides, Pavlovian conditioning would only work on something intelligent. ;)
Click to expand...
It is positive / negative reinforcements. It doesn't need intelligence, if they hard coded that result A is good and any other results is bad.
Yes there needs to be the ability to be able to connect A to B in an animal. But in this case the program is hard wired to expect result A and threat everything else as bad. So I think it has no intelligence attached to it. Intelligence only plays a role if the subject connects the dots itself. Here the dots are pre-programmed.
 
M76 said:
It is positive / negative reinforcements. It doesn't need intelligence, if they hard coded that result A is good and any other results is bad.
Yes there needs to be the ability to be able to connect A to B in an animal. But in this case the program is hard wired to expect result A and threat everything else as bad. So I think it has no intelligence attached to it. Intelligence only plays a role if the subject connects the dots itself. Here the dots are pre-programmed.
Click to expand...

Human learning is a pre-programmed response to external stimuli. Some may refer to that as instinct. We're all born with that instinct. That programming took place over the ages via evolution. How is this not the same thing but on a much shortened timeline?
 
theplaidfad said:
Human learning is a pre-programmed response to external stimuli. Some may refer to that as instinct. We're all born with that instinct. That programming took place over the ages via evolution. How is this not the same thing but on a much shortened timeline?
Click to expand...
I don't know how to articulate it.
With human learning the, expected result is not a known quantity. That's where intelligence and the ability to reason comes in, where the subject realizes that something is a direct result of their actions. And also realizes that the actions are beneficial to them.

These AIs cannot do either of those things. They're pre-programmed to keep trying until they get the desired result.

I'll call it intelligence when the AI figures out on it's own that beating the game is the desired result. Not just how to beat the game.
 
theplaidfad said:
Not to come across as argumentative, but that's exactly how the most basic learning for basic human behavior patterns work, which was my point.
Click to expand...

Give it emotions. Then, you'll have a depressed AI that just doesn't want to play.

I think people are wanting to compare this to human intelligence. Which it kind of should be. However, this is emotionless. It has something to do, and it does it. It learns from it's mistakes. It doesn't have the desire to go learn about boobies because it's focused on the task at hand. There is no desires, there is no emotion, there is only the task at hand and completing it.

I wouldn't want to give it emotion or a desire to be 'better'. Then, you get Skynet. :)
 
theplaidfad said:
Not to come across as argumentative, but that's exactly how the most basic learning for basic human behavior patterns work, which was my point.
Click to expand...
And you missed my point completely. This AI wouldn't know if a result is good or bad if it wasn't pre-programmed with it. A human can figure out when it beat the game. This AI can only beat the game when you tell it what beating the game looks like.
 
They showed the AI how to play the game, not how to beat the game. The AI learned how to beat the game based off of the initial programming it received. The ghost get you, that's bad... try something different. It doesn't get you... that's good, keep doing that. This is exactly how a human mind would approach the game.

I already showed you, with sources, that by definition, this was an AI. I find your reasoning for not considering it AI (which people a lot smarter than you or I would disagree with) pretty baffling.
 
M76 said:
It is positive / negative reinforcements. It doesn't need intelligence, if they hard coded that result A is good and any other results is bad.
Yes there needs to be the ability to be able to connect A to B in an animal. But in this case the program is hard wired to expect result A and threat everything else as bad. So I think it has no intelligence attached to it. Intelligence only plays a role if the subject connects the dots itself. Here the dots are pre-programmed.
Click to expand...

The thing is, the neural net already connected the dots. The plaidfad said it well. There's a lot of learning that we are not shown in the final result. It was slowly learning how to play while building it's own neural net long before this iteration was demonstrated. It's equivalent to long-term evolution, but the success of each iteration is more efficiently applied to the next. Some approximations of genetic models have also been modeled in computation for use in neural nets, but the truth is, a neural net yields results in a more apt way.

M76 said:
And you missed my point completely. This AI wouldn't know if a result is good or bad if it wasn't pre-programmed with it. A human can figure out when it beat the game. This AI can only beat the game when you tell it what beating the game looks like.
Click to expand...

I think you are expecting way too much of this AI. Satisfaction, achievement, task-reward, self-worth... come on. That's all frivolous stuff that will come later with more interlinked neural nets. This is just the core logic part.
 
theplaidfad said:
Then by your definition AI could never exist. Try again.
Click to expand...
Wrong, but being programmed and free thought might be different, imo.
Moving beyond the programming is the key. If it can become self aware, yes, thats ai. If it can play pacman, its not really. Rofl
 
Except that you're wrong and its called general AI. See my first post in this thread for reference.

You're entitled to your own opinion, but not your own set of facts.
 
theplaidfad said:
Except that you're wrong and its called general AI. See my first post in this thread for reference.

You're entitled to your own opinion, but not your own set of facts.
Click to expand...
Which is somebody's elses opinion. :) So we are all wrong and right.

I would like to have a nice chat with the AI machine, since it can play pacman........ I just hope it doesn't take over the world!
 
You must log in or register to reply here.
Back
Top