How does being Curious affect the actions of an algorithm?

By Nic LazThursday, Aug 15, 2019

“As human agents, we are accustomed to operating with rewards that are so sparse that we only experience them once or twice in a lifetime, if at all. To a three-year-old enjoying a sunny Sunday afternoon on a playground, most trappings of modern life – college, good job, a house, a family – are so far into the future, they provide no useful reinforcement signal. Yet, the three-year-old has no trouble entertaining herself in that playground using what psychologists call intrinsic motivation or curiosity.”

 

The researchers used statistical modeling of the exploration of a 2 and 3 dimensional video games by an experimental machine-learning algorithm. Their discoveries lead them to demonstrate the impact that curiosity has on navigation and learning by these algorithm. In this innovative approach, they altered whether the algorithm had internal or external rewards for exploring these virtual spaces, a system which is innate in humans.

 

Rewards are important to us humans in learning and discovery, psychologists call this intrinsic motivation, our brain is wired to fire off pleasurable neurotransmitters when we receive a stimulus we like – think about it next time you eat a piece of chocolate or enjoy the first sip of your favorite beverage.

 

These reward systems are built-into where “curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and learn skills that might be useful later in its life.” and in exploring how this would work for a computer algorithm they created the Intrinsic Curiosity Model and have tested it against two types of video games :

 

 

The researchers from University of California at Berkeley put it to the same test as putting a mouse through a maze with and without rewards – with the innovation of being able to control how the reward worked at a ‘cognitive’ level of the program and that we can see at a first-person view what the exploration was like – you can see the algorithm’s explorations here :

 

 

In comparing the results of their ‘curiosity-driven’ model with other algorithms available in the exploration of the video games, this model significantly outperforms them. Demonstrating the utility of being Curious and rewarded for the Curious exploration, leading to greater discoveries for the algorithm at play.

 

This is a reflection of how us humans exploring Curiously should be rewarded, it increases their curiosity and discovery – at TENCLUB you are rewarded for this sort of explorations. Through the Tables of Ten and Circles of Fascination you will meet people that are rewarding your efforts, along with feeding your Curiosity for future explorations.

 

You can read the full paper, download the algorithm code, and discover more by clicking here.

 

Join us in exploring anything Curious, it is your real-world playground.

 

We are here for you.