Exploration and Exploitation are two fundamental processes in reinforcement learning. In short, exploration is about learning something new, and exploitation is about making decisions based on known information. Balancing these two is crucial.
Exploration
Through this process, reinforcement learning involves learning new things. The more a model can explore, the better results it can provide.
Exploitation
Through this process, reinforcement learning involves learning in a new environment. Results are primarily determined through exploitation.
Dilemma
By now, we understand that exploitation is essential for receiving rewards; however, exploration is also necessary for making good and accurate decisions.
The question arises, which should the agent prioritize more, exploitation or exploration? Suppose a robot gets a reward for completing a task, and it gets another reward for charging its battery. Which reward will it choose? Without charging the battery, it can’t function, and without doing the task, it won’t get the reward.
Another major problem here is that if an agent doesn’t fail after taking an action, it won’t learn; the goal here is to make precise decisions.
Therefore, balancing exploration and exploitation is crucial. Agents must try different steps and progressively move towards the ones that seem the best.
Some Examples of Exploration and Exploitation
Here are some real-life examples of exploration and exploitation:
Restaurant Selection
Suppose you really like the food at a particular restaurant and go there every day. This doesn’t necessarily mean that this restaurant has the best food. There could be better restaurants out there. Since you’re not trying a new restaurant, you don’t know if there’s a better one.
Here, going to the same restaurant repeatedly is exploitation.
And if you go to another restaurant in search of better food, that would be exploration.
Advertising
Suppose you are an officer in the marketing department of a multinational company. You advertise your product on television, and currently, you are getting a good response from your customers. In this situation, you don’t know if you could get a better response from customers through other means. Suddenly, you place an online advertisement and get a tremendous response. This means your market is better online than on television.
Here, regularly advertising on television is exploitation.
And searching for an alternative online is exploration.
Many more examples can be given.