What Is Reinforcement Learning: How Machines Learn by Trial and Error

March 2016, second game between AlphaGo and Go champion Lee Sedol. On move 37 the program plays a position no human would have chosen. Commentators assume it is a mistake. It was instead a brilliant move that nobody had ever taught the machine. AlphaGo had discovered it on its own, playing millions of games against itself.

That is exactly what reinforcement learning is, a type of machine learning where an agent learns to act in an environment by receiving rewards or penalties. It is not given labeled examples. It discovers on its own which actions lead to the best outcome, much like a child learning by trial and error. The goal is to maximise the total reward over time.

In our article on machine learning reinforcement learning appears as one of the three main paradigms. Here we go inside that paradigm, to understand what it is, how it works, which methods it uses, and where it is already changing the world. If you sense this is your field, our Admissions Team can help you find the right path.

What is reinforcement learning: definition and core idea

Reinforcement learning is a way of training a machine that looks more like experience than study. It does not learn from a correct answer given in advance, but from the consequences of its own actions.

Learning from reward and penalty, not from examples

Think of training a dog. You do not show it a thousand photos of the right behaviour, you give it a treat when it does the right thing. A reinforcement learning agent reasons the same way. It tries an action, receives a positive or negative signal, and adjusts its behaviour accordingly.

Where it sits among the types of machine learning

In supervised learning the model learns from examples with the correct answer. In unsupervised learning it finds structure in unlabeled data. In reinforcement learning there is no correct answer given in advance. The agent experiments, makes mistakes, and improves based on the feedback it receives from the environment.

Type of machine learning	How it learns	Example
Supervised learning	From examples with a known answer	Detecting spam in email
Unsupervised learning	By finding structure in unlabeled data	Segmenting e-commerce customers
Reinforcement learning	From rewards and penalties, by trial and error	An agent learning to play Go

How it works: agent, environment, action, and reward

Four keywords are enough to understand reinforcement learning. Everything else is a variation on this basic scheme.

The learning loop step by step

The agent is the system that makes decisions. The environment is the setting it acts in and that responds to its actions. The reward is the numeric signal that says how good an action was. The loop is simple, the agent observes the state, chooses an action, receives a reward, and moves to a new state. Then it repeats, thousands or millions of times.

Policy, value function, and exploration

The strategy the agent follows to choose actions is called a policy. The value function estimates how good it is, in the long run, to be in a given state. The agent keeps updating its policy to earn the highest possible reward over time. This is where deep learning, explained in our guide to deep learning, comes in to handle complex problems.

The main methods of reinforcement learning

There are several families of algorithms, but two approaches cover most of the methods used today.

Value based and policy based methods

Value based methods learn to estimate how good each action is in each state, then pick the one with the highest value. Policy based methods instead optimise the behaviour strategy directly, without going through a value estimate. Each has strengths depending on the problem.

Deep reinforcement learning: when neural networks join in

When the possible states are too many to list, as with the pixels of a video game, we use deep reinforcement learning. Deep neural networks approximate the policy and the value function, letting the agent handle huge environments. It is the technique behind the most spectacular results of recent years.

The exploration versus exploitation problem

Every reinforcement learning agent faces a basic dilemma, the same one you face when choosing a restaurant.

When to try something new and when to use what already works

Do you always go to the place you know and like, or do you try a new one that might be better? Exploiting what works gives safe results, exploring can lead to bigger discoveries but with some risk. A good agent balances the two, exploring a lot at first and exploiting more as it learns.

Want to see up close how intelligent systems are built in a campus that thrives on innovation? Join the next Open Day and spend a day inside our classrooms.

Where reinforcement learning changes the world today

The applications of reinforcement learning are now concrete and widespread across very different sectors.

Robotics, autonomous driving, energy, and finance

In industrial robotics, agents learn to manipulate objects and move through warehouses. In autonomous driving they help make real time decisions. In energy, DeepMind used reinforcement learning to cut the energy spent cooling Google’s data centers. In finance it drives portfolio management strategies.

Video games and aligning large language models

In video games reinforcement learning has led programs to beat human champions, from AlphaGo onward. But its most famous application today is another one. The technique called reinforcement learning from human feedback, or RLHF, is what turned raw models like GPT-3 into the useful, conversational assistants we use, including ChatGPT and GPT-4.

Careers in reinforcement learning

It is an advanced area of AI and it looks for profiles with solid technical foundations.

Machine Learning Engineer, AI Researcher, and Robotics Engineer

The Machine Learning Engineer builds and deploys models into production. The AI Researcher works on methods and algorithms. The Robotics Engineer applies reinforcement learning to the movement and control of machines. These are among the most sought after and best paid roles in technology, with pay that rises quickly as you gain experience.

These roles need solid mathematical foundations in probability and statistics, knowledge of learning algorithms, command of Python and deep learning frameworks, and an understanding of policies, value functions, and exploration.

Build your AI skills at H-FARM College

Reinforcement learning is one of the most fascinating frontiers of artificial intelligence, and mastering it takes solid foundations. At H-FARM College we take students from those foundations through to applied projects.

The Bachelor’s Degree in AI & Data Science builds the fundamentals of mathematics, statistics, programming, and machine learning, then moves on to neural networks and intelligent systems on real data. If you want to bring these technologies into business strategy, the AI for Business Transformation master combines technical skills with a business vision.

Studying here means working on real challenges inside an ecosystem built on innovation and entrepreneurship, with an international community and a figure that speaks for itself, 92% of our students find a job within six months of graduating. Ready to build systems that learn on their own? The Bachelor’s in AI & Data Science is the right place to start building your career with us.

FAQ

frequently asked questions about Reinforcement Learning