AI alignment: teaching tech human language
Artificial intelligence used to provoke images of aliens and robots, now it corrects our spellings, helps us drive more safely and saves us time on everyday tasks. It’s well and truly integrated into our everyday lives and AI systems are only becoming more advanced with every passing day – some even exceeding human-level performance in some areas.
It’s safe to say that AI is one of the most impactful technologies of our time. For business and tech leaders alike it provides a whole new scope of opportunities to develop their businesses. However, to utilise AI’s full capabilities it’s crucial to ensure that the AI is aligned with human intent.
We can do this with AI Alignment.
Understanding AI Alignment
AI alignment is a field of AI safety research that aims to ensure that AI systems achieve their desired outcomes and work properly for humans. It aims to create an end-to-end process where an AI-based system can align its output with desired human preferences.
Imagine playing darts, or any game for that matter, but not agreeing on what the board looks like or what you get points for. If the designer of an AI system cannot express consistent and clear expectations through feedback, the system won’t know what to learn.
A great human example of this is trying to explain to someone what a road is. This seems relatively simple until we need to do it and realise that it’s quite difficult. Anyone looking into why AI Alignment is a critical step in business should answer the question where does a road begin? How do we explain the nuances of driveways, private roads, level crossings and dirt paths to cars if we can’t explain it accurately to another human?
At its core, alignment is about agreeing on expectations. Anyone who has managed a team knows how much communication is required to align a group of people – this is essentially what we need to do with the machines we teach, agree on the communication and instruction we give them and then ultimately what the ‘correct’ outcome looks like. With the emergence of more powerful AI, this alignment exercise will be extended to include algorithms. Human feedback is central to this process, and there is still much work to be done.
As business leaders, it is imperative that we act now to ensure AI is aligned with human values and intent.
It’s not easy to teach a machine…
Humans interpret things differently and develop preferences based on their personal perceptions. This makes it incredibly difficult to teach machines how humans think and how to tell your “machine” what high performance really is. In order to function properly, AI products need to learn the language of human preferences.
Up until now, most AI products have been trained using unsupervised learning where we let algorithms derive how to solve tasks by observing humans. But in an application as complex as driving, developers of autonomous vehicles must ask themselves two questions, how do we want this product to behave? And how do we make it behave that way? Agreeing on desired outcomes that align with human intent is crucial, particularly in a safety-critical environment.
For example, the autonomous vehicle industry has already faced many challenges in putting ML-enabled products on the road. AI has not lived up to consumer expectations in this sector. The problem is that there is no single way to drive – what makes a “good” driver? This is partially due to the complexity of the decision making associated with driving, and partially due to the fact that “programming by example” is so radically different from “programming by code.”
Just like when communicating with humans, the best way to express your intent is to review examples. It’s the same process with an algorithm, a human will review how it behaves and provide feedback, and from this, the algorithm learns and improves. Human feedback can be used to steer AI products very efficiently by shaping the evolving dataset to reflect the developers’ intentions and user expectations.
Taking an iterative approach
Contrary to common belief, AI alignment is not actually a technology problem, it’s a people problem. Ultimately, the ability of the AI system to learn the right kind of rules comes down to the ability of the product developer or service provider to express what it is that they want the product to do.
If we don’t figure out a better way to do this, we will see a lot of disappointment in the next few years and it’s going to be very difficult to realise the potential of AI. So, it’s in our collective interest to get this right. If business and technology leaders can collaborate closely on alignment, it will help create better products and in turn benefit humans day to day-to-day lives.
We live in a fast-changing world, and expectations evolve quickly. If you assemble a large dataset, you must expect it to evolve with the expectations of those who will benefit from it. The challenge developers now face is to shape the data they collect with this evolution in mind. This should in turn inform AI products and their future development.
Alignment is paving the way and is a necessary step when implementing AI. An iterative approach is the best one to navigate the ongoing challenges, expectations and developments of not just humans but AI itself. Ultimately, leaving us with better, smarter AI products.
