Neuro-symbolic AI is all about bringing together the best of both worlds: neural networks, which are super good at learning from huge amounts of data, and symbolic AI, known for its top-notch reasoning and handling symbols. This mix is aiming to get past the usual hang-ups, making AI systems that are stronger, more flexible, and think more like us.

The AI scene is always changing. Right now, a lot of smart folks are looking into Neuro-Symbolic Artificial Intelligence. It’s a fresh take on AI that blends the neural side of things with the symbolic, playing off the strengths of each to even out their weak spots.

Neural networks shine when it comes to learning from data, while symbolic AI is all about dealing with symbols or concepts instead of just crunching numbers.

The big goal for researchers is to crack open new levels of smarts, skills, and adaptability in AI, mixing up statistical models with clear-cut rules and knowledge to give AI systems a better grip on representing, reasoning, and generalizing concepts.

---

TAKEAWAYS

---

Towards Neuro-Symbolic Artificial Intelligence

The need for a different approach in developing Artificial Intelligence is well articulated by Gary Marcus, an American psychologist, cognitive scientist, and author renowned for his research at the crossroads of cognitive psychology, neuroscience, and artificial intelligence, and a professor emeritus of psychology and neural science at New York University.

Already in 2020, Marcus highlighted how research in the field of intelligence and machine learning had focused on general-purpose learning, employing increasingly larger datasets for training and requiring more and more computation.

According to Marcus, what’s needed is a different, hybrid approach, one that’s knowledge – and reasoning-based, centered around cognitive models, which could provide the foundation for a richer and more robust artificial intelligence than what’s currently achievable.

He defines this need through a trio of hybrid architecture, extensive prior knowledge, and sophisticated reasoning techniques, explaining:

«Let us call that new level robust artificial intelligence: intelligence that, while not necessarily superhuman or self-improving, can be counted on to apply what it knows to a wide range of problems in a systematic and reliable way, synthesizing knowledge from a variety of sources such that it can reason flexibly and dynamically about the world, transferring what it learns in one context to another, in the way that we would expect of an ordinary adult. In a certain sense, this is a modest goal, neither as ambitious or as unbounded as “superhuman” or “artificial general intelligence” but perhaps nonetheless an important, hopefully achievable, step along the way—and a vital one, if we are to create artificial intelligence we can trust, in our homes, on our roads, in our doctor’s offices and hospitals, in our businesses, and in our communities»

The foundations of Neuro-Symbolic AI

Neuro-Symbolic Artificial Intelligenceis built on two main pillars: neural networks, which learn patterns from data, and symbolic AI, which uses predefined rules and logic to make decisions.

Neural Networks

Neural networks are essentially a Machine Learning model designed to mimic the function and structure of the human brain. They’re made up of layers of interconnected nodes (or “neurons“) that learn from data and work together to tackle complex problems. They’re especially good at handling unstructured data like images, sounds, and text, laying the groundwork for what’s known as deep learning. Neural networks are stars at pattern recognition and can make predictions or classifications based on past examples.

In a neural network, the processors work in parallel and are arranged in layers. The first layer, similar to the optic nerves in human visual processing, receives raw incoming information. Each subsequent layer gets the output from the layer before it, rather than the raw input, just like neurons further from the optic nerve get signals from those closer to it. The final layer produces the system’s output.

Symbolic AI

Symbolic AI focuses on processing and manipulating symbols or concepts rather than numerical data. It aims to create intelligent systems that can reason and think like humans by representing and manipulating knowledge and reasoning based on logical rules. Rules and axioms are used to make inferences and deductions.

So, neuro-symbolic AI aims to take the best of both worlds (i.e., the pattern recognition and learning capabilities of neural networks and the structured reasoning and interpretability of symbolic AI) to create systems that can learn from data and reason in a way similar to humans. This could lead to a more powerful and versatile AI capable of tackling complex tasks that currently require human intelligence, and doing so in a way that’s more transparent and explainable than neural networks alone.

IBM, which is heavily investing in this research area, defines neuro-symbolic AI as «as a pathway to achieve artificial general intelligence. By augmenting and combining the strengths of statistical AI, like machine learning, with the capabilities of human-like symbolic knowledge and reasoning, we’re aiming to create a revolution in AI, rather than an evolution».

However, it should be clarified that the combination of the two main pillars can happen in different ways, and there is currently no single or dominant vision of how this should be done. Various strategies are being explored.

For example, an upper layer composed of familiar symbols can help provide results as more understandable explanations. This design also promotes “compositionality” which allows systematic creation of modules from smaller components, something with which pure neural networks often struggle. Alternatively, a neural network can also be built on a traditional control circuit that provides its inputs. The circuit acts as a regulator that limits the search problem for the neural network.

IBM, for instance, has focused on an architecture it calls logical neural networks, where there’s no differentiation between the neural and symbolic parts. Instead of using preset rules, they can be trained and learn new rules.

Different approaches to Neuro-Symbolic Artificial Intelligence

In the absence of a singular approach, many refer to the taxonomy developed by Henry Kautz, Professor Emeritus at the University of Rochester and founder of the university’s Institute for Data Science. Kautz refers to a series of models, among which are:

• Symbolic Neural symbolic
  Here, the inputs and outputs are presented in symbolic form, but all the actual processing is neural. This, in his words, is the “standard operating procedure” when inputs and outputs are symbolic. Examples include the LLMs BERT, RoBERTa, and GPT-3.
• Symbolic[Neural] 
  In this case, symbolic techniques are used to invoke neural techniques. We’re talking about a neural pattern recognition subroutine within a symbolic problem solver. Examples of this model are AlphaGo, AlphaZero (an artificial intelligence algorithm based on machine learning techniques developed by Google DeepMind), and current approaches to self-driving cars.
• Neural | Symbolic
  Uses a cascading neural architecture to interpret perceptual data as symbols and relationships on which to reason symbolically.
• Neural: Symbolic → Neural 
  Relies on symbolic reasoning to generate or label training data that is subsequently learned by a deep learning model.
• Neural_{Symbolic} 
  Here, a neural network is generated from symbolic rules.
• Neural[Symbolic] 
  Allows a neural model to directly make a call to a symbolic reasoning engine, for example, to perform an action or evaluate a state. Symbolic reasoning is embedded within a neural engine.

The future of Neuro-Symbolic Artificial Intelligence

At the heart of research in this field is the belief that Neuro-Symbolic Artificial Intelligence systems could, down the line, get to grips with and interact with the world in ways that are way more human-like.

Imagine a system that could spot and figure out objects in pictures using the smarts of neural networks and then dive deeper into understanding these objects with the brainy tactics of symbolic AI.

This could open doors to cooler AI stuff, like robots that can find their way around tricky places or chatbots that get what you’re saying and chat back just like a person would.

Glimpses of Futures

Though we’re stepping into relatively new territory here, it’s pretty clear that the future is already knocking. Let’s break it down with the STEPS (Social, Technological, Economic, Political, Sustainability) framework to see what’s up.

S – SOCIAL: a study from Shanghai University has shown that Neuro-Symbolic Artificial Intelligence is pretty handy for digging into the humanities and social sciences. But it’s not about choosing this tech over traditional methods that value deep dives into text. Instead, they should be buddies, offering each other a helping hand. Just leaning on tech to figure out complex human and social puzzles might not cut it.

T – TECHNOLOGICAL: tech-wise, neuro-symbolic AI is pretty slick at making sense of things from way fewer examples than what old-school deep learning needs. This is because it’s good at grabbing big ideas from just a handful of data. This could mean it’s smarter with data than the usual neural network gang. Even though we’re making strides towards mimicking human thought processes, there’s still a long way to go, especially with the really tricky math problems.

E – ECONOMIC: looking at the money side of things, in areas like health or finance, neuro-symbolic AI is making decisions clearer to see. Unlike the data-hungry traditional deep learning models, this new kid on the block can make smart choices with much less data, thanks to its knack for symbolic thinking. And its ability to adapt to new situations without starting from scratch is pretty cool.

P – POLITICAL: everyone’s eyeing neuro-symbolic AI for its potential to power up future machines, think self-running systems that can do their thing without us having to tell them what to do, especially in high-stakes scenarios like natural disasters or factory mishaps. But, it’s not just about the tech; it’s also about making sure these advances are for everyone, not just a select few.

S – SUSTAINABILITY: when it comes to keeping things green, neuro-symbolic AI has its role to play, like helping out with managing waste or tackling pollution. It’s also on the front lines for predicting wild weather, keeping tabs on wildlife and natural spaces to boost conservation efforts, and even in cutting down on greenhouse gases. But, it’s not all roses; we’ve got to keep an eye on the energy it uses and the electronic junk it might leave behind.