EXplainable Neural-Symbolic Learning X-NeSyL methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case

Semantic networks, conceptual graphs, frames, and logic are all approaches to modeling knowledge such as domain knowledge, problem-solving knowledge, and the semantic meaning of language. DOLCE is an example of an upper ontology that can be used for any domain while WordNet is a lexical resource that can also be viewed as an ontology. YAGO incorporates WordNet as part of its ontology, to align facts extracted from Wikipedia with WordNet synsets. The Disease Ontology is an example of a medical ontology currently being used. At the height of the AI boom, companies such as Symbolics, LMI, and Texas Instruments were selling LISP machines specifically targeted to accelerate the development of AI applications and research.

• An epoch of optimization consisted of 100,000 episode presentations based on the human behavioural data.
• Function 1 (‘fep’ in Fig. 2) takes the preceding primitive as an argument and repeats its output three times (‘dax fep’ is RED RED RED).
• These permutations induce changes in word meaning without expanding the benchmark’s vocabulary, to approximate the more naturalistic, continual introduction of new words (Fig. 1).
• Henry Kautz,[17] Francesca Rossi,[80] and Bart Selman[81] have also argued for a synthesis.
• By the mid-1960s neither useful natural language translation systems nor autonomous tanks had been created, and a dramatic backlash set in.
• Program tracing, stepping, and breakpoints were also provided, along with the ability to change values or functions and continue from breakpoints or errors.

Non-algebraic responses must be explained through the generic lapse model (see above), with a fit lapse parameter. Note that all of the models compared in Table 1 have the same opportunity to fit a lapse parameter. For successful optimization, it is also important to pass each study example (input sequence only) as an additional query when training on a particular episode. This effectively introduces an auxiliary copy task—matching the query input sequence to an identical study input sequence, and then reproducing the corresponding study output sequence—that must be solved jointly with the more difficult generalization task. There are now several efforts to combine neural networks and symbolic AI. One such project is the Neuro-Symbolic Concept Learner (NSCL), a hybrid AI system developed by the MIT-IBM Watson AI Lab.

Search code, repositories, users, issues, pull requests…

This transformation is applied to the query outputs before MLC and MLC (joint) process them. For each SCAN split, both MLC and basic seq2seq models were optimized for 200 epochs without any early stopping. For COGS, both models were optimized for 300 epochs (also without early stopping), which is slightly more training than the extended amount prescribed in ref. 67 for their strong seq2seq baseline. This more scalable MLC variant, the original MLC architecture (see the ‘Architecture and optimizer’ section) and basic seq2seq all have approximately the same number of learnable parameters (except for the fact that basic seq2seq has a smaller input vocabulary).

New deep learning approaches based on Transformer models have now eclipsed these earlier symbolic AI approaches and attained state-of-the-art performance in natural language processing. However, Transformer models are opaque and do not yet produce human-interpretable semantic representations for sentences and documents. Instead, they produce task-specific vectors where the meaning of the vector components is opaque. COGS is a multi-faceted benchmark that evaluates many forms of systematic generalization. To master the lexical generalization splits, the meta-training procedure targets several lexical classes that participate in particularly challenging compositional generalizations.

Methods

In symbolic reasoning, the rules are created through human intervention and then hard-coded into a static program. The two biggest flaws of deep learning are its lack of model interpretability (i.e. why did my model make that prediction?) and the large amount of data that deep neural networks require in order to learn. First, we evaluated lower-capacity transformers but found that they did not perform better. Second, we tried pretraining the basic seq2seq model on the entire meta-training set that MLC had access to, including the study examples, although without the in-context information to track the changing meanings.

Output symbols were replaced uniformly at random with a small probability (0.01) to encourage some robustness in the trained decoder. For this variant of MLC training, episodes consisted of a latent grammar based on 4 rules for defining primitives and 3 rules defining functions, 8 possible input symbols, 6 possible output symbols, 14 study examples and 10 query examples. Our use of MLC for behavioural modelling relates to other approaches for reverse engineering human inductive biases.

In the symbolic stage, knowledge is stored primarily as language, mathematical symbols, or in other symbol systems. We use symbols all the time to define things (cat, car, airplane, etc.) and people (teacher, police, salesperson). Symbols can represent abstract concepts (bank transaction) or things that don’t physically exist (web page, blog post, etc.). Symbols can be organized into hierarchies (a car is made of doors, windows, tires, seats, etc.). They can also be used to describe other symbols (a cat with fluffy ears, a red carpet, etc.). When deep learning reemerged in 2012, it was with a kind of take-no-prisoners attitude that has characterized most of the last decade.

A second flaw in symbolic reasoning is that the computer itself doesn’t know what the symbols mean; i.e. they are not necessarily linked to any other representations of the world in a non-symbolic way. Again, this stands in contrast to neural nets, which can link symbols to vectorized representations of the data, which are in turn just translations of raw sensory data. So the main challenge, when we think about GOFAI and neural nets, is how to ground symbols, or relate them to other forms of meaning that would allow computers to map the changing raw sensations of the world to symbols and then reason about them.

Languages

LISP is the second oldest programming language after FORTRAN and was created in 1958 by John McCarthy. LISP provided the first read-eval-print loop to support rapid program development. Program tracing, stepping, and breakpoints were also provided, along with the ability to change values or functions and continue from breakpoints or errors. It had the first self-hosting compiler, meaning that the compiler itself was originally written in LISP and then ran interpretively to compile the compiler code. It’s all fun and games until you start worrying about your child who isn’t engaging in symbolic play.

The encoder network (Fig. 4 (bottom)) processes a concatenated source string that combines the query input sequence along with a set of study examples (input/output sequence pairs). The encoder vocabulary includes the eight words, six abstract outputs (coloured circles), and two special symbols for separating the study examples (∣ and →). The decoder network (Fig. 4 (top)) receives messages from the encoder and generates the output sequence.

The first is that human compositional skills, although important, may not be as systematic and rule-like as Fodor and Pylyshyn indicated3,6,7. The second is that neural networks, although limited in their most basic forms, can be more systematic when using sophisticated architectures8,9,10. In recent years, neural networks have advanced considerably and led to a number of breakthroughs, including in natural language processing. In light of these advances, we and other researchers have reformulated classic tests of systematicity and reevaluated Fodor and Pylyshyn’s arguments1.

Powered by such a structure, the DSN model is expected to learn like humans, because of its unique characteristics. First, it is universal, using the same structure to store any knowledge. Second, it can learn symbols from the world and construct the deep symbolic networks automatically, by utilizing the fact that real world objects have been naturally separated by singularities. Third, it is symbolic, with the capacity of performing causal deduction and generalization. Fourth, the symbols and the links between them are transparent to us, and thus we will know what it has learned or not – which is the key for the security of an AI system. Fifth, its transparency enables it to learn with relatively small data.

Symbols also serve to transfer learning in another sense, not from one human to another, but from one situation to another, over the course of a single individual’s life. That is, a symbol offers a level of abstraction above the concrete and granular details of our sensory experience, an abstraction that allows us to transfer what we’ve learned in one place to a problem we may encounter somewhere else. In a certain sense, every abstract category, like chair, asserts an analogy between all the disparate objects called chairs, and we transfer our knowledge about one chair to another with the help of the symbol. Insofar as computers suffered from the same chokepoints, their builders relied on all-too-human hacks like symbols to sidestep the limits to processing, storage and I/O. As computational capacities grow, the way we digitize and process our analog reality can also expand, until we are juggling billion-parameter tensors instead of seven-character strings. Expert systems can operate in either a forward chaining – from evidence to conclusions – or backward chaining – from goals to needed data and prerequisites – manner.

For example, in teaching a particular concept, the teacher should present the set of instances that will best help learners develop an appropriate model of the concept. Bruner would likely not contend that all learning should be through discovery. He argued that schools waste time trying to match the complexity of subject material to a child’s cognitive stage of development. But symbolic AI starts to break when you must deal with the messiness of the world. For instance, consider computer vision, the science of enabling computers to make sense of the content of images and video. Say you have a picture of your cat and want to create a program that can detect images that contain your cat.

All of the query and study examples were drawn from the training corpus. Each episode was scrambled (with probability 0.95) using a simple word procedure30,65, and otherwise was not scrambled (with probability 0.05), meaning that the original training corpus text was used instead. Occasionally skipping the permutations in this way helps to break symmetries that can slow optimization; that is, the association between the input and output primitives is no longer perfectly balanced. Otherwise, all model and optimizer hyperparameters were as described in the ‘Architecture and optimizer’ section. This probabilistic symbolic model assumes that people can infer the gold grammar from the study examples (Extended Data Fig. 2) and translate query instructions accordingly.

Second, children become better word learners over the course of development60, similar to a meta-learner improving with training. It is possible that children use experience, like in MLC, to hone their skills for learning new words and systematically combining them with familiar words. Beyond natural language, people require a years-long process of education to master other forms of systematic generalization and symbolic reasoning6,7, including mathematics, logic and computer programming. Although applying the tools developed here to each domain is a long-term effort, we see genuine promise in meta-learning for understanding the origin of human compositional skills, as well as making the behaviour of modern AI systems more human-like. Over 35 years ago, when Fodor and Pylyshyn raised the issue of systematicity in neural networks1, today’s models19 and their language skills were probably unimaginable.

The grammars are not observed by the networks and must be inferred (implicitly) to successfully solve few-shot learning problems and make algebraic generalizations. The optimization procedures for the MLC variants in Table 1 are described below. A, During training, episode a presents a neural network with a set of study examples and a query instruction, all provided as a simultaneous input. The study examples demonstrate how to ‘jump twice’, ‘skip’ and so on with both instructions and corresponding outputs provided as words and text-based action symbols (solid arrows guiding the stick figures), respectively. The query instruction involves compositional use of a word (‘skip’) that is presented only in isolation in the study examples, and no intended output is provided.

Symbolic AI programs are based on creating explicit structures and behavior rules. If I tell you that I saw a cat up in a tree, your mind will quickly conjure an image. Similarly, Allen’s temporal interval algebra is a simplification of reasoning about time and Region Connection Calculus is a simplification of reasoning about spatial relationships. A more flexible kind of problem-solving occurs when reasoning about what to do next occurs, rather than simply choosing one of the available actions.

Read more about https://www.metadialog.com/ here.