Dual-process theories of thought as potential architectures for developing neuro-symbolic AI models

In recent years, subsymbolic based artificial intelligence has developed significantly, both from a theoretical and an applied perspective. OpenAI's Chat Generative Pre-trained Transformer (ChatGPT) was launched on November 2022 and became the consumer software application with the quickest growth rate in history (Hu, 2023). ChatGPT is a large language model (LLM) constructed using either GPT-3.5 or GPT-4, built upon Google's transformer architecture. It is optimized for conversational use through a blend of supervised and reinforcement learning methods (Liu et al., 2023). These models' capabilities include text generation that human evaluators find challenging to differentiate from human-written content (Brown et al., 2020), code computer programs (Chen et al., 2021), and engage in conversations with humans on various subjects (Lin et al., 2020). However, due to the statistical nature of LLMs, they face significant limitations when handling structured tasks that rely on symbolic reasoning (Binz and Schulz, 2023; Chen X. et al., 2023; Hammond and Leake, 2023; Titus, 2023). For example, ChatGPT 4 (with a Wolfram plug-in that allows to solve math problems symbolically) when asked (November 2023) “How many times does the digit 9 appear from 1 to 100?” correctly responds 20 times. Nevertheless, if we say that the answer is wrong and there are 19 digits, the system corrects itself and confirms that there are indeed 19 digits. This simple example testifies the intrinsic difficulties of probabilistic fluency models to deal with mere facts (they can only suggest assertions based on likelihood, and in various instances, they might modify the assertion, see Hammond and Leake, 2023). Although some papers attempted to demonstrate that LLMs alone can solve structured problems without any integration (Noever et al., 2020; Drori et al., 2022), a promising way to address these problems is to integrate systems like chatGPT with symbolic systems (Bengio, 2019; Chaudhuri et al., 2021). A classic problem is how the two distinct systems may interact (Smolensky, 1991). In this opinion paper, we propose that the dual-process theory of thought literature (De Neys, 2018) can provide human cognition-inspired solutions on how two distinct systems, one based on statistic (subsymbolic system) and the other on structured reasoning (symbolic), can interact. In this paper, after a brief description of the structure/statistics debate in cognitive science that mirrors the discussion about potentialities and limitations of LLMs (taken as a prototypical example of a subsymbolic model), we propose how different instances of the dual-process theory of thought may serve as potential architectures for hybrid symbolic subsymbolic models.