Large language models (LLMs) are designed to learn patterns from vast datasets, enabling them to generate responses. However, as noted by Goertzel, many of these models are unable to continuously adjust their fundamental parameters based on new experiences once their initial training is completed. While they can access previously stored information or reference earlier interactions, they lack a comprehensive memory that tracks their entire "life."
“They lack a long-term memory similar to that of a human,” he explained. “This limitation means they do not fully understand their identity, capabilities, or how they connect with the world.”
According to Goertzel, for a machine to exhibit general intelligence, it must possess a self-model—an internal representation that reflects its history, skills, constraints, and aspirations. Moreover, it should be capable of retaining useful insights and applying them to future choices.
Participants at a recent conference explored predictive coding as a potential method for advancing AI intelligence. This technique involves having a model forecast an outcome, assess its predictions against reality, and learn from the discrepancies.
Goertzel shared that his team has successfully implemented predictive coding within transformers at a scale similar to GPT-2, an earlier model by OpenAI. However, they have yet to scale this approach to match the capabilities of today's most advanced models.
While continual learning is a step forward, Goertzel argues that it addresses only a portion of the challenges. He advocates for a neural-symbolic AI approach, which merges pattern-recognizing neural networks with symbolic techniques that articulate rules, relationships, and objectives more transparently.
In addition, he emphasizes the importance of integrating ethical reasoning, self-awareness, and goal-oriented pursuits directly into the learning process, rather than retrofitting safeguards after the fact.
Establishing that an AI has achieved general intelligence may also pose a challenge. Companies can often train models to excel in popular tests without truly demonstrating the broader competencies those evaluations are meant to assess.
To tackle this issue, Goertzel suggests an ambitious measure he calls the "MIT test." In this scenario, an AI-driven robot would need to earn a doctorate from the university while adhering to the same regulations as human students. This would require the machine to navigate the campus, attend lectures, pass written and oral examinations, collaborate with professors, and generate original research. Achieving this would necessitate skills in communication, physical navigation, social reasoning, long-term planning, and knowledge creation.
“If I could develop a robot capable of attending MIT and obtaining a PhD while complying with all the same rules as human students, I would be genuinely impressed,” Goertzel remarked. “It needs to navigate hallways, perform an oral exam, complete a written test, and produce a thesis that represents an original contribution to knowledge.”
