Earlier this year, Tom Dietterich, BigML’s Chief Scientist and Emeritus Professor at Oregon State University, delivered a keynote speech at the ValgrAI event in Valencia, Spain. The presentation was titled “What’s wrong with LLMs and what we should be building instead”.

Large Language Models (LLMs) serve as a pre-trained foundation for developing various AI systems. They have achieved significant milestones, such as conducting conversations and answering questions across a vast spectrum of human knowledge. Dietterich highlights this as our first instance of creating a broadly knowledgeable AI system. Other notable capabilities of LLMs include document summarization and revision, writing code from English descriptions, and learning in context from a small number of training samples.

However, LLMs also have numerous drawbacks. They are costly to train and update, their non-linguistic knowledge is limited, they often make false and contradictory statements, and these statements can sometimes be socially and ethically inappropriate. Dietterich begins his keynote by discussing these well-documented deficiencies of LLMs and the current efforts to address them.

In the second half of his enlightening presentation, Dietterich suggests a more modular architecture that breaks down the functions of existing LLMs and introduces several additional components that could potentially address all their shortcomings. This modular architecture could be developed using a mix of cutting-edge machine learning and software engineering best practices.

The specifics of the proposed solution architecture and more can be found in the keynote on YouTube.

If interested, you can also access the slides for the keynote at your convenience. We welcome your thoughts on what the future holds for making LLMs increasingly robust and whether they are ready to meet the capital markets’ tremendous growth expectations in the near future.

If your organization is considering transitioning from a model-centric approach to machine learning and scaling machine learning solutions without adding unnecessary complexity, please contact us so we can arrange a demo of the BigML platform, which has been simplifying machine learning for everyone for over a decade.