SocraSynth is a groundbreaking platform that integrates the principles of "Socratic Synthesis" and "Socratic Symposium." Inspired by the Socratic method's rich tradition of stimulating critical thought and revealing hidden insights, SocraSynth aims to bring to light perspectives that are often overlooked. By encouraging constructive dialogue and fostering critical thinking, the platform bridges the gaps across various disciplines, providing users with an integrated, multidimensional tapestry of knowledge and understanding.
SocraSynth functions through two primary phases: the generative phase and the evaluative phase. In the generative phase, SocraSynth leverages the vast "polydisciplinary" capabilities of foundational models like GPT-4 and PaLM to produce content. These models are akin to AI agents with expertise equivalent to holding doctorates in every conceivable discipline. This breadth of knowledge raises a crucial question: How do we probe the unknown or uncover insights beyond our current awareness? SocraSynth tackles this challenge by orchestrating a symposium of Generative AI (GAI) agents engaged in interactive and adversarial dialogues.
During these discussions, the role of human moderators is pivotal yet restrained. They should act primarily as facilitators, guiding the flow of conversation rather than contributing their expertise. This approach is essential given the GAI agents' presumed superior knowledge base. The moderators' focus should be on directing the dialogue effectively, ensuring a comprehensive exploration of topics while harnessing the full potential of the GAI agents.
There are three key factors that significantly enhance the capabilities of SocraSynth:
The second phase of SocraSynth focuses on validating the information generated. Due to human limitations in understanding, evaluating the content's validity can be difficult and may vary between individuals. To address this, SocraSynth employs the Socratic method, enhanced by rigorous deductive and inductive reasoning, for critical evaluation. By thoroughly examining each reason-to-conclusion argument and introducing counter-arguments and "what-if" scenarios, the platform can assess both the validity and credibility of the generated content.
The notion of truth is intricate and subtle. While verification based on training data is one method to assess the accuracy of a statement, this strategy raises questions about the data's own veracity. The commonly used term "ground truth" doesn't necessarily signify absolute accuracy; it is often simply assumed to be accurate. SocraSynth tackles this issue of "reasonableness" through its rigorous evaluation process. Ultimately, it is up to humans to decide whether the insights produced are both valid and useful.
Following the positive reception of our July paper, titled "Exploring GPT-4: Abilities, Implications, and Future Avenues," recognized as one of the top 1% of reviewed papers on ResearchGate, we are delighted to present our latest research endeavors. These efforts aim to uncover insights beyond established knowledge boundaries, leveraging SocraSynth to illuminate information that might typically escape human observation. We have employed SocraSynth in various domains, including sales planning, healthcare, financial analysis, and geopolitical analysis. Our experiences demonstrate that SocraSynth effectively addresses the challenges of hallucination and statistical biases. For more in-depth information, please refer to our publications.
Each paper presents a unique experimental methodology, exploring areas that remain largely untapped. They are particularly centered around utilizing the profound capabilities of GPT-4 to reveal fresh insights and perspectives. Below, we furnish links and abstracts for these recent contributions along with their related works, all of which saw publication in 2023.
Full paper: SocraFin, Conditional Statistics for Financial Planning and Analysis, November 2023, in collaboration with AiBanker (under review).
Full paper: SocraHealth: Enhancing Medical Diagnosis and Correcting Historical Records, Jocelyn Chang and Edward Chang, October 2023. The 10th International Conference on Computational Science and Computationalm Intelligence, December 2023.
This study introduces SocraHealth, an innovative method using Large Language Models (LLMs) for medical diagnostics. By engaging LLM-based agents in structured debates, SocraHealth not only refines diagnoses but also corrects historical record inaccuracies, utilizing patient data effectively. The case study, featuring GPT-4 and Bard across two experiments, showcases this approach's success in producing logical, hallucination-free debates. Demonstrating a significant advancement over traditional diagnostic techniques, SocraHealth highlights the transformative power of LLMs in healthcare, especially in enhancing diagnostic accuracy and rectifying past diagnostic errors.
Full paper: Multi-Agent Reasoning with Large Language Models for Effective Corporate Planning, in collaboration with S. Tsao at TrendMicro, October 2023. The 10th International Conference on Computational Science and Computationalm Intelligence, December 2023.
Large Language Models (LLMs) have demonstrated significant capabilities in natural language processing tasks. In this paper, we explore the application of LLMs within a business context. Specifically, we employ LLMs to devise a sales strategy geared towards maximizing customer values (benefits and satisfaction). This sales plan encompasses five iterative stages: market landscape survey, customer profiling, product usage analysis, sales strategy formulation, and crafting persuasive pitches and materials. We leverage LLMs to supplement the limited data available to the company, aiming to enhance the efficacy of each stage and optimize KPIs, including the value-oriented sales conversion and profitability. Due to confidentiality and trade secret concerns, we blend artificial data with genuine data to ensure customer anonymity and protect sales playbooks. Despite these precautions, we effectively demonstrate our methodology of harnessing LLMs to refine the sales planning procedure.
Full paper: Multi-Agent Collaboration, October 2023 (under anonymous review).
Recent trends in formulating prompts have escalated to an unreasonable level. For instance, the utilization of chain-of-thought—an abductive or speculative learning approach—is employed to enhance reasoning. Another method oversimplifies Theory of Mind to determine the locations of objects. Regrettably, these strategies, often accompanied by complex mathematical notations to exaggerate their profundity and followed by contrived supervised fine-tuning, have become prevalent in research conferences. It is imperative to maintain a balance between innovation and rigor, ensuring that our pursuits contribute substantively to the field. We must critically examine the methodologies we employ, the conclusions we draw, and the knowledge we disseminate, keeping the pursuit of truth and understanding at the forefront of our endeavors.
Full paper: SocraSynth: Socratic Synthesis for Reasoning and Decision Making, September 2023.
SocraSynth, a portmanteau of "Socratic Synthesis" and "Socratic Symposium," is a digital forum designed to leverage the extensive knowledge base of foundation models like GPT-4, PaLM, and LLamA. It aims to uncover insights and knowledge that may be beyond human awareness. The platform's unique strength lies in its interdisciplinary nature, allowing foundation models to formulate questions and issues that might elude human intuition, thereby facilitating the emergence of novel perspectives and insights. The SocraSynth workflow is divided into two primary phases: knowledge generation and evaluation. In the generation phase, a human moderator assembles a committee featuring agent representatives from a selected foundation model. The moderator outlines a topic for discussion and debate, setting the context and parameters for the discourse that follows. Agents then engage in a structured debate , offering initial arguments, counter-arguments, and concluding statements. Subsequently, SocraSynth transitions to its evaluation phase, employing the CRIT algorithm-a methodology rooted in Socratic reasoning and formal logic. This comprehensive evaluation serves as an invaluable resource for decision-makers aiming to make informed, judicious choices. Through case studies, we demonstrate how SocraSynth enables rigorous research and reasoning , effectively supporting critical decision-making processes in domains such as management and governance.
Full paper: Examining GPT-4's Capabilities and Enhancement with SocraSynth, July 2023.
The 10th International Conference on Computational Science and Computational Intelligence (CSCI'23), December 2023.
(Top-1% assessed paper, over 6,680 reads on ResearchGate in 2023)
In this work, we investigate the capabilities and limitations of GPT-4, a large-scale, polydisciplinary, and polymodal language model. Despite its accomplishments across a range of tasks, GPT-4 exhibits key shortcomings, particularly in areas of reasoning and ethics, manifesting in tendencies like hallucination, imitation rather than understanding, and a lack of fact-checking ability. We propose several remedies to address these challenges. First, we introduce the CoCoMo framework, designed to incorporate reasoning into AI systems using Socratic methods and prompt ensembles. Second, we advocate for the use of demonstrations as a means to imbue AI agents with ethical behavior, building upon our experience with the Noora chatbot project. Lastly, we recommend adopting a more comprehensive approach to training ensemble members of GPT-4, shifting from an exclusive focus on optimizing for cross-entropy loss. Our end goal is the development of AI systems that not only enhance human abilities but also align with human values, thereby contributing constructively to society.
Full paper: Discovering Insights Beyond the Known: A Dialogue Between GPT-4 Agents, August 2023. (The most read paper in August on ResearcgGate)
Human knowledge, vast as it is, often falls short in grasping intricate interdisciplinary domains fully. In contrast, foundation models like GPT-4, endowed with extensive multidisciplinary knowledge, can potentially bridge this gap. Significantly, we leverage the vast expanses of GPT-4's knowledge, banking on its ability to frame questions that might elude human intuition, thus paving the way for the emergence of fresh insights and potentially novel knowledge. In this study, we convened a unique committee comprising a moderator (the authors) and two GPT-4 agents. The dialogue is ignited by the ancient narrative of Adam and Eve, setting the stage for a rich exchange between the GPT-4 agents. This conversation derives from the age-old tale, as the agents delve into three intertwined domains: the significance of myths in ecological interpretation, the intricate ethical and philosophical quandaries surrounding AI, and the enigmatic realm of the human brain as complemented by technology. This dialogue not only unveils captivating insights but also underscores the indispensable value of interdisciplinary exchanges. Foundation models, as demonstrated, can catalyze such dialogues, equipping us to traverse expansive knowledge landscapes and explore domains previously beyond human comprehension.
Full paper: Prompting Large Language Models With the Socratic Method, IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), March 2023 (The best presentation in the AI track)
This paper presents a systematic approach to using the Socratic method in developing prompt templates that effectively interact with large language models, including GPT-3. Various methods are examined, and those that yield precise answers and justifications while fostering creativity and imagination to enhance creative writing are identified. Techniques such as {\em definition}, {\em elenchus}, {\em dialectic}, {\em maieutics}, {\em generalization}, and {\em counterfactual reasoning} are discussed for their application in engineering prompt templates and their connections to inductive, deductive, and abductive reasoning. Through examples, the effectiveness of these dialogue and reasoning methods is demonstrated. An interesting observation is made that when the task's goal and user intent are conveyed to GPT-3 via ChatGPT before the start of a dialogue, the large language model seems to connect to the external context expressed in the intent and perform more effectively.
Full paper: CoCoMo: Computational Consciousness Modeling for Generative and Ethical AI, February 2023.
The CoCoMo model proposes a computational solution to the challenge of incorporating ethical and emotional intelligence considerations into AI systems, with the aim of creating AI agents that combine knowledge with compassion. To achieve this goal, CoCoMo prioritizes fairness, beneficence, non-maleficence, empathy, adaptability, transparency, and critical and exploratory thinking abilities. The model employs consciousness modeling, reinforcement learning, and prompt template formulation to support these desired traits. By incorporating ethical and emotional intelligence considerations, a generative AI model can potentially lead to improved fairness, reduced toxicity, and increased reliability.
Full paper: CRIT: An Inquisitive Prompt Template for Critical Reading, January 2023.
Critical reading, a pivotal element of education, necessitates active engagement with texts to delve deeper and form informed assessments about their validity and credibility. We introduce CRIT, a comprehensive prompt template designed to streamline this process. CRIT leverages pre-trained language models to critically evaluate texts, extracting their conclusions and supportive reasons, scrutinizing reason-to-claim arguments, suggesting counterarguments, and offering an overarching quality assessment. Notably, CRIT also possesses the capability to conduct fact-checking on the outputs of foundation models, ensuring accuracy and trustworthiness. With its structured and recursive prompts, CRIT facilitates a comprehensive and logical text analysis, providing insights into argument validity and source reliability. This makes CRIT an invaluable asset for K-12 education, fostering critical reading skills, and refining articles before public examination.
Full paper: Knowledge-Guided Data-Centric AI in Healthcare: Progress, Shortcomings, and Future Directions, December 2022; Chapter 2 in Artificial Intelligence, Machine Learning, and Deep Learning in Precision Medicine in Liver Diseases (ISBN: 9780323991360), Elsevier, August 22, 2023
The success of deep learning is largely due to the availability of large amounts of training data that cover a wide range of examples of a particular concept or meaning. In the field of medicine, having a diverse set of training data on a particular disease can lead to the development of a model that is able to accurately predict the disease. However, despite the potential benefits, there have not been significant advances in image-based diagnosis due to a lack of high-quality annotated data. This article highlights the importance of using a data-centric approach to improve the quality of data representations, particularly in cases where the available data is limited. To address this "small-data" issue, we discuss four methods for generating and aggregating training data: data augmentation, transfer learning, federated learning, and GANs (generative adversarial networks). We also propose the use of knowledge-guided GANs to incorporate domain knowledge in the training data generation process. With the recent progress in large pre-trained language models, we believe it is possible to acquire high-quality knowledge that can be used to improve the effectiveness of knowledge-guided generative methods.