Spring 2023 MW 4:30pm –
5:50pm, April 3rd – June 5th
Location:
Gates B12
Edward
Chang (echang@cs.stanford.edu)
Adjunct Professor, Computer
Science, Stanford University
CA: Henrik
Marklund
Office Hours: MW after class to
6:15pm at Gates B12 or 104.
Announcements
About
The 2023 edition of the
course will focus on topics related to Generative AI, including GPT-3/4,
ChatGPT, BioGPT, and consciousness modeling.
Artificial intelligence,
specifically deep learning, attention mechanisms, and foundation models, has
emerged as some of the most transformative technologies of the past decade. AI
can already outperform humans in various computer vision and natural language
processing tasks. However, we still face some limitations and obstacles
reminiscent of those that contributed to the decline of the first AI boom five
decades ago. This research-oriented course will initially examine the limitations
(e.g., iid assumption on training and testing data,
extensive training data requirements, and lack of interpretability) of widely
used AI algorithms, including convolutional neural networks (CNNs),
transformers, reinforcement learning, generative AI, LLMs, and AI safety. To
address these limitations, we will explore topics such as transfer learning to
alleviate data scarcity, knowledge-guided multimodal learning to enhance data
diversity, out-of-distribution generalization, subspace learning for interpretability,
privacy-preserving data management, emotion/behavior/ethics modeling, and more.
The course will be taught
through a combination of lectures and project sessions. Guest speakers from
academia and industry will present lectures on specialized AI applications
(e.g., cancer/depression diagnosis and treatment). Students will be assigned to
work on a term project relevant to their fields of study (e.g., CS, Medicine,
and Data Science). Projects may involve conducting literature surveys,
formulating ideas, and implementing these ideas. Students are encouraged to
develop a project that integrates their own graduate research.
Note:
The following prerequisite for taking this course have been waved because of
the ease-of-use of ChatGPT.
Level: Senior/graduate
of EE/CS, Medicine, Medical Informatics, Engineering; proficient in basic
programming
Perquisite:
Introductory course in AI, Statistics, or Machine Learning
Assignments
· No exams.
· Three light-weight assignments, using
ChatGPT to complete.
· Term project: A group project of two to three
involving prompt template design.
References (optional)
Probabilistic
Machine Learning, Kevin P. Murphy, March 2022
Projections: A Story of Human Emotion, Karl Deisseroth, 2021
Grading
· Assignments 30%
· Class participation 10%
· Literature survey and project
proposal 20%
· Project implementation and demo
40%
Syllabus (last updated 4/03/2023)
Date |
Description |
Course Materials |
Notes |
Week #1 4/3/2023 |
Course
Aims and Syllabus ChatGPT
Intro [slides]
|
Intro to
AI, GAI and why you should or should not take this course. |
E. Chang |
4/5/2023 |
LLMs
(Large Language Models) Part 1 of 4 Foundation
Models and ChatGPT [slides] |
Intro to GPT-3, GPT-4, ChatGPT, BioGPT
and Prompt engineering. [1] J. Wei, X. Wang, et al, Chain of thought
prompting elicits reasoning in large language models. Advances in Neural
Information Processing Systems, 2022. [link] [2] J. Jung, L. Qin, S. Welleck,
F. Brahman, C. Bhagavatula, R. L. Bras, and Y.
Choi. Maieutic prompting: Logically consistent reasoning with recursive
explanations. In Conference on Empirical Methods in Natural Language
Processing, 2022. [link] [3] Prompting Large Language Models with the Socratic Method, E. Y.
Chang, IEEE CCWC, March 2023. [link] [4] Sparks of Artificial General
Intelligence: Early experiments with GPT-4, Sébastien Bubeck, Varun Chandrasekaran, et al, March 2003. [link] |
E. Chang Assignment
#1: Design [your] chatbot functions, using ChatGPT Henrik’s [PPT] |
Week #2 4/10/2023 |
P4
Medicine, Part 1 of 3: History of
AI in Diagnosis [link] |
[1] Schwartz, W. B., R. S. Patil, and
P. Szolovits. "AI in Medicine: where do
we stand." New England Journal of Medicine 316 (1987):
685-688. [link]. [2] Wu, T. D. "Efficient
Diagnosis of Multiple Disorders Based on a Symptom Clustering
Approach." Proceedings of AAAI, 1990, pp. 357-364. [3] Universal Equivariant Multilayer Perceptrons, Siamak Ravanbakhsh, ICML, 2020. [4]
Tricorder (medical IoTs), E. Y. Chang, et al., "Artificial Intelligence
in XPRIZE DeepQ Tricorder. " ACM MM Workshop
for Personal Health and Health Care, 2017. [5] Szolovits, P., and S.
G. Pauker. "Categorical and Probabilistic
Reasoning in Medical Diagnosis.” Artificial
Intelligence 11(1-2), 1978: 115-144. [6] Patil, R. S., P. Szolovits,
and W. B. Schwartz. "Causal Understanding of Patient Illness in Medical
Diagnosis." In Proceedings of the Seventh International Joint
Conference on Artificial Intelligence. |
E. Chang |
4/12/2023 |
LLM, Part
2 of 4: Prompt
template design principles [link] + Noora
introduction |
[1] Prompting Large Language Models
with the Socratic Method, E. Y. Chang, IEEE CCWC, March 2023. [link] [2] CRIT: Critical Reading Inquisitive
Template, E. Y. Chang. [link] [3] Noora.cs.stanford.edu |
E. Chang
Assignment
#1 due Assignment
#2: Design and test prompting templates |
Week #3 4/17/2023 |
LLM, Part
2 of 4 (finish
the lecture) Prompt
template design principles + project
discussion |
[1] Prompting Large Language Models
with the Socratic Method, E. Y. Chang, IEEE CCWC, March 2023. [link] [2] CRIT: Critical Reading Inquisitive
Template, E. Y. Chang. [link] [3] Noora.cs.stanford.edu |
E. Chang |
4/19/2023 |
LLM, Part
3 of 4: History of
NLP in one lecture. From
one-hot vector, word2vec to attention, transformers, BERT, and GPT [slides][video] |
[1]
Attention is all you need, Ashish Vaswani, et al., [link]. [2]
BERT: Pre-training of Deep Bidirectional Transformers for Language
Understanding [link]. [3]
Reformer: The Efficient Transformer [link]. [4]
Perceiver: General Perception with Iterative Attention [link]. |
E. Chang
Assignment
#2 due Assignment
#3: Design chatbot states |
Week #4 4/24/2023 |
Intro to
Psychiatric Disorders, symptoms, and treatments [slides][video] |
NA |
Dr. Vikas Duvvuri, MD/PhD Stanford |
4/26/2023 |
ChatGPT for
Better Communication? Envisioning the Future of Speech Language
Therapy via Generative AI |
[1] Lee, S. A. S. (2019). Virtual
speech-language therapy for individuals with communication disorders:
Current evidence, limitations, and benefits. Current Developmental
Disorders Reports, 6, 119-125. [2] Du, Y., Grace, T. D., Jagannath, K.,
& Salen-Tekinbas, K. (2021). Connected play in
virtual worlds: communication and control mechanisms in virtual worlds for
children and adolescents. Multimodal Technologies and Interaction, 5(5),
27. [3] Edwards-Gaither, L., Harris, O., &
Perry, V. (2023). Viewpoint Telepractice 2025:
Exploring Telepractice Service Delivery During
COVID-19 and beyond. Perspectives of the ASHA Special Interest
Groups, 1-6. |
Assignment
#3 due Prof. Yao Do, Medical School, USC |
Week #5 5/1/2023 |
P4
Medicine, Part 2 of 3: Healthcare
with Small Data [slides] Part#2
Training Your Own ChatGPT [slides]
|
[1]
Daphne Koller: Biomedicine and
Machine Learning, AI Podcast #93 with Lex Fridman,
May 2020 [link] [2] Edward Y. Chang et al., "Context-Aware
Symptom Checking for Disease Diagnosis Using Hierarchical Reinforcement
Learning." AAAI (2018). [3] Edward Y. Chang et al., "REFUEL: Exploring
Sparse Features in Deep Reinforcement Learning for Fast Disease
Diagnosis." NIPS (2018). |
E. Chang
|
5/3/2023 |
Project proposal
and related work presentation |
|
Henrik Marklund E. Chang |
Week #6 5/8/2023 |
P4
Medicine, Part 3 of 3: Frontier
Research [slides] (Healthcare
is already in the 3rd AI winter due to data and regulations.) |
[1]
On
Bottleneck of Graph Neural Networks and its Practical Implications, Uri
Alon, Eran Yahav, ICRL 2021. [2] Panel: VR/AR for
Surgery and Medical Education, April 2018 [link]. [3]
Stanford SNI Talk: Advancing Healthcare w/ AI and VR, Edward Y. Chang [link] (Stanford ID required). [4]
The problem of Protein
Folding, Wikipedia. |
E. Chang
|
5/10/2023 |
LLM Part 4 of 4: Virtual Assistant and Augmented LM. [slides] |
[1] Genie: A Generator of Natural
Language Semantic Parsers for Virtual Assistant Commands [2] Augmented Language Model, a survey, Meta, 2023
[link]. |
E. Chang
|
Week #7 5/15/2023 |
Consciousness
& Mind, Part 1 of 3: What is Consciousness? Rule of
Nature in Philosophy and Physics (lecture #14) [slides] |
[1]
What is Life, Erwin Schrodinger, online book [link]. [2]
COCOMO: Computational Consciousness Modeling, E.Y. Chang, 2023 [link]. |
E. Chang
|
5/17/2023 |
Consciousness
& Mind, Part 2 of 3: Computational Consciousness, COCOMO (lecture #15) [slides].
|
[1] Projections, by Karl Deisseroth, Random House,
2021. [2] Discovery of a Perceptual Distance Function for
Measuring Image Similarity, Beitao Li, Edward
Chang, and Yi Wu, Multim. Syst., 2003 [link]. [3] COCOMO: Computational Consciousness Modeling, E.Y.
Chang, 2023 [link]. |
E. Chang
|
Week #8 5/22/2023 |
Cancer,
Part 1 od 2: Cancer Causes and Diagnosis (lecture #12) |
[1]
Principles and methods of integrative genomic analyses in cancer, Vessela N. Kristensen, Ole Christian Lingjærde,
Hege G. Russnes, Hans Kristian M. Vollan, et al., Nature Reviews, 2014. [2]
Biomarker development in the precision medicine era: lung cancer as a case
study, Ashley J. Vargas, and Curtis C. Harris, Natural Reviews, 2016. [optional]
Artificial intelligence in radiology, Ahmed Hosny,
Chintan Parmar, John Quackenbush, Lawrence H. Schwartz,and Hugo J. W. L. Aerts,
Nature Reviews, 2018. |
Dr. Melissa Ko, PhD,
Cancer Biology, Stanford E. Chang |
5/24/2023 |
Cancer,
Part 2 of 2: Cancer Treatment and How AI May Help (lecture #13) [video][Slides] |
[1]
Mass cytometry: blessed with the curse of dimensionality, Evan W Newell
& Yang Cheng, Nature Immunology, June 2016. [2]
Next-Generation Machine Learning for Biological Networks, Diogo M. Camacho, Katherine M. Collins, Rani K. Powers,
James C. Costello, and James J. Collins, Leading Edge Review, June 2018. [optional]
Personalized Cancer Models for Target, Discovery and Precision Medicine,
Carla Grandori1 and Christopher J. Kemp, Trends in Cancer, CellPress Reviews, September 2018. |
Dr. Melissa Ko, PhD,
Cancer Biology, Stanford Henrik Marklund |
Week #9 5/29/2023 |
Memorial
Holiday |
|
|
5/31/2023 |
Consciousness
& Mind, Part 3 of 3: GAI Alignment, Free Will, Ethics, and Mind; Course
Summary (lecture #16) |
[1]
COCOMO: Computational Consciousness Modeling, E.Y. Chang, 2023. |
E. Chang
|
5/31/2023 |
Project
Presentation I |
Please
sign up (sending us email) |
E. Chang
Henrik Marklund |
Week #10 6/5 (last
day of class) |
Project
Presentations II |
Presentation
time is flexible to arrange between 5/31 and 6/5. If later than 6/5, then online
presentation can be arranged. |
E. Chang
Henrik Marklund |