报告题目:LLMs Beyond Code: Multimodal Reasoning for Scratch Debugging and Feedback
主讲人: Dr. Jialu Zhang, Assistant Professor in the Department of Electrical and Computer Engineering at the University of Waterloo
报告时间:2026年3月6日(星期五)15:00-16:00
报告地点:信息学院3号楼109会议室
讲座摘要:Large Language Models (LLMs) have achieved impressive results in text-based programming, yet they struggle in block-based environments such as Scratch, where semantics are tightly intertwined with visual structure, event-driven concurrency, and runtime behavior. This talk presents a line of research on multimodal LLM systems for Scratch. ViScratch integrates block code and gameplay video to diagnose semantic bugs and generate minimal, execution-verified repairs. Stitch transforms automated repair into step-by-step LLM-guided tutoring rather than direct-answer feedback. ScratchEval introduces an executable benchmark for rigorously evaluating multimodal reasoning and repair quality. Together, this work demonstrates that effective LLM-based programming assistance must be grounded in multimodal execution signals, not code alone.
报告人简介:Jialu Zhang is an Assistant Professor in Electrical and Computer Engineering at the University of Waterloo. His research lies at the intersection of large language models, multimodal reasoning, and software engineering for education. He develops LLM-based systems for block-based programming environments, focusing on reliable debugging, verified repair, and interactive tutoring. He received his Ph.D. from Yale University under the supervision of Ruzica Piskac and earned his undergraduate degree from Shanghai Jiao Tong University’s IEEE Honor Class.
邀请人:计算机科学与技术系 向乔 教授