BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201805Z
LOCATION:241
DTSTART;TZID=America/Chicago:20251116T144000
DTEND;TZID=America/Chicago:20251116T150000
UID:submissions.supercomputing.org_SC25_sess209_ws_fgai119@linklings.com
SUMMARY:Beyond End-to-End: Understanding the Limits of LLMs in Scientific 
 Problem Solving
DESCRIPTION:Youyuan Liu (Temple University); Sheng Di, Neil Getty, Tanwi M
 allick, and Robert Underwood (Argonne National Laboratory (ANL)); and Sian
  Jin (Temple University)\n\nMultimodal large language models (MLLMs) are n
 ow widely used across many applications, including scientific question ans
 wering that requires combining visual and textual inputs. However, existin
 g benchmarks in this area are mostly end-to-end, making it difficult to pi
 npoint where models fail. To address this gap, we design an evaluation fra
 mework that decomposes scientific question answering into subtasks for fin
 e-grained assessment. We evaluate two MLLMs, Gemini 2.5 Pro and Qwen2.5-VL
 -32B-Instruct, on questions involving high-resolution visual data. Results
  show that accurate answers are unattainable without scripting or tool use
 . Although both models can solve individual subtasks, such as mapping citi
 es to coordinates or computing pixel positions, they often fail to integra
 te these abilities in end-to-end reasoning, producing large deviations. Ou
 r findings highlight the importance of benchmarks that expose reasoning bo
 ttlenecks and suggest that agent-based or multi-model approaches may be re
 quired to achieve reliable performance on complex scientific tasks.\n\nRec
 ording: Livestreamed, Recorded\n\nRegistration Category: Technical Program
  Reg Pass, Workshop Reg Pass\n\nSession Chairs: Charlie Catlett (Argonne N
 ational Laboratory (ANL), University of Chicago); Javier Aula-Blasco (Barc
 elona Supercomputing Center (BSC)); Valerie Taylor (Argonne National Labor
 atory (ANL), University of Chicago); Laura Morselli (CINECA); and Rio Yoko
 ta (Institute of Science Tokyo)\n\n
END:VEVENT
END:VCALENDAR