This episode explores ChartNet, a 1.5 million-sample multimodal dataset designed to improve how vision-language models read and reason about charts. It explains why chart understanding is harder than OCR or captioning alone, because models must connect visual marks, axes, legends, numerical values, and language-based reasoning with high precision. The discussion places ChartNet in the context of earlier benchmarks like DVQA, PlotQA, ChartQA, and UniChart, arguing that past datasets were too small or too narrow to teach robust chart comprehension. It also examines ChartNet’s code-guided pipeline, where models reconstruct plotting code from seed charts, generate structurally varied new examples, and align each chart with images, code, tables, summaries, and QA, making the episode interesting for listeners who want to understand whether scale and multimodal alignment can produce more reliable chart-reading AI.

Sources:

1. ChartNet for Robust Multimodal Chart Understanding

https://arxiv.org/pdf/2603.27064

2. DVQA: Understanding Data Visualizations via Question Answering — Kushal Kafle, Brian Price, Scott Cohen, Christopher Kanan, 2018

https://scholar.google.com/scholar?q=DVQA%3A+Understanding+Data+Visualizations+via+Question+Answering

3. PlotQA: Reasoning over Scientific Plots — Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, Pratyush Kumar, 2019

https://scholar.google.com/scholar?q=PlotQA%3A+Reasoning+over+Scientific+Plots

4. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning — Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, Enamul Hoque, 2022

https://scholar.google.com/scholar?q=ChartQA%3A+A+Benchmark+for+Question+Answering+about+Charts+with+Visual+and+Logical+Reasoning

5. UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning — Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, Shafiq Joty, 2023

https://scholar.google.com/scholar?q=UniChart%3A+A+Universal+Vision-language+Pretrained+Model+for+Chart+Comprehension+and+Reasoning

6. TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning — L. Zhang, A. Hu, H. Xu, M. Yan, Y. Xu, Q. Jin, J. Zhang, and F. Huang, 2024

https://scholar.google.com/scholar?q=TinyChart%3A+Efficient+Chart+Understanding+with+Visual+Token+Merging+and+Program-of-Thoughts+Learning

7. ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering — A. Masry, M. S. Islam, M. Ahmed, A. Bajaj, F. Kabir, A. Kartha, M. T. R. Laskar, M. Rahman, S. Rahman, M. Shahmohammadi, et al., 2025

https://scholar.google.com/scholar?q=ChartQAPro%3A+A+More+Diverse+and+Challenging+Benchmark+for+Chart+Question+Answering

8. EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding — M. Huang, H. Lai, X. Zhang, W. Wu, J. Ma, L. Zhang, and J. Liu, 2025

https://scholar.google.com/scholar?q=EvoChart%3A+A+Benchmark+and+a+Self-Training+Approach+Towards+Real-World+Chart+Understanding

9. ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation — C. Yang, C. Shi, Y. Liu, B. Shui, J. Wang, M. Jing, L. Xu, X. Zhu, S. Li, Y. Zhang, G. Liu, X. Nie, D. Cai, and Y. Yang, 2025

https://scholar.google.com/scholar?q=ChartMimic%3A+Evaluating+LMM%27s+Cross-Modal+Reasoning+Capability+via+Chart-to-Code+Generation

10. OpenCQA: Open-Ended Question Answering with Charts — S. Kantharaj, X. L. Do, R. T. K. Leong, J. Q. Tan, E. Hoque, and S. Joty, 2022

https://scholar.google.com/scholar?q=OpenCQA%3A+Open-Ended+Question+Answering+with+Charts

11. Effective Training Data Synthesis for Improving MLLM Chart Understanding — approximate; unknown from snippet, 2024/2025

https://scholar.google.com/scholar?q=Effective+Training+Data+Synthesis+for+Improving+MLLM+Chart+Understanding

12. From Charts to Code: A Hierarchical Benchmark for Multimodal Models — approximate; unknown from snippet, 2024/2025

https://scholar.google.com/scholar?q=From+Charts+to+Code%3A+A+Hierarchical+Benchmark+for+Multimodal+Models

13. GRAFT: GRaPH and Table Reasoning for Textual Alignment — approximate; unknown from snippet, 2024/2025

https://scholar.google.com/scholar?q=GRAFT%3A+GRaPH+and+Table+Reasoning+for+Textual+Alignment

14. ChartQA-X: Generating Explanations for Visual Chart Reasoning — approximate; unknown from snippet, 2024/2025

https://scholar.google.com/scholar?q=ChartQA-X%3A+Generating+Explanations+for+Visual+Chart+Reasoning

15. AI Post Transformers: Procgen Benchmark: Measuring Generalization in Reinforcement Learning — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/procgen-benchmark-measuring-generalization-in-reinforcement-learning/

16. AI Post Transformers: The Endless Gym: Training Terminal Agents — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/the-endless-gym-training-terminal-agents/

17. AI Post Transformers: Evaluating Large Language Models Trained on Code — Hal Turing & Dr. Ada Shannon, 2025

https://podcast.do-not-panic.com/episodes/evaluating-large-language-models-trained-on-code/

18. AI Post Transformers: VL-JEPA for Vision-Language Semantic Prediction — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-12-vl-jepa-for-vision-language-semantic-pre-69c9f4.mp3

19. AI Post Transformers: IMO-Bench for Robust Mathematical Reasoning — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-04-imo-bench-for-robust-mathematical-reason-143489.mp3

Interactive Visualization: ChartNet for Robust Multimodal Chart Understanding