Testing LLM reasoning abilities with SAT is not an original idea; there is a recent research that did a thorough testing with models such as GPT-4o and found that for hard enough problems, every model degrades to random guessing. But I couldn't find any research that used newer models like I used. It would be nice to see a more thorough testing done again with newer models.
Opens in a new window
，推荐阅读WPS官方版本下载获取更多信息
莫納漢（Monaghan）解釋，這類實驗性研究的目的，是了解人們如何在一門語言中逐漸站穩腳步。，更多细节参见搜狗输入法下载
(you can advance these simulations using the step and play buttons)，更多细节参见WPS下载最新地址