That chat with a comp sci professor wrecked my views on neural networks

I was at a coffee shop downtown and this older guy starts picking apart my argument about GPT models being smart. He said they are just fancy autocomplete with zero understanding. At first I got defensive, but then he showed me how the same model fails on basic logic puzzles if you reword them slightly. It hit me that we are building these massive systems without checking if they actually grasp anything. Now I wonder if all this AI hype is just us projecting intelligence onto math. Has anyone else had a conversation that made you question the whole field?

2 comments

2 Comments

the_thomas20d ago

Wait, did he show you any specific examples of those logic puzzles getting messed up? Because I keep hearing this argument but I've never actually seen someone break down exactly how the rewording trips it up.

mila_craig420d ago

Try looking up the "GSM8K vs GSM-Symbolic" test results, that's basically the same thing the prof probably showed you. @the_thomas the trick is they swap out numbers and contexts in math problems and accuracy nosedives like 30%. Showing those failures in person makes the whole "understanding" claim just fall apart.