Loading…
7
I thought bigger AI models were always better until I hit a wall with a 70B model last month
Last month I was trying to build a little model that reads wiring diagrams for my shop and I kept fighting with a 70B model on my one GPU. A guy at the local makerspace in Tucson showed me his setup running a small 3B model fine-tuned on just 800 of his own labeled images, and it nailed the same task in under 2 seconds. I was sure scale was everything, but for narrow jobs like mine the small tuned model beat the giant one on speed and accuracy both. Anyone else here find that a small fine-tuned model did better than a big general one for your specific task?
1 comments
Log in to join the discussion
Log In1 Comment
jason1125d ago
800 labeled images is the catch though. That guy in Tucson didn't just grab a 3B and win, he spent weeks labeling wiring diagrams. Most people don't have that. I ran a small model on invoice parsing for a while and it fell apart the second a vendor changed their layout, while the 70B at least guessed right on stuff it had never seen. Speed is real, sure, but a tuned model only knows what you taught it and that bites you later.
5