Loading…
22
A guy at the Columbus maker fair told me to stop training my own chatbot and just use a smaller model, and he was dead right
Spent 3 weekends fine tuning a model on 40k lines of my own notes for a support bot. He watched it fail once and asked why I was paying for compute to teach it stuff it could look up. Swapped to retrieval over the same notes and cut my cost from $210 a month to under $30. Anyone else get stubborn about building the hard way first?
1 comments
Log in to join the discussion
Log In1 Comment
theas949d ago
$210 a month to teach a bot your own notes? That's wild. Like paying a tutor to memorize your diary when you could just hand them the book and say "look it up." Three weekends of fine tuning and the guy spots the whole problem in one failed run, brutal. The cost drop alone would make me sick thinking about all those months. Retrieval just feels so obvious in hindsight but I bet it didn't feel that way while you were deep in it.
3