Loading…
25
Spent a full weekend trying to get a local AI model to stop repeating itself
I was trying to set up a small language model on my own computer to help write project notes. The thing kept saying the same phrase over and over, like a broken record. I tried changing the temperature setting, the top_p value, and even the max token length, but nothing worked for hours. Turns out the problem was in the repetition penalty setting in the config file; I had it set way too low. Has anyone else run into this with running models locally?
3 comments
Log in to join the discussion
Log In3 Comments
gibson.oliver6mo ago
Set repetition penalty to 1.2 fixed mine.
9
paul1176mo ago
What if the small stuff is the whole problem?
8
bettykim6mo ago
Is it really that big of a deal? Gibson.oliver's fix might work for some, but in my experience most people can just ignore the small stuff.
3