3
PSA: I picked a local LLM over a cloud API for a project last month
Honestly, I needed to process about 500 documents and the privacy thing was a big deal. Has anyone else gone local and found a good setup for under $100 a month?
3 comments
Log in to join the discussion
Log In3 Comments
nina_hall2mo ago
Yeah, the power cost was my worry too. I just run the model on an old laptop and only fire it up when I need to batch process stuff.
3
john4301mo ago
Smaller quantized models help a ton. 4-bit or 5-bit versions run way faster on older hardware. My laptop barely breaks a sweat now.
1
julia_burns92mo ago
Wait, you got a local model to work for under a hundred bucks a month... how? The electricity alone for my old gaming rig would blow past that if it was running all the time. Did you find some magic setup that sips power?
1