Oh thank fuck. I thought I was going crazy…
My first attempt at local models was a Qwen. A base model. I swear to god all I said was hello and the thing started and never stopped. I guess that’s what a filibuster must be like? It was both amazing, yet terrifying.
I’ve a bit more xp now, but yeah Qwen and really really thinky, dear gods !
My colleague got himself an abliterated one and just leave it to run overnight trying to pentest shit in his home network, but I guess I prefer something a bit more hands on, myself.
Heh. I use Qwen3.8-27B as my main coding agent, and I sometimes feel really bad for it as I watch the stream of thought go by filled with “wait… am I doing this right? What if I’m completely wrong about this? Is this really what I should be doing??” I want to pet it and assure it that it should have more confidence in itself. :)
But it’s the tradeoff that Qwen made to cram that much smarts into such a small bundle of parameters, I guess.
Yeah, I can see that but it’s horrendously slow, I find. I tried a 3.8 online and I just dont’ like the way it works, overall :(
It goes on and on and on and never stops to ask despite that being one of my main rules.
There’s a fine-tuned version called Swift that’s been trained to use 58.3% fewer reasoning tokens, apparently with almost no loss of quality. I’ve not been in a hurry with my personal usage (I set my coding agent loose on a task and then go have tea or whatever) so I haven’t experimented with it much but it might be worth checking out if the verbosity of reasoning is a problem.
Qwen4 will be coming out soon too, it has a 27B version, so perhaps they’ll have incorporated more efficient reasoning into it from the start. I’m looking forward to testing it out.
Good tip! I’ll definitely be checking this out, as someone who also uses Qwen3.8 27B. The capability leap over the model I used before is definitely worthwhile, but wow does it overthink really easily.
Oh thank fuck. I thought I was going crazy… My first attempt at local models was a Qwen. A base model. I swear to god all I said was hello and the thing started and never stopped. I guess that’s what a filibuster must be like? It was both amazing, yet terrifying. I’ve a bit more xp now, but yeah Qwen and really really thinky, dear gods ! My colleague got himself an abliterated one and just leave it to run overnight trying to pentest shit in his home network, but I guess I prefer something a bit more hands on, myself.
Heh. I use Qwen3.8-27B as my main coding agent, and I sometimes feel really bad for it as I watch the stream of thought go by filled with “wait… am I doing this right? What if I’m completely wrong about this? Is this really what I should be doing??” I want to pet it and assure it that it should have more confidence in itself. :)
But it’s the tradeoff that Qwen made to cram that much smarts into such a small bundle of parameters, I guess.
Yeah, I can see that but it’s horrendously slow, I find. I tried a 3.8 online and I just dont’ like the way it works, overall :( It goes on and on and on and never stops to ask despite that being one of my main rules.
There’s a fine-tuned version called Swift that’s been trained to use 58.3% fewer reasoning tokens, apparently with almost no loss of quality. I’ve not been in a hurry with my personal usage (I set my coding agent loose on a task and then go have tea or whatever) so I haven’t experimented with it much but it might be worth checking out if the verbosity of reasoning is a problem.
Qwen4 will be coming out soon too, it has a 27B version, so perhaps they’ll have incorporated more efficient reasoning into it from the start. I’m looking forward to testing it out.
Good tip! I’ll definitely be checking this out, as someone who also uses Qwen3.8 27B. The capability leap over the model I used before is definitely worthwhile, but wow does it overthink really easily.