• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

What LLM/Model you using?

kram182 said:
It can vary just giving it the same task, repeatedly. Downside of probabilistic vs determinsistic.
Click to expand...
It's like calling an indian call center, hoping for the most intelligent least arrogant operator, but often that's not what you get.
 
Qwen 3.5 122b moe replaced gpt oss 120b for me. I don't know if it's more accurate, but I like it's output better and I like that it has vision. Also, it has huge context which I like because most of the conversations I have are iterative. If I need a quick answer, I usually throw it into google/Gemini, claude, or chatgpt, depending on what it is. Google ai has provided tons of bad into that I call it out on though for technical things.

It will be nice when windows updates how it uses unified memory to allow more to be used by the gpu, later this year.

Also looking to start using medgemma to track things I want to bring up with my Dr between physicals. I seem to always forget things like my shoulder issue when it's fine the week of the appointment, etc. But it might be cool to see if ai has any insights and medgemma seems solid. ***only local of course, don't be discussing your personal health with cloud ai***
 
Last edited:
Curious if folks complaining about all local models being dumb actually tune the parameters of their model. It is not insignificant by a long stretch.
 
Most models are simply dumb. They start to resemble intelligence around 120b or so, where maybe if you use them right the can be usefull. Anything under 8b is ofc useless for almost all things.

The problem is, to do tasks like advanced coding, you dont need a model that is just good enough, you need pretty much the best model avalible and even then you must prompt it and check work and use a good workflow. A cloud model is like 1.7t there really is no comparison between that and a local model.

It is still cool to see the usefulness many can get with over 32gb of vram or fast mem. The sparks and macs and good gpus are definitely capable of some work.
 
Last edited:
cdabc123 said:
Most models are simply dumb. They start to resemble intelligence around 120b or so, where maybe if younusebthem right the can be usefull. Anything under 8b is ofc useless for almost all things.

The problem is, to do tasks like advanced coding, you dont need a model that is just good enough, you need pretty much the best model avalible and even then you must prompt it and check work and use a good workflow. A cloud model is like 1.7t there really is no comparison between that and a local model.

It is still cool to see the usefulness many can get with over 32gb of vram or fast mem. The sparks and macs and good gpus are definitely capable of some work.
Click to expand...
The small models can still be good for tasks instead of knowledge. And if they're able to search the web, they can be pretty useful.
 
With tooling and agentic harness small model can be quite useful and with 512GB Mac connected together and a lot of money you can run large model, the distinction is a bit more cheap to run vs expensive than local vs non local.
 
"Advanced Coding" can only be advanced with developer guidance. Developers would not be needed if there's true advanced coding but we aren't there yet. If we get there then there will be no more software development careers. But a good developer knows how to solve very advanced problems by using the AI skillfully on focused prompts.
 
LukeTbk said:
With tooling and agentic harness small model can be quite useful and with 512GB Mac connected together and a lot of money you can run large model, the distinction is a bit more cheap to run vs expensive than local vs non local.
Click to expand...
100% Think a harness alone would solve many of the "My model is dumb" experiences. Don't want to say people are doing it wrong, but I don't have these issues from 2.5b up to 35b locally and being properly utilized for the intended tasks.
 
SecretStash said:
"Advanced Coding" can only be advanced with developer guidance. Developers would not be needed if there's true advanced coding but we aren't there yet. If we get there then there will be no more software development careers. But a good developer knows how to solve very advanced problems by using the AI skillfully on focused prompts.
Click to expand...
Goes back to using the right tool for the job, and knowing how to properly use said tool to be a force multiplier, vs just a crutch for lack of actual knowledge and skill.
 
mashie said:
I have no interest in Colibri, but you can just about fit GLM-5.2 on a cluster of 4 on a 4bit quant.

Hermes + DSv4 Flash 0731 is a match made in heaven. Last night I asked it to figure out why a MySQL database in my system always errored out when trying to make a backup. Well the issue was found, a backup was made, a MariaDB was installed on a Raspberry Pi to verify it could be restored. Then after a pro/con discussion about MySQL vs MariaDB it was decided to migrate the database to MariaDB as it was under support. Oh and now I have a monthly scheduled backup in place. All in all about 40 minutes and no external LLM used.

Colour me impressed.
Click to expand...
Well, I tried it...0.3tok/sec.

I too have been using the deepseek 0731. I've been doing the q3, but considering the way the made the model, i do t think that is too bad. Also at the full 1m context. Only now getting it cranking on some stuff.
 
You must log in or register to reply here.
Back
Top