how do these deepseek Quants measure against IQ3_S Qwen3.8-flash-next RCO GSK
IQ3_S Qwen3.8-flash-next RCO GSK quant is just fabulous, at times can match frontier models at many coding task and is generally very smart for reasoning with internet+RAG grounding... Qs is how does the deepseek v4.1 1017 Istalab's quant compare to Qwen3.8-flash-next quants... since this quant will need hardware upgrades, want to see the economic justification based on the gap
Since they claimed IQ2_XXS is pretty much = FP8, you can just compare it with the hosted DSV4 flash and get a good feel of the difference. Spend a lunch money of $5 and test same prompts/tasks against Qwen3.8 FN.
BIG CAVEAT this is if we trust their IQ2_XXS = FP8 claim of course, but these big 700B+ models are resilient to low-bit quantizations to begin with so I guess there are some truth.
@mayankiit04
We just had the opportunity to test the DS model with a reasoning effort of 50 (it goes up to 100!) and compared it to QFN with xhigh reasoning effort. It was both more accurate and more token-efficient on our benchmarks. Which isn't surprising, given that this model is much larger than QFN.
it better should be more token efficient given they will likely come at a slow pace on real world local hardware..
I think i pass even on disk usage alone. And qfn is in deed very good and ridiculous fast for the size.
Who'd have though that 3 months ago... When i tried a 120b model in lmstudio
first qfn IQ3_S from istalab and then strata has made working quite fun that usage of external frontier api have dropped for me... I understand that given dps IQ2_XS speed on strata is still slow, but wanted to understand the leap in improvement to decide if i need to upgrade to 128 gb ram from 64 gb... slow speed makes comparision very painful... i guess will wait for new qwen4 flash hope it is also open weight
it better should be more token efficient given they will likely come at a slow pace on real world local hardware..
I think i pass even on disk usage alone. And qfn is in deed very good and ridiculous fast for the size.Who'd have though that 3 months ago... When i tried a 120b model in lmstudio
Not just disk usage, disk wear too. With Deepseek, you'd be constantly reading experts from SSD (which is waaay larger than just the PLE table, GBs of experts vs just less than 2MiB of reads per token). And SSD having only limited RW lifespan will wear so much faster with deepseek here (unless you fit all experts in RAM of course)
You don't wear out an SSD from reads - and reads is what you will be doing a lot of to serve 4.1Flash on a "normal" system.
What would the system need to write to disk? If this is happening then it will be due to OS paging happening - and that is a configuration mistake.
Fast SSDs in RAID0 is quickly becoming more an more attractive as a third-tier memory layer, and you can get a surprisingly high speed and fast latency out of 2-3 decent NVME's
You don't wear out an SSD from reads - and reads is what you will be doing a lot of to serve 4.1Flash on a "normal" system.
What would the system need to write to disk? If this is happening then it will be due to OS paging happening - and that is a configuration mistake.
Fast SSDs in RAID0 is quickly becoming more an more attractive as a third-tier memory layer, and you can get a surprisingly high speed and fast latency out of 2-3 decent NVME's
It does wear, but you're right, it's not any meaningful wear (probably less than ~1% of what write wears). Write is what I'm concerned as most OS pages aggressively when RAM is tight (and with DeepSeek you'd be tight).