Show HN: OS Megakernel that match M5 Max Tok/w at 2x the Throughput on RTX 3090Hey there, we fused all 24 layers of Qwen3.5-0.8B (a hybrid DeltaNet + Attention model) into a single CUDA kernel launch and made it open-source for everyone to try it.On an RTX 3090 power-limited to 220W: - 411 tok/s vs 229 tok/s on M5 Max (1.8x) - 1.87 tok/J, beating M5 Max efficiency - 1.55x faster decode than llama.cpp on the same GPU - 3.4x faster prefillThe RTX 3090 launched in 2020. Everyone calls it power-hungry. It isn't, the software is. The conventional wisdom NVIDIA is fast but thirsty. Apple Silicon is slow but sips power. Pick a side.With stock frameworks, the numbers back that up: Setup | tok/s | Power | tok/J RTX 3090 (llama.cpp) | 267 | 350W | 0.76 M5 Max (LM Studio) | 229 | ~130W | 1.76Case closed. Except the 3090 has 936 GB/s of bandwidth and 142 TFLOPS of FP16 compute, and llama.cpp extracts 267 tok/s out of it. That ratio is absurd.Traditional inference dispatches one kernel per operation. For 24 layers, that's roughly 100 launches per token. Every boundary means:
Forest exhibits resilience after California mega fireIn 2019 and again in 2021, Penn State researchers in the Department of Geography walked a series of 1,000 square foot plots in California's Lassen Volcanic National Park. The goal was to see how the ...
Mesterséges intelligencia: buborék vagy megatrend?A 2025-ös év meghatározó trendje a mesterséges intelligencia felfutása, a legkülönbözőbb szoftverek használatának berobbanása, az újabb és újabb fejlettebb modellek harcba állítása, az egymással ...
Megan Thee Stallion Blocks Nicki Minaj From A New No. 1 HitForbes contributors publish independent expert analyses and insights. Hugh McIntyre covers music, with a focus on the global charts. “Lover Girl” is the Grammy-winning rap artist’s latest focus track, ...
Steve Bannon Reignites Musk Feud After Billionaire Bashes Trump’s MegabillSteve Bannon knocked Elon Musk’s underwhelming effort to save billions in government spending after the departed DOGE head trashed Donald Trump’s spending bill. On Monday’s broadcast of his War Room ...
How Josh Became a Wine Megabrand—and Which Brand Might Be Next“I love that it’s called Josh,” says Mrs. K (played by Kate McKinnon) in a Saturday Night Live skit from a few years back. Regarding her wineglass wonderingly, she adds, “It’s a neat name for a wine!” ...