GGUF

• Community aggregator
• Country: United States

Show HN: Reflex – a cold-start GGUF engine that runs Aleph Alpha's Kolibri-1

• Community aggregator
• Country: United States

Cómo correr un modelo de 180B sin GPU: POCKET-Darwin-180B en GGUF con llama.cpp

TL;DR POCKET-Darwin-180B-GGUF es la versión cuantizada del modelo abierto Darwin-180B-RSI, pensada para correr sin GPU. Puntos clave para quien va a desplegarlo: Pesa 111 GB en GGUF (el original en BF16 ocupa 360 GB) y se ejecuta con llama.cpp . Hasta 21 tokens por segundo solo en CPU (1 socket, 16…

• Community aggregator
• Country: United States

Running a 180B Model on a Laptop With No GPU: How 4-bit GGUF Keeps Full Accuracy

A frontier-class model used to mean a rack of data-center GPUs. That assumption is what this post takes apart. POCKET-Darwin-180B-GGUF is a 4-bit build of Darwin-180B-RSI, a 180-billion-parameter model, packaged so it runs without a GPU . It ships as GGUF on Hugging Face and on ModelScope. The…