Runtimes
MobiBench paper benchmarks small LLMs in llama.cpp on two laptops and an RTX 3060
MobiBench is a llama.cpp benchmark suite for speed, memory and accuracy of small LLMs, but the reported tests ran on two laptops and not on phones.
Cutting edge on edge devices.
MobiBench is a llama.cpp benchmark suite for speed, memory and accuracy of small LLMs, but the reported tests ran on two laptops and not on phones.