We walk through a local build of Llama.cpp on a Windows machine! And we use the word "build" because it's not just a download and install but rather a download and build from source. Very cool process!
Llama.cpp is a command line inference engine and the basis for many of the user-interfaces that you will find yourself using.
Being lightweight, llama.cpp can run on Android devices, old-machines, and absolutely blaze on machines with 8+gb of VRAM.
You will need the Github CLI installed
Links:
https://github.com/ggerganov/llama.cpp
https://github.com/skeeto/w64devkit/r... (get the w64devkit-fortran-1.23.0.zip)
https://central.github.com/deployment...
コメント