Josh Rising

Building the Future of Local AI, One Local Model at a Time.

Practical deep-dives into AI engineering, local LLMs, and custom software infrastructure optimized for consumer hardware.

Learn from my experiments.
local_inference.py 
import llama_cpp

llm = llama_cpp.Llama(
    model_path="gemma-4-12b-it-qat.gguf",
    n_ctx=131072,
    n_gpu_layers=-1
)

print("Initializing local session...")

Contact