Josh Rising

Building the Future of Local AI, One Local Model at a Time.

Practical deep-dives into AI engineering, local LLMs, and custom software infrastructure optimized for consumer hardware.

Learn from my experiments.

local_inference.py

import llama_cpp

llm = llama_cpp.Llama(
    model_path="gemma-4-12b-it-qat.gguf",
    n_ctx=131072,
    n_gpu_layers=-1
)

print("Initializing local session...")

About

Hi. I’m Josh. I build things that break in interesting ways, then I write about how they broke.

By day, I’ve been a WordPress developer for 15+ years — long enough to remember when “custom theme” meant editing functions.php at 2am and hoping. I’ve architected 2+ dozen international WordPress sites with 10,000+ pages each, built a GraphQL server to keep them all in sync, migrated an entire franchise’s infrastructure to Docker on Azure in three months, and defended sites against 100,000+ daily botnet attacks without losing sleep over it (mostly). These days I do that work as an independent consultant for clients ranging from cannabis retail platforms to international student services: custom Gutenberg blocks, WooCommerce builds, API integrations, the whole enterprise-WordPress toolkit.

By night (and increasingly by day too), I’m trying to answer a much dumber-sounding question: can I write code without sending my keystrokes to someone else’s data center?

That question turned into a whole project. I’m running local LLMs (currently Gemma, on a consumer GPU I definitely didn’t need to check the Newegg price of mid-blog-post) as an actual coding assistant, not a toy.

It meant learning the hard way that when your AI keeps “fixing” a test by rewriting the program instead, the bug is sometimes in the test itself. It meant building a chain of skills (sdlc-plan, sdlc-implement, sdlc-update-tests, and friends) that turned a pile of copy-pasted prompts into something closer to actual infrastructure. And it meant a lot of yelling at llama-server for dying in its sleep; fair, same, but also please stop.

The skills are open source: sdlc-skills on GitHub. If you’re trying to figure out the line between “local-only experiment” and “just use the cloud model, you fool,” I’m somewhere in that fight too, and I write it up as I go.

What you’ll find on this blog

Two threads, mostly:

AI Development & Local LLMs — the local coding assistant project, what’s working, what’s on fire this week, and the actual configs and numbers behind it.
Technical Notes — WordPress, PHP, infrastructure, and whatever bug decided to ruin my afternoon.

Work with me

If you need enterprise WordPress work, that’s the day job: custom blocks, API integrations, performance and security hardening, or someone who’s already made most of the mistakes on a 10,000-page site so you don’t have to. Get in touch or find me on LinkedIn.

If you just want to talk about running models locally, I’m always up for that too.