Josh Rising

Building the Future of Local AI, One Local Model at a Time.

Practical deep-dives into AI engineering, local LLMs, and custom software infrastructure optimized for consumer hardware.

Learn from my experiments.
local_inference.py 
import llama_cpp

llm = llama_cpp.Llama(
    model_path="gemma-4-12b-it-qat.gguf",
    n_ctx=131072,
    n_gpu_layers=-1
)

print("Initializing local session...")

opencode.json

Written by

zero4281

in

{
  "$schema": "https://opencode.ai/config.json",
  "autoupdate": true,
  "server": {
    "port": 4096
  },
  "permission": "allow",
  "provider": {
    "llama-server": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "LLama Server (Local)",
      "options": {
        "baseURL": "http://192.168.1.151:11235/v1"
      },
      "models": {
        "Jackrong/Qwen3.5-9B-Neo-GGUF:Q5_K_M": {
          "name": "Qwen 3.5 Coder Neo (9B Q5)",
          "options": {
            "presence-penalty": 0.2,
            "repeat-penalty": 1.2,
            "temp": 0.7,
            "top-k": 20,
            "top-p": 0.95,
            "min-p": 0.1
          }
        },
        "Jackrong/Qwopus3.5-9B-Coder-GGUF:Q5_K_M": {
          "name": "Qwopus 3.5 Coder (9B Q5)",
          "options": {
            "presence-penalty": 0.0,
            "repeat-penalty": 1.05,
            "temp": 0.4,
            "top-k": 40,
            "top-p": 0.9,
            "min-p": 0.05
          }
        },
        "unsloth/gemma-4-12B-it-qat-GGUF:Q4_K_XL": {
          "name": "Gemma 4 QAT Unsloth (12B Q4)",
          "options": {
            "presence-penalty": 0.0,
            "repeat-penalty": 1.05,
            "temp": 1.0,
            "top-k": 64,
            "top-p": 0.95,
            "min-p": 0.05
          }
        },
        "unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M": {
          "name": "Qwen 3.6 (35B A3B Q4)",
          "options": {
            "presence-penalty": 0.0,
            "repeat-penalty": 1.0,
            "temp": 0.4,
            "top-k": 40,
            "top-p": 0.9,
            "min-p": 0.05
          }
        },
        "Jackrong/Qwopus3.6-35B-A3B-Coder-MTP-GGUF:Q4_K_M": {
          "name": "Qwopus 3.6 (35B A3B Q4)",
          "options": {
            "presence-penalty": 0.0,
            "repeat-penalty": 1.0,
            "temp": 0.4,
            "top-k": 40,
            "top-p": 0.9,
            "min-p": 0.05
          }
        }
      }
    },
    "openrouter": {
      "name": "OpenRouter",
      "models": {
        "nvidia/nemotron-3-super-120b-a12b:free": {
          "name": "NVIDIA: Nemotron 3 Super (free - custom)"
        }
      }
    }
  }
}

Comments

2 responses to “opencode.json”

  1. I Built a Local AI Coding Assistant on Consumer Hardware…and It Works. I think. – Josh Rising

    […] Here’s my config for OpenCode: opencode.json […]

  2. Wait… How Do I Write Code Now? – Josh Rising

    […] opencode.json […]

More posts