> ## Content Index > Fetch the complete content index at: https://blog.sweden-remote.synology.me/llms.txt > Use this file to discover other available public pages before exploring further. # Fully working local LLM setup using DeepSeek harness & llama.cpp - URL: https://blog.sweden-remote.synology.me/fully-working-local-llm-setup-using-deepseek-harness-llama-cpp/ - Published: 2026-08-30T13:23:25.000Z - Updated: 2026-08-30T13:23:25.000Z - Author: Julius P Disclaimer: This is going to be a highly opinionated setup. There is a plethora of alternative options with different harnesses out there, and many of them might work as well as or better than this setup. This article started out as an attempt to cover everything, but the landscape is vast. The goal is simply to **get you started with a usable setup**; everything else is up to you (and Reddit). --- **Is this necessary? Why not just use Anthropic/Open AI?** Fair question. You could just use OpenRouter with a little money and be productive. But relying on a closed-source model means it could become unavailable or unaffordable. It also means handling sensitive code or data will always feel iffy. Also, what else is a nerd to do in their free time? **The setup** I do own both a MacBook Pro 14 (M4, 32GB) and a Mac mini (M4 Pro, 64GB). We will go through the setup, and I will do my best to explain why I use these tools. This will be done on MacOS, but most steps can be translated to a Windows (using [WSL2](https://learn.microsoft.com/en-us/windows/wsl/install?ref=blog.sweden-remote.synology.me)) or Linux environment (with CUDA/Nvidia) We are going to use: - **git** must be [installed](https://git-scm.com/install/windows?ref=blog.sweden-remote.synology.me) for your OS - [node.js](https://nodejs.org/en/download?ref=blog.sweden-remote.synology.me) or [bun](https://bun.sh/?ref=blog.sweden-remote.synology.me) (my preference) must be installed - [llama.cpp](https://github.com/ggml-org/llama.cpp?ref=blog.sweden-remote.synology.me) as the LLM backend - [DeepSeek](https://github.com/deepseek-ai/deepseek-harness?ref=blog.sweden-remote.synology.me) as the harness --- More advanced setup (in this article): - [GSD methodology](https://www.opengsd.net/?ref=blog.sweden-remote.synology.me) - [caveman](https://github.com/JuliusBrussee/caveman?ref=blog.sweden-remote.synology.me) skill, [semble](https://github.com/MinishLab/semble?ref=blog.sweden-remote.synology.me), [RTK](https://github.com/rtk-ai/rtk?ref=blog.sweden-remote.synology.me), and [donsetch](https://github.com/dondai44423/donsetch?ref=blog.sweden-remote.synology.me) for improved token efficiency and speed I have tested a lot of different models and setups these last couple of months. I have failed with setups many times. Start with this, and explore from here. **llama.cpp** I think this is the most widely used open-source project to drive local LLMs. You are sure to stumble upon it once you want to run locally. Before starting this process, make sure you have **git** installed. Navigate to a folder of your choice in the terminal of your choice. All of the following instructions assume that your terminal is **at the root** of your chosen folder! This script will pull the latest changes from the llama.cpp repository and build the proper variant matching your OS (Metal on MacOS, CUDA on Windows). If you want to know more or install it a different way, check the [quick start](https://github.com/ggml-org/llama.cpp?ref=blog.sweden-remote.synology.me#quick-start). ```zsh #!/usr/bin/env bash set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" cd "$SCRIPT_DIR" LLAMA_DIR="repos/llama.cpp" # this updates (pulls) the latest version, # or if the target folder does not exist, # clones the files from the repo if [ ! -d "$LLAMA_DIR/.git" ]; then echo "Cloning llama.cpp into $LLAMA_DIR..." mkdir -p repos rm -rf "$LLAMA_DIR" git clone https://github.com/ggml-org/llama.cpp.git "$LLAMA_DIR" cd "$LLAMA_DIR" else cd "$LLAMA_DIR" echo "Pulling latest changes..." git checkout master git pull origin master fi rm -rf build OS_TYPE="$(uname -s 2>/dev/null || echo "Unknown")" CMAKE_ARGS=("-DCMAKE_BUILD_TYPE=Release") if [[ "$OS_TYPE" == "Darwin"* ]]; then CMAKE_ARGS+=("-DGGML_METAL=ON") NCPU=$(sysctl -n hw.ncpu 2>/dev/null || echo 4) elif command -v nvcc &>/dev/null; then CMAKE_ARGS+=("-DGGML_CUDA=ON") NCPU=$(nproc 2>/dev/null || echo "${NUMBER_OF_PROCESSORS:-4}") else NCPU=$(nproc 2>/dev/null || echo "${NUMBER_OF_PROCESSORS:-4}") fi cmake -B build "${CMAKE_ARGS[@]}" cmake --build build -j"$NCPU" echo "Build complete." ``` This is for UNIX-style systems or WSL Once the build is done, we need to download a model for testing. I will be using [Tiel-Coder](https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP/tree/main?ref=blog.sweden-remote.synology.me) for this tutorial, basically Ornith 1.5 35B with the chat template exchanged. Use one that fits your memory; you can read more about [quantization](https://localllm.in/blog/quantization-explained?ref=blog.sweden-remote.synology.me). I will be using a 4-bit quant, which is the sweet spot between quality and memory requirements. Download the file *Tiel-Coder-35B-A3B-MTP-UD-Q4\_K\_XL.gguf* and the vision adapter (if you want to work with images), *mmproj-BF16.gguf,* and place them in your working directory in a **/models** folder. Back in the root folder, we will create a server startup script: **start\_server.sh** ```zsh nano ./start_server.sh ``` ```zsh #!/usr/bin/env bash set -euo pipefail # ========================================= # RAM allocation for llama.cpp on macOS, not needed on Windows or Linux # ========================================= if [[ "$(uname)" == "Darwin" ]]; then ram_mb="$(/usr/sbin/sysctl -n hw.memsize | awk '{print int($1/1024/1024)}')" headroom_mb=$(( ram_mb >= 32768 ? 8192 : 4096 )) wired_mb=$(( ram_mb - headroom_mb )) (( wired_mb < 12288 )) && wired_mb=12288 (( wired_mb > ram_mb - headroom_mb )) && wired_mb=$(( ram_mb - headroom_mb )) echo "RAM: ${ram_mb}MB, headroom: ${headroom_mb}MB, setting iogpu.wired_limit_mb=${wired_mb}" sudo sysctl "iogpu.wired_limit_mb=${wired_mb}" fi # ========================================= ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SESSION_NAME="${SESSION_NAME:-llm-server}" HOST="${HOST:-127.0.0.1}" PORT="${PORT:-8081}" CTX_SIZE="${CTX_SIZE:-131072}" API_KEY="${API_KEY:-nokey}" if [ ! -f "$MODEL" ]; then echo "Error: Model not found at $MODEL" >&2 exit 1 fi LLAMA_SERVER="$ROOT_DIR/repos/llama.cpp/build/bin/llama-server" LOG_FILE="$ROOT_DIR/logs/llama-server.log" mkdir -p "$ROOT_DIR/logs" # Load the models we downloaded before MODEL="$ROOT_DIR/models//models/Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf" MMPROJ="$ROOT_DIR/models/mmproj-BF16.gguf" # The following command looks slightly weird, but I wanted to add some explanations. SERVER_CMD="$LLAMA_SERVER" # Multimodal projector for vision/image support. You can remove this if you want text only, saving some VRAM SERVER_CMD+=" --mmproj $MMPROJ" # Speculative decoding (MTP) for faster inference SERVER_CMD+=" --spec-type draft-mtp --spec-draft-n-max 2" # Model weights (.gguf) file path SERVER_CMD+=" -m $MODEL" # API key for authentication SERVER_CMD+=" --api-key $API_KEY" # Context size (max tokens in memory, you need to adopt this to the amount of ram you have) SERVER_CMD+=" -c $CTX_SIZE" # Network host / IP to listen on SERVER_CMD+=" --host $HOST" # Network port for HTTP API SERVER_CMD+=" --port $PORT" # Max tokens to generate per response SERVER_CMD+=" -n 32768" # Logical batch size for prompt evaluation SERVER_CMD+=" -b 2048" # Physical micro-batch size, remove this flag on CUDA (using the default instead) SERVER_CMD+=" -ub 2048" # Offload everything to GPU SERVER_CMD+=" -ngl 99" # Flash Attention (faster, saves VRAM) SERVER_CMD+=" -fa on" # Quantize KV cache to 8-bit to save VRAM # You can disable this if you have a lot of VRAM # If using NVIDIA/CUDA there are better settings available SERVER_CMD+=" --cache-type-k q8_0" SERVER_CMD+=" --cache-type-v q8_0" # Number of parallel request slots, 1 is optimal for a local agent/coding harness SERVER_CMD+=" -np 1" # Max tokens allocated for thinking SERVER_CMD+=" --reasoning-budget 16384" # Keep thinking blocks in context history SERVER_CMD+=" --reasoning-preserve" # Temperature (lower = focused, higher = creative), for coding mostly 0.6 - 0.7 worked well for me SERVER_CMD+=" --temp 0.7" # top-p: Filtering tokens by cummulative probability SERVER_CMD+=" --top-p 0.95" # min-p: Discard tokens below a certain probability (0.0 = disabled) SERVER_CMD+=" --min-p 0.05" # top-k: Filtering tokens absolutely, keep only top k tokens (0 = disabled), do not combine this with min-p SERVER_CMD+=" --top-k 0" # Minimum number of tokens to generate for an image SERVER_CMD+=" --image-min-tokens 1024" echo "Starting llama-server in foreground..." # Runs server, prints to terminal AND saves to log file simultaneously eval "$SERVER_CMD 2>&1 | tee '$LOG_FILE'" ``` Content and explanations for the start\_server.sh file Save the file, and make it executable using the terminal: ```bash chmod +x ./start_server.sh ``` Next, run the file: ```zsh ./start_server.sh ``` --- **Deepseek harness** We will use this recently popular harness to get started. There are many variants of this (like [pi.dev](https://pi.dev/?ref=blog.sweden-remote.synology.me), [hermes](https://hermes-agent.nousresearch.com/?ref=blog.sweden-remote.synology.me), [opencode](https://opencode.ai/?ref=blog.sweden-remote.synology.me), and more). The setup will vary slightly depending on which one you choose, but the basic steps will be the same (assuming the up-and-running Llama server) In a new terminal window or tab, navigate to a different folder, which we are going to use for some basic prompt evaluations Start the harness with: ```zsh bunx @deepseek-ai/dsh web # or npx @deepseek-ai/dsh web ``` Once loaded and installed, you will be taken to the web UI at [http://127.0.0.1:3080](http://127.0.0.1:3080/?ref=blog.sweden-remote.synology.me) Here, you need to go to **Settings** \-> **Models** \-> **Add a custom provider** Fill in the settings (we use the api key "**nokey**" above) Once you have filled in the info, click on **Fetch available models.** Select the Tiel-Coder model. Click on **Create provider**. ![](https://blog.sweden-remote.synology.me/content/images/2026/08/Screenshot-2026-08-30-at-15.21.31.png) Apply the changes. We are ready to use DeepSeek. Click on **New Session.** Then select the model picker and choose **llama > Tiel-Coder.** ![](https://blog.sweden-remote.synology.me/content/images/2026/08/Screenshot-2026-08-30-at-14.44.43.png) We are ready to get started now. In your selected folder, start an agent task as a little demo. You can use your own, or try the one I use for model evaluation. ```zsh Task Create a complete, fully functional, self-contained 3D Flight Simulator rendered inside a single index.html file using vanilla HTML5, CSS3, and JavaScript (using WebGL directly or Three.js loaded dynamically via CDN). Procedural Terrain & World Generation: Generate an infinite or large bounded 3D terrain using multi-frequency Simplex/Perlin noise. Render a dynamic heightmap featuring mountains, valleys, and an ocean floor. Color terrain dynamically based on altitude (e.g., sandy beaches -> green grass -> rocky cliffs -> snowcaps). Add procedurally generated low-poly trees and clouds scattered across the world. Flight Physics Engine: Implement realistic 6-DOF (Degrees of Freedom) airplane physics, including thrust, pitch, roll, yaw, drag, and aerodynamic lift proportional to airspeed. Include stall physics: if velocity drops below a critical threshold, lift fails, and gravity takes over. Handle collision detection against terrain, trees, and water with an automated crash/reset state. Procedural Aircraft Model: Build a detailed low-poly airplane geometry (fuselage, wings, tail fin, spinning propeller) programmatically—do not load external .gltf or .obj files. Lighting & Atmospheric Effects: Implement a dynamic day/night cycle with a moving sun/directional light. Include atmospheric distance fog matching the sky color for depth perception. HUD & User Interface: Render a functional Heads-Up Display (HUD) using a 2D Canvas overlay or styled SVG/CSS showing: Airspeed indicator, Altimeter (altitude), Throttle percentage (0% - 100%). A working mini-map showing aircraft position and heading. Controls & Accessibility: We want arcade controls. Full keyboard support (W/S or Up/Down for pitch, A/D or Left/Right for turning (like a car), Q/E for roll left/right, Shift/Ctrl for throttle). A plane should always follow its nose direction. Toggleable camera views: Chase Camera (behind aircraft, 3rd person), Cockpit/First-Person Camera, and Free Look Orbit Camera. Responsive canvas scaling (100vw x 100vh) with resizing event handling. Make sure NO HUD elements are centered on the screen to ensure full visibility for the pilot. Postprocessing and optimizing game feel: Once the basic file is working and tested, continue and make it ultra-high-fidelity, meaning optimize visuals, lighting, and sounds. Constraints: Output only valid, executable HTML with embedded CSS (