Run an LLM locally on a Mac: what to download, how much RAM

Running LLMs locally on a Mac mini is surprisingly viable, but you must carefully monitor how memory is actually allocated. Testing this on a 48GB Mac mini, my biggest takeaway is that you cannot trust configuration settings blindly; always verify the runtime behavior.

Run an LLM locally on a Mac: what to download, how much RAM

Regarding disk space, five different models occupied 56.60GB. Note that parameter counts do not map linearly to file size: a 4B model is 6.86GB, a 26B model is 22.79GB, and a 35B model is 26.23GB. Performance-wise, the 35B model is impressive, loading in about 10 seconds and generating text at 28.7 tok/s.

The real catch is memory management. I noticed that while my configuration set the context to 262,144, the actual running value was 136,704. LM Studio silently shrinks the context window to ensure the model fits into available memory, so you should always check the active value during execution.

Another critical point for Mac users is that Apple Silicon uses unified memory, meaning the CPU and GPU share the same pool. If you load a large model and start a separate GPU-intensive task, you can easily exhaust your RAM. When this happens, macOS may kill a process to reclaim memory without leaving a single log entry. To track active models, use the following command:

lms ps

Overall, it is a great experience, but the lack of error logs during memory-related crashes means you must proactively monitor your resource usage.

Related posts

Comments

Post a Comment

Popular posts from this blog

npm command not found on Windows: fix the PATH

Image
I recently hit a wall while configuring my development environment on Windows. It's one of those subtle issues that can waste an afternoon if you don't know what to look for. Here's exactly how I diagnosed and fixed a 'command not found' error after a successful global install. The Problem I decided to add a markdown processing tool to my toolkit by running: npm install -g markdown-it-cli The terminal output confirmed the installation succeeded, displaying a message about added packages. Satisfied, I tried to use it by typing markdown-it-cli in the command line. Windows immediately responded with: 'markdown-it-cli is not recognized as an internal or external command, operable program or batch file.' This error means Windows looked through every folder in its search list and couldn't find an executable with that name. Attempts and Failures My initial reaction was to suspect the package itself might be broken. I tried running it using npx , wh...
Read more

Claude CLI 401 Unauthorized Refresh Token Issue

Symptom: claude commands fail with 401 Unauthorized and the message "refresh token was already used. Please log out and sign in again." Cause: The refresh token expired after being consumed by another session or a background process. Refresh tokens are single-use credentials. Fix: Run claude auth login to refresh authentication without running a logout command first. Why claude login alone did not clear the 401 I was running a pipeline command when every subsequent CLI call failed. The terminal returned this exact output: Error: 401 Unauthorized Message: "Authentication failed: refresh token was already used. Please log out and sign in again." Code: "AUTH_ERROR" The error text explicitly instructs me to log out. I spent roughly ten minutes searching the terminal for a logout function. The prompt only showed AUTH_ERROR . I assumed clearing old sessions was required first. That assumption blocked progress. The instructions in the error message ...
Read more

Tailscale without sudo: what userspace mode actually costs you

At first, I installed Tailscale on macOS knowing I did not have admin rights to write to system directories. The CLI package installed cleanly enough, landing entirely within my user space without triggering any permission errors or requiring elevated prompts. The installation process felt seamless, but the friction appeared immediately during usage. Every single command I typed required appending --socket=$HOME/.tailscale/tailscaled.sock to talk to the daemon. I found myself adding this flag to status checks, key generation, and connection queries across multiple sessions. The repetition was exhausting for a tool I expected to run in the background, turning routine operations into a chore. I eventually wrote a shell function to wrap this flag, allowing me to call tailscale directly without typing the socket path every time. This workaround saved my sanity but highlighted that userspace mode demands extra configuration just to use the CLI, and any automation scripts I maintained had...
Read more