Running LLMs Locally with Ollama

curl -fsSL https://ollama.com/install.sh | sh
>>> Installing ollama to /usr/local
[sudo] password for alexshank:
>>> Downloading Linux amd64 bundle
######################################################################## 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> NVIDIA GPU installed.
alexshank@alex-desktop:~/Desktop/main/repos/blog-astro$ ollama run mistral-small3.1
pulling manifest
pulling 1fa8532d986d... 100% ▕████████████████████████████████▏  15 GB
pulling 6db27cd4e277... 100% ▕████████████████████████████████▏  695 B
pulling 70a4dab5e1d1... 100% ▕████████████████████████████████▏ 1.5 KB
pulling a00920c28dfd... 100% ▕████████████████████████████████▏   17 B
pulling 9b6ac0d4e97e... 100% ▕████████████████████████████████▏  494 B
verifying sha256 digest
writing manifest
success
>>> hello
Hello! How can I assist you today?

> > > What's the capital of italy?
> > > The capital of Italy is Rome.

ollama run deepseek-r1:14b "What are the next three numbers in the sequence: 2, 4, 8, 16, ...?"
Thinking...
The given sequence starts with 2 and each subsequent number is obtained by multiplying the previous one by 2.

To find the next three numbers:
- The fifth number will be 32 (16 × 2).
- The sixth number will be 64 (32 × 2).
- The seventh number will be 128 (64 × 2).

Therefore, the next three numbers in the sequence are 32, 64, and 128.
...done thinking.

**Solution:**

The given sequence is:
\[ 2,\ 4,\ 8,\ 16,\ \ldots \]

To find the pattern:
- **First term:** \(2\)
- **Second term:** \(4\) (which is \(2 \times 2\))
- **Third term:** \(8\) (which is \(4 \times 2\))
- **Fourth term:** \(16\) (which is \(8 \times 2\))

We observe that each term is obtained by multiplying the previous term by **2**. This indicates a **geometric sequence** with a common ratio of \(2\).

To find the next three terms:
1. **Fifth term:**
   \[
   16 \times 2 = 32
   \]
2. **Sixth term:**
   \[
   32 \times 2 = 64
   \]
3. **Seventh term:**
   \[
   64 \times 2 = 128
   \]

**Final Answer:**
\[
\boxed{32,\ 64,\ 128}
\]

# base64 encode image
export IMAGE_DATA=$(base64 -w 0 severance-test-image.jpg)

# make HTTP request (with formatting help from jq)
curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
    "model": "mistral-small3.1",
    "messages": [{
    "role": "user",
    "content": "What is in this image?",
    "images": ["'"$IMAGE_DATA"'"]
    }],
    "stream": false
}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 87822  100  1179  100 86643     83   6156  0:00:14  0:00:14 --:--:--   305
{
  "model": "mistral-small3.1",
  "created_at": "2026-02-10T22:37:39.975971606Z",
  "message": {
    "role": "assistant",
    "content": "The image appears to be a promotional or title card for a TV show or movie called \"Severance.\" The central figure in the image is a man dressed in business attire, consisting of a suit, white dress shirt, and tie. He has a serious or contemplative expression on his face.\n\nThe most striking element of the image is that the man has a small, detailed scene perched on his head. This scene depicts an office environment with another man sitting at a desk with a computer and other office equipment. The setting appears to be a modern office with fluorescent lighting visible in the background.\n\nThe title \"Severance\" is prominently displayed in white, uppercase letters to the right of the man's head. The background is a muted green with a pattern of rectangular light fixtures, contributing to the overall corporate and somewhat dystopian atmosphere of the image."
  },
  "done": true,
  "done_reason": "stop",
  "total_duration": 14072339404,
  "load_duration": 115410488,
  "prompt_eval_count": 1029,
  "prompt_eval_duration": 95425089,
  "eval_count": 172,
  "eval_duration": 13781950876
}

nvidia-smi
Tue Feb 10 13:39:12 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09             Driver Version: 580.126.09     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti     Off |   00000000:01:00.0  On |                  N/A |
|  0%   51C    P3             21W /  165W |    1381MiB /  16380MiB |     32%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A          264831      G   /usr/lib/xorg/Xorg                      333MiB |
|    0   N/A  N/A          265179      G   /usr/bin/gnome-shell                     63MiB |
|    0   N/A  N/A          267015      G   .../7672/usr/lib/firefox/firefox        434MiB |
|    0   N/A  N/A         1136024      G   /usr/bin/nautilus                        15MiB |
|    0   N/A  N/A         1151182      G   /usr/bin/ghostty                         67MiB |
|    0   N/A  N/A         1152661      G   /proc/self/exe                           61MiB |
|    0   N/A  N/A         2719223      G   ...ice --variations-seed-version         51MiB |
+-----------------------------------------------------------------------------------------+

# add this function to your shell profile
reload_nvidia_uvm() {
    echo "Stopping Ollama service..."
    sudo systemctl stop ollama

    echo "Identifying processes using nvidia_uvm..."
    processes=$(sudo lsof /dev/nvidia-uvm | awk 'NR>1 {print $2}' | sort -u)

    if [[ -n "$processes" ]]; then
        echo "Terminating processes..."
        for pid in $processes; do
            echo "Killing process $pid..."
            sudo kill -9 $pid
        done
    fi

    echo "Unloading nvidia_uvm module..."
    sudo rmmod nvidia_uvm

    echo "Reloading nvidia_uvm module..."
    sudo modprobe nvidia_uvm

    echo "Restarting Ollama service..."
    sudo systemctl start ollama

    echo "Done!"

}