Differences

This shows you the differences between two versions of the page.

--- linux64 [2026/07/18 14:32] – [ollama in podman for ai] bengt
+++ linux64 [2026/07/18 21:37] (current) – [ollama in podman for ai] bengt
@@ Line 866: / Line 866: @@
   cd $HOME
   loginctl enable-linger ollama
-  podman run -d --rm -v /home/ollama/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
+  podman run -d --rm -v /home/ollama/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama # See below for flags when using gpu
   $ chmod u+x start.sh
 Then setup a start.sh-script containing the startup command line and maybe start this using "lxterminal -t ollama -e sudo -u ollama /home/ollama/start.sh". Replace "-d" with "-it" in the podman command line to see the output if starting this way.
@@ Line 893: / Line 893: @@
   # dnf install nvidia-container-toolkit
   # nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
+  # nvidia-ctk cdi list
+  # podman run --rm --security-opt label=disable --device nvidia.com/gpu=1 ubuntu nvidia-smi # Check gpu index to the one you want to use
+  # vi start.sh
+  Add "--security-opt label=disable --device nvidia.com/gpu=1" with the index of the wanted gpu or all to use the 1660 ti for some layers in my case. Doubled the performance compared to spilling over to the cpu. But models splits early.
+  # nvidia-smi # Check ram usage for gpus. Watch out for too large context windows spilling over as it will spread not prioritizing the 5060 ti.
+If you run into context size problems (it got capped at 4096) you can add "-e OLLAMA_CONTEXT_LENGTH=32768" to the podman line. But then you will probably want to cap this client side for the larger models using both gpus. Using "ollama ps" will show the active max context size. Check "nvidia-smi" for when the context makes the model split.
 ===== Software =====