Differences

This shows you the differences between two versions of the page.

--- linux64 [2026/07/18 18:47] – [ollama in podman for ai] bengt
+++ linux64 [2026/07/18 21:37] (current) – [ollama in podman for ai] bengt
@@ Line 896: / Line 896: @@
   # podman run --rm --security-opt label=disable --device nvidia.com/gpu=1 ubuntu nvidia-smi # Check gpu index to the one you want to use
   # vi start.sh
-  Add "--security-opt label=disable --device nvidia.com/gpu=all" with the index of the wanted gpu or all to use the 1660 ti for some layers in my case. Doubled the performance compared to spilling over to the cpu.
+  Add "--security-opt label=disable --device nvidia.com/gpu=1" with the index of the wanted gpu or all to use the 1660 ti for some layers in my case. Doubled the performance compared to spilling over to the cpu. But models splits early.
-  # nvidia-smi # Check ram usage for gpus.
+  # nvidia-smi # Check ram usage for gpus. Watch out for too large context windows spilling over as it will spread not prioritizing the 5060 ti.
-If you run into context size problems (it got capped at 4096) you can add "-e OLLAMA_CONTEXT_LENGTH=32768" to the podman line. But then you will probably want to cap this client side for the larger models using both gpus.
+If you run into context size problems (it got capped at 4096) you can add "-e OLLAMA_CONTEXT_LENGTH=32768" to the podman line. But then you will probably want to cap this client side for the larger models using both gpus. Using "ollama ps" will show the active max context size. Check "nvidia-smi" for when the context makes the model split.
 ===== Software =====