Differences

This shows you the differences between two versions of the page.

--- linux64 [2026/05/30 18:52] – [Moving boot disk from sata to m.2] bengt
+++ linux64 [2026/07/18 21:37] (current) – [ollama in podman for ai] bengt
@@ Line 78: / Line 78: @@
 Removed a kernel and tried this instead:
-echo -e 'add_drivers+=" nvme ahci libata sd_mod "' | sudo tee /etc/dracut.conf.d/generic.conf
+  # echo -e 'add_drivers+=" nvme ahci libata sd_mod "' | tee /etc/dracut.conf.d/generic.conf
-dracut --force
+  # dracut --force
+Then clone again. Remove sata and boot from nvme to check that it works. This time it did. Then boot livecd again, install gparted and use this to expand /boot and move and expand / putting the swap at the end as well. Reboot
-Then clone again. Remove sata and boot from nvme to check that it works. This time it did.
-Then boot livecd again, install gparted and use this to expand /boot and move and expand / putting the swap at the end as well. Reboot
 Also took the time to get new rescue images as my old ones were from 2016, remove old vmlinuz/initramfs for rescue first:
@@ Line 869: / Line 866: @@
   cd $HOME
   loginctl enable-linger ollama
-  podman run -d --rm -v /home/ollama/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
+  podman run -d --rm -v /home/ollama/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama # See below for flags when using gpu
   $ chmod u+x start.sh
 Then setup a start.sh-script containing the startup command line and maybe start this using "lxterminal -t ollama -e sudo -u ollama /home/ollama/start.sh". Replace "-d" with "-it" in the podman command line to see the output if starting this way.
@@ Line 875: / Line 872: @@
 Test it as the ollama user:
   # su - ollama
-  $ podman exec -it ollama ollama run llama2
+  $ podman exec -it ollama bash
+  $ ollama run llama3.2
+Backup images and update:
+  # su - ollama
+  $ podman save ollama/ollama > image_backup.tar
+  $ podman pull ollama/ollama
+  Restart container...
 You can add the ollama integration to home assistant. On issues check your firewall config. Be sure to have "conversation:" "assist_pipeline:" in configuration.yaml if you do not use the default config before trying to setup assist pipelines.
-TODO: GPU support. Need to use nvidia drivers first...
+After buying a 5060 16GB ti for AI to have next to my 1660 ti super used for the console I needed to switch to nvidia drivers and configure podman. I already have rpmfusion-nonfree repo enabled so I just did this:
+  # dnf update # To avoid issues below. Reboot if new kernel.
+  # dnf install -y akmod-nvidia xorg-x11-drv-nvidia-cuda
+  # akmods --force
+  Wait for akmod to complete building drivers before rebooting. Check with top?
+Note that I lost audio after this for the new kernel that got installed at the same time. Turns out that those modules were not installed as they were not needed to compile the nvidia driver. Update and reboot before or install lacking kernel-modules and kernel-modules-extra. This should also work "dnf install kernel-$(uname -r)".
+Configure podman:
+  # dnf install nvidia-container-toolkit
+  # nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
+  # nvidia-ctk cdi list
+  # podman run --rm --security-opt label=disable --device nvidia.com/gpu=1 ubuntu nvidia-smi # Check gpu index to the one you want to use
+  # vi start.sh
+  Add "--security-opt label=disable --device nvidia.com/gpu=1" with the index of the wanted gpu or all to use the 1660 ti for some layers in my case. Doubled the performance compared to spilling over to the cpu. But models splits early.
+  # nvidia-smi # Check ram usage for gpus. Watch out for too large context windows spilling over as it will spread not prioritizing the 5060 ti.
+If you run into context size problems (it got capped at 4096) you can add "-e OLLAMA_CONTEXT_LENGTH=32768" to the podman line. But then you will probably want to cap this client side for the larger models using both gpus. Using "ollama ps" will show the active max context size. Check "nvidia-smi" for when the context makes the model split.
 ===== Software =====