Photon OS

  • 1.  Ollama.ai on Photon OS on WSL2

    Posted Nov 27, 2023 07:02 PM
      |   view attached

    Hi,

    Lately I've tinkered around with Ollama.ai on Photon OS, also in a special use case with WSL2 in a Microsoft environment.

    The provisioning of the NVidia GPU drivers on the Windows 11 test environment and installed WSL2  is straight forward.

    The strange thing is that ollama-runner throws an issue  error while loading shared libraries: libstdc++.so.6 on Photon OS. Any idea?


    The logfile is attached.

     

    Photon OS on WSL2 recipe

    dism /Online /Enable-Feature /All /FeatureName:Microsoft-Hyper-V # reboot
dism /Online /Enable-Feature /All /FeatureName:Microsoft-Windows-Subsystem-Linux # reboot
bcdedit /set hypervisorlaunchtype auto # reboot
start "Docker Desktop Installer" /wait "Docker Desktop Installer.exe" # reboot

docker pull photon:5.0-20231028 # docker pull photon release from https://hub.docker.com/_/photon/
docker run -it -d photon:5.0-20231028 # run container in background

docker container list # get docker container id
docker export -o photon-5.0-20231028.tar <container id> # docker export
docker container stop <container id> # docker container stop and prune
docker container prune # docker container stop and prune

wsl --set-default-version 2 # set wsl version
wsl --import Ph5 <target dir> photon-5.0-20231028.tar # wsl import
wsl --set-default Ph5 # set wsl default
wsl # start

     

    Ollama setup recipe:

    tdnf install -y awk libstdc++
    curl https://ollama.ai/install.sh | sh
    ollama serve &
    ollama run llama2

     

    Thanks for good suggestions!
    Daniel

     

    Attachment(s)

    txt
    logfile.txt   10 KB 1 version


  • 2.  RE: Ollama.ai on Photon OS on WSL2

    Posted Nov 28, 2023 10:19 AM

    Okay it works now. The package libstdc++ wasn't installed. Added it to the post above.

    tdnf install libstdc++

     

    Some observations:

    WSL2 is weired. It somehow uses the NVidia GPU drivers installed on Windows and interacts with the WSL2'ified Photon OS. From a security perspective I would say there are concerns.

    Ollama speed is more than 20x faster than in CPU-only mode. This is great news.

    With Photon OS on Workstation 17 Pro, the resources used are lower than in comparison to WSL2. Especially, the fan doesn't kick-in in idle mode. However actually it cannot make use of the NVidia adapter.

    Photon OS on vSphere or baremetal provisioned still is the best fit for lowest resource consumption, security and speed.

     

    edited: One culprit on WSL2 is the complexity of make use of audio. An audio device is missing.

    I've tried to get it run by using a component called AudioPulse, but actually it does not work.

    Good suggestions are welcome! I leave the code snippets here if someone is interested.

    # WSL2 seems to make use of virtio driver bits
    lspci -k
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 00) dxgkrnl
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 00) dxgkrnl
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci
    00:00.0 : (rev 01) virtio-pci

    # Correct time is needed as prerequisite.
    # This snippet however is uncurated. ntpd does not work in WSL2 because systemd isn't the booted init system.
    tdnf install -y ntp iptables
    # add your servers to /etc/ntp.conf
    iptables -A INPUT -i eth0 -p udp --dport 123 -j ACCEPT
    iptables-save >/etc/systemd/scripts/ip4save
    ip6tables-save >/etc/systemd/scripts/ip6save
    systemctl enable ntpd
    set <continent/town> timezone
    ln -sf /usr/share/zoneinfo/<continent>/<town> /etc/localtime
    ntpdate -s <your ntp server>
    hwclock --systohc --localtime

    # A few bits have to be compiled and downloaded. Hence, build-essential and pip is installed.
    tdnf install -y build-essential python3-pip

    # On WSL2 an audio device is missing.
    # An option could be to use PulseAudio. PulseAudio uses sndfile, but the installation of sndfile fails.
    pip3 install sndfile # THIS FAILS. The Linux distro isn't recognized and the headerfile sndfile.h is missing.
    # As alternative, compiling from source https://github.com/libsndfile/libsndfile, fails, too.

    # The following recipe does not work without sndfile.

    tdnf install -y meson cmake
    git clone https://github.com/pulseaudio/pulseaudio
    cd pulseaudio/
    meson build
    meson compile -C build
    build/src/daemon/pulseaudio -n -F build/src/daemon/default.pa -p $(pwd)/build/src/modules/
    cd ..

    # install google_speech. PulseAudio and libsox are prerequisites.
    git clone https://github.com/dmkrepo/libsox
    cd libsox/
    autoreconf -i
    ./configure
    make -s
    make install
    cd ..
    pip3 install google_speech
    google_speech "Hello world."