/closes #6 /claim #6

A video demonstrating the test process: Link to video Includes both the worker invoke and worker stream

To run the tests, follow these steps:

1. Ensure Docker is installed and running.
2. Pull the required Ollama models:

   docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
   docker exec -it ollama ollama pull moondream # for image test
   docker exec -it ollama ollama pull qwen3:1.7b # for text tests
   

3. In the main project directory, build the application:

   cargo make build
   

4. In the test directory, deploy the application using the specific build profile:

   golem-cli app deploy --build-profile ollama-debug
   

5. No need for env, base url defaults to localhost:11434, can be overriden if needed. Create worker and run tests.

• Test 1,2,3,5,6: ✅

• Test 4: ⚠️ Partial Success

• Test 4 (Tool Use): Ollama’s streaming doesn’t fully support tool calls, Refer to API. The code is there, when Ollama supports this, our code should work as is.

• Test 5 (Vision with Moondream): This test uses the moondream model (only one that will work in my system) to see if image understanding (vision) works via Ollama. moondream is an older, small model, which works in my system, it doesn’t know Hungarian, but it does reply with some vague idea of the image. (Repeat the test, it gets it somewhat right in 3-4 tries) Default response for test5 is empty by default, but you can see the reply in the stream. Shouldn’t it output the reply ?

Sending Images to Ollama:

• Url Image is not supported in Ollama, but base64 is, golem-llm doesnt support base64 image
• We first download the image, turns it into the Base64 text format, and then send that text to Ollama the way it needs.
• Since Anthropic API can also support Base64, golem-llm having support for Base64 would be nice

Ollama API

• Ollama has 2 API: its own API (/generate /chat) and an experimental one that acts like OpenAI’s (/v1).
• We use the /v1 one (like OpenAI) for chat, vision (/v1/chat/completions), as streaming in default api uses ndjson and not sse