my mugshot
ESSAY

SLM Image Recognition Test and Impressions

Are Small Language Models Ready for Prime Time?

Large Language Models get all the press and all the excitement but the real revolution occurs when we lower the barrier to entry and increase the number of participants leveraging Artificial Intelligence.  This will not come with the large, expensive, paid models but with the open source, free models.  These Small Language Models (SLMs) are the future.  Why?  Because we need many tries against many models, many sources of data, and many estimations to increase the reliability and utility.  You can't experiment that freely when you are paying for every token sent into an LLM and every token received from the LLM.

SLM Experiment #1

So, with that principle in mind, I decided to experiment with the ability of a vision language model (VLM) to examine the spines of books on my bookshelf and tell me what is there.  This would be a starter experiment for a larger vibe-coded application: A video inventory app.  

So, to do this experiment there are three prerequisites:

  1. Know what models I could run.
  2. Setup the local software to run SLMs locally.
  3. Setup the local software to query SLMs locally.  

Let's go through each step and then I will present the results of the experiment.

Assess Your Hardware

The key to what local models you can run is the amount of memory you have on your GPU (8GB) and CPU (32 GB).  Both will be pushed to their limits running local models.  My anecdotal evidence so far is that I can run 4 Billion (4B) parameter models efficiently and I can run 8 Billion (8B) parameter models very slowly.  Yes, I need more experimentation and in future articles I will clarify this in greater detail as I have only just barely scratched the surface of this.  For example, the # of parameters is not the only factor (though it is the most important one).  Another key factor in the models you can run is the quantization of the models.  The quantization means that the 32 or 64 bit floating point numbers that make up the weights and biases of the model are mapped to a smaller number of bits even as low as 4 bit quantization.  While it is evident that such a drastic mapping of values works, I do not yet know how that affects the quality.  Intuitively, it probably effects it drastically; however, I do not have any quantifiable data on that yet.  

Software Setup to Run Local models

There are two front-runner choices relating to running models locally: ollama and LM studio.  I had used ollama before and so I went with that.  In a later article, I will do a more detailed comparison of these two choices (and maybe a few others).  Here is what ollama looks like after you install it:



You can see the drop down list of models to choose from at the bottom right corner of the main prompt text field.  In the Figure above the local model to run is qwen3-vl:4b.  There are hundreds of models to choose from and choosing the right model for your application is key.  More on this later and that is an area in desperate need of both robust metadata on each model and a search tool to match a user (or agent's) needs to the available models.  In general, language model metadata is an extremely weak area that needs to be improved and standardized.  I plan to do more work in that area soon.

Software Setup to Query Local Models

While ollama has a rudimentary user interface (which is a fairly new addition to that project ... it is impressive how fast that team is adding features), I wanted to experiment with some other front-end applications for running local models.  I examined three applications/SLM interfaces:

  • Open Web UI - this interface (as shown below) offers good integration with a number of models and a large number of agent features.  It requires docker desktop to run so you have two installs.  In testing its Retrieval Augmented Generation (RAG) features it did not perform as well as Anything LLM (see below).  The test for that was to have it index my philosophy book (Philosophy of Intentional Living) and list the five stability layers.  
  • Jan - this is a simple user interface with some builtin models but it lacks any agent, RAG or VLM integration.  So, while it was interesting to test, it was not suitable for my needs.
  • Anything LLM - this tool has both a nice interface, robust agent capabilities and an intuitive RAG interface.  Besides its easy setup and easy integration with ollama, this tool performed better even with the Gemma 4B model in its RAG queries (as seen below).

The Vision Language Model Experiment

So, now that we have a setup capable of running local Small Language Models we need to test some models against the Large Language models and see how they perform.  The test was to analyze the image below of one section of my bookshelf.

As you can see most of the spines are easy to read but a few of them are obscured in shadow or difficult to read.  As an interesting aside, none of the small models were able to read the spine of the Bruce Lee book.  They all thought the spine said "Tao of Jeff Kune Do".  LOL!  So, the test is simple, given this photo, list out all the titles of the books in the image.  Here is the prompt that was used for every model:

"Analyze this image. OCR any text you see. Create a list of items in the image." 

So, here are the results of the test (scores are between 0.0 and 1.0 with 1.0 being 100% accuracy).

 Model Score 
 gemma3-4b  0.27  (few seconds to run)
 Claude Sonnet 4.5  0.33  (LLM at claude.ai)
 Qwen3-vl:8b  0.66  ** Took 25 minutes to run
 OpenAI GPT5 0.83  (LLM at chatgpt.com) 
 llava-llama3  0.22

If you want the full details of the test, see the bottom of this article.

Conclusion (for now)

So looking at the results (and these are only preliminary as alot more testing be done), four things stand out.

  1. The 4B SLM models performed poorly (gemma, llava).
  2. Suprisingly Claude also performed poorly.  This definitely warrants further investigation.
  3. OpenAI performed the best on this task.
  4. While the qwen model took the longest to run (on my hardware), it gave promising results.  It almost tempts me enough to quickly buy a better GPU!  

An interesting question would be to test further with a more detailed prompt.  Prompt engineering coud improve the results by itself.  Athough the fact we need to do experimentation on the prompt, proves the point that paid models are costly to be doing large number of experiments with.  

In the end, this was a worthwhile experiment that showed SLMs are promising but not there yet for typical low-cost personal computers. Fortunately, I think this situation will change quickly because the improvements with SLMs are still happening at an exponential pace.  So, I am confident that SLMs will continue to grow as an important part of the AI ecosystem!  

The next steps for my SLM experimentation is to integrate them into my XLM (cross language model) framework on github.  

Test Details

Here are the exact results for each model:

----------------------------------------

Model: gemma3-4b  

Results: 0.27


Response:

Okay, let's analyze the image and create a list of the books visible on the shelf. I’ve used OCR (Optical Character Recognition) to extract the text from the book covers.


**List of Books (as identified from the image):**


1.  *Man’s Search for Meaning* by Viktor Frankl, M.D.

2.  *You Can Be a Stock Market Genius* by Joel Greenblatt, M.D.

3.  *The Psychology of Money* by Morgan Housel

4.  *Atomic Habits* by James Clear

5.  *The Innovator’s Dilemma* by Clayton M. Christensen

6.  *The Art of Thinking Clearly* by Rolf Dobelli

7.  *How to Write a Book: A Proven System* by K.M. Weiland

8.  *The Writers Journey: Mythic Structure For Writers* by Christopher Vogler

9. *Toyota Production System* by Shigeo Shingo

10. *The Art of Thinking Clearly* by Rolf Dobelli

11. *Tao Of Jest: The Secret of Happiness* by  Paul Hoffman

12. *The Purpose Driven Life* by Rick Warren

13. *A Different Kind of Wise* by  Nathaniel Branden

14. *The Purpose Driven Life* by Rick Warren

15. *A Field Guide to the Senses* by Richard Etches


**Notes & Observations:**


*   There's a duplication of *The Purpose Driven Life* by Rick Warren.

*   The books are organized somewhat haphazardly on the shelf.


Let me know if you would like me to do anything else with this image, such as categorizing the books by genre, etc.!

------------------------------------------

Model: Claude Sonnet 4.5

Score: 0.33 


Response: 

Looking at this bookshelf, I can see a collection of books. Here's what I can identify: 

OCR Text (readable book spines from left to right):

Middle Shelf:


1. "MAN'S SEARCH FOR MEANING" - Viktor E. Frankl

2. "SHERLOCK HOLMES[?]" (partially visible)

3. "CLARK'S SPACE" - Arthur C. Clarke

4. "YOU CAN BE A STOCK MARKET GENIUS" - Joel Greenblatt

6. "the challenges of life"

7. A book by "C.S. Lewis"

9. "Timeless"

10. "THE ROADLESS TRAVELED"

11. "A WORLD WAITING TO BE BORN" - M. Scott Peck, M.D.

12. "THE ROAD LESS TRAVELED BEYOND" - M. Scott Peck

13. "THE FINISHING SCHOOL" (appears to be Gail Godwin)

14. "THE MASTER'S MUSE" - Gail Godwin/Miller

15. A dark spine book (text unclear)

16. "TOUGH-MINDED OPTIMIST" - Norman Vincent Peale

17. "THE UNKNOWN MATISSE" - Hilary Spurling

18. "THE PURPOSE DRIVEN LIFE" - Rick Warren

19. "MAHATMA GANDHI"

20. "THE MONK WHO SOLD HIS FERRARI" (appears to be Robin S. Sharma)

----------------------------------------------------------------------

Model:  Qwen3-vl:8b

score: 0.66


Response:

The provided context does not contain information about the bookshelf image or its contents. However, based on the visual description of the image, here is a list of the visible book titles:


Mans Search for Meaning

The Psychology of Self-Steem

Mark Spence: You Can Be a Stock Market Genius

The Challenges of Life

Further Along the Roadless Traveled

A World Waiting to Be Born

The Roadless Traveled Beyond

The Writers Journey

How to Write Best-Selling Fiction

Toyota Production System

A Life of Henri Matisse

The Unknown Matisse

The Purpose Driven Life

Marriane Williamson: A Blessing to Love

Tao of Jeff Kune Do

---------------------------------------------------------------

Model: OpenAI GPT5

Score: 0.83


Response:

The OCR result was partial and noisy, but by visually analyzing the image, here’s a clear list of identifiable books and items on the shelf:


Books (Left to Right)


1. Man’s Search for Meaning — Viktor E. Frankl

2. Who Moved My Cheese? — Spencer Johnson, M.D.

3. You Can Be a Stock Market Genius — Joel Greenblatt

4. An Untimely Frost — Mark Spencer

5. The Challenges of Life — C.S. Lewis

6. The Road Less Traveled — M. Scott Peck, M.D.

7. The Road Less Traveled and Beyond — M. Scott Peck, M.D.

8. A World Waiting to Be Born — M. Scott Peck, M.D.

9. The Writer’s Journey — Christopher Vogler

10. How to Write Best-Selling Fiction — Dean Koontz

11. Toyota Production System — Taiichi Ohno

12. The Unknown Matisse: A Life of Henri Matisse – The Early Years, 1869–1908 — Hilary Spurling

14. The Purpose Driven Life — Rick Warren

15. A Return to Love — Marianne Williamson

16. Tao of Jeet Kune Do — Bruce Lee

-------------------------------------------------------


Model: qwen3-vl:8b on ollama

Score: 


Response: 

Here is the list of items (books) visible in the image, based on OCR of the book spines:


MANS SEARCH FOR MEANING

THE PSYCHOLOGY OF SELF-STEERING

AN UNTIMELY FROST

YOU CAN BE A STOCK MARKET GENIUS

the challenges of life

Further Along the Roadless Traveled

A WORLD WAITING TO BE BORN M.SCOTT PECK M.D.

THE ROADLESS TRAVELED BEYOND

EASTING FOR SPIRITUAL BREAKTHROUGH

THE WRITERS JOURNEY

How to Write Best-Selling Fiction

TOTOA PRODUCTION SYSTEM

A LIFE OF HENRI MATISSE: THE EARLY YEARS, 1869-1908

THE PURPOSE DRIVEN

MARIAHNE WILLIAMSON A BRIEFING TO LOVE

TAO OF JEFF KUNE DO

Note: "TOTOA PRODUCTION SYSTEM" appears to be a likely OCR/typographical variant of "TOYOTA PRODUCTION SYSTEM"


----------------------------------------

Model: llava-llama3

Score: 0.22


Response:

Here is the list of books visible in the image, based on their titles and authors:


1. The Roadless Traveled by Mariah Wilson

2. The Purpose Driven Life by Rick Warren

3. The Art Of Possibilities by Rosamund Stone Zander

4. The New Science of Procrastination by Daniel Goleman and Mark Twain

5. The Power of Now by Eckhart Tolle

6. An Unabridged Woman by Monica Wood

7. A Life Of Henri Matisse: The Early Years, 1869-1908 by Louis Ettmann

8. In Search Of Our Selves: An Aesthetic Approach to the Human Spirit by Erich Fromm

9. The Hero's Journey by Joseph Campbell

10. The World As Home by Michael Lecker

11. Mans Search For Meaning by Viktor E. Frankl

12. Rebel at Work: 12 Steps To Transform Your Career, Improve Your Life by Alastair Scott

13. The Art Of Living by Leonid Hurvich

14. The World As Family by Kuan Tzu

15. The New World Order: The Unabridged Woman's Guide to the Politics of 16. 21st Century Feminism and Capitalism by Monica Wood

*Note: The titles and authors listed are as visible in the image, with OCR transcription where applicable.

-----------------------------------------------