How to choose an open AI model: a practical decision guide

Choose an open model by task, hardware, license and capabilities. A step-by-step decision guide with shortlists for chat, coding, documents, images, video, speech, music, translation and search.

  • Beginner
  • 5 min read
  • Published
  • Updated

Short answer. Decide in this order: what you need the model to do, what hardware you have, which license you can accept, and which capabilities matter. Then test two or three candidates on your own examples. Do not rank models by a single benchmark score, and do not assume that bigger is better.

Step 1: what do you need it to do?

Task Model type Start here
Chat, writing, reasoning, summaries Language model Best open-source LLMs
Coding help, coding agents, tool use Language model with agent skills Coding and agents and the coding agents guide
Understanding photos, screenshots and charts Vision-language model Multimodal models, such as Gemma 4 and MiniCPM-V 4.7
Reading scans, PDFs and forms OCR or document model OCR and document models and the OCR guide
Translating text or text in images Translation model Translation models and the translation guide
Making or editing images and video Image or video model Image and video
Transcribing speech Speech-to-text model Speech recognition and the speech-to-text guide
Generating speech Text-to-speech model Text-to-speech and the text-to-speech guide
Making songs, music or sound effects Music or sound model Music and sound generation and the music guide
Searching documents by meaning Embedding model, plus a language model for answers Embedding models and the embedding guide

Step 2: what hardware do you have?

Memory is a hard limit: a model that does not fit will not run well, or at all. Work out what you have, then look only at models that fit.

Step 3: which license can you accept?

  • Personal use and learning: almost any open license is fine.
  • A product or a business: prefer permissive licenses such as Apache 2.0, MIT and OpenMDW-1.1, because they leave you few conditions to track.
  • Conditional licenses: check revenue limits, credit lines and rules for hosting businesses against your own situation.
  • Non-commercial licenses: fine for research and evaluation, but you need a separate license for products.

AI licenses explained covers the details, and models you can use commercially sorts the catalog by license type.

Step 4: which capabilities matter?

  • Input types. If you need audio input, look at the smaller Gemma 4 sizes, Muse Spark Open, MiMo-V2.6 and Inkling. Many models accept images, and some also accept video.
  • Context length. Many of the large 2026 releases offer a 1-million-token window, while smaller ones offer 128K to 256K tokens. Check that your documents fit, and remember that long contexts need extra memory.
  • Languages. Use the languages the developer lists. For example, Mistral Large 4 lists more than 160, Gemma 4 more than 35, and Qwen3-Embedding more than 100.
  • Tool calling and agents. If you are building an agent, look for explicit function-calling support, as in gpt-oss and Granite 4.2.
  • Reasoning mode. Many models can think step by step before answering. Some, such as gpt-oss, let you choose how much effort to spend.
  • Speed. Mixture-of-experts models activate only some of their parameters for each token, so they can generate faster than a dense model of the same memory size.

Step 5: test your shortlist

Pick two or three candidates and run the same set of your own examples through each. Compare quality, speed and memory use. How to evaluate models yourself explains how to build a small test set and score it fairly.

Four example decisions

A private assistant on a 16 GB laptop. Look at Gemma 4, which has sizes for phones up to a 31B model, and at the 21B version of gpt-oss, which OpenAI says fits in 16 GB. Both are Apache 2.0, so there are no license conditions to track.

A coding assistant for a small team on one 80 GB GPU. The 117B version of gpt-oss is about 65 GB and runs on a single 80 GB GPU. A dense model such as Qwen3.8-27B leaves far more headroom for long contexts and several users. Serve either with a server engine such as vLLM.

Interview transcripts with speaker labels. Whisper large-v3-turbo is the light option at about 6 GB, but it does not label speakers. VibeVoice-ASR does, takes up to an hour of audio in one pass and is MIT-licensed, but it has 8.7B parameters and needs about 20 GB by our estimate. Nemotron 3.5 Transcribe also labels speakers, but only in 13 of its 40 languages, and it needs a large GPU.

A startup that wants an image feature. Start with the license. FLUX 3 Image Klein is free for commercial use below US$5 million of annual revenue and comes with output-filtering duties, Qwen-Image-2.1 is research-only, and MiniMax H3-Image is not licensed in the US, the EU, the UK or South Korea (a territory restriction). Z-Image and MOSS-Image are Apache 2.0, so they leave the fewest conditions to track, but they only generate images from text and cannot edit them. FLUX.3 Image Klein vs Z-Image compares the two.

Common mistakes

  • Choosing by one leaderboard number. Benchmark scores in announcements come from the developers and may not reflect your task.
  • Choosing the biggest model you can run. It will be slower and costlier, and a smaller one may do the job.
  • Reading the license last. Check it before you build, not after.
  • Forgetting the context cache. A model that fits with a short prompt can run out of memory with a long one.
  • Testing only easy prompts. Include the hard and unusual cases from your real work.

Frequently asked questions

What is the best open-source LLM?
There is no single best one. The right choice depends on your hardware, your task and the license you can accept. Start from a shortlist for your situation, such as small models for laptops or models for coding and agents, and test two or three candidates on your own examples.
Should I pick the biggest model I can run?
Not necessarily. Larger models are slower and more expensive to run. Start with the smallest model that does the job and move up only if it falls short on your examples.
Do newer models always beat older ones?
Often, but not always for your task. Older models can be better supported by tools, and newer ones sometimes need recent software versions. Test them on your own work.
Can I switch models later?
Yes, but plan for it. Prompts may need retuning, licenses differ between models, and embedding models are not interchangeable, so switching one means re-indexing your documents.

Key terms in this guide

  • Benchmark A benchmark is a standardized test used to compare models, such as a set of maths, coding or knowledge questions; scores are useful but easy to over-read.
  • Embedding model An embedding model turns text, images or other data into a list of numbers that captures its meaning, so that similar items end up close together.
  • Open weights An open-weights model is an AI model whose trained parameters, called weights, are published so that anyone can download and run it, usually under a license that sets the terms of use.
  • Permissive license A permissive license, such as Apache 2.0 or MIT, lets you use, modify and sell a model with very few conditions, usually just keeping a notice.
  • Territory restriction A territory restriction limits where a license is valid: using the model in the excluded countries or regions is not licensed.
  • VRAM VRAM is the memory on a graphics card; it decides which models a GPU can hold, because the weights must fit in memory for the model to run quickly.

Models mentioned in this guide

Model Released Parameters Context License Smallest variant needs
Gemma 4 Google DeepMind 5.1B – 30.7B 256K Apache 2.0 (Permissive) 3.52 GB Laptop or phone
gpt-oss OpenAI 21B – 117B 5.1B active 128K Apache 2.0 (Permissive) 15 GB Laptop or phone
Qwen3.8-27B Alibaba Qwen 27.8B 256K Apache 2.0 (Permissive) 19 GB Gaming GPU
Whisper large-v3-turbo OpenAI 810M – 1.54B – MIT (Permissive) 6 GB Laptop or phone
VibeVoice-ASR Microsoft 2.81B – 8.67B 64K MIT (Permissive) 6.47 GB Laptop or phone
Nemotron 3.5 Transcribe NVIDIA 34B 4.3B active 64K OpenMDW-1.1 (Permissive) 25 GB Gaming GPU
FLUX 3 Image Klein Black Forest Labs 5.6B – FLUX Kommunity (Conditional) 8 GB Laptop or phone
Z-Image Tongyi MAI 6.15B – Apache 2.0 (Permissive) 16 GB Laptop or phone
MOSS-Image OpenMOSS 5.08B – Apache 2.0 (Permissive) 4.2 GB Laptop or phone
MiniMax H3-Image MiniMax 33.1B – MiniMax H3-Image Community (Conditional) 155 GB Big-memory desktop