Multimodal describes an AI model that works across more than one kind of data — text, images, audio, video — rather than text alone. The term covers both directions: understanding non-text input (reading a screenshot, hearing a voice memo) and generating non-text output (producing an image or a video clip). Under the hood, everything is converted into tokens or embeddings the same network can process — an image becomes a grid of patch tokens much like a sentence becomes word tokens.
”Natively multimodal” vs. bolted together
The qualifier worth noticing is natively multimodal: one network trained on mixed media from the start, versus a text LLM with separate encoders or a diffusion generator attached after the fact. Native training lets capabilities transfer across modalities — a model that sees a chart can reason about it in the same forward pass, rather than reading a lossy caption from a separate vision module. GPT-5.5 is natively omnimodal across text, image, audio and video; MiniMax M3 was trained multimodally from step zero and is open-weight; Kimi K2.7 Code reads images and video natively for coding-from-screenshot workflows.
Why it matters in practice
Multimodality decides which workflows a model can even attempt: debugging from a screenshot, extracting tables from a scanned PDF, voice agents that hear tone, video QA. It also matters for agents — a computer-use agent must see the screen. Check the input/output row on any model page before assuming; “supports images” ranges from genuine native understanding to a thin OCR wrapper, and pricing for image or audio tokens is often separate from text.
The output side
Generation-first multimodal models are their own category — see the image, video and audio directories. The boundary is blurring from both ends: LLMs gained eyes and ears, generators gained language understanding, and hybrid systems like GPT Image 2 sit deliberately in the middle.