DS being unable to precisely view Playwright screenshots is the only thing I really miss from Sonnet. This is promising.
> Images are converted into tokens based on their dimensions, and these tokens are billed together with your text tokens.
> Before inference, every image is automatically resized:
> - Images with a total pixel count below roughly 384×384 are scaled up while preserving their aspect ratio.
> - Larger images are scaled down while preserving their aspect ratio so that the total pixel count after resizing is roughly that of an 800×800 image.
> As a result, there is an upper bound of 384 tokens per image: for example, a 2000×2000 image and a 5000×5000 image consume the same number of tokens after resizing. When a request contains multiple images, each image is counted independently under the same rule—there is no separate calculation for multi-image requests.
400 tokens per image results in 2,500 images per dollar, if I’m not mistaken.
Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.
For most use cases you can fix that in the harness. Just give the model a tool to request a crop of specific coordinates of any image it has in its context. Call the tool "zoom" and it should be intuitive for the model
Maybe there are some use cases where you need high detail everywhere at once, but for OCR of small text and the like a zoom ability should be sufficient
For really dumb models I've also had success automatically cropping it into a grid of N images with the max size, then processing each cell individually, then once all been processed, do one final call with resized image + all other context previously generated per cell. Basically a workaround to the image dimension restrictions without loosing fidelity. Works well with even dumb 7B models.
Can't remember if I stole this idea from some existing public harness though, can't remember. If someone knows of public harnesses that do this already, please share them :)
It might also be due to its experimental status. Wouldn't surprise me if the GA version allows for larger input. Either that or the eventual pro version.
I main V4 Pro at work now, and at home I route between Pro and Flash based on task. Switched to Opus 4.6 for some tasks at work because I needed image input - horrible. So nice to get image input with DS.
Edit: I see it has limited resolution. Luckily I just built a vision worker plugin for DSH that routes image input to Kimi K2.6 on Cloudflare.
I've heard that DeepSeek v4 Flash 0731 has frequently assumed that it has vision capabilities and then resorts to inventing text-based image analysis tools when it finds that it actually can't see. In that case, this is a great upgrade for the model.
Anecdotally, I had to tell 0731 to refrain from viewing screenshots since it kept breaking its sessions by trying to read images.
800x800 is 640,000 pixels, or 0.64 Megapixels. That is less than the resolution of computer screens from 1995, Super VGA which has around 0.79 MPs.
This is useful for a reasonable amount of use-cases, but I think the watershed rez will be around triple that, ~1080p, which is enough for almost anything, except small text and subtle details.
I typically provide small screenshots to llms so this seems fine for that usecase, providing an entire screens context seems cause confusion with a lot of llms.
Interesting. Wasn't Deepseek's founder saying that they had explicitly decided not to focus on multimodal models at all and were going text-only because they believed it was enough to achieve AGI?
Nevertheless, as a component, we will undoubtedly implement multimodal support — and we are already doing so. We plan to develop relevant models, ensuring that versions like V4 and subsequent iterations will natively support multimodal functionality.
Earlier, the following was said, which might match more what you had in mind.
Achieving excellence in AI training does not require a global model or even multimodal approaches—by narrowing the scope of AI training and eliminating multimodality, certain tasks may remain unachievable without compromising the algorithm's validity.
Multimodal approaches ultimately need to be implemented.
It is difficult to tell who said what, since the speaker ids are missing.
DS being unable to precisely view Playwright screenshots is the only thing I really miss from Sonnet. This is promising.
> Images are converted into tokens based on their dimensions, and these tokens are billed together with your text tokens.
> Before inference, every image is automatically resized:
> - Images with a total pixel count below roughly 384×384 are scaled up while preserving their aspect ratio.
> - Larger images are scaled down while preserving their aspect ratio so that the total pixel count after resizing is roughly that of an 800×800 image.
> As a result, there is an upper bound of 384 tokens per image: for example, a 2000×2000 image and a 5000×5000 image consume the same number of tokens after resizing. When a request contains multiple images, each image is counted independently under the same rule—there is no separate calculation for multi-image requests.
400 tokens per image results in 2,500 images per dollar, if I’m not mistaken.
edit: format.
Oof 800 by 800 kills a lot of use cases
Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.
For most use cases you can fix that in the harness. Just give the model a tool to request a crop of specific coordinates of any image it has in its context. Call the tool "zoom" and it should be intuitive for the model
Maybe there are some use cases where you need high detail everywhere at once, but for OCR of small text and the like a zoom ability should be sufficient
For really dumb models I've also had success automatically cropping it into a grid of N images with the max size, then processing each cell individually, then once all been processed, do one final call with resized image + all other context previously generated per cell. Basically a workaround to the image dimension restrictions without loosing fidelity. Works well with even dumb 7B models.
Can't remember if I stole this idea from some existing public harness though, can't remember. If someone knows of public harnesses that do this already, please share them :)
It might also be due to its experimental status. Wouldn't surprise me if the GA version allows for larger input. Either that or the eventual pro version.
flash vs fine details. Pick one.
Gemini "flash" models have an option for media resolution, including a high resolution option for screenshots.
Benchmarks got a little bump from this: https://xcancel.com/deepseek_ai/status/2087864585504305397?s...
> Larger images are scaled down while preserving their aspect ratio, so that the total pixel count after resizing is roughly that of an 800×800 image.
It's useful but for OCR and a lot of other applications it needs to be a bit higher (eg: putting in a full A4 / Letter sized page)
Can split and feed?
that's difficult as well, how do you k ow where to split?
There are models specifically for splitting an image into text regions, e.g. PP-DocLayoutV3 https://huggingface.co/PaddlePaddle/PP-DocLayoutV3
I am using a stripped-down minimal version of it which I uploaded here, since I am not a fan of huge dependency trees: https://github.com/99991/simple-pp-doclayoutv3
Another recent model for this task is Unlimited-OCR: https://github.com/baidu/Unlimited-OCR
Let the model do the splitting. A 800x800px image should be enough to make those decisions
Overlap the splits?
Presumably a small cheap model could do that part?
I main V4 Pro at work now, and at home I route between Pro and Flash based on task. Switched to Opus 4.6 for some tasks at work because I needed image input - horrible. So nice to get image input with DS.
Edit: I see it has limited resolution. Luckily I just built a vision worker plugin for DSH that routes image input to Kimi K2.6 on Cloudflare.
I've heard that DeepSeek v4 Flash 0731 has frequently assumed that it has vision capabilities and then resorts to inventing text-based image analysis tools when it finds that it actually can't see. In that case, this is a great upgrade for the model.
Anecdotally, I had to tell 0731 to refrain from viewing screenshots since it kept breaking its sessions by trying to read images.
It tried to recreate vision by analyzing pixels on 3 separate projects I had it working on.
800x800 is 640,000 pixels, or 0.64 Megapixels. That is less than the resolution of computer screens from 1995, Super VGA which has around 0.79 MPs.
This is useful for a reasonable amount of use-cases, but I think the watershed rez will be around triple that, ~1080p, which is enough for almost anything, except small text and subtle details.
You'd expect a tool-enabled model to leverage crop and zoom tools to inspect and validate what it thinks it's seeing, though.
I typically provide small screenshots to llms so this seems fine for that usecase, providing an entire screens context seems cause confusion with a lot of llms.
Interesting. Wasn't Deepseek's founder saying that they had explicitly decided not to focus on multimodal models at all and were going text-only because they believed it was enough to achieve AGI?
It was explicitly said that they are pursuing multimodal support. A quote from the meeting transcript: https://github.com/demo-zexuan/liang-wenfeng-investor-meetin...
Earlier, the following was said, which might match more what you had in mind. It is difficult to tell who said what, since the speaker ids are missing.I think you're thinking of Dario saying this about image generation.
Hello Ox Alpha?
Congratulations! DeepSeek has finally gained eyes — the dark days are about to be behind us.
will this be open weights?
I believe so. Openness has always been a consistent tradition of DeepSeek
This is something I would like to know as well.
But if not, does anybody know a recommended way to attach vision to deepseek flash (on a self-hosted infrastructure)?
Generally you just add a vision model as an MCP server like this: https://github.com/DavidEasden/opencode-vision