Release fine-tuned MLLMs on Hugging Face

[NielsRogge]

Hello,

Niels here from the open-source team at Hugging Face. I discovered your work on Arxiv and was wondering whether you would like to submit it to hf.co/papers to improve its discoverability. If you are one of the authors, you can submit it at https://huggingface.co/papers/submit.

The paper page lets people discuss about your paper and lets them find artifacts about it (your models, datasets or demo for instance), you can also claim
the paper as yours which will show up on your public profile at HF, add Github and project page URLs.

It's great to see that you've already hosted the VisNLModel dataset on the 🤗 hub! This is fantastic for visibility and discoverability.

It'd be awesome if you'd also consider making the fine-tuned MiniCPM-V and Qwen-VL checkpoints, which leverage your new dataset, available on https://huggingface.co/models. Hosting them will further improve their discoverability and visibility within the community. We can add tags so that people find them when filtering https://huggingface.co/models.

Uploading models

See here for a guide: https://huggingface.co/docs/hub/models-uploading.

In this case, we could leverage the PyTorchModelHubMixin class which adds from_pretrained and push_to_hub to any custom nn.Module. Alternatively, one can leverages the hf_hub_download one-liner to download a checkpoint from the hub.

We encourage researchers to push each model checkpoint to a separate model repository, so that things like download stats also work. We can then also link the checkpoints to the paper page.

You can also build a demo for your models on Spaces, we can provide you a ZeroGPU grant, which gives you A100 GPUs for free.

Let me know if you're interested/need any help regarding this!

Cheers,

Niels
ML Engineer @ HF 🤗