Support for microsoft/Florence-2-large-ft vision model

[AshD (AshD)]

microsoft/Florence-2-large-ft works really well and it is much smaller than Phi-3 vision.

Would it be possible to convert this model in Onnx format?

Thanks,
Ash

[Zain Shah (tarzain)]

+1! Would love to have this in ONNX

[BALACHANDAR S (balachandarsv)]

+1 It would be great to have this in ONNX.

[aciddelgado]

Thanks all for the suggestion; we'll add this model to our roadmap and we'll discuss the priority internally. Thanks!

[AshD (AshD)]

Xenova created an Onnx version of the Florence model
https://huggingface.co/onnx-community/Florence-2-base-ft/tree/main/onnx

[Nitin Kumar (nogifeet)]

Is it possible to get access to scripts/branch used to convert to onnx ?

I want to convert some of my fine-tuned checkpoints to onnx

[Youho99]

I believe you can create your own onnx now

https://github.com/microsoft/onnxruntime-genai/blob/main/examples/python/awq-quantized-model.md

[Nitin Kumar (nogifeet)]

Youho99 Looks like florence2 is still not supported using AWQ.

Traceback (most recent call last):
File "/content/awq-quantized-model.py", line 140, in
main()
File "/content/awq-quantized-model.py", line 117, in main
quantize_model(args)
File "/content/awq-quantized-model.py", line 53, in quantize_model
model = AutoAWQForCausalLM.from_pretrained(
File "/content/AutoAWQ/awq/models/auto.py", line 70, in from_pretrained
model_type = check_and_get_model_type(
File "/content/AutoAWQ/awq/models/auto.py", line 48, in check_and_get_model_type
raise TypeError(f"{config.model_type} isn't supported yet.")
TypeError: florence2 isn't supported yet.

[Timothy Carambat (timothycarambat)]

AshD (@AshD) Been looking for that, thank you!

[Alessandro T. (trincadev)]

Xenova created an Onnx version of the Florence model https://huggingface.co/onnx-community/Florence-2-base-ft/tree/main/onnx

Thank you. Any news for support on python? Xenova's onnx florence-2 model requires transformers.js

[Nat Kershaw (MSFT) (natke)]

Thank you for your interest in Florence-2 V. We are looking into it. Moving this issue into Discussions

[Youho99]

+1

[callmehanyu]

+1

[css (qingfengcss)]

Could the author write a document about how to convert florence model to onnx format?

[Youho99]

Add same support request for new phi-3.5-vision
Discussion here : #827

[快乐的我531 (happyme531)]

I made an onnxruntime inference script for Florence2:
https://huggingface.co/happyme531/Florence-2-base-ft-ONNX-RKNN2

[AshD (AshD)]

Do you have a C# version of this script?

[Armin Mehri (Rm1n90)]

Nat Kershaw (MSFT) (@natke) Any update regarding Florence 2?

[Ievgenii Larin (ievgeniilarin)]

Please provide an official onnx for the model 🙏

Using this in the meantime.