Hi, I'm Jambo a Microsoft Learn Student Ambassador. This article aims to run the quantized Phi-3-vision model in ONNX format on the Jetson platform and successfully perform inference for image+te...
Updated Jul 29, 2024
Version 3.0
Blog Post
RBrown955 I have not yet posted this process on the NVIDIA forums.
For other models, you can still try running GGUF models using LlamaEdge (vision models are still not supported for conversion). It has better support on Jetson compared to Ollama (but Jetson currently supports CUDA 12).
I also greatly appreciate your efforts in trying out my process, as it helps me significantly in refining the entire workflow.
