Skip to content

Add OpenCV DNN option for ONNX inference#5136

Merged
glenn-jocher merged 3 commits into
masterfrom
update/dnn
Oct 11, 2021
Merged

Add OpenCV DNN option for ONNX inference#5136
glenn-jocher merged 3 commits into
masterfrom
update/dnn

Conversation

@glenn-jocher

@glenn-jocher glenn-jocher commented Oct 11, 2021

Copy link
Copy Markdown
Member

Usage:

# Export
python export.py --weights yolov5s.pt --include onnx --simplify

# Inference
python detect.py --weights yolov5s.onnx  # ONNX Runtime inference
python detect.py --weights yolov5s.onnx -dnn  # OpenCV DNN inference

NOTE: Requires cv2>=4.5.4

OpenCV(4.5.3)

Need to use latest version i.e. 4.5.4 which was released just a few days ago. I have added a fix for this exact error in opencv opencv/opencv#20713 which will be present in the 4.5.4 version.

opencv-python is still on 4.5.3 so we need to wait till the latest version is released which contains the fix. Once it is released,
pip install -U opencv-python should solve the issue.

Meanwhile, the following onnx models should also work with OpenCV 4.5.3:

python export.py --weights yolov5s.pt --include onnx --opset 11
or
python export.py --weights yolov5s.pt --include onnx --simplify

Originally posted by @SamFC10 in #4811 (comment)

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Added OpenCV DNN support for ONNX models in YOLOv5 detection.

📊 Key Changes

  • Introduced a new parameter dnn to enable OpenCV's DNN module for ONNX model inference.
  • Modified weights parameter conversion to string to ensure compatibility.
  • Adjusted ONNX model loading to use OpenCV DNN when dnn flag is set.
  • Updated ONNX inference path to perform a forward pass with OpenCV DNN if chosen.
  • Added the dnn flag to the command-line argument parser.

🎯 Purpose & Impact

  • Purpose:
    • To provide an alternative inference backend using OpenCV's DNN module for users with ONNX models.
    • To improve compatibility and flexibility in model deployment scenarios.
  • Impact:
    • Users can now leverage the OpenCV DNN module for potentially faster inference on CPU or compatibility with OpenCV processing pipelines.
    • Offers additional deployment options that could be beneficial in various environments without relying solely on ONNX Runtime.

Usage:

```bash
python detect.py --weights yolov5s.onnx  # ONNX Runtime inference
python detect.py --weights yolov5s.onnx -dnn  # OpenCV DNN inference
```
@glenn-jocher glenn-jocher self-assigned this Oct 11, 2021
@glenn-jocher glenn-jocher added the enhancement New feature or request label Oct 11, 2021
@glenn-jocher glenn-jocher merged commit 0bf24cf into master Oct 11, 2021
@glenn-jocher glenn-jocher deleted the update/dnn branch October 11, 2021 19:39
@glenn-jocher glenn-jocher mentioned this pull request Oct 12, 2021
@msly

msly commented Oct 14, 2021

Copy link
Copy Markdown

python detect.py --weights yolov5s.onnx --dnn # OpenCV DNN inference

BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
* Add OpenCV DNN option for ONNX inference

Usage:

```bash
python detect.py --weights yolov5s.onnx  # ONNX Runtime inference
python detect.py --weights yolov5s.onnx -dnn  # OpenCV DNN inference
```

* DNN prediction to tensor

* Update detect.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Change broadcast Add/Mul to element-wise Add/Mul in Detect layer

2 participants