本地模型部署工具:OpenVINO™

openlab_96bf3613 更新于 9天前

OpenVINO™ 是一个深度学习模型的部署工具,可实现针对特定模型结构的优化与推理任务提升,目前 OpenVINO™ 已适配x86,ARM 在内的异构硬件终端,支持 CNN,Transformer 等多种网络结构,提供 C/C++/Python 等多类 API 接口语言,不管是传统的机器视觉 NLP 模型,或是最新的 LLM 多模态模型,都可以利用 OpenVINO™ 工具包中提供的丰富接口进行部署。

安装方式:
OpenVINO runtime:pip install openvino

OpenVINO with GenAI: pip install openvino-genai

更多安装方式可以参考:

https://docs.openvino.ai/2024/get-started/install-openvino.html

官方文档:

Documents:https://docs.openvino.ai/

Github:https://github.com/openvinotoolkit

Gitee:https://gitee.com/openvinotoolkit-prc

Notebooks示例:

https://openvinotoolkit.github.io/openvino_notebook****r/>
视频教程:

https://space.bilibili.com/38566875

文字版教程可关注 微信公众号

Python示例:
YOLO示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov11-optimization

https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/yolov10-optimization/yolov10-optimization.ipynb

SAM分割示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sam2-image-segmentation

文生图示例:

1)性能最优:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/latent-consistency-models-image-generation

https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/sdxl-turbo/sdxl-turbo.ipynb

2)效果最佳:

https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/flux.1-image-generation/flux.1-image-generation.ipynb

https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/stable-diffusion-v3/stable-diffusion-v3.ipynb

Whisper语音识别示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/distil-whisper-asR

Chatbot示例(支持多种模型):

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot

RAG示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-llamaindex

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-rag-langchain

多模态示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/qwen2-vl

https://github.com/openvinotoolkit/openvino_notebook***lob/latest/notebooks/qwen2-audio/qwen2-audio.ipynb

Agent示例:

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-agent-functioncall

https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-agent-react

更多实践示例可以参考:

https://github.com/openvinotoolkit/openvino_notebook***r/>
C/C++示例:

基础用法:

https://github.com/openvinotoolkit/openvino/tree/master/sample***r/>
GenAI示例:

https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/cpp

英特尔 AI PC

介绍及使用案例:

https://www.intel.com/content/www/us/en/products/docs/processors/core-ultra/ai-pc.html

哪吒开发套件

哪吒开发板及英特尔边缘开发套件使用案例**: https://inteldevkit.csdn.net/

0个评论