Skip to content

Latest commit

 

History

History
945 lines (806 loc) · 68.7 KB

README_en.md

File metadata and controls

945 lines (806 loc) · 68.7 KB

PaddleX

🔍 Introduction

PaddleX 3.0 is a low-code development tool for AI models built on the PaddlePaddle framework. It integrates numerous ready-to-use pre-trained models, enabling full-process development from model training to inference, supporting a variety of mainstream hardware both domestic and international, and aiding AI developers in industrial practice.

Image Classification Multi-label Image Classification Object Detection Instance Segmentation
Semantic Segmentation Image Anomaly Detection OCR Table Recognition
PP-ChatOCRv3-doc Time Series Forecasting Time Series Anomaly Detection Time Series Classification

🌟 Why PaddleX ?

🎨 Rich Models One-click Call: Integrate over 200 PaddlePaddle models covering multiple key areas such as OCR, object detection, and time series forecasting into 19 pipelines. Experience the model effects quickly through easy Python API calls. Also supports more than 20 modules for easy model combination use by developers.

🚀 High Efficiency and Low barrier of entry: Achieve model full-process development based on graphical interfaces and unified commands, creating 8 featured model pipelines that combine large and small models, semi-supervised learning of large models, and multi-model fusion, greatly reducing the cost of iterating models.

🌐 Flexible Deployment in Various Scenarios: Support various deployment methods such as high-performance inference, serving, and lite deployment to ensure efficient operation and rapid response of models in different application scenarios.

🔧 Efficient Support for Mainstream Hardware: Support seamless switching of various mainstream hardware such as NVIDIA GPUs, Kunlun XPU, Ascend NPU, and Cambricon MLU to ensure efficient operation.

📣 Recent Updates

🔥🔥 2025.2.14, PaddleX v3.0.0rc0 major upgrade. This version fully adapts to PaddlePaddle 3.0rc0, with the following core upgrades:

  • Added 12 high-value pipelines, launching self-developed Layout Parsing v2 Pipeline, PP-ChatOCRv4-doc Pipeline, Table Recognition v2 Pipeline. Additionally, new pipelines for document processing, rotated box detection, open vocabulary detection/segmentation, video analysis, multilingual speech recognition, 3D, and other scenarios have been added.

  • Expanded 48 cutting-edge models, including the major releases in the OCR field such as Document Layout Detection Model PP-DocLayout, Formula Recognition Model PP-FormulaNet, Table Structure Recognition Model SLANeXt, Text Recognition Model PP-OCRv4_server_rec_doc. In the CV field, models for 3D detection, human keypoints, open vocabulary detection/segmentation, and in the speech recognition field, models from the Whisper series, among others.

  • Optimized and upgraded the inference APIs for models and pipelines, supporting more parameter configurations to enhance the flexibility of model and pipeline inference. Details.

  • Expanded hardware support: added support for Suoyuan GCU (90+ models), and significantly increased the number of models for Ascend NPU/Kunlunxin XPU/Cambricon MLU/Hygon DCU.

  • Upgraded full-scenario deployment capabilities:

    • High-performance inference supports one-click installation, Windows systems, and 220+ models, with the core library ultra-infer open-sourced;
    • Serving deployment added a highly stable solution, supporting dynamic configuration optimization.
  • Enhanced system compatibility: adapted to Windows training/inference, fully supporting Python 3.11/3.12.

🔥🔥 11.15, 2024, PaddleX 3.0 Beta2 open source version is officially released, PaddleX 3.0 Beta2 is fully compatible with the PaddlePaddle 3.0b2 version. This update introduces new pipelines for general image recognition, face recognition, vehicle attribute recognition, and pedestrian attribute recognition. We have also developed 42 new models to fully support the Ascend 910B, with extensive documentation available on GitHub Pages.

🔥🔥 9.30, 2024, PaddleX 3.0 Beta1 open source version is officially released, providing more than 200 models that can be called with a simple Python API; achieve model full-process development based on unified commands, and open source the basic capabilities of the PP-ChatOCRv3 pipeline; support more than 100 models for high-performance inference and serving (iterating continuously), more than 7 key visual models for edge-deployment; more than 70 models have been adapted for the full development process of Ascend 910B, more than 15 models have been adapted for the full development process of Kunlun chips and Cambricon

🔥 6.27, 2024, PaddleX 3.0 Beta open source version is officially released, supporting the use of various mainstream hardware for pipeline and model development in a low-code manner on the local side.

🔥 3.25, 2024, PaddleX 3.0 cloud release, supporting the creation of pipelines in the AI Studio Galaxy Community in a zero-code manner.

🔠 Explanation of Pipeline

PaddleX is dedicated to achieving pipeline-level model training, inference, and deployment. A pipeline refers to a series of predefined development processes for specific AI tasks, which includes a combination of single models (single-function modules) capable of independently completing a certain type of task.

📊 What can PaddleX do?

All pipelines of PaddleX support online experience on AI Studio and local fast inference. You can quickly experience the effects of each pre-trained pipeline. If you are satisfied with the effects of the pre-trained pipeline, you can directly perform high-performance inference / serving / edge deployment on the pipeline. If not satisfied, you can also Custom Development to improve the pipeline effect. For the complete pipeline development process, please refer to the PaddleX pipeline Development Tool Local Use Tutorial.

In addition, PaddleX provides developers with a full-process efficient model training and deployment tool based on a cloud-based GUI. Developers do not need code development, just need to prepare a dataset that meets the pipeline requirements to quickly start model training. For details, please refer to the tutorial "Developing Industrial-level AI Models with Zero Barrier".

Pipeline Online Experience Local Inference High-Performance Inference Serving Edge Deployment Custom Development Zero-Code Development On AI Studio
OCR Link
PP-ChatOCRv3 Link 🚧
Table Recognition Link 🚧
Object Detection Link
Instance Segmentation Link 🚧
Image Classification Link
Semantic Segmentation Link
Time Series Forecasting Link 🚧
Time Series Anomaly Detection Link 🚧
Time Series Classification Link 🚧
Small Object Detection Link 🚧
Multi-label Image Classification Link 🚧
Pedestrian Attribute Recognition Link 🚧
Vehicle Attribute Recognition Link 🚧
Formula Recognition Link 🚧
Seal Recognition Link 🚧
Image Anomaly Detection 🚧 🚧 🚧
Human Keypoint Detection 🚧 🚧 🚧 🚧
Open Vocabulary Detection 🚧 🚧 🚧 🚧 🚧
Open Vocabulary Segmentation 🚧 🚧 🚧 🚧 🚧
Rotated Object Detection 🚧 🚧 🚧 🚧
3D Bev Detection 🚧 🚧 🚧 🚧
Table Recognition v2 🚧 🚧 🚧 🚧
Layout Parsing 🚧 🚧 🚧
Layout Parsing v2 🚧 🚧 🚧 🚧 🚧
Document Image Preprocessing 🚧 🚧 🚧 🚧
Image Recognition 🚧 🚧 🚧
Face Recognition 🚧 🚧 🚧
Multilingual Speech Recognition 🚧 🚧 🚧 🚧 🚧
Video Classification 🚧 🚧 🚧 🚧
Video Detection 🚧 🚧 🚧 🚧

❗Note: The above capabilities are implemented based on GPU/CPU. PaddleX can also perform local inference and custom development on mainstream hardware such as Kunlunxin, Ascend, Cambricon, and Haiguang. The table below details the support status of the pipelines. For specific supported model lists, please refer to the Model List (Kunlunxin XPU)/Model List (Ascend NPU)/Model List (Cambricon MLU)/Model List (Haiguang DCU). We are continuously adapting more models and promoting the implementation of high-performance and serving on mainstream hardware.

🔥🔥 Support for Domestic Hardware Capabilities

Pipeline Ascend 910B Kunlunxin R200/R300 Cambricon MLU370X8 Haiguang Z100
OCR 🚧
Table Recognition 🚧 🚧 🚧
Object Detection 🚧
Instance Segmentation 🚧 🚧
Image Classification
Semantic Segmentation
Time Series Forecasting 🚧
Time Series Anomaly Detection 🚧 🚧 🚧
Time Series Classification 🚧 🚧 🚧
Multi-label Image Classification 🚧 🚧
Pedestrian Attribute Recognition 🚧 🚧 🚧
Vehicle Attribute Recognition 🚧 🚧 🚧
Image Recognition 🚧
Seal Recognition 🚧 🚧 🚧
Image Anomaly Detection
Face Recognition

⏭️ Quick Start

🛠️ Installation

❗Before installing PaddleX, please ensure you have a basic Python runtime environment (Note: Currently supports running under Python 3.8 to Python 3.10, with more Python versions under adaptation). The PaddlePaddle version required by PaddleX

  • Installing PaddlePaddle
# CPU
python -m pip install paddlepaddle==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/

# gpu,requires GPU driver version ≥450.80.02 (Linux) or ≥452.39 (Windows)
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

# gpu,requires GPU driver version ≥545.23.06 (Linux) or ≥545.84 (Windows)
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/

❗No need to focus on the CUDA version on the physical machine, only the GPU driver version needs attention. For more information on PaddlePaddle Wheel versions, please refer to the PaddlePaddle Official Website.

  • Installing PaddleX
pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/whl/paddlex-3.0.0rc0-py3-none-any.whl

❗For more installation methods, refer to the PaddleX Installation Guide.

💻 CLI Usage

One command can quickly experience the pipeline effect, the unified CLI format is:

paddlex --pipeline [Pipeline Name] --input [Input Image] --device [Running Device]

Each Pipeline in PaddleX corresponds to specific parameters, which you can view in the respective Pipeline documentation for detailed explanations. Each Pipeline requires specifying three necessary parameters:

  • pipeline: The name of the Pipeline or the configuration file of the Pipeline
  • input: The local path, directory, or URL of the input file (e.g., an image) to be processed
  • device: The hardware device and its index to use (e.g., gpu:0 indicates using the 0th GPU), or you can choose to use NPU (npu:0), XPU (xpu:0), CPU (cpu), etc.

For example, using the OCR pipeline:

paddlex --pipeline OCR \
        --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png \
        --use_doc_orientation_classify False \
        --use_doc_unwarping False \
        --use_textline_orientation False \
        --save_path ./output \
        --device gpu:0
👉 Click to view the running result
{'res': {'input_path': 'general_ocr_002.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': False, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'model_settings': {'use_doc_orientation_classify': True, 'use_doc_unwarping': False}, 'angle': 0},'dt_polys': [array([[ 3, 10],
       [82, 10],
       [82, 33],
       [ 3, 33]], dtype=int16), ...], 'text_det_params': {'limit_side_len': 960, 'limit_type': 'max', 'thresh': 0.3, 'box_thresh': 0.6, 'unclip_ratio': 2.0}, 'text_type': 'general', 'textline_orientation_angles': [-1, ...], 'text_rec_score_thresh': 0.0, 'rec_texts': ['www.99*', ...], 'rec_scores': [0.8980069160461426,  ...], 'rec_polys': [array([[ 3, 10],
       [82, 10],
       [82, 33],
       [ 3, 33]], dtype=int16), ...], 'rec_boxes': array([[  3,  10,  82,  33], ...], dtype=int16)}}

The visualization result is as follows:

alt text

To use the command line for other pipelines, simply adjust the pipeline parameter to the name of the corresponding pipeline and modify the parameters accordingly. Below are the commands for each pipeline:

👉 More CLI usage for pipelines
Pipeline Name Command
General Image Classification paddlex --pipeline image_classification --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_image_classification_001.jpg --device gpu:0 --save_path ./output/
General Object Detection paddlex --pipeline object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_object_detection_002.png --threshold 0.5 --save_path ./output/ --device gpu:0
General Instance Segmentation paddlex --pipeline instance_segmentation --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_instance_segmentation_004.png --threshold 0.5 --save_path ./output --device gpu:0
General Semantic Segmentation paddlex --pipeline semantic_segmentation --input https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/application/semantic_segmentation/makassaridn-road_demo.png --target_size -1 --save_path ./output --device gpu:0
Image Multi-label Classification paddlex --pipeline image_multilabel_classification --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_image_classification_001.jpg --save_path ./output --device gpu:0
Small Object Detection paddlex --pipeline small_object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/small_object_detection.jpg --threshold 0.5 --save_path ./output --device gpu:0
Image Anomaly Detection paddlex --pipeline anomaly_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/uad_grid.png --save_path ./output --device gpu:0
Pedestrian Attribute Recognition paddlex --pipeline pedestrian_attribute_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/pedestrian_attribute_002.jpg --save_path ./output/ --device gpu:0
Vehicle Attribute Recognition paddlex --pipeline vehicle_attribute_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/vehicle_attribute_002.jpg --save_path ./output/ --device gpu:0
3D Multi-modal Fusion Detection paddlex --pipeline 3d_bev_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/det_3d/demo_det_3d/nuscenes_demo_infer.tar --device gpu:0 --save_path ./output/
Human Keypoint Detection paddlex --pipeline human_keypoint_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/keypoint_detection_001.jpg --det_threshold 0.5 --save_path ./output/ --device gpu:0
Open Vocabulary Detection paddlex --pipeline open_vocabulary_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/open_vocabulary_detection.jpg --prompt "bus . walking man . rearview mirror ." --thresholds "{'text_threshold': 0.25, 'box_threshold': 0.3}" --save_path ./output --device gpu:0
Open Vocabulary Segmentation paddlex --pipeline open_vocabulary_segmentation --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/open_vocabulary_segmentation.jpg --prompt_type box --prompt "[[112.9,118.4,513.8,382.1],[4.6,263.6,92.2,336.6],[592.4,260.9,607.2,294.2]]" --save_path ./output --device gpu:0
Rotated Object Detection paddlex --pipeline rotated_object_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/rotated_object_detection_001.png --threshold 0.5 --save_path ./output --device gpu:0
General OCR paddlex --pipeline OCR --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0
Document Image Preprocessor paddlex --pipeline doc_preprocessor --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/doc_test_rotated.jpg --use_doc_orientation_classify True --use_doc_unwarping True --save_path ./output --device gpu:0
General Table Recognition paddlex --pipeline table_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0
General Table Recognition v2 paddlex --pipeline table_recognition_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0
General Layout Parsing paddlex --pipeline layout_parsing --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/demo_paper.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0
General Layout Parsing v2 paddlex --pipeline layout_parsing_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout_parsing_v2_demo.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0
Formula Recognition paddlex --pipeline formula_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/general_formula_recognition.png --use_layout_detection True --use_doc_orientation_classify False --use_doc_unwarping False --layout_threshold 0.5 --layout_nms True --layout_unclip_ratio 1.0 --layout_merge_bboxes_mode large --save_path ./output --device gpu:0
Seal Text Recognition paddlex --pipeline seal_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png --use_doc_orientation_classify False --use_doc_unwarping False --device gpu:0 --save_path ./output
Time Series Forecasting paddlex --pipeline ts_forecast --input https://paddle-model-ecology.bj.bcebos.com/paddlex/ts/demo_ts/ts_fc.csv --device gpu:0 --save_path ./output
Time Series Anomaly Detection paddlex --pipeline ts_anomaly_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/ts/demo_ts/ts_ad.csv --device gpu:0 --save_path ./output
Time Series Classification paddlex --pipeline ts_classification --input https://paddle-model-ecology.bj.bcebos.com/paddlex/ts/demo_ts/ts_cls.csv --device gpu:0 --save_path ./output
Multilingual Speech Recognition paddlex --pipeline multilingual_speech_recognition --input https://paddlespeech.bj.bcebos.com/PaddleAudio/zh.wav --save_path ./output --device gpu:0
General Video Classification paddlex --pipeline video_classification --input https://paddle-model-ecology.bj.bcebos.com/paddlex/videos/demo_video/general_video_classification_001.mp4 --topk 5 --save_path ./output --device gpu:0
General Video Detection paddlex --pipeline video_detection --input https://paddle-model-ecology.bj.bcebos.com/paddlex/videos/demo_video/HorseRiding.avi --device gpu:0 --save_path ./output

📝 Python Script Usage

A few lines of code can complete the quick inference of the pipeline, the unified Python script format is as follows:

from paddlex import create_pipeline

pipeline = create_pipeline(pipeline=[Pipeline Name])
output = pipeline.predict([Input Image Name])
for res in output:
    res.print()
    res.save_to_img("./output/")
    res.save_to_json("./output/")

The following steps are executed:

  • create_pipeline() instantiates the pipeline object
  • Passes the image and calls the predict() method of the pipeline object for inference prediction
  • Processes the prediction results

To use the Python script for other pipelines, simply adjust the pipeline parameter in the create_pipeline() method to the name of the corresponding pipeline and modify the parameters accordingly. Below are the parameter names and detailed usage explanations for each pipeline:

👉 More Python script usage for pipelines
pipeline Name Corresponding Parameter Detailed Explanation
PP-ChatOCRv3-doc PP-ChatOCRv3-doc PP-ChatOCRv3-doc Pipeline Python Script Usage Instructions
Image Classification image_classification Image Classification Pipeline Python Script Usage Instructions
Object Detection object_detection Object Detection Pipeline Python Script Usage Instructions
Instance Segmentation instance_segmentation Instance Segmentation Pipeline Python Script Usage Instructions
Semantic Segmentation semantic_segmentation Semantic Segmentation Pipeline Python Script Usage Instructions
Image Multi-Label Classification multilabel_classification Image Multi-Label Classification Pipeline Python Script Usage Instructions
Small Object Detection small_object_detection Small Object Detection Pipeline Python Script Usage Instructions
Image Anomaly Detection image_classification Image Anomaly Detection Pipeline Python Script Usage Instructions
Image Recognition PP-ShiTuV2 Image Recognition Pipeline Python Script Usage Instructions
Face Recognition face_recognition Face Recognition Pipeline Python Script Usage Instructions
Pedestrian Attribute Recognition pedestrian_attribute Pedestrian Attribute Recognition Pipeline Python Script Usage Instructions
Vehicle Attribute Recognition vehicle_attribute Vehicle Attribute Recognition Pipeline Python Script Usage Instructions
3D Multi-modal Fusion Detection 3d_bev_detection Instructions for Using the 3D Multi-modal Fusion Detection Pipeline Python Script
Human Keypoint Detection human_keypoint_detection Instructions for Using the Human Keypoint Detection Pipeline Python Script
Open Vocabulary Detection open_vocabulary_detection Instructions for Using the Open Vocabulary Detection Pipeline Python Script
Open Vocabulary Segmentation open_vocabulary_segmentation Instructions for Using the Open Vocabulary Segmentation Pipeline Python Script
Rotated Object Detection rotated_object_detection Instructions for Using the Rotated Object Detection Pipeline Python Script
OCR OCR Instructions for Using the General OCR Pipeline Python Script
Document Image Preprocessing doc_preprocessor Instructions for Using the Document Image Preprocessing Pipeline Python Script
General Table Recognition table_recognition Instructions for Using the General Table Recognition Pipeline Python Script
General Table Recognition v2 table_recognition_v2 Instructions for Using the General Table Recognition v2 Pipeline Python Script
General Layout Parsing layout_parsing Instructions for Using the General Layout Parsing Pipeline Python Script
General Layout Parsing v2 layout_parsing_v2 Instructions for Using the General Layout Parsing v2 Pipeline Python Script
Formula Recognition formula_recognition Instructions for Using the Formula Recognition Pipeline Python Script
Seal Text Recognition seal_recognition Instructions for Using the Seal Text Recognition Pipeline Python Script
Time Series Forecasting ts_forecast Instructions for Using the Time Series Forecasting Pipeline Python Script
Time Series Anomaly Detection ts_anomaly_detection Instructions for Using the Time Series Anomaly Detection Pipeline Python Script
Time Series Classification ts_classification Instructions for Using the Time Series Classification Pipeline Python Script
Multilingual Speech Recognition multilingual_speech_recognition Instructions for Using the Multilingual Speech Recognition Pipeline Python Script
General Video Classification video_classification Instructions for Using the General Video Classification Pipeline Python Script
General Video Detection video_detection Instructions for Using the General Video Detection Pipeline Python Script

📖 Documentation

⬇️ Installation
🔥 Pipeline Usage
⚙️ Module Usage
🏗️ Pipeline Deployment
🖥️ Multi-Hardware Usage
📝 Tutorials & Examples

🤔 FAQ

For answers to some common questions about our project, please refer to the FAQ. If your question has not been answered, please feel free to raise it in Issues.

💬 Discussion

We warmly welcome and encourage community members to raise questions, share ideas, and feedback in the Discussions section. Whether you want to report a bug, discuss a feature request, seek help, or just want to keep up with the latest project news, this is a great platform.

📄 License

The release of this project is licensed under the Apache 2.0 license.