Tensor Issue with infer_request.set_tensor()

[Maxi-9]

Description:
I am encountering a General Error when calling infer_request.set_tensor() in my Rust code. Despite following the example code and using the OpenVINO Rust bindings, I am unable to pass the tensor to the inference request successfully. I am new to rust and I've checked the docs and haven't found anything useful for the problem. If there were examples of using YOLO and converting raw images to the correct formatting, that would be very helpful.

Expected Behavior:
The tensor should be set correctly without errors and inference should run successfully.

Actual Behavior:

infer_request.set_tensor() results in a General Error without further detail.

Environment:
• OS: macOS
• Rust Version: 1.83.0
• OpenVINO Version: openvino-2024.5.0
• Rust Crate Version: "0.8.0"
• Model: YOLO11s

Here is the code that produces the error:

use openvino::{Core, DeviceType, ElementType, Shape, Tensor};
use image::{GenericImageView, imageops::FilterType};
use std::path::Path;

fn prepare_image_for_inference(image_path: &str, target_size: u32) -> Result<Tensor, Box<dyn std::error::Error>> {
    let img = image::open(&Path::new(image_path))?;
    let resized_img = image::imageops::resize(&img, target_size, target_size, FilterType::Triangle);
    let mut bgr_image: Vec<f32> = Vec::with_capacity((target_size * target_size * 3) as usize);
    for pixel in resized_img.pixels() {
        let rgb = pixel.to_rgb();
        bgr_image.push(rgb[2] as f32 / 255.0); // Blue
        bgr_image.push(rgb[1] as f32 / 255.0); // Green
        bgr_image.push(rgb[0] as f32 / 255.0); // Red
    }
    let input_shape = Shape::new(&[1, 3, target_size as i64, target_size as i64])?;
    let mut tensor = Tensor::new(ElementType::F32, &input_shape)?;
    let buffer = tensor.get_data_mut()?;
    buffer.copy_from_slice(&bgr_image);
    Ok(tensor)
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut core = Core::new()?;
    let model = core.read_model_from_file("yolo11s_openvino_model/yolo11s.xml", "yolo11s_openvino_model/yolo11s.bin")?;
    let tensor = prepare_image_for_inference("images/64.jpg", 640)?; // I'm using a 480x640 image for testing

    let mut executable_model = core.compile_model(&model, DeviceType::CPU)?;
    let mut infer_request = executable_model.create_infer_request()?;
    
    infer_request.set_tensor(&*model.get_input_by_index(0)?.get_name()?, &tensor)?; // Error here
    infer_request.infer()?;

    Ok(())
}

The following error is encountered:

Error: General Error

What I Have Tried:
• Ensured that the tensor’s shape is [1, 3, 640, 640] (NCHW format).
• Checked the element type of the tensor and model input to ensure they match (f32).
• Confirmed that the input name passed to set_tensor() matches the model’s input name.

Additional Notes:
• The error occurs only when calling set_tensor(), and I am unable to retrieve any further details about the issue due to #154.
• I suspect it may be related to the model’s preprocessing expectations or a mismatch in input formats.
• I created my own pre-processing function to ensure that I got the pre-processed data in the right format for Yolo, but I also tried the prepostprocess as well.

[abrown]

Thanks for the report, but as you point to in #154, it's extremely hard to figure out what is going on without logging. One technique I've used in the past is to rewrite a minimal reproducer of a problem in C++ and run that directly against the OpenVINO library. The exception tends to have more information and if that is not enough, it can be easier to debug what is going wrong (might need a debuggable version of OpenVINO, e.g., by building from source). Sorry, that's not a great solution but that's where things are at with the C/Rust bindings to the OpenVINO library; as another plug as in #154, opening an upstream issue to enable logging or more verbose errors in the OpenVINO C bindings could be the push we need to solve this recurring problem.

[Maxi-9]

I checked the OpenVino repository and it seems like no issue was opened, but I may have missed it. I didn't convey this in my issue but I thought that openvino-rs could potentially try to validate the tensor if it finds an issue and let the user know, as it has all the info to do so. But it would be a lot more work for OpenVino. For now, I’ve moved on to OnnxRuntime (which is much slower). OpenVino was definitely one of the nicer libraries I've worked with in the ML rust space

[abrown]

I didn't convey this in my issue but I thought that openvino-rs could potentially try to validate the tensor if it finds an issue and let the user know, as it has all the info to do so.

Interesting, do you mind sketching out what you were thinking there? I assume at some point you were able to create the Yolo tensors in the right format or no?

[Maxi-9]

Thanks for following up. What I meant was that Rust could detect when the C API gives a general error and then manually check tensor sizing, data format, or other mismatches. The Rust bindings could potentially validate the tensor and provide more specific feedback. Since the C API already has the necessary model specifications (from what I can tell), OpenVINO-rs might be able to leverage that to help debug issues. I know this would involve extra work and might not be ideal, but it could be very helpful for cases like mine.

After trying for a while, I couldn’t get the Yolo tensors in the correct format. I usually don’t post issues because they’re usually user errors or misunderstandings, but since I was stuck, I wanted to try here and see if anyone had any ideas. I tried different tensor input names (“input”, “image”, and the Rust API-provided name), verified the sizes and data formats, and even double-checked the RGB vs. GBR format (although it doesn't matter at this stage). Everything seemed to match, but the “General Error” persisted, and I couldn’t identify the issue.

Let me know if you’d like me to expand on the validation idea further or help in any other way. I know this idea is a Band-Aid on the problem, but still seems helpful even if it wasn't implemented directly in the library but instead as an example that users can copy and paste.