Skip to content

Commit

Permalink
OnnxStaticQuantization: Remove quant_preprocess=False default for q…
Browse files Browse the repository at this point in the history
…nn ep (#1565)

## Describe your changes
`quant_preprocess` was originally set to `False` by default because I
thought it might not be compatible with QNN EP. However, it is needed
for most models for the quantization to work properly:
- The quantizer needs a shape inferred model to quantize some tensors.
- Model quality is bad if constants are not made into initializers (see
#1552).

## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.

## (Optional) Issue link
  • Loading branch information
jambayk authored Jan 24, 2025
1 parent 92cd3ac commit fb0d228
Showing 1 changed file with 0 additions and 1 deletion.
1 change: 0 additions & 1 deletion olive/passes/onnx/quantization.py
Original file line number Diff line number Diff line change
Expand Up @@ -608,7 +608,6 @@ def _default_config(cls, accelerator_spec: AcceleratorSpec) -> Dict[str, PassCon
config["activation_type"].search_defaults = Categorical(["QInt8", "QUInt8", "QUInt16", "QInt16"])
config["weight_type"].search_defaults = Categorical(["QInt8", "QUInt8", "QUInt16", "QInt16"])
config["prepare_qnn_config"].default_value = True
config["quant_preprocess"].default_value = False
# in QNN EP, the default value WeightSymmetric is None
# but in base quantizer, the default value is True.
config["WeightSymmetric"].default_value = None
Expand Down

0 comments on commit fb0d228

Please sign in to comment.