Skip to content

Unexpected warnings presented for default masking behavior #2141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jasonbcox opened this issue Mar 14, 2025 · 0 comments
Open

Unexpected warnings presented for default masking behavior #2141

jasonbcox opened this issue Mar 14, 2025 · 0 comments
Assignees
Labels
type:Bug Something isn't working

Comments

@jasonbcox
Copy link

jasonbcox commented Mar 14, 2025

Describe the bug
UserWarnings are presented when a mask is propagated via mask_zero=True to TransformerDecoder. Considering this is the default behavior, a warning is confusing and a nuisance.

To Reproduce

import keras_nlp
import numpy as np
seq = keras_nlp.layers.TokenAndPositionEmbedding(3,3,4,mask_zero=True)(np.ones((1,3)))

Observe:

env/keras/src/layers/layer.py:938: UserWarning: Layer 'position_embedding' (of type PositionEmbedding) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
keras_nlp.layers.TransformerDecoder(8,1)(decoder_sequence=seq)

Observe:

env/keras/src/layers/layer.py:938: UserWarning: Layer 'query' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
env/keras/src/layers/layer.py:938: UserWarning: Layer 'key' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.
env/keras/src/layers/layer.py:938: UserWarning: Layer 'value' (of type EinsumDense) was passed an input with a mask attached to it. However, this layer does not support masking and will therefore destroy the mask information. Downstream layers will not see the mask.

Expected behavior
These warnings should not appear when letting a mask propagate by default. I would only expect to see warnings if I'm doing something unusual or something I should probably not be doing.

Additional context
Others have also encountered this issue:
https://stackoverflow.com/q/79053957
https://stackoverflow.com/q/78868403

@dhantule dhantule added the type:Bug Something isn't working label Mar 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants