Pascal support #223
-
|
The description mentions flash attention which is not supported on Pascal. Does anyone know if flash attention is optional such that it is still possible to run on Pascal class GPUs e.g. P40, P100 or whethet it is required so that these older cards are not supported? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hey @cduk , it used torch s functional F.sdpa. This falls to FA2 or memory efficent attention, or plain MHA depending on your hardware (cpu, cuda compute capability / cache). Pascal therefore is supported. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the quick reply! TEI says older compute not supported, so maybe this is a selling point for Infinity! |
Beta Was this translation helpful? Give feedback.
Hey @cduk ,
it used torch s functional F.sdpa. This falls to FA2 or memory efficent attention, or plain MHA depending on your hardware (cpu, cuda compute capability / cache). Pascal therefore is supported.