Skip to content

Issues: Dao-AILab/flash-attention

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

523 Open 511 Closed
523 Open 511 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

CUDA versions > 12.3 do not correctly compile H100 Flash Attention 3
#1243 opened Sep 21, 2024 by rohany
1
Partial success with build from source for Windows 11, but the resulting wheel needed work
#1242 opened Sep 20, 2024 by jim-plus
【HDIM=96】head dim = 96 ?
#1238 opened Sep 19, 2024 by SunNy820828449
WHEN can we get the flash-attention 2.x for Turing GPU ?
#1235 opened Sep 17, 2024 by eileen2003-w
2
2.6.4 and FA3 release .whl for CUDA 12.4 torch2.4.1 python 3.11?
#1234 opened Sep 17, 2024 by tqangxl
fp8 not enabled for mha_varlen_fwd
#1232 opened Sep 16, 2024 by goldhuang
[BUG]2 tests failed...?
#1231 opened Sep 16, 2024 by ziyuhuang123
Turing architecture error on Nvidia Quadro T1000
#1230 opened Sep 15, 2024 by Tortoise17
2
ERROR [12/13] RUN pip install flash-attn --no-build-isolation
#1229 opened Sep 15, 2024 by promaprogga
1
Avoid padding computation with cu_seqlens
#1228 opened Sep 14, 2024 by imoneoi
3
ImportError: fused_dense is not installed
#1227 opened Sep 14, 2024 by kanebay
Softmax (particularly exp operations) becomes a major bottleneck in full FP16 pipeline
#1225 opened Sep 13, 2024 by phantaurus
4
[Question]Computation and register/shared memory wasted during decoding phase?
#1224 opened Sep 13, 2024 by sleepwalker2017
You are attempting to use Flash Attention 2.0 without specifying a torch dtype. This might lead to unexpected behaviour
#1223 opened Sep 12, 2024 by AraratSaribekyan
Which file is the source code of flash_attn_varlen_qkvpacked_func?
#1221 opened Sep 12, 2024 by scuizhibin
Unable to install flash attention in docker
#1220 opened Sep 11, 2024 by shivance
Additive Bias in Flash Attention
#1219 opened Sep 11, 2024 by kkh517
Support sliding window attention in FA3
#1218 opened Sep 10, 2024 by lin-ht
3
Is the combination of var-len, paged KV and split KV supported?
#1217 opened Sep 10, 2024 by masahi
export onnx issue
#1216 opened Sep 10, 2024 by scuizhibin
Can't compile from source on ROCm 6.1.3 wtih gfx1100... error : "static assertion failed" 2.6.3
#1215 opened Sep 9, 2024 by nktice
[FA3][Varlen] bug for head_dim not in [64, 128, 256] for varlen
#1214 opened Sep 9, 2024 by YLGH
[FP8][FA3] Is there a plan to support _flash_attn_varlen_forward with fp8
#1213 opened Sep 9, 2024 by baoleai
Question about FA3 supporting (256, 256)
#1212 opened Sep 9, 2024 by YTianZHU
Failed to build installable wheels for some pyproject.toml based projects (flash-attn)
#1211 opened Sep 9, 2024 by danielchang1985
ProTip! Add no:assignee to see everything that’s not assigned.