DCA comment

This commit is contained in:
Eren Gölge 2021-04-04 03:55:37 +02:00
parent b86e7fb2e8
commit 44b4cb5ba5
1 changed files with 1 additions and 0 deletions

View File

@ -354,6 +354,7 @@ class MonotonicDynamicConvolutionAttention(nn.Module):
Dynamic convolution attention is an alternation of the location senstive attention with Dynamic convolution attention is an alternation of the location senstive attention with
dynamically computed convolution filters from the previous attention scores and a set of dynamically computed convolution filters from the previous attention scores and a set of
constraints to keep the attention alignment diagonal. constraints to keep the attention alignment diagonal.
DCA is sensitive to mixed precision training and might cause instable training.
Args: Args:
query_dim (int): number of channels in the query tensor. query_dim (int): number of channels in the query tensor.