Transformer Models for Processing Biological Signal
Date Issued
2023-07
Author(s)
Kuzmanov, Ivan
Madevska Bogadnova, Ana
Abstract
The transformer neural network architecture is a
deep learning model, that has been developed recently and as
such it’s potential is still being investigated. It is a powerful
model due to the their self-attention mechanism that finds use
in several domains, but our focus is on transformers used for
biological signals processing. Various hybrid model architectures
suitable for this type of task are considered in this study:
the basic transformer, temporal fusion transformer, time series
transformer, convolutional vision transformer and informer. A
brief description of the architecture is given. The reasons why
they are appropriate for processing biological signals, what makes
them unique, along with their strengths and weaknesses, are
discussed. Finally, a literature review is made involving actual
studies that use these model types for biosignal processing.
deep learning model, that has been developed recently and as
such it’s potential is still being investigated. It is a powerful
model due to the their self-attention mechanism that finds use
in several domains, but our focus is on transformers used for
biological signals processing. Various hybrid model architectures
suitable for this type of task are considered in this study:
the basic transformer, temporal fusion transformer, time series
transformer, convolutional vision transformer and informer. A
brief description of the architecture is given. The reasons why
they are appropriate for processing biological signals, what makes
them unique, along with their strengths and weaknesses, are
discussed. Finally, a literature review is made involving actual
studies that use these model types for biosignal processing.
Subjects
File(s)![Thumbnail Image]()
Loading...
Name
CIIT2023_paper_8.pdf
Size
8.97 MB
Format
Adobe PDF
Checksum
(MD5):3c68163718a84a2446d9f38edb515502