Stories for images-in-sequence by using visual and narrative components
Date Issued
2018-09-17
Author(s)
Smilevski, Marko
Lalkovski, Ilija
Madjarov, Gjorgji
Abstract
Recent research in AI is focusing towards generating narrative stories about visual scenes. It has the potential to achieve more
human-like understanding than just basic description generation of imagesin-sequence. In this work, we propose a solution for generating stories for
images-in-sequence that is based on the Sequence to Sequence model. As
a novelty, our encoder model is composed of two separate encoders, one
that models the behaviour of the image sequence and other that models
the sentence-story generated for the previous image in the sequence of
images. By using the image sequence encoder we capture the temporal
dependencies between the image sequence and the sentence-story and by
using the previous sentence-story encoder we achieve a better story flow.
Our solution generates long human-like stories that not only describe the
visual context of the image sequence but also contains narrative and evaluative language. The obtained results were confirmed by manual human
evaluation.
human-like understanding than just basic description generation of imagesin-sequence. In this work, we propose a solution for generating stories for
images-in-sequence that is based on the Sequence to Sequence model. As
a novelty, our encoder model is composed of two separate encoders, one
that models the behaviour of the image sequence and other that models
the sentence-story generated for the previous image in the sequence of
images. By using the image sequence encoder we capture the temporal
dependencies between the image sequence and the sentence-story and by
using the previous sentence-story encoder we achieve a better story flow.
Our solution generates long human-like stories that not only describe the
visual context of the image sequence but also contains narrative and evaluative language. The obtained results were confirmed by manual human
evaluation.
Subjects
File(s)![Thumbnail Image]()
Loading...
Name
1805.05622.pdf
Size
2.69 MB
Format
Adobe PDF
Checksum
(MD5):7847d17bc7e09cf024aca7d10cea9d84