University of Illinois Urbana-Champaign

Adopting the two-branch network to video-text tasks

Chang, Hsiao-Ching

Content Files
CHANG-THESIS-2018.pdf

Permalink

https://hdl.handle.net/2142/101210

Description

Title
Adopting the two-branch network to video-text tasks
Author(s)
Chang, Hsiao-Ching
Issue Date
2018-04-23
Director of Research (if dissertation) or Advisor (if thesis)
Lazebnik, Svetlana
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2018-09-04T20:36:51Z
Keyword(s)
  • Computer vision
  • Video captioning
Abstract
Modeling visual context and its corresponding text description with a joint embedding network has been an effective way to enable cross-modal retrieval. However, while abundant work has been done for image-text tasks, not much exists with regards to the video domain. We hope to adopt a nonlinear embedding model, the two-branch network, to the video-text tasks in order to show its robustness. Two kinds of tasks are explored, bidirectional video-sentence retrieval and video description generation. For the retrieval task, we use nearest neighbor search to get the corresponding video or text with respect to the query. For video captioning, we incorporate the two-branch network in a traditional LSTM model with an additional embedding loss term in order to demonstrate its ability of preserving a semantic structure between video and text.
Graduation Semester
2018-05
Type of Resource
text
Permalink
http://hdl.handle.net/2142/101210
Copyright and License Information
Copyright 2018 Hsiao-Ching Chang

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science