BRAIN and NERVE－神経研究の進歩71巻1号

2019年01月発行

特集 人工知能の医療応用Update

深層学習と自然言語処理

著者： 鶴岡慶雅¹

所属機関： ¹東京大学大学院情報理工学系研究科電子情報学専攻

ページ範囲：P.45 - P.55

文献概要

近年，自然言語処理分野は深層学習技術の導入により急速に進展している。構文解析，機械翻訳，文書要約といった様々な自然言語処理タスクが，リカレントネットワークなどの汎用ニューラルネットワークモデルとアテンション機構の比較的簡単な組合せで実現できるようになっている。本稿では，深層学習の基礎とそれに基づく自然言語処理技術の最先端を簡単に紹介する。

参考文献

1）Mikolov T, Sutskever I, Chen K, Corrado G, Dean J: Distributed representations of words and phrases and their compositionality. NIPS 2013

2）Hochreiter S, Schmidhuber J: Long short-term memory. Neural Computation 9: 1735-1780, 1997

3）Kim Y: Convolutional neural networks for sentence classification. EMNLP: 1746-1751, 2014

4）Ma X, Hovy E: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. ACL 1: 1064-1074, 2016

5）Andor D, Alberti C, Weiss D, Severyn A, Presta A, et al: Globally normalized transition-based neural networks, ACL 2016

6）Lee K, He L, Lewis M, Zettlemoyer L: End-to-end neural coreference resolution. EMNLP 2017

7）Sutskever I, Vinyals O, Le QV: Sequence to sequence learning with neural networks. NIPS 2014［https://arxiv.org/abs/1409.3215］

8）Vinyals O, Toshev A, Bengio S, Erhan D: Show and tell: a neural image caption generator. CVPR 2015

9）Bahdanau D, Cho K, Bengio Y: Neural machine translation by jointly learning to align and translate. ICLR 2015

10）Luong MT, Pham H, Manning CD: Effective approaches to attention-based neural machine translation. EMNLP 2015

11）Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, et al: Google's neural machine translation system: bridging the gap between human and machine translation. arXiv: 1609.08144, 2016

12）Artetxe M, Labaka G, Agirre E, Cho Ke: Unsupervised neural machine translation. ICLR 2018

13）Lample G, Ott M, Conneau A, Denoyer L, Ranzato M: Phrase-based ＆ neural unsupervised machine translation. EMNLP 2018

14）Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, et al: Attention is all you need. NIPS 2017

15）See A, Liu PJ, Manning CD: Get to the point: summarization with pointer-generator networks. ACL 2017

16）Paulus R, Xiong C, Socher R: A deep reinforced model for abstractive summarization. ICLR 2018

17）Rajpurkar P, Zhang J, Lopyrev K, Liang P: SQuAD: 100,000+ questions for machine comprehension of text. EMNLP 2016

18）Rajpurkar P, Jia R, Liang P: Know what you don't know: unanswerable questions for SQuAD. ACL 2018

19）Seo M, Kembhavi A, Farhadi A, Hajishirzi H: Bidirectional attention flow for machine comprehension. ICLR 2017

20）Berant J, Chou A, Frostig R, Liang P: Semantic parsing on freebase from question-answer pairs. EMNLP: 1533-1544, 2013

21）Choi E, He H, Iyyer M, Yatskar M, Yih WT, et al: QuAC: question answering in context. EMNLP 2018

22）Ling W, Grefenstette E, Hermann KM, Kočiský T, Senior A, et al: Latent predictor networks for code generation. ACL 2016

23）Yin P, Neubig G: A syntactic neural model for general-purpose code generation. ACL 2017

24）Zhong V, Xiong C, Socher R: Seq2SQL: generating structured queries from natural language using reinforcement learning. arXiv: 1709.00103, 2017

25）Kiros R, Zhu Y, Salakhutdinov RR, Zemel R, Urtasun R, et al: Skip-thought vectors. NIPS 28, 2015

26）Logeswaran L, Lee H: An efficient framework for learning sentence representations. ICLR 2018

27）Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, et al: Deep contextualized word representations. NAACL 2018

28）Devlin J, Chang MW, Lee K, Toutanova K: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv: 1810.04805, 2018

29）Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, et al: Natural language processing (almost） from scratch. Machine Learning 2011

30）Hashimoto K, Xiong C, Tsuruoka Y, Socher R: A joint many-task model: growing a neural network for multiple NLP tasks, EMNLP 2017

31）McCann B, Keskar NS, Xiong C, Socher R: The natural language decathlon: multitask learning as question answering. arXiv: 1806.08730, 2018