“NLP Status Report 2017-3-13”版本间的差异

来自cslt Wiki
跳转至： 导航、 搜索
（以“{| class="wikitable" !Date !! People !! Last Week !! This Week |- | rowspan="6"|2017/1/3 |Yang Feng || * ran experiments on the CS-EN data set (200k pairs) with tota...”为内容创建页面）
 
第4行： 第4行：
 
| rowspan="6"|2017/1/3
 
| rowspan="6"|2017/1/3
 
|Yang Feng ||
 
|Yang Feng ||
−
* ran experiments on the CS-EN data set (200k pairs) with totally identical initialization as the paper. on the sampled 2k training sentences, the bleu is 19.5 (not converged yet). (the bleu on the test set expected to be 26)
+
* tested and analyzed the results on the cs-en data set (30.4 on the heldout-training set and 7.3 on the dev set);
−
* add the alpha and gamma score and do multi-task training. Without multi-task training, the loss didn't decline on the training data, but with multi-task training, the loss did decline.
+
* added masks to the baseline (44.4 on the cn-en);
−
* prepared for Huilan's inspection.
+
* added masks to alpha-gamma method and fixed the bugs. Got an improvement of 0.5 again the masked baseline [[http://cslt.riit.tsinghua.edu.cn/mediawiki/images/b/b8/Nmt_mn_report_continue.pdf report]];
 +
* To avoid doing softmax twice, rewrite the softmax_cross_entropy function myself. (under-training)
 
||
 
||
−
* Analyze the reason that the loss didn't decline with alpha and gamma
+
* analyze and improve the alpha-gamma method.
−
* test for multi-task training;
+
−
* improve the baseline for the CS-EN
+
 
|-
 
|-
 
|Jiyuan Zhang ||
 
|Jiyuan Zhang ||
−
*  reproduced planning neural network [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/3/38/Planning_neural_network_initial_decode.pdf results]
+
 
 
||  
 
||  
−
*reproduce planning neural network
 
 
|-
 
|-
 
|Andi Zhang ||
 
|Andi Zhang ||
−
* added source masks in attention_decoder where calculates attention and in gru_cell where calculates new states.
+
 
−
* found the attribute sentence_length, perhaps it works better than my code
+
 
||
 
||
  
 
|-
 
|-
 
|Shiyue Zhang ||  
 
|Shiyue Zhang ||  
−
* figured out the problem of attention: the initial value of V should be around 0
+
 
−
* tested different modification, such as add mask, init b with 0.
+
−
* Compared the results, and concluded only change the initial value of V is the best.
+
 
||
 
||
−
* try to get right attention on memory
+
 
 
|-
 
|-
 
|Peilun Xiao ||
 
|Peilun Xiao ||

2017年3月13日 (一) 05:33的版本

Date People Last Week This Week
2017/1/3 Yang Feng
  • tested and analyzed the results on the cs-en data set (30.4 on the heldout-training set and 7.3 on the dev set);
  • added masks to the baseline (44.4 on the cn-en);
  • added masks to alpha-gamma method and fixed the bugs. Got an improvement of 0.5 again the masked baseline [report];
  • To avoid doing softmax twice, rewrite the softmax_cross_entropy function myself. (under-training)
  • analyze and improve the alpha-gamma method.
Jiyuan Zhang
Andi Zhang
Shiyue Zhang
Peilun Xiao
取自“http://cslt.org/mediawiki/index.php?title=NLP_Status_Report_2017-3-13&oldid=26420”