2014-10-20 - 版本历史

Lr：/* Text Processing */

2014-10-20T07:23:35Z

‎Text Processing

Lr：/* LM development */

2014-10-20T07:20:58Z

‎LM development

Lr：/* LM development */

2014-10-20T07:15:27Z

‎LM development

Lr：/* Text Processing */

2014-10-20T07:12:33Z

‎Text Processing

2014年10月20日 (一) 07:06 Zhangzy

2014-10-20T07:06:17Z

2014年10月20日 (一) 07:04 Zhangzy

2014-10-20T07:04:43Z

2014年10月20日 (一) 07:02 Zhangzy

2014-10-20T07:02:14Z

Zhangzy：/* Speech rate training */

2014-10-20T06:32:32Z

‎Speech rate training

Zhangzy：/* low resource language AM training */

2014-10-20T06:29:25Z

‎low resource language AM training

2014年10月20日 (一) 06:28 Zhangzy

2014-10-20T06:28:08Z

@@ 第158行： / 第158行： @@
 ===RNN LM===
 *rnn
-: get baseline on nbest rescore of wer.
+:* get baseline on nbest rescore of wer.
 *lstm+rnn
-: trained the RNN+LSTM lm on wsj_np_data about 200M. the neural net work is 100*100(lstm cell)*10000 with 100 classes. it cost about 200 minutes each epoch.
+:* trained the RNN+LSTM lm on wsj_np_data about 200M. the neural net work is 100*100(lstm cell)*10000 with 100 classes. it cost about 200 minutes each epoch on 2 cpu kernels.
-: get baseline on nbest rescore of wer.
+:* get baseline on nbest rescore of wer.
-: more detail on LSTM
+:* more detail on LSTM
 ===Translation===
@@ 第173行： / 第173行： @@
 * search method:
-: add the vsm and BM25 to improve the search. and the strategy of selecting the answer.
+:* add the vsm and BM25 to improve the search. and the strategy of selecting the answer.
 * spell check
-: get ngram tool and make a simple demo.
+:* get ngram tool and make a simple demo.
-: get domain word list and pingyin tool from huilan.
+:* get domain word list and pingyin tool from huilan.
-* new inter will install SEMPRE
+** new inter will install SEMPRE

@@ 第127行： / 第127行： @@
 ====Domain specific LM====
-h2. lm based on baidu_hi and baiduzhidao is done, test on shujutang test set.
+* dongxu get good vocabulary from big data. Train 5-gram LM using Baiduzhidao_corpus(~30GB after preprocess) with new lexicon. There is a mistake when counted possiblity after merge.
-h2. weibo lm were training with pruning on counts(5,10,10,20,20),because it is too large. the ppl is twice as high than baidu_hi && baidu_zhidao.
-===tag LM===:
+===tag LM===
 * use HIT's LTP tool to segment,pos and ner. the program is running(about 3 days) on baiduHi and baiduzhidao(total 365G)
 * will use the small test set from xiaoxi for address-tag..
 * now about more 1M address,will prune it using frequency.
 ===Word2Vector===

@@ 第131行： / 第131行： @@
 h2. weibo lm were training with pruning on counts(5,10,10,20,20),because it is too large. the ppl is twice as high than baidu_hi && baidu_zhidao.
-h2. NUM tag LM:
+===tag LM===:
 * use HIT's LTP tool to segment,pos and ner. the program is running(about 3 days) on baiduHi and baiduzhidao(total 365G)
 * will use the small test set from xiaoxi for address-tag..
 * now about more 1M address,will prune it using frequency.
 ===Word2Vector===

@@ 第128行： / 第128行： @@
 ====Domain specific LM====
-h2. ngram generation is on going
+h2. lm based on baidu_hi and baiduzhidao is done, test on shujutang test set.
-h2. look the memory and baidu_hi done
+h2. weibo lm were training with pruning on counts(5,10,10,20,20),because it is too large. the ppl is twice as high than baidu_hi && baidu_zhidao.
 h2. NUM tag LM:
-* maxi work is released.
+* use HIT's LTP tool to segment,pos and ner. the program is running(about 3 days) on baiduHi and baiduzhidao(total 365G)
-* yuanbin continue the tag lm work.
+* will use the small test set from xiaoxi for address-tag..
-* add the ner to tag lm .
+* now about more 1M address,will prune it using frequency.
-* Boost specific words like wifi if TAG model does not work for a particular word.
@@ 第149行： / 第148行： @@
 * Knowledge vector started
 :* format the data
 * Character to word conversion
@@ 第155行： / 第155行： @@
 * Google word vector train
-:* improve the sampling method
+:* some ideal will discuss on weekly report.
-−
 ===RNN LM===
 *rnn
 *lstm+rnn
-: install the tool and prepare the data of wsj
+: trained the RNN+LSTM lm on wsj_np_data about 200M. the neural net work is 100*100(lstm cell)*10000 with 100 classes. it cost about 200 minutes each epoch.
-: prepare the baseline.
+: get baseline on nbest rescore of wer.
 ===Translation===
 * v3.0 demo released
 :* still slow
-:* re-segment the word using new dictionary.
+:* re-segment the word using new dictionary.will use the tencent-dic about 11w.
 :* check new data.
@@ 第172行： / 第173行： @@
 * search method:
-:* add the vsm and BM25 to improve the search. and the strategy of selecting the answer
+: add the vsm and BM25 to improve the search. and the strategy of selecting the answer.
-:* segment the word using minimum granularity for lucene index and bag-of-words method.
+* spell check
 * new inter will install SEMPRE

@@ 第61行： / 第61行： @@
 * Add more silence tag "#" in pure-silence utterance text(train).
 :* xEntropy model be training
-: need to test baseline.
+:* need to test baseline.
 * Sum all sil-pdf as the silence posterior probability.
@@ 第75行： / 第75行： @@
 :* Various the used Chinese trained DNN layer numbers.
 :**  feature_transform = 6000h_transform + 6000_N*hidden-layers
-   nnet.init = rand
+   nnet.init = random (4-N)*hidden-layers + output-layer
-om (4-N)*hidden-layers + output-layer
-−
    | N / learn_rate | 0.008         | 0.001 | 0.0001 |
    |   baseline     | 17.00(14*2h)  |       |        |
@@ 第88行： / 第86行： @@
    nnet.init = random (4-N)*hidden-layers + output-layer
    Note: This is reproduced Yinshi's experiment
    | N / learn_rate | 0.008 | 0.001 | 0.0001 |
    |   baseline     | 17.00 |       |        |

@@ 第36行： / 第36行： @@
 .5 |     5.39    |    4.80    |   4.36     |    -
-:* Test on noisy AURORA 4 dataset
+:* Test on noisy AURORA4 dataset
-:* Continue the droptout on normal trained XEnt NNET , eg wsj.
+        std |  dropout0.4 | dropout0.5 | dropout0.7 | dropout0.8
-:* Draft the dropout-DNN weight distribution.
+     -------------------------------------------------------------
 * Rectification
-:* Still NAN error, need to debug.
+:* Still NAN error, need to debug. (+)
-* MaxOut
+* MaxOut (+)
 * Convolutive network

@@ 第48行： / 第48行： @@
 :*Test more configurations
 :* Yiye will work on CNN
 ====Denoising & Farfield ASR====
@@ 第57行： / 第58行： @@
 * Add more silence tag "#" in pure-silence utterance text(train).
 :* xEntropy model be training
 * Sum all sil-pdf as the silence posterior probability.
 ====Speech rate training====
 * Seems ROS model is superior to the normal one with faster speech
-* Suggest to extract speech data of different ROS, construct a new test set
+* Suggest to extract speech data of different ROS, construct a new test set(+)
-* Suggest to use Tencent training data
+* Suggest to use Tencent training data(+)
 ==== low resource language AM training ====
@@ 第69行： / 第72行： @@
 :* Various the used Chinese trained DNN layer numbers.
 :**  feature_transform = 6000h_transform + 6000_N*hidden-layers
-   nnet.init = random (4-N)*hidden-layers + output-layer
+   nnet.init = rand
    | N / learn_rate | 0.008         | 0.001 | 0.0001 |
@@ 第79行： / 第83行： @@
    |       0        | 16.88         |       |        |
 :**  feature_transform = uyghur_transform + 6000_N*hidden-layers
-nnet.init = random (4-N)*hidden-layers + output-layer
+  nnet.init = random (4-N)*hidden-layers + output-layer
-*Note: This is reproduced Yinshi's experiment
+  Note: This is reproduced Yinshi's experiment
    | N / learn_rate | 0.008 | 0.001 | 0.0001 |
@@ 第91行： / 第95行： @@
 :** feature_transform = 6000_transform + 6000_N*hidden-layers
-nnet.init = uyghur (4-N)*hidden-layers + output-layer
+  nnet.init = uyghur (4-N)*hidden-layers + output-layer
    | N / learn_rate | 0.008 | 0.001 | 0.0001 |
    |   baseline     | 17.00 |       |        |

@@ 第85行： / 第85行： @@
 *Note: This is reproduced Yinshi's experiment
-| N / learn_rate | 0.008 | 0.001 | 0.0001 |
+  | N / learn_rate | 0.008 | 0.001 | 0.0001 |
-|   baseline     | 17.00 |       |        |
+  |   baseline     | 17.00 |       |        |
-|       4        | 28.23 | 30.72 | 37.32  |
+  |       4        | 28.23 | 30.72 | 37.32  |
-|       3        | 22.40 |       |        |
+  |       3        | 22.40 |       |        |
-|       2        | 19.76 |       |        |
+  |       2        | 19.76 |       |        |
-|       1        | 17.41 |       |        |
+  |       1        | 17.41 |       |        |
-|       0        |       |       |        |
+  |       0        |       |       |        |
 :** feature_transform = 6000_transform + 6000_N*hidden-layers
 nnet.init = uyghur (4-N)*hidden-layers + output-layer
-| N / learn_rate | 0.008 | 0.001 | 0.0001 |
+  | N / learn_rate | 0.008 | 0.001 | 0.0001 |
-|   baseline     | 17.00 |       |        |
+  |   baseline     | 17.00 |       |        |
-|       4        | 17.80 | 18.55 | 21.06  |
+  |       4        | 17.80 | 18.55 | 21.06  |
-|       3        | 16.89 | 17.64 |        |
+  |       3        | 16.89 | 17.64 |        |
-|       2        |       |       |        |
+  |       2        |       |       |        |
-|       1        |       |       |        |
+  |       1        |       |       |        |
-|       0        |       |       |        |
+  |       0        |       |       |        |
 ====Scoring====

@@ 第70行： / 第70行： @@
 ==== low resource language AM training ====
 * Use Chinese NN as initial NN, change the last layer
 :* Various the used Chinese trained DNN layer numbers.
 ====Scoring====
 * global scoring done.
 * Pitch & rhythm done, need testing
-* Harmonics hold
+* Harmonics program done, experiment to be done.
-−
 ====Confidence====