2013-05-10 - 版本历史

2013年5月14日 (二) 01:11 Liuc

2013-05-14T01:11:54Z

2013年5月13日 (一) 07:57 Cslt

2013-05-13T07:57:52Z

2013年5月13日 (一) 07:44 Cslt

2013-05-13T07:44:43Z

2013年5月13日 (一) 07:37 Cslt

2013-05-13T07:37:03Z

2013年5月13日 (一) 07:34 Cslt

2013-05-13T07:34:27Z

Cslt：/* Embedded progress */

2013-05-13T07:33:44Z

‎Embedded progress

2013年5月13日 (一) 07:32 Cslt

2013-05-13T07:32:50Z

2013年5月13日 (一) 07:25 Cslt

2013-05-13T07:25:14Z

Cslt：/* Data sharing */

2013-05-13T07:17:50Z

‎Data sharing

Liuc：以内容“== Data sharing == * status? == DNN progress == === Experiments === * setups for input layer : s1: mfcc(13), splice +-5[143] : s2: mfcc(13), splice +-5(143), LDA[143]...”创建新页面

2013-05-10T07:58:35Z

以内容“== Data sharing == * status? == DNN progress == === Experiments === * setups for input layer : s1: mfcc(13), splice +-5[143] : s2: mfcc(13), splice +-5(143), LDA[143]...”创建新页面

新页面

== Data sharing ==

* status?

== DNN progress ==
=== Experiments ===
* setups for input layer
: s1: mfcc(13), splice +-5[143]
: s2: mfcc(13), splice +-5(143), LDA[143]
: s3: mfcc(13), delta(39), splice +-5(429), LDA[143]
: s4: mfcc(13), delta(39), splice +-5(429), LDA[300]
* setups for alignment
: tri1: triphone training, feature input: mfcc(13), delta[39]. #pdfs 1651, #gaussians 10028
: tri2: LDA/MLLT training, feature input: mfcc(13), delta(39), splice +-4(351), LDA[40]. #pdfs 3536, #gaussians 39995
* other notes
: about 100 hours training data
: 88k LM, biglm decoding (1e-5 / 1e-9)
: gpu-based nnet training, in-1200-1200-1200-1200-out
* results
{| class="wikitable"
! Test Set !! fMMI !! s1/tri1 !! s2/tri1 !! s3/tri1 !! s4/tri1 !! s2/tri2 !! s4/tri2 !! cpu-based (like s4/tri1)
|-
|map || || 25.38 || 24.47 || 26.16 || 26.20 || 22.85 || 24.27 || 26.45
|-
|2044 || || 23.58 || 22.82 || 23.84 || 24.13 || 21.45 || 22.76 || 24.66
|-
|notetp3|| || 16.08 || 14.89 || 15.92 || 15.97 || 14.89 || 14.79 || 16.14
|-
|1900 || || 8.55 || 8.43 || 8.66 || 8.90 || 7.30 || 7.91 || 8.23
|-
|general|| || 36.18 || 34.79 || 35.88 || 35.90 || 33.06 || 33.79 || 38.02
|-
|online1|| || 34.68 || 33.90 || 33.45 || 33.38 || 32.93 || 32.43 || 33.00
|-
|online2|| || 27.27 || 26.61 || 26.26 || 26.36 || 25.94 || 25.69 || 26.63
|-
|speedup|| || 24.97 || 24.40 || 24.55 || 25.42 || 23.04 || 23.67 || 27.17
|-
|}
* conclusion
TODO

=== Tencent exps ===
TODO

=== GPU & CPU merge ===
TODO

=== L-1 sparse initial training ===
* experiments on L1 penalty
{| class="wikitable"
! Test Set !! 0 !! 1.00E-06 !! 1.00E-05 !! 2.50E-05 !! 5.00E-05 !! 7.50E-05 !! 1.00E-04
|-
|map || 61.06 || 61.18 || 61.31 || 61.72 || 62.89 || 62.84 || 62.48
|-
|2044 || 49.53 || 49.58 || 49.84 || 49.94 || 50.71 || 51.08 || 51.08
|-
|notetp3|| 43.44 || 43.44 || 43.55 || 44.58 || 45.22 || 44.95 || 45.76
|-
|1900 || 38.50 || 38.54 || 39.00 || 39.21 || 40.33 || 40.53 || 40.60
|-
|general|| 61.24 || 61.22 || 61.69 || 61.84 || 62.71 || 62.83 || 62.93
|-
|online1|| 58.02 || 58.05 || 58.31 || 58.23 || 58.83 || 59.06 || 59.46
|-
|online2|| 53.62 || 53.70 || 54.03 || 53.93 || 54.65 || 54.94 || 55.51
|-
|speedup|| 57.51 || 57.49 || 57.93 || 58.31 || 59.75 || 60.08 || 59.52
|-
|}
* TODO

=== To be done ===
TODO

== Kaldi/HTK merge ==
* HTK2Kaldi: hold.
* Kaldi2HTK: stuck. various sp models tried but don't help
* To be done
# TODO

== Embedded progress ==
*Status:
:# training acoustic model for sphinx

*To be done
:# finish AM training
:# run test

@@ 第64行： / 第64行： @@
 # The conventional recipe seems the best: PLP without HLDA. Referring to the CSLT results, the identical-dimension LDA should work, while ould not be used as dimension reduction.
 # some linear transform, DCT/LDA should contribution, due to the prior knowledge to avoid the network overtraining.
+# the PLP & FBank comparision seems strange: we should not regulate the same dimension, instead, the best performance. The 41 frame splice is a bit too long. Comparing to the CSLT result, fbank-dct (which is the same as mfcc) 12 * 11 frame slice is good enough. The large dimension of PLP-11 is the delta feature, which has been verified to be harmful in the CSLT experiments.

@@ 第42行： / 第42行： @@
 #the fine training leads to significant better performance than the rough training (see s2/tri1 vs s2/tri2 & s4/tri1 vs s4/tri2)
 #the delta features do not help, actually harm the perfromance (see s2/tri1 vs s3/tri1 & s4/tri1)
-#the linear LDA helps the performance (see s1/tri1 vs s2/tri1)
+#the identical-dimension LDA helps the performance (see s1/tri1 vs s2/tri1)
 '''the best system is s2/tri2: without delta, apply a linear LDA.'''
@@ 第60行： / 第60行： @@
 #  Momentum实验：模型参数更新中，sgd算法添加动量项，同样的收敛准则，添加动量项的试验收敛次数变少，但性能变差
 #  Minibatch size实验：128：51.6，1024：51.7，minibatch size较大性能略好。

@@ 第43行： / 第43行： @@
 #the delta features do not help, actually harm the perfromance (see s2/tri1 vs s3/tri1 & s4/tri1)
 #the linear LDA helps the performance (see s1/tri1 vs s2/tri1)
-'''#the best system is s2/tri2: without delta, apply a linear LDA.'''
 === Tencent exps ===
-TODO
 === GPU & CPU merge ===

@@ 第43行： / 第43行： @@
 #the delta features do not help, actually harm the perfromance (see s2/tri1 vs s3/tri1 & s4/tri1)
 #the linear LDA helps the performance (see s1/tri1 vs s2/tri1)
-#the best system is s2/tri2: with out delta, apply a linear LDA.
+'''#the best system is s2/tri2: without delta, apply a linear LDA.'''
 === Tencent exps ===

@@ 第95行： / 第95行： @@
 :# first embedded demo done, 1000 words take 3.2M memory.
 :# accuracy test not yet finished
-:# training acoustic model for sphinx
+:# training acoustic model for sphinx. The toolkit runs well, need prepare data and start parallel training.
 *To be done
 :# finish AM training
 :# run offline test

@@ 第21行： / 第21行： @@
 ! Test Set !! fMMI !! s1/tri1 !! s2/tri1 !! s3/tri1 !! s4/tri1 !! s2/tri2 !! s4/tri2 !! cpu-based (like s4/tri1)
 |-
-|map    || || 25.38 || 24.47 || 26.16 || 26.20 || 22.85 || 24.27 || 26.45
+|map    || 28.58 || 25.38 || 24.47 || 26.16 || 26.20 || 22.85 || 24.27 || 26.45
 |-
-|2044   || || 23.58 || 22.82 || 23.84 || 24.13 || 21.45 || 22.76 || 24.66
+|2044   || 24.79 || 23.58 || 22.82 || 23.84 || 24.13 || 21.45 || 22.76 || 24.66
 |-
-|notetp3|| || 16.08 || 14.89 || 15.92 || 15.97 || 14.89 || 14.79 || 16.14
+|notetp3|| 21.64 || 16.08 || 14.89 || 15.92 || 15.97 || 14.89 || 14.79 || 16.14
 |-
-|1900   || ||  8.55 ||  8.43 ||  8.66 ||  8.90 ||  7.30 ||  7.91 ||  8.23
+|1900   ||  8.19 ||  8.55 ||  8.43 ||  8.66 ||  8.90 ||  7.30 ||  7.91 ||  8.23
 |-
-|general|| || 36.18 || 34.79 || 35.88 || 35.90 || 33.06 || 33.79 || 38.02
+|general|| 39.63 || 36.18 || 34.79 || 35.88 || 35.90 || 33.06 || 33.79 || 38.02
 |-
-|online1|| || 34.68 || 33.90 || 33.45 || 33.38 || 32.93 || 32.43 || 33.00
+|online1|| 35.19 || 34.68 || 33.90 || 33.45 || 33.38 || 32.93 || 32.43 || 33.00
 |-
-|online2|| || 27.27 || 26.61 || 26.26 || 26.36 || 25.94 || 25.69 || 26.63
+|online2|| 28.30 || 27.27 || 26.61 || 26.26 || 26.36 || 25.94 || 25.69 || 26.63
 |-
-|speedup|| || 24.97 || 24.40 || 24.55 || 25.42 || 23.04 || 23.67 || 27.17
+|speedup|| 28.45 || 24.97 || 24.40 || 24.55 || 25.42 || 23.04 || 23.67 || 27.17
 |-
 |}

@@ 第43行： / 第43行： @@
 #the delta features do not help, actually harm the perfromance (see s2/tri1 vs s3/tri1 & s4/tri1)
 #the linear LDA helps the performance (see s1/tri1 vs s2/tri1)
+#the best system is s2/tri2: with out delta, apply a linear LDA.
 === Tencent exps ===
@@ 第49行： / 第49行： @@
 === GPU & CPU merge ===
-TODO
 === L-1 sparse initial training ===
-* experiments on L1 penalty
+* experiments on L1 and L2 penalty
 {| class="wikitable"
 ! Test Set !! 0 !! 1.00E-06 !! 1.00E-05 !! 2.50E-05 !! 5.00E-05 !! 7.50E-05 !! 1.00E-04
@@ 第73行： / 第79行： @@
 |-
 |}
-* TODO
+* Conclusions
 == Kaldi/HTK merge ==
 * HTK2Kaldi: hold.
 * Kaldi2HTK: stuck. various sp models tried but don't help
-* To be done
+* Need thorough debug this week.
-# TODO
 == Embedded progress ==
 *Status:
 :# training acoustic model for sphinx
 *To be done
 :# finish AM training
-:# run test
+:# run offline test

@@ 第39行： / 第39行： @@
 |}
 * conclusion
-TODO
+#the GPU approach is comparable with the CPU approach (see s4/tri1 & CPU results). The former works slightly better in most cases.
 === Tencent exps ===

@@ 第1行： / 第1行： @@
 == Data sharing ==
-* status?
+* LM count files still undelivered!
 == DNN progress ==