2013-10-25

Data sharing

• LM count files still undelivered!

DNN progress

Sparse DNN

• Optimal Brain Damage(OBD).

1. The initial observation shows that, after direct/obd cutting, obd is better than direct cutting in terms of training and testing accuracy.
2. The sticky training goes.

Tencent exps

N/A

Noisy training

1. The 863 clean test shows consistent performance.
2. Investigation shows that the car noise corrupted the speech patterns more seriously with the current fb-based noise adding. This leads to worse performance with car noises than with white noises. Note this observation is valid only with the fb-based noise adding

Continuous LM

1. The lattice rescore toolkit is done. The efficiency problem is solved by DP.
2. The perplexity of 10-NN is worse than 1-NN
3. Initial lattices rescoring shows that the rescoring needs remove the old ngram score and then insert the new score. The 1st nn works better than the 10-nn.
4. The CSLM does not improve the performance so far. This is conflict to the ppl results.
5. Investigate better ways to combine multiple NNs. Investigate rescoring n-best results instead of lattices.

lattice-rescore: traing-data:500M,dict:11w,test:tencent

id        cslm(lopp) cslm(-logp)   n-gram  n-gram(replace) cslm(-old+cslm) cslm((0-1024)) cslm((0-1024),(1024-2048))
map:        53.86         38.44      34.40    37.38          35.38                34.94      34.91
2044:       47.81         32.20      27.18    31.18          28.53                27.65      27.87
notetp3:    43.01         24.61      18.78    22.34          20.18                19.64      19.81
record1900: 32.14         20.42      12.89    20.10          13.97                13.52      13.81
general:    62.86         49.17      43.23    47.50          45.55                44.28      44.62
online1:    62.79         46.76      39.49    46.40          40.26                39.91      40.01       
online2:    58.20         39.21      32.15    38.75          33.04                32.61      32.65
speedup:    52.50         38.74      31.47    37.72          34.01                32.65      32.81

note: cslm(logp) --replace lattice value1 with cslm lnp
      cslm(-logp)--replace lattice value1 with cslm -lnp
      ngram(replace)--replace lattice vaule1 with ngram -lnp
      cslm((-old+cslm)--repace lattice value1 = -old_ngram+cslm 
      cslm((0-1024))--replace lattice value1=-old_ngram+cslm (slist:0-1024,mach_num=1)
      cslm((0-1024),(1024-2048))--replace lattice value1=-old_ngram+cslm (slist:0-1024,1024-1048,mach_num=2)

QA LM

1. Tencent word segmentation system ready.
2. Collecting data for Q-LM training.