http://cslt.org/mediawiki/index.php?action=history&feed=atom&title=2014-02-28 2026-04-15T05:13:19Z 本wiki的该页面的版本历史 MediaWiki 1.23.3 http://cslt.org/mediawiki/index.php?title=2014-02-28&diff=9235&oldid=prev 2014-02-28T02:24:21Z

<p>以内容“==Resoruce Building== * Current text resource has been re-arranged and listed == AM development == === Sparse DNN === * Optimal Brain Damage(OBD). # GA-based block...”创建新页面</p> <p><b>新页面</b></p><div>==Resoruce Building==<br /> * Current text resource has been re-arranged and listed<br /> <br /> == AM development ==<br /> <br /> === Sparse DNN ===<br /> <br /> * Optimal Brain Damage(OBD). <br /> <br /> # GA-based block sparsity<br /> <br /> === Efficient DNN training ===<br /> <br /> # Asymmetric window: Great improvement on training set(WER 34% to 24%), however the improvement is lost on test. Overfitting? <br /> <br /> ===Multi GPU training===<br /> * Error encountered<br /> <br /> ===GMM - DNN co-training===<br /> * Error encountered<br /> <br /> === Multilanguage training===<br /> <br /> # Pure Chinese training reached 4.9%<br /> # Chinese + English reduced to 7.9%<br /> # English phone set should discriminate beginning phone and ending phone<br /> # Should set up multilingual network structure which shares low layers but separate languages at high layers<br /> <br /> ===Noise training===<br /> <br /> * Train with wsj database by corrupting data with various noise types<br /> :* White noise training completed. All results are fine<br /> :* Car noise training almost finished. Large-variance training on progress<br /> <br /> <br /> ===Engine optimization===<br /> <br /> * Investigating LOUDS FST. <br /> <br /> <br /> ==Word to Vector==<br /> <br /> * Test a training toolkit Standford University, which can involve global information into word2vector training<br /> :* C++ implementation (instead of python) for data pre-processing. Failed. Just use python.<br /> <br /> * Basic wordvector plus global sense<br /> :* 1 MB corpus costs 5 mins,vocab size 16698<br /> :* 10 MB corpus costs about 82 mins vocab size 56287<br /> <br /> * Improved wordvector with multi sense<br /> :* Almost impossible with the toolkit<br /> :* Can think of pre-training vectors and then do clusering<br /> <br /> <br /> * WordVecteor-based keyword extraction<br /> :* wordvector keyword extraction seems more reasonable if the keywords are in the lexicon<br /> :* For oov words, wv-based extraction is limited by the vocabulary<br /> :* Need a standard new word extraction<br /> <br /> <br /> * Investigating Senna toolkit from NEC. Intending to implement POS tagging based on word vectors. <br /> <br /> ==LM development==<br /> <br /> ===NN LM===<br /> <br /> * Character-based NNLM (6700 chars, 7gram), 500M data training done.<br /> :* 3hours per iteration<br /> :* For word-based NNLM, 1 hour/iteration for 1024 words, 4 hours/iteration for 10240 words<br /> :* Performance lower than word-based NNLM<br /> <br /> * WordVector-based word and char NNLM training done<br /> :* Google wordvecotr-based NNLM is worse than random initialized NNLM<br /> <br /> <br /> ===3T Sogou LM===<br /> <br /> <br /> * Improved training<br /> :* re-segmentation by Tencent 110k lexicon<br /> :* re-train with 4G text blocks<br /> :* 1/6 merge done. PPL reduced to 466(vs Tencent 8w8 213.74)<br /> :* Need to check the OOV problem<br /> :* Need to finish the final merge.<br /> <br /> <br /> ==Embedded development==<br /> <br /> * CLG embedded decoder is almost done. Online compiler is on progress.<br /> * Zhiyong is working on layer-by-layer DNN training.<br /> <br /> <br /> ==Speech QA==<br /> <br /> * N-best with entity LM was analyzed<br /> * Entity-class LM comparision<br /> :* re-segmentation &amp; re-train<br /> :* SRILM class-based LM ???<br /> :* Subgraph integration from Zhiyong</div>

Cslt