http://cslt.org/mediawiki/index.php?action=history&feed=atom&title=2014-02-21 2026-04-14T11:24:09Z 本wiki的该页面的版本历史 MediaWiki 1.23.3 http://cslt.org/mediawiki/index.php?title=2014-02-21&diff=9210&oldid=prev 2014-02-21T02:22:19Z

<p>以内容“==Resoruce Building== * Current text resource has been re-arranged and listed == AM development == === Sparse DNN === * Optimal Brain Damage(OBD). # GA-based block...”创建新页面</p> <p><b>新页面</b></p><div>==Resoruce Building==<br /> * Current text resource has been re-arranged and listed<br /> <br /> == AM development ==<br /> <br /> === Sparse DNN ===<br /> <br /> * Optimal Brain Damage(OBD). <br /> <br /> # GA-based block sparsity<br /> <br /> === Efficient DNN training ===<br /> <br /> # Asymmetric window: Great improvement on training set(WER 34% to 24%), however the improvement is lost on test. Overfitting? <br /> <br /> === Multilanguage training===<br /> <br /> # Pure Chinese training reached 4.9%<br /> # Chinese + English reduced to 7.9%<br /> # English phone set should discriminate beginning phone and ending phone<br /> # Should set up multilingual network structure which shares low layers but separate languages at high layers<br /> <br /> ===Noise traing===<br /> <br /> * Train with wsj database by corrupting data with various noise types<br /> :* baseline system ready<br /> :* noise data ready, selected 5 noise which is noise in reality<br /> :* Liuchao's noise-adding toolkit ready<br /> <br /> <br /> ===Engine optimization===<br /> <br /> * Investigating LOUDS FST. <br /> <br /> <br /> === Adaptation ===<br /> <br /> * Tested adaptation performance with adapted utterances from 10 to 40.<br /> <br /> <br /> ==Word to Vector==<br /> <br /> * Test a training toolkit Standford University, which can involve global information into word2vector training<br /> :* C++ implementation (instead of python) for data pre-processing, problem encountered<br /> <br /> * Basic wordvector plus global sense<br /> :* Training 100M data (with global sense), memory overflow<br /> :* Split the data into small pieces<br /> <br /> * Improved wordvector with multi sense<br /> :* Prepare scripts <br /> <br /> * Keyword extraction based on wordvectors<br /> :* Using google word vectors<br /> :* Using k-mean to cluster<br /> <br /> * Investigating Senna toolkit from NEC. Intending to implement POS tagging based on word vectors. <br /> <br /> ==LM development==<br /> <br /> ===NN LM===<br /> <br /> * Character-based NNLM (6700 chars, 7gram), 500M data training done.<br /> :* 3hours per iteration<br /> :* For word-based NNLM, 1 hour/iteration for 1024 words, 4 hours/iteration for 10240 words<br /> :* Performance lower than word-based NNLM<br /> <br /> * WordVector-based word and char NNLM training done<br /> :* Google wordvecotr-based NNLM is worse than random initialized NNLM<br /> <br /> <br /> ===3T Sogou LM===<br /> <br /> * Naive training <br /> :* all-word in lexicon <br /> :* split into 9G text blocks<br /> :* Merge one-by-one<br /> :* Cutting to 110k lexicon<br /> :* Test on QA <br /> :* Performance reduced compared to Liurong's previous LM<br /> <br /> * Improved training<br /> :* re-segmentation by Tencent 110k lexicon<br /> :* re-train with 4G text blocks<br /> :* sub-model training done, ready for merge based Tencent online1 test set.<br /> <br /> <br /> ==Embedded development==<br /> <br /> * CLG embedded decoder is almost done. Online compiler is on progress.<br /> * Zhiyong is working on layer-by-layer DNN training.<br /> <br /> <br /> ==Speech QA==<br /> <br /> * Current N-best results<br /> :* N-best search plus pinyin correction<br /> :* Total 2718 QA requests<br /> :* default 1844 QA correct<br /> :* no-entity 1650 QA correct<br /> :* with-entity 1884 QA correct<br /> <br /> * Analyze error patterns for Nbest match<br /> <br /> :* 10.8% song transcriptions errors<br /> :* 18.3% English error<br /> :* 38.7% entity (song name, singer name) recognition lost<br /> :* 32.3% non-entity recognition error<br /> <br /> * Computing complexity<br /> :* 11000 entity has 23000 different pronunciations<br /> :* Use tree to improve efficiency<br /> <br /> * Entity-class LM comparision<br /> :* re-segmentation &amp; re-train<br /> :* SRILM class-based LM<br /> :* Subgraph integration from Zhiyong</div>

Cslt