ASR:2014-12-08

Text Processing

LM development

Domain specific LM

• domain lm

• train some more LMs with Zhenlong (dianzishu sogou bbs chosen),put result on cvss.
• Sougou2T : kn-count.
• lm v2.0 continue(this week)

• new dict.

• Released vocab v2.0 (mainly done by Dongxu) to JieTong.

tag LM

• summary done
• need to do

• tag Probability should test add the weight(hanzhenglong) and handover to hanzhenglong (hold)
• make a summary about tag-lm and journal paper(wxx and yuanb)(this weeks).

• Reviewed papers and begin to write paper (this week)

RNN LM

• rnn

• test wer RNNLM on Chinese data from jietong-data(this week)
• generate the ngram model from rnnlm and test the ppl with different size txt.[1]

• lstm+rnn

• check the lstm-rnnlm code about how to Initialize and update learning rate.(hold)

Word2Vector

W2V based doc classification

• Initial results variable Bayesian GMM obtained. Performance is not as good as the conventional GMM.(hold)
• Non-linear inter-language transform: English-Spanish-Czch: wv model training done, transform model on investigation

Knowledge vector

• Knowledge vector started

• Analysis the wiki infomation of category and link into jso done, knowledge vector build graph done.
• begin to code for train

relation

• Accomplish transE with almost the same performance as the paper did(even better)[2]

Character to word

• Character to word conversion(hold)

• prepare the task: word similarity
• prepare the dict.

Translation

• v5.0 demo released

• cut the dict and use new segment-tool

QA

deatil:

Spell mistake

• add the xiaoI pingyin correct to framework.

improve fuzzy match

• add Synonyms similarity using MERT-4 method(hold)

improve lucene search

• using MERT-4 method to get good value of multi-feature.like IDF,NER,baidu_weight,keyword etc.(liurong this month)
• now test the performance.

Multi-Scene Recognition

• done

XiaoI framework

• ner from xiaoI
• new inter will install SEMPRE

patent

• done