Hulan-2014-11-06

Dialog system

Algorithm

Spell mistake

• retrain the ngram model(caoli)
• prepare the test and development set(caoli)

improve fuzzy match

• add Synonyms similarity using MERT-4 method

improve lucene search

• our vsm method

different result in lucene

method	lucene	vsm_idf(haiguan)	VSM_idf(baidu)	vsm_idf(tain)	vsm_idf(calculate)
Accary	0.6628	0.6228	0.6197	0.5827	0.5426

• lucene top

• top10(82.95%),top20(86.34),top50(90.23%),top100(94.11%),top200(96.18%),top1000(97.31%),top2000(97.87%),top5000(98.75%),top10000(99.06)
• test the result of top(100,200,1000) in full qa(lucene+fuzzymatch)(caoli)

• lucene Optimization(liurong)

• rewrite the method to select the 50 standard question not same template.(liurong)
• check the word segment for template.(liurong)
• boost the query keyword using IDF

boost keyword in lucene

method	Default	idf_train	idf_train_norm	idf_baidu	idf_baidu_norm
Accary	0.66228	0.651629	0.57644	0.647869	0.65288

• using MERT-4 method to get good value of multi-feature.like IDF,NER,baidu_weight,keyword etc.(liurong this month)

Multi-Scene Recognition

• add the triples search to QA engine

• discuss the detail and give a report.(liurong)

• demo (liurong two week)

Knowledge Management and labeling system

• continue coding.

Patent

• the GA method to improve QA .(liurong this month)

plan to discuss

• how to add the spell check method to QA engine.