http://cslt.org/mediawiki/api.php?action=feedcontributions&feedformat=atom&user=Zhangzy 2026-04-14T11:06:10Z 用户贡献 MediaWiki 1.23.3 http://cslt.org/mediawiki/index.php/Sinovoice-2016-2-2 2021-09-09T05:42:43Z

<p>Zhangzy：保护“Sinovoice-2016-2-2”（[编辑=CSLT users]（无限期）[移动=CSLT users]（无限期）[Read=CSLT users]（无限期））</p> <hr /> <div>'''警告：'''“Sinovoice-2016-2-2”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-02-25 2019-02-25T01:30:12Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-02-25”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:12:58Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> :*mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> :*mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> :*fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> mlpg | 130000 | 500000 | 6200000 <br /> fast-mlpg | 60000 | 300000 | 3580000<br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> :*text<br /> ::*5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> ::*13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * no-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_13.wav]<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:11:55Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> *mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> *mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> *fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> mlpg | 130000 | 500000 | 6200000 <br /> fast-mlpg | 60000 | 300000 | 3580000<br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> 5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> 13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * no-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_13.wav]<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:11:29Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> mlpg | 130000 | 500000 | 6200000 <br /> fast-mlpg | 60000 | 300000 | 3580000<br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> 5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> 13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * no-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg-no_13.wav]<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:09:12Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> mlpg | 130000 | 500000 | 6200000 <br /> fast-mlpg | 60000 | 300000 | 3580000<br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> 5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> 13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:08:50Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> -----------------------------------------------------------------<br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> mlpg | 130000 | 500000 | 6200000 <br /> fast-mlpg | 60000 | 300000 | 3580000<br /> <br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> 5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> 13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/TTS-project-synthesis 2019-02-18T12:08:00Z

<p>Zhangzy：</p> <hr /> <div>=Project name=<br /> Text To Speech<br /> <br /> =Project members=<br /> Dong Wang, Zhiyong Zhang<br /> <br /> =Introduction=<br /> We are interested in a flexible syntehsis based on neural model . The basic idea is that since the neural model can be <br /> traind with multiple conditions, we can treat speaker and emotion as the conditional factors. We use the speaker vector<br /> and emotion vector as addiiontal input to the model, and then train a single model that can produce sound of different<br /> speakers and different emotions. <br /> <br /> In the following experiments, we use a simple DNN architecture to implement the training. The vocoder is WORD. <br /> <br /> =Experiments=<br /> <br /> ==Mono-speaker==<br /> <br /> The first step is mono-speaker systems. We trained three systems: a female, a male and a child, each with a <br /> single network. The performance is like the ofllowing.<br /> <br /> Synthesis text:好雨知时节，当春乃发声，随风潜入夜，润物细无声<br /> <br /> *Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/female01/female01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/male01/male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/huilian/child01.neutral/child01-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker==<br /> <br /> Now we combine all the data from male, female and child to train a single model.<br /> <br /> ===Without Speaker-vector===<br /> <br /> The first experiment is that the data are blindly combined, without any indicator of speakers. <br /> <br /> *Female &amp; Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-male01/female01-male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Female &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/female01-child01.neutral/female01-child.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male &amp; Child[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/male01-child01.neutral/male01_child01.neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> <br /> ===With Speaker-vector===<br /> <br /> Now we use speaker vector as an indicator of the speaker trait. <br /> <br /> *Specific person<br /> <br /> Firstly, use the speaker fector to specifiy a particular person:<br /> <br /> :*Female[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/female01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> :*Male[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/all.dvector40/male01.dvec40_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolate of different person<br /> <br /> Now let's produce interpolated voice by interpolating two speakers: female and amle.<br /> <br /> :* Female &amp; Male with different ratio<br /> <br /> ::*(1) 0.0:1.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_0_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(2) 0.1:0.9[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_1_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(3) 0.2:0.8[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_2_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(4) 0.3:0.7[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_3_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(5) 0.4:0.6[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_4_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(6) 0.5:0.5[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_5_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(7) 0.6:0.4[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_6_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(8) 0.7:0.3[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_7_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(9) 0.8:0.2[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_8_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(10) 0.9:0.1[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_9_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ::*(11) 1.0:0.0[http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speakers/mix/iterpolation/female01_male01/iterpolation_10_female01_male01_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Mono-speaker Multi-Emotion==<br /> <br /> Using emotion vectors can specify which emotio to use, and the emotion can be also interpolated. <br /> <br /> *Specific emotion<br /> :* Neutral emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-neutral_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Happy emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-happy_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Sorrow emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-sorrow_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* Angry emotion [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Interpolation emotion<br /> :* Angry &amp; neutral with different ratio<br /> ::*(1) 0.0:1.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_0_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(2) 0.1:0.9 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(3) 0.2:0.8 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_2_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(4) 0.3:0.7 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_3_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(5) 0.4:0.6 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_4_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(6) 0.5:0.5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(7) 0.6:0.4 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_6_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(8) 0.7:0.3 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_7_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(9) 0.8:0.2 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_8_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(10) 0.9:0.1 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/mix-emotion-angry-neutral_1_9_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> ::*(11) 1.0:0.0 [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/emotion/roobo.child/x-angry_1_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> ==Multi-speaker Multi-emotion==<br /> <br /> Finally, all the data (different speakers and different emotions) are combined together. Note that only the child voice<br /> has different emotions of training data. We hope that this emotion can be learned so that we can generate voice of <br /> other speakers with emotion, although they do not have any training data with emtoions. <br /> <br /> *Female<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/female01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> *Male<br /> :* angry [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_angry_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* happy [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_happy_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* neutral [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_neutral_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> :* sorrow [http://zhangzy.cslt.org/categories/tts/sample-wav/mimic-wangd-front-end/multi-speaker_multi-emotion/male01_sorrow_final_5_amdurTanh_acTanh_mlpg1_postfilter1.world.wav01.wav]<br /> <br /> =MLPG Comparation=<br /> We compare the different implementation of mlpg AS merlin does(mlpg.py and fast_mlpg.py).<br /> There are three implementations:<br /> mlpg: As mlpg.py while compute all the dimension of delta features(including lf0/bap/mgc, the dim is 1/5/60 respectively)<br /> mlpg-lossy: Wrong implementation of mlpg.py by only considering the first dimension of global co-variance.<br /> fast-mlpg: As fast_mlpg.py in merlin.<br /> <br /> *Computation Time(Estimation)<br /> -----------------------------------------------------------------<br /> alg. | lf0(dim=1) | bap(dim=5) | mgc(dim=60) <br /> -----------------------------------------------------------------<br /> mlpg-lossy | 100000 | 130000 | 160000 <br /> -----------------------------------------------------------------<br /> mlpg | 130000 | 500000 | 6200000 <br /> -----------------------------------------------------------------<br /> fast-mlpg | 60000 | 300000 | 3580000<br /> -----------------------------------------------------------------<br /> avg-rate | 1:1.3:0.6 | 1:4:2+ | 1:40:20+<br /> -----------------------------------------------------------------<br /> <br /> * Synthesis waves<br /> 5='好雨知时节，当春乃发声，随风潜入夜，润物细无声。'<br /> 13='大熊猫最大的愿望就是拍一张自己的照片。'<br /> <br /> * mlpg-lossy<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg01_13.wav]<br /> <br /> * mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/mlpg60_13.wav]<br /> <br /> * fast-mlpg<br /> :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_5.wav]<br /> :*13 :*5 [http://zhangzy.cslt.org/categories/tts/sample-wav/mlpg-cmp/fast-mlpg_13.wav]</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-02-18 2019-02-18T04:18:54Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-02-18”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-28 2019-01-28T05:02:45Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-28”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-21 2019-01-21T03:08:59Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-21”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-14 2019-01-14T03:17:51Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-14”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-14 2019-01-14T02:43:33Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-14”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-14 2019-01-14T02:38:45Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-14”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-07 2019-01-07T03:25:10Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-07”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2019-01-02 2019-01-02T02:57:29Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2019-01-02”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-12-24 2018-12-24T03:05:52Z

<p>Zhangzy：</p> <hr /> <div>This Week:<br /> {| class=&quot;wikitable&quot;<br /> !People !! Last Week !! This Week !! Meet Minutes !! Task Tracing(&lt;font color=&quot;red&quot;&gt;DeadLine&lt;/font&gt;)<br /> |-<br /> |Mengyuan Zhao ||<br /> 本周:<br /> * 工程化<br /> # 熟悉语音分割流程<br /> ||<br /> 下周：<br /> * 工程化<br /> # 继续梳理demo list，并上线。<br /> ||<br /> <br /> ||<br /> |-<br /> |Zhiyong Zhang||<br /> 本周：<br /> # 新嵌入式板子验证及串口输出测试；<br /> # 归档英语/日语模型整理，已完成英语<br /> # 国网语音切分工具部署--虚拟机模式<br /> <br /> ||<br /> 下周：<br /> # ASR-decoder重置<br /> # 嵌入式语音识别板子测试<br /> # 归档日语模型<br /> <br /> ||<br /> ||<br /> |-<br /> |Yang Wei ||<br /> 本周：<br /> * 外包声纹demo分数问题定位<br /> ||<br /> 下周：<br /> * 完成外包声纹demo测试<br /> * 新版asr socket server部署测试<br /> <br /> ||<br /> ||<br /> |-<br /> |Zhenlong Han||<br /> 本周：<br /> # 整理项目工具框架<br /> # 跟进国网标注<br /> # 双猴京华项目支持<br /> # 分音塔标注检查<br /> <br /> ||<br /> 下周：<br /> # 训练国网模型<br /> # 整理工具脚本<br /> ||<br /> <br /> ||<br /> |-<br /> |Shuai Zhang||<br /> 本周：<br /> # asr服务端更换模型<br /> # release x-vector Demo<br /> # vad engine 需求更新修改<br /> ||<br /> 下周：<br /> # vad engine bug修改<br /> # 所有demo更新引擎<br /> # 助残项目计划<br /> ||<br /> <br /> ||<br /> <br /> <br /> |-<br /> |Yanchi Jin||<br /> 本周：<br /> * 完成初版自适应训练平台，lm训练部分<br /> * 支持日本同方vpr server部署<br /> <br /> ||<br /> 下周：<br /> * 持续完成分音塔月度计划<br /> # 中文最终测试集整理，确定<br /> # 第二批次中文100h训练<br /> * 国家电网<br /> # datax第5、6批次标注数据整理<br /> # 数据分发<br /> ||<br /> ||<br /> |-<br /> |Rong Liu||<br /> 上周<br /> 1. 京华合同落地<br /> 2. 需求沟通，友杰智新离线asr和语种识别；海天瑞声的离线asr匹配<br /> 3. 国网项目沟通，结项情况推进；提供维语语音识别相关项目资料<br /> 4. 智能助残demo需求确定，开发对接禹为，初步方案确定<br /> ||<br /> 下周：<br /> 1. 品牌共享+自适应产品推进<br /> 2. 友杰智新需求和合作模式沟通<br /> 3. 智能助残demo完成<br /> ||<br /> <br /> ||<br /> ||<br /> |-<br /> |Dong Wang||<br /> 本周:<br /> # Free宝、债转股、年终奖分配方案，合作公司方案计划确定。<br /> # 《机器学习》引用检查完成 @云麒<br /> # Attention系统设计@蓝天@嘉威<br /> # 基于VAE的说话人特征提取设计<br /> ||<br /> 下周：<br /> #. 基于VAE的说话人特征提取方案确定<br /> #. 语音识别手册：《说话人自适应》和《环境鲁棒性》两章完成<br /> <br /> ||<br /> |-<br /> |Zhiyuan Tang<br /> ||<br /> 上周：<br /> 1. PyTorch/TensorFlow 工具探索和使用；<br /> 2. 歌词生成（古风）进一步落实；<br /> 3. ASR 技术报告准备（delayed）。<br /> ||<br /> 本周：<br /> 1. PyTorch/TensorFlow Speech Recipe 整理上传；<br /> 2. 歌词生成进一步推动；<br /> 3. 技术报告。<br /> <br /> ||<br /> <br /> ||<br /> |-<br /> |Lantian Li||<br /> 上周：<br /> # 完成 Nnet-vad 和 Energy-vad 的训练与对比测试<br /> # 完成九天微联声纹测试（嵌入式）<br /> # 调研当前市场声纹 API 发布情况<br /> # 开展模型压缩测试<br /> ||<br /> 本周：<br /> # Nnet-vad 优化<br /> # 完成模型压缩测试<br /> ||<br /> <br /> ||<br /> |-<br /> |Yating Peng||<br /> 上周：<br /> *财务：参加海淀税务组织的新个税法培训；<br /> *行政：设计制作公司名片、总结会议纪要、续费、报销、合同盖章处理；<br /> <br /> ||<br /> 本周：<br /> *确定年会地点；<br /> *收羽绒服<br /> ||<br /> ||<br /> |-<br /> |Shiying||<br /> 上周：<br /> * local ASR 结果release（am: clean chain model graph: graph1e-5 graph1e-6 graph1e-7 graph1e-9）<br /> * 14000h 中文数据clean up（预计还需要超过一周的时间）<br /> * 与勇哥一起做语音识别小模型探索（目前最小的模型：1.2M 10种命令词识别结果为：8.81%）<br /> ||<br /> 本周<br /> * 继续中文14000h 中文数据clean up<br /> * 语音识别小模型（尽量将模型控制在1M以内）<br /> ||<br /> ||<br /> <br /> |-<br /> |-<br /> |Wenqiang Du ||<br /> 上周:<br /> * 8K数据（9400h）chain模型训练（预计最快还要11天）<br /> * 日语graph以及大小语言模型解码<br /> * 日语的chain模型做区分性训练（问题排查中）<br /> * 16k-8k的数据各项测试（结果持续更新到bugdb上）<br /> ||<br /> 本周：<br /> * 继续8K数据大模型训练<br /> * 16k-8k方法验证<br /> ||<br /> ||<br /> |-<br /> |}</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-12-17 2018-12-17T02:45:32Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-12-17”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-12-10 2018-12-10T03:05:38Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-12-10”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/How_to_install_driver_of_optical-network-card_and_setup_the_ip-config 2018-12-05T12:14:16Z

<p>Zhangzy：</p> <hr /> <div>== Optical port network card plugging ==<br /> # First verify the type of your Optical-network, usually PCI-E (x8/x16).<br /> # Scan the mother-board to check if there are any PCI-E(x8/x16) left, then plug the card to the match one.<br /> # Run &quot;lspci |grep net&quot; to see the numbers of network-card.<br /> # Run &quot;ethtool enp4s0&quot; to check the speed property of the network-card. 1000 or 10000bits/s<br /> <br /> == Driver install(ixgbe) ==<br /> # First check the chip-type and adapter-type of the network-card.<br /> # Go to https://www.intel.com/content/www/us/en/support/products/36773/network-and-i-o/ethernet-products.html to find the corresponding drivers.<br /> # Download the source code of driver and then compile it on your target machine according to the README in src<br /> :# make &amp; make install<br /> :# modinfo ./ixgbe.ko<br /> :# remove ixgbe<br /> :# insmod ixgbe<br /> :# modprobe ixgbe<br /> <br /> == Ip-config settings ==<br /> # Run &quot;systemctl start NetworkManager&quot; and &quot;nmtui&quot; to grab the name of the new card.<br /> # Copy raw electric-network-card config to the optical one.&quot;cp ifcfg-enp0s31f6 ifcfg-enp4s0&quot;<br /> # ifdown the electric-network-card &amp;&amp; ifup optical-network-card<br /> # Restart the network &quot;systemctl restart network&quot;</div>

Zhangzy http://cslt.org/mediawiki/index.php/How_to_install_driver_of_optical-network-card_and_setup_the_ip-config 2018-12-05T12:03:47Z

<p>Zhangzy：</p> <hr /> <div>== Optical port network card plugging ==<br /> # First verify the type of your Optical-network, usually PCI-E (x8/x16).<br /> # Scan the mother-board to check if there are any PCI-E(x8/x16) left, then plug the card to the match one.<br /> # Run &quot;lspci |grep net&quot; to see the numbers of network-card.<br /> <br /> == Driver install(ixgbe) ==<br /> * First check the chip-type and adapter-type of the network-card.<br /> * Go to https://www.intel.com/content/www/us/en/support/products/36773/network-and-i-o/ethernet-products.html to find the corresponding drivers.<br /> * Download the source code of driver and then compile it on your target machine according to the README in src<br /> ** make<br /> ** make install<br /> <br /> #</div>

Zhangzy http://cslt.org/mediawiki/index.php/How_to_install_driver_of_optical-network-card_and_setup_the_ip-config 2018-12-05T12:00:08Z

<p>Zhangzy：</p> <hr /> <div># Optical port network card plugging<br /> * First verify the type of your Optical-network, usually PCI-E (x8/x16).<br /> * Scan the mother-board to check if there are any PCI-E(x8/x16) left, then plug the card to the match one.<br /> # Driver install(ixgbe)<br /> * First check the chip-type and adapter-type of the network-card.<br /> * Go to https://www.intel.com/content/www/us/en/support/products/36773/network-and-i-o/ethernet-products.html to find the corresponding drivers.<br /> * Download the source code of driver and then compile it on your target machine according to the README in src<br /> ** make<br /> ** make install<br /> #</div>

Zhangzy http://cslt.org/mediawiki/index.php/How_to_install_driver_of_optical-network-card_and_setup_the_ip-config 2018-12-05T10:57:56Z

<p>Zhangzy：</p> <hr /> <div># Optical port network card plugging<br /> * First verify the type of your Optical-network, usually PCI-E (x8/x16).<br /> * Scan the mother-board to check if there are any PCI-E(x8/x16) left, then plug the card to the match one.<br /> <br /> # Driver install(ixgbe)<br /> * First check the chip-type and adapter-type of the network-card.<br /> * Go to https://www.intel.com/content/www/us/en/support/products/36773/network-and-i-o/ethernet-products.html to find the corresponding drivers.<br /> * Download the source code of driver and then compile it on your target machine according to the README in src<br /> ** make<br /> ** make install<br /> <br /> #</div>

Zhangzy http://cslt.org/mediawiki/index.php/How_to_install_driver_of_optical-network-card_and_setup_the_ip-config 2018-12-05T10:57:22Z

<p>Zhangzy：以“# Optical port network card plugging * First verify the type of your Optical-network, usually PCI-E (x8/x16). * Scan the mother-board to check if there are any PCI-E...”为内容创建页面</p> <hr /> <div># Optical port network card plugging<br /> * First verify the type of your Optical-network, usually PCI-E (x8/x16).<br /> * Scan the mother-board to check if there are any PCI-E(x8/x16) left, then plug the card to the match one.<br /> <br /> # Driver install(ixgbe)<br /> * First check the chip-type and adapter-type of the network-card.<br /> * Go to https://www.intel.com/content/www/us/en/support/products/36773/network-and-i-o/ethernet-products.html to find the corresponding drivers.<br /> * Download the source code of driver and then compile it on your target machine according to the README in src<br /> ** make<br /> ** make install<br /> <br /> #</div>

Zhangzy http://cslt.org/mediawiki/index.php/Computing 2018-12-05T10:42:14Z

<p>Zhangzy：/* FAQ */</p> <hr /> <div>==General info==<br /> [[Grid coumputing]]<br /> <br /> [[Use CSLT cluster]]<br /> <br /> [[CSLT cluster queues]]<br /> <br /> [[CSLT cluster nodes]]<br /> <br /> [[CSLT Central Storage (CCS)]]<br /> <br /> [[ASR-publication process|CSLT Publication]]<br /> <br /> ==FAQ==<br /> [[Steps of adding a new grid node]]<br /> <br /> [[How to setup SGE]]<br /> <br /> [[How to setup your homepage]]<br /> <br /> [[How to use SSH tunnel to access the campus network]]<br /> <br /> [[How to publish tools,data,code]]<br /> <br /> [[How to access cvss from outside]]<br /> <br /> [[Several service alias you may want to known ]]<br /> <br /> [[How to connect to cvss if the web server fails]]<br /> <br /> [[Using cvs]]<br /> <br /> [[Using neighbour hood browser]]<br /> <br /> [[Access outside from behind firewall using socks5]]<br /> <br /> [[What to do if our website can not access from outside? ]]<br /> <br /> [[How to mount grid disks]]<br /> <br /> [[How to reboot the grid]]<br /> <br /> [[How to repair super blocks]]<br /> <br /> [[How to build a centos-7 node]]<br /> <br /> [[How to setup Samba on centos 7]]<br /> <br /> [[Centos7: ERROR: could not insert 'nvidia': Required key not available]]<br /> <br /> [[Centos7: After reboot the grid, how to reset the NIS]]<br /> <br /> [[Ubuntu: set domain name]]<br /> <br /> [[Ubuntu: set nfs server]]<br /> <br /> [[Centos: config mysql server]]<br /> <br /> [[Look at me when failing to configure service on Linux]]<br /> <br /> [[convert sql to csv]]<br /> <br /> [http://wiki.ubuntu.com.cn/Wiki%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95 How to edit wiki pages?]<br /> <br /> [http://cslt.riit.tsinghua.edu.cn/mediawiki/images/0/02/Github_%E7%AE%80%E6%98%93%E6%95%99%E7%A8%8B_.pdf Github Simple Guide]<br /> <br /> [[How if gird-n can not be found by ping]]<br /> <br /> [[How to reset a pasword for a wiki user]]<br /> <br /> [[How if a machine can not ping outside]]<br /> <br /> [[How to solve the mistmatch between nvidia-smi and driver]]<br /> <br /> [[How to install driver of optical-network-card and setup the ip-config ]]</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-12-03 2018-12-03T01:13:18Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-12-03”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-11-26 2018-11-26T01:25:26Z

<p>Zhangzy：</p> <hr /> <div>This Week:<br /> {| class=&quot;wikitable&quot;<br /> !People !! Last Week !! This Week !! Meet Minutes !! Task Tracing(&lt;font color=&quot;red&quot;&gt;DeadLine&lt;/font&gt;)<br /> |-<br /> |Mengyuan Zhao ||<br /> 本周:<br /> * 工程化<br /> # 完善nnet3-to-nnet1转换工具，加入对StatisticalExtraction、StatisticalPooling的支持<br /> # local VPR engine:<br /> ## 实现了cmvn和PLDA打分，但与kaldi执行结果不同，还需进一步debug。<br /> * 服务器维护<br /> # corpus1创建完成<br /> # 协助之勇修理tiger01<br /> ||<br /> 下周：<br /> * 工程化<br /> # local VPR engine:<br /> ## 继续debug cmvn和plda打分模块。<br /> <br /> ||<br /> <br /> ||<br /> |-<br /> |Zhiyong Zhang||<br /> 本周：<br /> # TTS-海峡研究院特定说话人语音合成--Failed , 需做adaptation，重新合成<br /> # TTS-大规模数据训练--整理数据中<br /> <br /> ||<br /> 下周：<br /> # ASR-decoder重置<br /> # TTS-海峡研究院特定说话人语音合成<br /> # TTS-大规模数据训练<br /> <br /> ||<br /> ||<br /> |-<br /> |Yang Wei ||<br /> 本周：<br /> * 测试vad引擎<br /> * 测试使用tdnn-f chain模型的asr引擎rt<br /> ||<br /> 下周：<br /> * 完成vad引擎测试<br /> * 测试i-vector vpr引擎<br /> <br /> ||<br /> ||<br /> |-<br /> |Zhenlong Han||<br /> 本周：<br /> # 整理项目工具框架<br /> # 跟进国网标注<br /> # 双猴京华项目支持<br /> # 测试分音塔日语识别率<br /> <br /> ||<br /> 下周：<br /> # 分音塔项目<br /> # 国网项目<br /> ||<br /> <br /> ||<br /> |-<br /> |Shuai Zhang||<br /> 本周：<br /> #. vad engine 修改功能需求<br /> #. vpr打包<br /> #. asr服务<br /> ||<br /> 下周：<br /> #. vad engine<br /> #. asr服务压测<br /> ||<br /> <br /> ||<br /> <br /> <br /> |-<br /> |Yanchi Jin||<br /> 本周：<br /> # 支持roobo语音识别项目，更新v3.6模型。<br /> # 评估分音塔标准测试集<br /> # 分析国网训练模型提升效果<br /> ||<br /> 下周：<br /> # 优化分音塔日语识别模型<br /> ||<br /> ||<br /> |-<br /> |Rong Liu||<br /> 上周<br /> 1. 黄淮学院AI实验室落地沟通，由于内部问题，沟通进度不如预期<br /> 2. 秒针费用结算流程及后续合作方式<br /> 3. 协助roobo、分音塔、国网项目状态沟通和推进<br /> ||<br /> 下周：<br /> 1. 继续推进黄淮学院AI资源落地<br /> 2. Roobo、分音塔和国网项目状态推进<br /> 3. roobo专利<br /> 4. 其它前期项目需求沟通<br /> ||<br /> <br /> ||<br /> ||<br /> |-<br /> |Dong Wang||<br /> 本周:<br /> #. 实习生课题讨论部分完成<br /> #. 入台证办理（失败）<br /> #. 日本演示、DataX进展等相关项目讨论<br /> ||<br /> 下周：<br /> #. 完成实习生课题声纹识别部分讨论<br /> #. BP讨论<br /> #. 研讨阿汤提出的品牌共享计划<br /> ||<br /> |-<br /> |Zhiyuan Tang<br /> ||<br /> 上周：<br /> Attended APSIPA.<br /> ||<br /> 本周：<br /> 1. Deep compression 调研与实现.<br /> 2. pair-wise 后端设计.<br /> <br /> ||<br /> <br /> ||<br /> |-<br /> |Lantian Li||<br /> 上周：<br /> # 支持秒针声纹项目 @zs<br /> # 完成 i/d/x-vector 的 CMN 验证<br /> # 支持 @zmy x-vector 工程化<br /> # 开展若干声纹产品原型设计<br /> # 组织实习生学习讨论<br /> ||<br /> 本周：<br /> # 开启 d-x-vector 模型的 串行训练<br /> # 尝试 xi-vector 模型实现<br /> ||<br /> <br /> ||<br /> |-<br /> |Yating Peng||<br /> 上周：<br /> *看政府2019年科技计划和基金，可报2019年中关村国家自主创新示范区科技型小微企业研发费用支持资金和国际合作研发项目，在找性价比高的可靠代理中，希望本周能确定下来；<br /> *去社区开租房发票；<br /> *日常财务报销处理。<br /> <br /> <br /> ||<br /> 本周：<br /> *准备政府资金支持材料；<br /> *完善员工档案excel；<br /> <br /> ||<br /> ||<br /> |-<br /> |Shiying||<br /> 上周：<br /> * 参加APSIPA 2018会议，两份口头报告<br /> <br /> ||<br /> 本周<br /> * local ASR model （context 为3的倍数的ASR model）<br /> * ASR model 综合测试<br /> ||<br /> ||<br /> <br /> |-<br /> |-<br /> |Wenqiang Du ||<br /> 上周:<br /> * roobo口语打分模型训练<br /> * 日语NHK新闻数据重新加入训练<br /> * 用新训练的8k模型对16K转8K数据做自适应<br /> <br /> ||<br /> 本周：<br /> * 对roobo口语模型进行多组实验<br /> * 16k转8k新模型训练<br /> <br /> ||<br /> ||<br /> |-<br /> |}</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-11-19 2018-11-19T01:04:19Z

<p>Zhangzy：</p> <hr /> <div>This Week:<br /> {| class=&quot;wikitable&quot;<br /> !People !! Last Week !! This Week !! Meet Minutes !! Task Tracing(&lt;font color=&quot;red&quot;&gt;DeadLine&lt;/font&gt;)<br /> |-<br /> |Mengyuan Zhao ||<br /> 本周:<br /> * 工程化<br /> # 完成ivector-based声纹识别引擎开发<br /> * 服务器维护<br /> # 备份了/freeneb/release目录<br /> * 项目<br /> # roobo口语打分<br /> ## 按照roobo的需求，增加了输出phone串，和phone级别打分的接口函数，更新了word级别打分的算法。<br /> ||<br /> 下周：<br /> * 工程化<br /> # local VPR engine:<br /> ## 实现StatisticalPooling component，以实现对x-vector的支持<br /> <br /> ||<br /> <br /> ||<br /> |-<br /> |Zhiyong Zhang||<br /> 本周：<br /> # TTS-化学论文摘要合成<br /> # release目录model整理<br /> # TTS-前端/模型训练调研<br /> <br /> ||<br /> 下周：<br /> # ASR-decoder重置<br /> # TTS-海峡研究院特定说话人语音合成<br /> # TTS-大规模数据训练<br /> <br /> ||<br /> ||<br /> |-<br /> |Yang Wei ||<br /> 本周：<br /> * vad引擎部分测试<br /> ||<br /> 下周：<br /> * 完成vad引擎测试<br /> * TDNN-F chain 模型rt测试<br /> <br /> ||<br /> ||<br /> |-<br /> |Zhenlong Han||<br /> 本周：<br /> # 再测试汽车之家<br /> # 整理秒针数据完成，正在训练完成<br /> # 跟进国网标注，分析训练问题<br /> # 双猴京华项目支持<br /> # 马老师本地识别项目支持<br /> ||<br /> 下周：<br /> # 发布秒针模型<br /> # 分音塔项目<br /> # 国网项目<br /> ||<br /> <br /> ||<br /> |-<br /> |Shuai Zhang||<br /> 本周：<br /> #. vad engine 修改功能需求<br /> ||<br /> 下周：<br /> #. vad engine<br /> ||<br /> <br /> ||<br /> <br /> <br /> |-<br /> |Yanchi Jin||<br /> 本周：<br /> # 处理训练国网数据。<br /> # 支持预演项目。<br /> ||<br /> 下周：<br /> # 整理所有项目测试集<br /> ||<br /> ||<br /> |-<br /> |Rong Liu||<br /> 上周<br /> 1. 黄淮学院AI实验室沟通，待细节协议确定<br /> 2. 国网数据结构化分析，解析出客服（1.6k）、客户(9.7W)、及对应地区（分布）标签，可用于声纹<br /> 3. 配合誉为科技windows离线输入法联调，基本完成<br /> ||<br /> 下周：<br /> 1. 推动黄淮学院AI实验室协议确定，启动招标流程<br /> 2. 京华电子合同签订<br /> 3. 嵌入式语音产品调研<br /> ||<br /> <br /> ||<br /> ||<br /> |-<br /> |Dong Wang||<br /> 本周:<br /> #. ML book通过出版社审查，准备签定合同。<br /> #. ICASSP论文提交。<br /> #. 黄淮学院AI实验室进展顺利。<br /> #. DataX场所、启动资金、数据库采集方案等完成，近期开始采集声纹。<br /> #. DataX代FreeNeb收集文本和网上数据。<br /> ||<br /> 下周：<br /> #. 参加ICASSP会议<br /> ||<br /> |-<br /> |Zhiyuan Tang<br /> ||<br /> 上周：<br /> 1. 口语打分交付计划，及 phone/word 参考 likelihood 生成；<br /> 2. ICASSP 论文查写；<br /> 3. FreeNeb Logo 设计与整理。<br /> ||<br /> 本周：<br /> 1. 模型压缩方法调研与实现；<br /> 2. pair-wise 后端设计<br /> ||<br /> <br /> ||<br /> |-<br /> |Lantian Li||<br /> 上周：<br /> # 完成 ICASSP 论文<br /> # 完成 x-vector 模型的 解码调参（chunk_size）<br /> # 完成 d-vector 模型的 训练调参（nnet_structure, dropout, batch_size）<br /> # 跟进 声纹明星-微信小程序<br /> # 支持 秒针声纹项目<br /> ||<br /> 本周：<br /> # 开启 d-x-vector 模型的 串行训练<br /> # 尝试 xi-vector 模型实现<br /> # 阅览 ICASSP18 论文<br /> ||<br /> <br /> ||<br /> |-<br /> |Yating Peng||<br /> 上周：<br /> *汇总十月账，做账，报税，发工资；<br /> *公司布置；<br /> <br /> <br /> ||<br /> 本周：<br /> *准备19年政府资金支持项目，物色合适代理；<br /> *去社区开租房发票，继续完善公司布置；<br /> *日常财务报销。<br /> ||<br /> ||<br /> |-<br /> |Shiying||<br /> 上周：<br /> * 汉语大模型noise training(no skip)<br /> * 汉语大模型( clean skip)<br /> * 熟悉fnscore代码<br /> * 完善汉语模型release<br /> * rnnlm<br /> ||<br /> 本周<br /> * 启动汉语rnnlm训练<br /> * 继续熟悉fnscore代码<br /> ||<br /> ||<br /> <br /> |-<br /> |-<br /> |Wenqiang Du ||<br /> 上周:<br /> * 中文8K训练（iter=2900）共计4260<br /> * 日语项目的配合<br /> * 实习生demo的整理，文档整理<br /> <br /> ||<br /> 本周：<br /> * 继续8K中文模型训练<br /> * 实习生文档整理<br /> ||<br /> ||<br /> |-<br /> |}</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-11-12 2018-11-12T01:36:33Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-11-12”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-11-05 2018-11-05T01:23:57Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-11-05”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-29 2018-10-29T01:30:56Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-29”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-29 2018-10-29T01:28:02Z

<p>Zhangzy：保护“FreeNeb status Report 2018-10-29”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-29”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-22 2018-10-22T01:19:16Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-22”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-15 2018-10-15T01:46:06Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-15”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-15 2018-10-15T01:44:29Z

<p>Zhangzy：保护“FreeNeb status Report 2018-10-15”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-15”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-10-08 2018-10-08T01:47:33Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-10-08”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-25 2018-09-25T00:58:43Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-25”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-25 2018-09-25T00:58:10Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-25”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-17 2018-09-17T01:24:43Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-17”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/Annual-report-2015 2018-09-12T02:39:34Z

<p>Zhangzy：</p> <hr /> <div>[[媒体文件:2015年 梦想的步伐.pptx|Wang Dong: Towards the future]]<br /> <br /> [[媒体文件:2016-01-10 Tianyi Luo's language technology research group annual report.pdf| Luo Tianyi: Language processing team annual report]]<br /> <br /> [[媒体文件:20160110_asr_annual_summary.pdf|Zhang Zhiyong: 2016 ASR group Annual Summary]]<br /> <br /> [[媒体文件:2015年 总结 财务报告 zhangxw.ppt|Zhang Xuewei]]<br /> <br /> [[媒体文件:Zengxy.pptx|Zeng Xiangyu]]<br /> <br /> [[媒体文件:Zhaomy-summary of 2015.pptx|Zhao Mengyuan]]<br /> <br /> [[媒体文件:2015-Lilt's_Annual_Summary.pdf|Lantian Li: Speaker recognition]]</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-10 2018-09-10T01:45:35Z

<p>Zhangzy：保护“FreeNeb status Report 2018-09-10”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-10”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-10 2018-09-10T01:45:17Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-10”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-08-27 2018-09-03T01:23:40Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-08-27”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-03 2018-09-03T01:23:00Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-03”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-03 2018-09-03T01:12:49Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-03”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-08-27 2018-09-03T01:05:34Z

<p>Zhangzy：保护“FreeNeb status Report 2018-08-27”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-08-27”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-09-03 2018-09-03T01:05:18Z

<p>Zhangzy：保护“FreeNeb status Report 2018-09-03”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-09-03”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_status_Report_2018-08-20 2018-08-20T05:36:50Z

<p>Zhangzy：</p> <hr /> <div>'''警告：'''“FreeNeb status Report 2018-08-20”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_Status_Report_2016-12-19 2018-08-14T04:54:15Z

<p>Zhangzy：保护“FreeNeb Status Report 2016-12-19”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb Status Report 2016-12-19”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy http://cslt.org/mediawiki/index.php/FreeNeb_Status_Report_2016-12-26 2018-08-14T04:54:08Z

<p>Zhangzy：保护“FreeNeb Status Report 2016-12-26”（[编辑=FreeNeb users]（无限期）[移动=FreeNeb users]（无限期）[Read=FreeNeb users]（无限期））</p> <hr /> <div>'''警告：'''“FreeNeb Status Report 2016-12-26”指向这里，但您没有足够的权限来访问它。</div>

Zhangzy