Introduction

This paper presented a speech information factorization method based on a novel deep generative model that we called factorial discriminative normalization flow.

Qualitative and quantitative experimental results show that compared to all other models, the proposed factorial DNF can retain the class structure corresponding to multiple information factors, and changing one factor will cause little distortion on other factors. This demonstrates that factorial DNF can well factorize speech signal into different information factors.

Members

Haoran Sun, Lantian Li, Yunqi Cai, Yang Zhang, Thomas Fang Zheng, Dong Wang

Publications

Haoran Sun, Lantian Li, Yunqi Cai, Yang Zhang, Thomas Fang Zheng, Dong Wang, "Deep Generative Factorization For Speech Signal", 2020. pdf

Source Code

xxx

Factorial DNF

xxx

Experiments

Data

xx

Encoding

xx

Factor manipulation

Sampling: figs wavs (right click)

                        Phone Manipulation
 Model | p(q₂|x)  | p(q₂|x') |  δ(q₂)  ||  p(s|x) |  p(s|x') | δ(s)
  VAE  |   0.013  |   0.312  |   0.299  ||  0.612  |  0.454   |  -0.158 
  NF   |   0.013  |   0.410  |   0.397  ||  0.612  |  0.489   |  -0.123 
  DNF  |   0.013  |   0.619  |   0.606  ||  0.612  |  0.335   |  -0.277  
 f-DNF |   0.013  |   0.636  |   0.623  ||  0.612  |  0.536   |  -0.076

Future Work

Test factorial DNF on larger datasets.
Establish general theories for deep generative factorization.

Deep Generative Factorization For Speech Signal(ICASSP21)

目录