“2013-07-22”版本间的差异

2013年7月22日 (一) 12:38的最后版本

Data sharing

• LM count files still undelivered!

DNN progress

Experiments

• Sparse DNN on the ARM board

1200-1200-1200-3536                 1200-1200-1200-3536-sparse0.3 (sparsity 1/5)
original atlas:  RT 2.3                         RT 2.3
atlas sparse:    RT 54                          RT 14  
NIST smatmat:    RT 27.3                        RT 5.98

800-800-800-2108                    800-800-800-2108-sparse0.3 (sparsity 2/5):
original atlas: RT 1.1                          RT 1.1
NIST smatmat:   RT 11.9                         RT 5.5

Conclusions:
1. the atlas works well for both non-sparse and sparse.
2. sparsity does not work if the sparsity rate is low. It looks the sparsity computing can 
outperform the non-sparsity computing only if the sparsity rate is higher than 1/15.
3. In another words, to employ sparsity, the cost that first should be taken is the error rate 
increase with the 1/15 compression.
4. The sparse approach seems more useful for storage: if the sparsity is higher than 1/2, then the 
storage of CSR/CSC will start to save storage.
5. Possibly unit-based sparsity instead of weight sparsity.

Tencent exps

GPU & CPU merge

1. Hold

Embedded progress

• Tested various PS models:

ID            model        feature    WER       RT          storage

semi_10000    semi HMM    s2-4x    6.30%       0.80          10.2M
semi_5000     semi HMM    s2-4x    6.70%       0.74          5.2M
semi_5000     semi HMM    1c-d-dd  9.11%       0.91          1.3M
ptm_5000      PTM HMM     s2-4x    6.47%       2.15          1.3M

So there is not a perfect which wins in terms all the criteria. Looks like semi-5000 is an acceptable trade-off.