Welcome to Yi Yu's Homepage

Yi YU Ph.D.

Associate Professor (Researchgate Google Scholar)

DBLP (Yi Yu 0001) https://dblp.org/pid/99/111-1.html

Graduate School of Advanced Science and Engineering, Hiroshima University, Japan

yiyu at hiroshima-u.ac.jp

Yi Yu is an associate professor with Graduate School of Advanced Science and Engineering and Education and Research Center for Artificial Intelligence and Data Innovation, Hiroshima University. She was an assistant professor with National Institute of Informatics and a senior research fellow with School of Computing, National University of Singapore. She has been working with lots of researchers (see students, collaborators, and Photo2＠2023 and photo1＠2019) with different research backgrounds. Her research interests focus on multimedia content understanding and artificial intelligence, particularly in multimodal representation learning, generative modeling, and multimodal information fusion for visual, audio, speech, language, and music information (see ongoing works). She and her team received best poster award from IEEE ISM 2018 and best paper runner up in APWeb-WAIM 2017, were recognized as finalist of the World’s FIRST 10K best paper award in ICME17, won the 2^nd prize in Yahoo Flickr grand challenge 2015, were in the top winners (out of 29 teams) awarded by ACM SIGSPATIAL GIS Cup 2013, and received a best paper award from IEEE ISM 2012. She is a co-chair @ IEEE ICDM19 Ph.D Forum and IEEE ICDM18 Ph.D Forum, an associate editor of IEEE Transactions on Multimedia.

Source Codes and Data

JustLMD for dance generation from music and lyrics

https://github.com/yy1lab/LMD

Neural melody generation from lyrics

https://github.com/yy1lab/Lyrics-Conditioned-Neural-Melody-Generation

Selected Publications by years (Full)

1. Zhe Zhang, Yi Yu*, and Atsuhiro Takasu, “Controllable syllable-level lyrics generation from melody with prior attention,” accepted by IEEE Transactions on Multimedia, https://ieeexplore.ieee.org/document/10637751, DOI: 10.1109/TMM.2024.3443664, 2024.

2. Zehang Lin, Jiayuan Xie, Zhenguo Yang, Yi Yu, and Qing Li, “Generalized news event discovery via dynamic augmentation and entropy optimization,” accepted by ACM Multimedia (MM), 2024.

3. Shuai Yu, Xiaoliang He, Ke Chen, Yi Yu*, “HKDSME: Heterogeneous knowledge distillation for semi-supervised singing melody extraction using harmonic supervision,” accepted by ACM Multimedia (MM), 2024.

4. Haonan Jin, Guangwei Gao, Juncheng Li, Zhenhua Guo, and Yi Yu, “Efficient Dual-branch Information Interaction Network for Lightweight Image Super-Resolution”, accepted by IEEE Transactions on Instrumentation and Measurement.

5. Donghuo Zeng, Yanan Wang, Kazushi Ikeda, and Yi Yu, “Anchor-aware deep metric learning for audio-visual retrieval,” ACM International Conference on Multimedia Retrieval (ICMR), 2024, pp.211-219. [CORE B]

6. Zhe Zhang, Karol Lasocki, Yi Yu*, Atsuhiro Takasu, “Syllable-level lyrics generation from melody exploiting character-level language model,” accepted by European Chapter of the Association for Computational Linguistics (EACL) 2024.

7. Wenjie Yin, Yi Yu*, Hang Yin, Dannica Kragic, and Mårten Björkman, “Scalable motion style transfer with constrained diffusion generation,” accepted by The Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI) 2024. [CORE A*]

8. Yang, Zhuopan, Zhenguo Yang, Xiaoping Li, Yi Yu, Qing Li, and Wenyin Liu “A Progressive Placeholder Learning Network for Multimodal Zero-Shot Learning”, accepted by IEEE Transactions on Multimedia, 2024. IF: 6.513

9. Yiming Tang, Yi Yu, and Yan Qiu Chen “Prototype-guided prior enhancement and rectification in few-shot semantic segmentation”, accepted by IEEE Conference on Multimedia Expo (ICME) 2024.

10. Yulun Wu, Weixing Wei, Dichucheng Li, Mengbo Li, Yongwei Gao, Yi Yu, and Wei Li “Harmonic frequency-separable Transformer for instrument-agnostic music transcription”, accepted by IEEE Conference on Multimedia Expo (ICME) 2024.

11. Chenyang Lyu, Wenxi Li, Tianbo Ji, Yi Yu, Longyue Wang, “Semantic enrichment for video question answering with gated graph neural networks,” accepted by IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024. [CORE B]

12. Shuai Yu, Jun Liu, Yi Yu, Wei Li, “A scalable sparse transformer model for singing melody extraction,” accepted by IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024. [CORE B]

13. Chenyang Lyu, Wenxi Li, Tianbo Ji, Longyue Wang, Liting Zhou, Cathal Gurrin, Linyi Yang, Yi Yu, Yvette Graham, and Jennifer Foster, “Graph-based video-language learning with multi-grained audio-visual alignment,” ACM Multimedia (MM), 2023. [CORE A*]

14. Yulun Wu, Jiahao Zhao, Yi Yu, and Wei Li “MFAE: Masked frame-level features autoencoder with hybrid-supervision for low-resource music Transcription,” IEEE International Conference on Multimedia and Expo (ICME), 2023, pp.1109-1114.

15. Xinlu Liu, Jiale Qian, Qiqi He, Yi Yu, and Wei Li, “LC-Beating: An online system for beat and downbeat tracking using latency-controlled mechanism,” IEEE International Conference on Multimedia and Expo (ICME), 2023, pp. 1098-1103.

16. DichuCheng Li, Mingjin Che, Wenwu Meng, Yulun Wu, Yi Yu, Fan Xi, and Wei Li, “Frame-level multi-label playing technique detection using multi-scale network and self-attention mechanism,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp.1-5. [CORE B]

17. Wei Duan, Yi Yu*, and Keizo Oyama, “Semantic dependency network for lyrics generation from melody”, accepted by Neural Computing and Applications, 2023.

18. Jiwei Zhang, Yi Yu*, Suhua Tang, Wei Li, and Jianming Wu, “Multi-scale network with shared cross attention for audio-visual correlation learning,” https://rdcu.be/dhgSH, Journal of Neural Computing and Applications, 2023.

19. Zhe Zhang, Yi Yu*, and Astsuhiro Takasu, “Controllable lyrics-to-melody generation,” https://rdcu.be/dgozv, Journal of Neural Computing and Applications, 2023.

20. Jiale Qian, Xinlu Liu, Yi Yu, and Wei Li, “Stripe-Transformer: deep stripe feature learning for music source separation”, EURASIP Journal on Audio, Speech, and Music Processing, 2023:2, https://doi.org/10.1186/s13636-022-00268-1. IF: 2.114

21. Wei Duan, Yi Yu*, Xulong Zhang, Suhua Tang, Wei Li, and Keizo Oyama, “Melody Generation from Lyrics with Local Interpretability” accepted by ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 2022. IF: 3.275, JCR Q1

22. Jiwei Zhang, Yi Yu*, Suhua Tang, Jianming Wu, and Wei Li, “Variational Autoencoder with CCA for Audio-Visual Cross-Modal Retrieval,” accepted by ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 2022. IF: 3.275

23. Yi Yu, Zhe Zhang, Wei Duan, Abhishek Srivastava, Rajiv Shah, and Yi Ren, “Conditional Hybrid GAN for Melody Generation from Lyrics,” Journal of Neural Computing and Applications, https://rdcu.be/cXazS , doi: 10.1007/s00521-022-07863-5, 2022. IF: 5.102

24. Donghuo Zeng, Jianming Wu, Gen Hattori, Rong Xu, and Yi Yu* “Learning Explicit and Implicit Dual Common Subspaces for Audio-Visual Cross-Modal Retrieval,” accepted by ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), https://dl.acm.org/doi/10.1145/3564608, 2022. IF: 3.275

25. Guangwei Gao, Yi Yu*, Huimin Lu, Jian Yang, and Dong Yue, “Context-Patch Representation Learning with Adaptive Neighbor Embedding for Robust Face Image Super-resolution,” IEEE Transactions on Multimedia, 10.1109/TMM.2022.3192769, 2022. IF: 6.513

26. Shuai Yu, Yi Yu*, and Wei Li, “A Neural Harmonic-Aware Network with Gated Attentive Fusion for Singing Melody Extraction,” accepted by Journal of Neurocomputing, 2022. IF: 4.072

27. Yiming Tang and Yi Yu, “Query-Guided Prototype Learning with Decoder Alignment and Dynamic Fusion in Few-Shot Segmentation,” accepted by ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), https://doi.org/10.1145/3555314, 2022. IF: 3.275

28. Guangwei Gao, Guoan Xu, Juncheng Li, Yi Yu*, Huimin Lu and Jian Yang “FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation,” IEEE Transactions on Multimedia, 10.1109/TMM.2022.3157995, 2022. IF: 6.513

29. Dawei Zhou, Nannan Wang, Chunlei Peng, Yi Yu, Xi Yang, and Xinbo Gao “Towards Multi-domain Face Synthesis via Domain-Invariant Representations and Multi-level Feature Parts,” accepted by IEEE Transactions on Multimedia, 2021. [CORE A*], IF: 5.452

30. Guangwei Gao, Guoan Xu, Yi Yu, Jin Xie, Jian Yang, and Dong Yue. “MSCFNet: A Lightweight Network with Multi-Scale Context Fusion for Real-Time Semantic Segmentation,” accepted by IEEE Transactions on Intelligent Transportation Systems, 2021. IF: 6.49

31. Shuai Yu, Yi Yu, Xi Chen, and Wei Li, “HANME: Hierarchical attention network for singing melody extraction,” IEEE Signal Processing Letters, vol. 28, pp.1006-1010, 2021. IF:4.18

32. Yi Yu, Abhishek Srivastava, and Simon Canales, “Conditional LSTM-GAN for Melody Generation from Lyrics,” https://dl.acm.org/doi/fullHtml/10.1145/3424116, ACM Transaction on Multimedia Computing Communication and Applications (TOMCCAP), 2020. IF: 3.275

33. Guangwei Gao, Yi Yu, Jian Yang, Guo-Jun Qi, and Meng Yang, “Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition,” 10.1109/TCSVT.2020.3042178, IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2020. IF: 4.133

34. Guangwei Gao, Dong Zhu, Huimin Lu, Yi Yu, and Dong Yue, “Robust Facial Image Super-Resolution by Kernel Locality-Constrained Coupled-Layer Regression,” accepted by ACM Transactions on Internet Technology (TOIT), 2020. IF: 3.750

35. Guangwei Gao, Yi Yu, Jin Xie, Jian Yang, Meng Yang, and Jian Zhang, “Constructing Multilayer Locality-Constrained Matrix Regression Framework for Noise Robust Face Super-Resolution,” https://doi.org/10.1016/j.patcog.2020.107539, journal of Pattern Recognition, 2020. IF: 7.196

36. Weijian Ruan, Chao Liang, Yi Yu, and Zheng Wang, “Correlation Discrepancy Insight Network for Video Re-identification,” accepted by ACM Transaction on Multimedia Computing Communication and Applications (TOMCCAP), 2020. IF: 3.275

37. Xulong Zhang, Yi Yu, Yongwei Gao, Xi Chen, and Wei Li, “Research on Singing Voice Detection Based on a Long-term Recurrent Convolutional Network with Vocal Separation and Temporal Smoothing,” accepted by Electronics in MDPI open access journal, 2020. IF:2.41

38. Guangwei Gao, Yi Yu, Meng Yang, Pu Huang, Qi Ge, and Dong Yue, “Multi-scale Patch based Representation Feature Learning for Low-Resolution Face Recognition,” Journal of applied Soft Computing, https://doi.org/10.1016/j.asoc.2020.106183. IF: 4.873

39. Hangyu Li, Nannan Wang, Yi Yu, Xi Yang, and Xinbo Gao, “LBAN-IL: A Novel Method of High Discriminative Representation for Facial Expression Recognition,” V.432, pp.159-169, Journal of Neurocomputing, 2020. IF: 4.072

40. Donghuo Zeng, Yi Yu, and Keizo Oyama, “Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-Modal Retrieval,” ACM Transaction on Multimedia Computing Communication and Applications (TOMCCAP), Vol.16, No. 3, article76, pp.1-23, 2020. IF: 3.275

41. Xinrui Jiang, Nannan Wang, Jingwei Xin, and Xi Yang, Yi Yu, Xinbo Gao, “Image Super-Resolution via Multi-view Information Fusion Networks”, V.402: pp.29-37, Journal of Neurocomputing, 2020. IF: 4.072

42. Guangwei Gao, Yi Yu, Meng Yang, Heyou Chang, Pu Huang, and Dong Yue, “Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression,” V.506: pp.19-36, Journal of Information Sciences, 2020. IF: 5.524

43. Weijian Ruan, Chao Liang, Yi Yu, Jun Chen, and Ruimin Hu, “Online Scale-Adaptive Object Tracking with Stepwise Insight”, V.384, pp.200-212, Journal of Neurocomputing, 2020. IF: 4.072

44. Yi Yu, Suhua Tang, Kiyoharu Aizawa, and Akiko Aizawa, “Category-based deep CCA for fine-grained venue discovery from multimodal data,” IEEE Transaction on Neural Network and Learning System (TNNLS), Vol. 30, No.4, pp.1250-1258, 2019. [CORE A*], IF: 11.683

45. Yi Yu, Suhua Tang, Francisco Raposo, and Lei Chen, “Cross-modal correlation learning for audio and lyrics in music retrieval,” ACM Transaction on Multimedia Computing Communication and Applications (TOMCCAP), Vol.15, No. 1, article 20, pp.1-16, 2019. IF: 2.25

46. Junjun Jiang, Yi Yu, Zheng Wang, Suhua Tang, and Ruimin Hu, “Ensemble super-resolution with a reference dataset,” IEEE Transactions on Cybernetics, pp.1-15, 2019. [CORE A], IF: 11.47

47. Junjun Jiang, Yi Yu, Zheng Wang, Xianming Liu, and Jiayi Ma, “Graph-regularized locality-constrained joint dictionary and residual learning for face sketch synthesis,” IEEE Transactions on Image Processing, Vol. 28, No. 2, pp.628-641, 2019. [CORE A*], IF: 6.79

48. Zheng Wang, Junjun Jiang, Yi Yu, and Shin'ichi Satoh, “Incremental re-identification by cross-direction and cross-ranking adaption,” IEEE Transactions on Multimedia, Vol.21, No. 9, pp.2376-2386, 2019. [CORE A*], IF: 5.452　

49. Junjun Jiang, Jiayi Ma, Suhua Tang, Yi Yu, and Kiyoharu Aizawa, “Face hallucination through differential evolution parameter map learning with facial structure prior,” Journal of Information Sciences, Vol. 481, pp.174-188, 2019. IF: 5.524

50. Junjun Jiang, Yi Yu, Suhua Tang, Jiayi Ma, Akiko Aizawa, and Kiyoharu Aizawa, “Context-patch face hallucination based on thresholding locality-constrained representation and reproducing learning,” IEEE Transactions on Cybernetics, pp.1-14, 2019. [CORE A], IF: 11.47

51. Zheng Wang, Ruimin Hu, Chen Chen, Yi Yu, Junjun Jiang, Chao Liang, and Shin’ichi Satoh, “Person re-identiﬁcation via discrepancy matrix and matrix metric,” IEEE Transactions on Cybernetics, Vol. 48, No.10, pp.3006-3020, 2018. [CORE A], IF: 11.47

52. Zhifeng Bao, Yi Yu, Jian Shen, and Zhangjie Fu, “A query refinement framework for XML keyword search,” Journal of World Wide Web, 2017, Vol. 20, Issue 6, pp.1469-1505. [CORE A], IF: 1.77

53. Junjun Jiang, Chen Chen, Yi Yu, Xinwei Jiang, and Jiayi Ma, “Spatial-aware collaborative representation for hyperspectral remote sensing image classification,” IEEE Geoscience and Remote Sensing Letters, Vol.14, No.3, pp. 404-408, 2017. IF: 3.534

54. Mang Ye, Chao Liang, Yi Yu, and Zheng Wang, “Person re-identification via ranking aggregation of similarity pulling and dissimilarity pushing,” IEEE Transactions on Multimedia, Vol. 18, No. 12, pp.2553-2566, 2016. [CORE A*], IF: 5.452　

55. Rajiv Ratn Shah, Yi Yu, Akshay Verma, Suhua Tang, Anwar Dilawar Shaikhe, and Roger Zimmermann, “Leveraging multimodal information for event summarization and concept-level sentiment analysis,” Journal of Knowledge-Based Systems, Vol.108, pp.102-109, 2016. IF: 5.101

56. Zheng Wang, Ruimin Hu, Chao Liang, Yi Yu, Junjun Jiang, Many Ye, Jun Chen, and Qingming Leng, “Zero-shot person re-identification visa cross-view consistency,” IEEE Transaction on Multimedia, Vol. 18, No. 2, pp.260-272, 2016. [CORE A*], IF: 5.452

57. Yifang Yin, Yi Yu, and Roger Zimmermann, “On generating content-oriented geo features for sensor-rich outdoor video search,” IEEE Transaction on Multimedia, Vol. 17, No. 10, pp.1760-1772, 2015. [CORE A*], IF: 5.452

58. Suhua Tang, Yi Yu, Roger Zimmermann, and Sadao Obana, “Efficient geo-fencing via hybrid hashing: A combination of bucket selection and in-bucket binary search,” ACM Transactions on Spatial Algorithms and Systems, Volume 1, Issue 2, Article No. 5, 2015. IF: 1.69

59. Luming Zhang, Yi Yang, Yue Gao, Yi Yu, Changbo Wang, and Xuelong Li, “A probabilistic Associative Model for Segmenting Weakly Supervised Images,” IEEE Transaction on Image Processing, Vol. 23, No. 9, pp.4150-4159, 2014. [CORE A*], IF: 6.79

60. Yi Yu, Roger Zimmermann, Ye Wang, and Vincent Oria, “Scalable content-based music retrieval using chord progression histogram and tree-structure LSH,” IEEE Transaction on Multimedia, Vol. 15, No. 8, pp.1969-1981, 2013. [CORE A*], IF: 5.452

61. Zhonghua Li, Bingjun Zhang, Yi Yu, Jialie Shen, and Ye Wang, “Query-document-dependent fusion: A case study of multimodal music retrieval,” IEEE Transaction on Multimedia, Vol. 15, No. 8, pp.1830-1942, 2013. [CORE A*], IF: 5.452 　

62. Weixing Wei, Peilin Li, Yi Yu, and Wei Li, “HPPNet: Modeling the Harmonic Structure and Pitch Invariance in Piano Transcription,” accepted by International Society for Music Information Retrieval Conference (ISMIR), 2022.

63. Dichucheng Li, Yulun Wu, Qinyu Li, Jiahao Zhao, Yi Yu, Fan Xia, and Wei Li, “Playing Technique Detection by Fusing Note Onset Information in Guzheng Performance,” accepted by International Society for Music Information Retrieval Conference (ISMIR), 2022.

64. Guangwei Gao, Zhengxue Wang, Juncheng Li, Wenjie Li, Yi Yu, and Tieyong Zeng, “Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer,” accepted by International Joint Conference on Artificial Intelligence (IJCAI) 2022.

65. Weixing Wei,Peilin Li, Yi Yu, and Wei Li, “HarmoF0: Logarithmic Scale Dilated Convolution for Pitch Estimation,” accepted by IEEE International Conference on Multimedia and Expo (ICME), 2022.

66. Jiahao Zhao, Ganghui Ru, Dichucheng Li, Yi Yu, and Wei Li, “Multimodal Music Emotion Recognition with Hierarchical Cross-Modal Attention Network,” accepted by IEEE International Conference on Multimedia and Expo (ICME), 2022.

67. Qiqi He, Xiaoheng Sun, Yi Yu, and Wei Li, “DEEPCHORUS: A Hybrid Model of Multi-scale Convolution and Self-attention for Chorus Detection,” accepted by IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

68. Guangwei Gao, Wnejie Li, Juncheng Li, Fei Wu, Huimin Lu, Yi Yu, “Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution,” accepted by AAAI 2022. [CORE A*]

69. Xingcai Wu, Yucheng Xie, Jiaqi Zeng, Zhenguo Yang, Yi Yu, Qing Li, Wenyin Liu, “Adversarial learning with mask reconstruction for text-guided image inpainting,” accepted by ACM MM 2021. [CORE A*]

70. Abhishek Srivastava, Wei Duan, Rajiv Ratn Shah, Jianming Wu, Suhua Tang, Wei Li, and Yi Yu, “Melody Generation from Lyrics Using Three Branch Conditional LSTM-GAN,” 28th International Conference on Multimedia Modeling (MMM), 2022.

71. Xuejiao Tang, Wenbin Zhang, Yi Yu, Kea Turner, Tyler Derr, Mengyu Wang, “Interpretable visual understanding with cognitive attention network,” International Conference on Artificial Neural Networks (ICANN) 2021.

72. Zhengxue Wang, Guangwei Gao, Juncheng Li, Yi Yu, and Huimin Lu, “Lightweight Image Super-Resolution with Multi-scale Feature Interaction Network”, accepted by IEEE ICME 2021. [CORE B]

73. Xulong Zhang, Jiale Qian, Yi Yu, Yifu Sun, and Wei Li, “Singer identification using deep timbre feature learning with KNN-net,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.3380-3384, 2021.

74. Shuai Yu, Xiaoheng Sun, Yi Yu, and Wei Li, “Frequency-temporal attention network for singing melody extraction,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp.251-255.

75. Hemant Yadav, Sreyan Ghosh, Yi Yu, and Rajiv Ratn Shah, “End-to-end Named Entity Recognition from English Speech”, Interspeech 2020, pp.4268-4272. [CORE A]

76. Shagun Uppal, Anish Madan, Sarthak Bhagat, Yi Yu, and Rajiv Ratn Shah, “C3VQG: Category Consistent Cyclic Visual Question Generation”, accepted by ACM MM Asia, 2020.

77. Wei Duan, Lin Li, and Yi Yu, “A Relation learning hierarchical framework for multi-label charge prediction,” accepted by the 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2020.

78. Yi Yu, Florian Harscoët, Simon Canales, Gurunath Reddy, Suhua Tang and Junjun Jiang, “Lyrics-conditioned neural melody generation,” The 26th International Conference on Multimedia Modeling (MMM’20), vol 11962, pp.709-714.

79. Osaid Rehman Nasir, Shailesh Kumar Jha, Manraj Singh Grover, Yi Yu, Ajit Kumar and Rajiv Ratn Shah, “Text2Face GAN:Face Generation from Fine Grained Textual Descriptions,” IEEE International Conference on Multimedia Big Data 2019, pp.58-67.

80. Sein Minn, Yi Yu, Michel Desmarais, Feida Zhu, and Jill-Jenn Vie, “Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing”, accepted by the 2018 IEEE International Conference on Data Mining (ICDM'18).

81. Yi Yu, Samuel Beuret, Donghuo Zeng, Keizo Oyama, “Deep Learning of Human Perception in Audio Event Classification”, accepted by IEEE ISM2018.

82. Donghuo Zeng, Yi Yu, Keizo Oyama, “Audio-Visual embedding for cross-modal music video retrieval through Supervised Deep CCA”, accepted by IEEE ISM2018.

83. Junjun Jiang,Yi Yu, Jinhui Hu, Suhua Tang, Jiayi Ma, “Deep CNN Denoiser and Multi-layer Neighbor Component Embedding for Face Hallucination”, accepted by International Joint Conference on Artificial Intelligence (IJCAI) 2018.

84. Junjun Jiang, Yi Yu, Zheng Wang, Jiayi Ma, “Residual Learning for Face Sketch Synthesis”, accepted by ICASSP2018.

85. Wenjun Huang, Chao Liang, Yi Yu, Zheng Wang, Weijian Ruan, Ruimin Hu, “Video-based Person Re-identification Self Paced Weighting”, accepted by AAAI 2018.

86. Yi Yu, Suhua Tang, Kiyoharu Aizawa, Akiko Aizawa, “VenueNet: Fine-Grained Venue Discovery by Deep Correlation Learning,” accepted by IEEE ISM2017.

87. Zheng Wang, Ruimin Hu, Yi Yu, Junjun Jiang, Jiayi Ma, Shin'Ichi Satoh, “Statistical Inference of Gaussian-Laplace Distribution for Person Verification”, accepted by ACM Multimedia (MM) 2017.

88. Chunlin Zhong, Yi Yu, Suhua Tang, Shin’ichi Satoh, Kai Xing, “Deep Multi-Label Hashing for Large-Scale Visual Search Based on Semantic Graph,” accepted by APWeb-WAIM Joint Conference on Web and Big Data 2017 (best paper runner up).

89. Junjun Jiang, Yi Yu, Suhua Tang, Jiayi Ma, Guo-Jun Qi, Akiko Aizawa, “Context-Patch based Face Hallucination via Thresholding Locality-Constrained Representation with Reproducing Learning”, accepted by IEEE International Conference on Multimedia and Expo (ICME) 2017 (finalist of the World’s First 10K best paper award).

90. Soo-Chang Pei, Mei-Shuo Chen, Yi Yu, Suhua Tang, Chunlin Zhong, “Compact LBP and WLBP Descriptor with Magnitude and Direction Difference for Face Recognition”, accepted by IEEE International Conference on Image Processing (ICIP) 2017.

91. Zheng Wang, Ruimin Hu, Yi Yu, Chao Liang, and Chen Chen, “TAICHI Distance for Person Re-identification,” accepted by IEEE International Conference on Acoustics, Speech and Signal Processing 2017.

92. Tian Huat Tan, Yinxing Xue, Manman Chen, Shuang Liu, Yi Yu, Jun Sun, “JSFox: Integrating Static and Dynamic Type Analysis of JavaScript Programs, ” 39th International Conference on Software Engineering (ICSE 2017).

93. Rajiv Ratn Shah, Yi Yu, Suhua Tang, Shin’ichi Satoh, Akshay Verma, Roger Zimmermann, “Concept-Level Multimodal Ranking of Flickr Photo Tags via Recall Based Weighting”, Multimedia COMMONS Workshop at ACM Multimedia 2016, pp.19-26.

94. Zheng Wang, Ruimin Hu, Yi Yu, Chao Liang. “Scale-adaptive Low-resolution Person Re-identification via Learning a Discriminating Surface”, the 25^th International Joint Conference on Artificial Intelligence (IJCAI), 2016, pp.2669-2675.

95. Rajiv Ratn Shah, Anupam Samanta, Deepak Gupta, Yi Yu, Suhua Tang, Roger Zimmermann, “PROMPT: Personalized User Tag Recommendation for Social Media Photos Leveraging Personal and Social Contexts”, IEEE International Symposium on Multimedia, 2016, pp.486-492.

96. Amir Hossein Nabizadeh Rafsanjani, Alípio Mário Guedes Jorge, Suhua Tang, and Yi Yu, “Predicting User Preference Based on Matrix Factorization by Exploiting Music Attributes”, accepted by Ninth International C* Conference on Computer Science & Software Engineering (C3S2E), 2016, pp.61-66.

97. Subhasree Basu, Yi Yu, and Roger Zimmermann, “Fuzzy Clustering of Lecture Videos Based on Topic Modeling”, 14th Workshop on Content-Based Multimedia Indexing (CBMI), 2016, pp.1-6.

98. Subhasree Basu, Yi Yu, Vivek K. Singh, and Roger Zimmermann, “VIDEOPEDIA: Lecture Video Recommendation for Educational Blogs Using Topic Modeling”, The 22th International Conference on Multimedia Modeling (MMM’16), 2016, pp.238-250.

99. Rajiv Ratn Shah, Mohamed Hefeeda, Roger Zimmermann, Khaled Harras, Cheng-Hsin Hsu, and Yi Yu, “NEWSMAN: Uploading Videos over Adaptive Middleboxes to News Servers InWeak Network Infrastructures”, The 22th International Conference on Multimedia Modeling (MMM’16), 2016, pp.100-113.

100. Wenxin Huang, Ruimin Hu, Chao Liang, Yi Yu, Zheng Wang, Xian Zhong, Chunjie Zhang, “Camera network based person re-identification by leveraging spatial-temporal constraint and multiple cameras relations”, The 22th International Conference on Multimedia Modeling (MMM’16), 2016, pp.174-186.

101. Zheng Wang, Ruimin Hu, Yi Yu, Chao Liang, and Wenxin Huang, “Multi-Level Fusion for Person Re-identification with Incomplete Marks”, ACM international conference on Multimedia (ACM MM’15), 2015, pp.1267-1270.

102. Rajiv Ratn Shah, Anwar Shaikh, Yi Yu, Wenjing Geng, Roger Zimmermann, and Gangshan Wu, “EventBuilder: Real-time Multimedia Event Summarization by Visualizing Social Media”, accepted by ACM international conference on Multimedia (ACM MM’15), 2015, pp.185-188. Runner up in Yahoo Flickr Grand Challenge 2015.

103. Rajiv Ratn Shah, Yi Yu, Anwar Shaikh, and Roger Zimmermann, “TRACE: A linguistic-based approach for automatic lecture video segmentation leveraging Wikipedia texts”, The IEEE International Symposium on Multimedia, 2015, pp.217-220.

104. Lei Yao, Jun Chen, Yi Yu, Zheng Wang, Wenxin Huang, Mang Ye, and Ruimin Hu, “Adaptive Margin Nearest Neighbor for Person Re-identification”, Pacific-Rim conference on Multimedia (PCM’15), 2015, pp.75-84.

105. Rajiv Ratn Shah, Yi Yu, and Roger Zimmermann. “ADVISOR - Personalized Video Soundtrack Recommendation by Late Fusion with Heuristic Rankings”, Pdf, ACM international conference on Multimedia (ACM MM’14), 2014, pp.607-616. (Codes are available)

106. Yi Yu, Suhua Tang, Roger Zimmermann, and Kiyoharu Aizawa. “Empirical Observation of User Activities: Check-ins, Venue Photos and Tips in Foursquare”, Pdf, WISMM in ACM MM, 2014, pp.31-34. (Codes and Data are available)　Tips (NYC, LA) (Singapore and Nanjing)

107. Subhasree Basu, Yi Yu, and Roger Zimmermann. “Student Performance Evaluation of Multimodal Learning via a Vector Space Model”, Pdf, WISMM in ACM MM, 2014, pp.27-30. (Codes are available)

108. Rajiv Ratn Shah, Yi Yu, Anwar Shaikh, Suhua Tang, and Roger Zimmermann. “ATLAS: Automatic Temporal Segmentation and Annotation of Lecture Videos based on Modelling Transition Time”, Pdf, ACM international conference on Multimedia (ACM MM’14), 2014, pp.209-212.

109. Rajiv Ratn Shah, Yi Yu, and Roger Zimmermann. “User Preference-Aware Video Generation Based on Modeling Scene Moods”, in Proc. ACM MMSys 2014, 2014, pp.156-159.

110. Yi Yu, Kiyoharu Aizawa, Toshihiko Yamasaki and Roger Zimmermann. “Emerging Topics on Personalized and Localized Multimedia Information Systems”, Pdf, a full-day tutorial in ACM international conference on Multimedia (ACM MM’14), pp.1233-1234.

111. Roger Zimmermann, Yi Yu. “WISMM14 -- First International Workshop on Internet-Scale Multimedia Management”, ACM MM, 2014, pp.1257-1258. Workshop summary

112. Yi Yu, Suhua Tang, Roger Zimmermann. “Edge-Based Locality Sensitive Hashing for Efficient Geo-Fencing Application”, Pdf, pp.576-579, in Proc. ACM SIGSPATIAL GIS 2013 (ACM GIS Cup 13). Slides

113. Yi Yu, Roger Zimmermann, Ye Wang, and Vincent Oria, “Recognition and Summarization of Chord Progressions and Their Application to Music Information Retrieval”, in Proc. IEEE International Symposium on Multimedia (IEEE ISM’12), Pdf, pp.9-16, Best paper award 2012. (Codes and Data are available)

114. Yi Yu, Zhijie Shen, Roger Zimmermann, “Automatic Music Soundtrack Generation for Outdoor Videos from Contextual Sensor Information”, in Proc. ACM international conference on Multimedia (ACM MM12), Multimedia Grand Challenge, Pdf, pp.1377-1378, 2012.

115. Yi Yu, Michel Crucianu, Vincent Oria, and Ernesto Damiani, “Combing Multi-Probe Histogram and Order-Statistics Based LSH for Scalable Audio Content Retrieval,” in Proc. ACM international conference on Multimedia (ACM MM10), Pdf, pp.381-390, 2010. (Codes and Data are available)

116. Yi Yu, Michel Crucianu, Vincent Oria, and Lei Chen, “Local Summarization and Multi-Level LSH for Retrieving Multi-Variant Audio Tracks,” in Proc. ACM international conference on Multimedia (ACM MM09), Pdf, pp.341-350, 2009.

Ph.D candidates

Jiuxiang You, Hiroshima University (October 2024 – September, 2027)

Luis Vilaça, University of Porto (January 2024 – February, 2027, co-supervised with Prof. Paula Viana)

Zhe Zhang, SOKENDAI (October 1, 2021 – September 30, 2024, co-supervised with Prof. Atsuhiro Takasu)

Wei Duan, SOKENDAI (April 1, 2021 – September 30, 2024, co-supervised with Prof. Keizo Oyama)

Donghuo Zeng, SOKENDAI (October 1, 2017 – September 30, 2020, co-supervised with Prof. Keizo Oyama)

Researchers in NII

Dr. Zhuoqi Ma (June 1, 2021 – March 30, 2022, co-supervised with Prof. Keizo Oyama)

Dr. Donghuo Zeng (October 1, 2020 – March 30, 2021)

Dr. Jun Wu (April 1, 2020 – March 30, 2021)

Dr. Guangwei Gao (April 1, 2019 – March 30, 2021)

Dr. Nannan Wang (April 1, 2019 – March 30, 2020)

Dr. Junjun Jiang, first project researcher received Outstanding Doctoral Dissertation Award from China Computer Federation in 2016 (October 1, 2016 – September 30, 2018)

Dr. Kazunari Sugiyama, first visiting researcher (November 15, 2017 – September 30, 2018)

Interns and RA

Divyam Srivastava, NII intern, Saarland University, (October 24, 2023 – April 9, 2024)

Chenyang Lyu, NII intern, Dublin City University (January 4,2023 – March 31, 2024)

Yulun Wu, NII intern, Fudan University, (January 4,2024 – March 31, 2024)

Yiming Tang, NII intern, Fudan University, (February 1,2024 – March 31, 2024)

Janak Kapuriya, NII intern, IIIT-Delhi, India, (February 1,2024 – March 31, 2024)

Anam Fatima, NII intern, IIIT-Delhi, India, (September 20, 2023 – March 15, 2024)

Qingyuan Yao, NII intern, Sorbonne Université, (March 3, 2023 – August 22, 2023)

Wenjie Yin, NII intern, KTH Royal Institute of Technology, (March 2, 2023 – August 1, 2023)

Julien Lalanne, NII intern, Grenoble INP – Ensimag, (February 28, – July 28, 2023)

Raphael Bournet, NII intern, Université Paris-Saclay, (March 6, 2023 – September 1, 2023)

Macarena Piaggio, NII intern, Buenos Aires University (March 8, 2023 – May 26, 2023)

Karol Lasocki, NII intern, Aalto University (February 10, 2023 – July 19, 2023)

Louis Ngo, NII intern, University of Freiburg (September 8, 2022 – December 26, 2022)

Monish Reddy Nallapareddy, NII intern, University of Freiburg, (November 30, 2021 – May 28, 2022)

Luís Vilaça, NII intern, University of Porto, (September 14, 2021 – March 13, 2022)

Jiwei Zhang, NII RA, (April 1, 2021 – March 30, 204)

Abhishek Srivastava, NII intern, IIIT-Delhi, India, (January 16, 2020 – July 10, 2020)

Vu Hoang Le, NII intern, School of Computer Science and Statistics, Trinity College Dublin (September 30, 2019 – February 26, 2020)

Florian Harscoët, NII intern, ISIMA, France, (March 29 – August 30, 2019)

Gurunath Reddy Madhumani, NII intern, Indian Institute of Technology Kharagpur, India, (March 11 – August 27, 2019)

Canales Simon, NII intern, Ecole polytechnique fédérale de Lausanne（EPFL), Switzerland, (March 1 – August 27, 2019)

Chengru Li, NII intern, Peking University, China, (October 10, 2018 – February 19, 2019)

Salo Aaro, NII intern, Aalto University, Espoo, Finland, (October 17, 2018 – November 5, 2019)

Himanshu Aggarwal, NII intern, IIIT-Delhi, India, (December 17, 2018 – June 7, 2019)

Yun Hao, UIUC, USA

Haoting Liang, Saarland University, (February 19-August 17, 2018)

HongThi Vuong, VNU University of Engineering and Technology, (March 9-August 31, 2018)

Weijian Ruan, Wuhan University, (March 5-August 31, 2018)

Tong Li, Berlin Institute of Technology (TUB, TU Berlin), (February 13-August 27, 2018)

Samuel Denys, École Polytechnique Fédérale de Lausanne（EPFL), (February 19, – August 10, 2018)

Raposo Francisco Afonso, INESC-ID, (August 28, 2017 – April 23, 2018)

Huaxuan Zhu, University of Southern California, (July 15, 2017 – April 26, 2018)

Minn Sein, Ecole Polytechnique de Montreal, Canada, (November 21, 2017 – May 17, 2018)

Maria-Eleni Kyriakidi, University of Athens/"Athena" Research & Innovation Centre, (March 7, 2017 – June 30, 2017)

Clément Maliet, Institute National Polytechnique de Toulouse, (March 24, 2017 –September 1, 2017)

Mai Trong, Khang, University of Information Technology – Vietnam National University Ho Chi Minh City Khang (February 28, 2017 – August 23, 2017)

Wei Zhou, University of Science and Technology of China, (February 24, 2017 – May 22, 2017)

Junpeng Xiao, University of Southern California, (September 5, 2016 – November 20, 2017)

Rajiv Ratn Shah, National University of Singapore (November 7, 2015 – May 3, 2016), first student who worked with me more than three years in NUS and NII.

Mei-Shuo Chen, National Taiwan University, (July 8, 2016 – September 8, 2016), first female intern.

Chunlin Zhong, University of Science and Technology of China, (August 25, 2016 – November 23, 2016)

Zhen Yu, University of Science and Technology of China, (October 15, 2016 – March 30, 2017)

Muthu Kumar Chandrasekaran, National University of Singapore (March 10, 2016 – June 10, 2016)

Kuan-Yu Lin, National Taiwan University (January 16, 2016 – March 14, 2016)

Amir Hossein Nabizadeh Rafsanjani, University of Porto (October 31, 2014 – April 23, 2015)

Tim Ziemer, University of Hamburg (January 4, 2016 – April 3, 2016)

Marcelo E. Rodríguez-López, Utrecht University (December 19, 2015 – March 17, 2016)

Visitors/ Collaborators

Prof. Shah Rajiv Ratn (Visitor, December 1, 2022 – December 16, 2022)

Prof. Jean-Pierre.Briot (Visitor, LIP6, Sorbonne University, October 20 – November 7, 2022)

Prof. Atsuhiro Takasu (Collaborator, NII)

Dr. Jianming Wu (Collaborator, KDDI)

Prof. Keizo Oyama (Collaborator, NII)

Prof. Shin’ichi Satoh (Collaborator, NII)

Prof. Jiayi Ma (Visitor, Wuhan University, July 28, 2019)

Prof. Jun Huang (Visitor, Wuhan University, July 28, 2019)

Prof. Xin Tian (Visitor, Wuhan University, July 28, 2019)

Dr. Rong Xu (Visitor, Data section, Feb. 12, 2019)

Prof. Tat-Seng Chua (Visitor, National University of Singapore, December 21, 2018)

Prof. Nannan Wang (Visitor, Xidian University, August 10 – 14, 2018)

Prof. Denis Parra Santander (Visitor, Pontificia Universidad Catolica de Chile, March 9, 2018)

Prof. Tengjiao Wang, (Visitor, Peking University, March 7, 2018)

Prof. Stephen Downie, (Visitor, University of Illinois Urbana-Champaign, USA, February 16, 2018)

Prof. Lin Li (Visitor, Wuhan University of Technology, January 4, 2018)

Prof. Tat-Seng Chua (Visitor, National University of Singapore, August 9, 2017)

Prof. Denis Parra Santander (Visitor, Pontificia Universidad Catolica de Chile, August 8, 2017)

Dr. Kazunari Sugiyama (Visitor, National University of Singapore, August 21-23, 2017)

Prof. Lei Chen, (Visitor/Collaborator, Hong Kong University of Science and Technology, Jan. 21, 2017 – Jan. 26, 2017)

Prof. Feida Zhu, (Visitor/Collaborator, Singapore Management University, Feb. 12, 2017 – Feb. 20, 2017)

Prof. Zhifeng Bao, (Visitor/Collaborator, RMIT University, Australia, Feb. 8, 2017 – Feb. 16, 2017)

Prof. Stephen Downie, (Visitor, University of Illinois Urbana-Champaign, USA, September 16, 2016)

Prof. Alípio Mário Guedes Jorge (Visitor/Collaborator, University of Porto INESC, Portugal, July 25, 2016 – August 3)

Prof. Guo-Jun Qi (Visitor, University of Central Florida, USA, May 13, 2016 – May 28, 2016)

Dr. Tian Huat Tan (Visitor, Singapore University of Technology and Design, Singapore, March 13, 2016 – March 23, 2016)

Prof. Roger Zimmermann (Collaborator, National University of Singapore, Singapore)

Prof. Suhua Tang (Collaborator/Visitor, The University of Electro-Communications, Japan)