Music's Cover generator[WIP]

discription

this is a fuse model to make midi+lyric to a fuse embedding that can generate this music's cover at this project, I use ...

how to use

first use source ./modelEnv/env388/bin/activate into virtual envirment

use python3 generate_image.py

and input artist and song name

it will auto download song's lyrics and music

and generate its cover image

performance

the similarity of fuse embedding and original cover the loss of decoder

reference

• dataset
• bert
• diffusion model
• microsoft musicBert
• YourMT3 this is a model for transform wav to midi
• LP-MusicCaps
• Qiu_Image_Generation_Associated_CVPR_2018_paper
• Performance-MIDI to Score
• DALI

issue

• image output size is only 64x64
• decoder overfitting

Thanks

• 葉梅珍 教授
• 王鈞右 教授
• 王兆偉
• 鍾詠傑
• 吳文元