I have a friend who's a music producer who does the exact same thing with sounds, certain sounds will be certain shades of different colours and then I think he sort of pieces together music from colours that match together or something, he never explained it a lot.
I didn't think smelling colour was that odd, the same friend who hears colours also thought my deodorant smelled like blue. Most bad things smell like yellow to me, which happens to be a colour I can't stand.
From what I know it's actually quite common for humans to associate information gained through the senses with specific colours, not quite sure why but I think it's just one of the ways we figure things out, we add in a sense, in my case vision to support the other one, smell in my case