Well, if we take a stone age axe to be a blunt tool more than an actual axe, such as one of these:
Then it would seem that it doesn't matter what type of stone it is so much as if it's the right size or not.
That said, being able to use just any rock makes it a bit easy. Also, even if that stone axe is more of a club, you'd probably be better off using fire hardened arrows over ones with heads similar to that.
I think if I were doing it, I'd make it so some weapons can be made from anything (axes, clubs, etc.) and some can only be made from knappable stone. I'd add hardened wood equivalents for things that can't be made from any stone (spears and arrows) so that there is an early game option there. Knappable stone is better than plain/hardened wood, and gems would be better than knappable stone.