Why do I take things so literally all the time?

If someone says ‘A’ but what they really meant was ‘B’, I interpret it literally as ‘A’ and nothing else. The only way I can take it as ‘B’ is when that person thoroughly explains to me that they actually meant ‘B’. Then I tend to say “why didn’t you just say ‘B’?”, and both of us get frustrated. I myself get frustrated because I wish people just said what they meant more precisely.

To use a real-life example, I told someone today that I can’t seem to find suitable topics for conversation when meeting people. They advised me to talk about ‘useless’ things (the wording is not exact because I had this conversation in my native language, not in English). I immediately responded ‘no’ because I didn’t see the point in talking about useless things. But it turned out, after they explained to me, that what they really meant was to talk about relatively unimportant things such as daily life, as opposed to heavier topics like philosophy or politics. I responded that talking about daily life is not useless at all since it could be pleasurable, so they should have worded things more precisely.

I’ve had countless conversations like this and frustrated both myself and other people. For the lack of a better phrase, what’s wrong with me? How do I stop taking things so literally?

p.s. Things get even more confusing when I consider the fact that I love poetry, in which metaphorical thinking is crucial. I would write poems with layers and layers of metaphors and then go on to be painfully literal with other people.