Is it rude to meetup with friends only to have them go to a restaurant? How to ask for heads-up?

This has happened a few times with certain friends. I agree to hangout with them on a certain street. We I arrive, they tell me they had ordered food at a restaurant. This kind of annoys me because it's like I'm being forced to order food with them. I tried talking to one over text about this and he was very defensive. Any suggestions? In fairness, he said they might go for fast food. But fast food to me means a place where you have to wait 3 minutes and order at the front, not have a waiter take your order. They had gone to more of a hybrid restaurant. Of course I don't want my friends to be hungry, but how can I explain my perspective without being demanding?