Both are correct, but you need to understand the difference between “in” and “at”. If you apply a vague “commonly used” rule here, you’ll definitely make mistakes.
If you are “in” a place, that area has limits (eg. walls, a fence, a border etc) and you are surrounded by those limits. Even a general area will work (in the corner, in the centre).
If you are “at” a place, it means you have arrived there. “At” is a point in your movements through time and space.
When you write “he was having dinner at a restaurant”, you are telling me that he had gone to the restaurant for dinner.