Ok so small company, has a certain culture, and everyone is at least a few years older. Also this is your first office job? And you aren’t fitting in, to the point that others are talking about you.
So definitely do not keep doing what you’re doing. It’s not working. I would say put your head down for a while, work hard at doing your job well, ask your boss for feedback, be polite and respectful, ask people how they’re doing and smile, and then outside of work join at least one club or league or meetup.