“We were embarrassed,” the father of four said with a laugh, recalling the awkward moment in public

mark wahlberg sells las vegas home

Mark Wahlberg and family. PHOTO: MARK WAHLBERG/INSTAGRAM

Mark Wahlberg is a good sport about his kids’ constant jabs.

The actor joked about how his four kids — Grace, 13, Brendan, 15, and Michael, 17, and Ella, 20 — “always make fun” of him while appearing on Wednesday’s episode of The Late Show with Stephen Colbert.

He then opened up about how he once took Brendan with him to the Super Bowl, though the father-son duo left at halftime.

“I went to do some stuff to promote the movie and then I promised my wife I would be home,” he explained.

“I took my son and I was a little worried because my son, the last time we went to a Super Bowl, it was the Patriots versus the Falcons.”

He continued, “And people always wonder why I left at halftime, but he was 8 years old and he was screaming and cursing at the top of his lungs because they were down 28-3 and we were embarrassed. So I didn’t know what he was going to do at the game.”

Wahlberg noted the family lives “literally 15 minutes away from the stadium” and was able to watch the game with “no stress.”

When Colbert asked Wahlberg if it’s hard for him and his wife Rhea Durham to see their kids dating, he said, “It’s the worst.”

“My 18-year-old has his girlfriend and they were cuddled up on the couch watching the Super Bowl.

My wife didn’t like that at all. She had no problem with the boyfriends coming and going.”

Wahlberg clarified that they do like his son’s girlfriend, adding, “We already consider her part of the family.”

“But she goes upstairs, the door shuts, and my wife is like, ‘Open the door, go in there,’ ” which he objected to.

“I’m not going in there. It’s not my place, and nothing’s happening.”

Mark Wahlberg and Kids Celebrate Wife Rhea's 45th Birthday in Family Boat Outing: '45 Is Giving'

Mark Wahlberg and family. INSTAGRAM/RHEA WAHLBERG

In December, Wahlberg shared a relatable parenting moment at the Las Vegas premiere of his movie The Family Plan.

“I’ve got a teenager who won’t stop playing the video games. I’ve got a daughter who thinks she knows everything and [me and wife] are complete idiots in many, many ways,” he told PEOPLE.