The budding couple separately appeared on the season 49 premiere on Saturday night

Travis Kelce, Taylor Swift. Photo:JC Olivera/Getty Images; Amy Sussman/Getty Images

Travis Kelce is showing appreciation for Taylor Swift in real life — and on social media.

On Instagram Sunday, the Kansas City Chiefs star was among the 174,000-plus who liked a Saturday Night Live post showing the pop superstar introducing Ice Spice as the musical guest on the show over the weekend.

Kelce, 34, also made a surprise appearance on the season 49 premiere in a skit about the new couple seemingly taking over NFL broadcasts.

Elsewhere in New York City over the weekend, the duo indulged in other forms of affection, where they were photographed holding hands at two restaurants on Saturday and Sunday, as well as enjoying one another’s company at the Saturday Night Live afterparty, where a source told PEOPLE that the pair was “so happy.”

“Taylor and Travis were there all night and they were so smiley, they were so happy,” the source said. “They were so talkative. They were really fun. They talked to every single person in the room that talked to them.”

“They were super cool. Super nice, super happy,” the insider added, noting that Swift and Kelce stayed at the party at Catch Steak “until after 4 a.m.”

Saturday Night Live Instagram

Although Swift, 33, and Kelce “mingled separately,” the pro athlete “kept going over to her and checking in on her every little while, like, ‘Are you good?’ ” the source said.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

According to the source, Kelce was “going over and touching her, putting his hand on her lower back or putting his hands on her waist and then they would do a little kiss and then they would go back to talking to whoever they were talking to.”

“He kept checking in with her, but gave her space to talk to other people in the room, both male and female, and he talked to a ton of people in the room too,” the insider added.

Travis Kelce and Taylor Swift depart the SNL Afterparty.Gotham/GC Images

On Sunday, after Travis traveled to New Jersey to watch brother Jason Kelce and the Philadelphia Eagles take on the New York Jets, he returned to the city for dinner with Swift.

The Grammy winner and the tight end were spotted walking hand in hand while leaving the Waverly Inn restaurant, where Swift looked chic in a sheer black and green floral long-sleeved top teamed with a black miniskirt, while Kelce sported a black T-shirt, beige jacket and matching pants.

The pair appeared to be in great spirits as they smiled while walking to their car after leaving the restaurant.