Billy Joel put Kevin Spacey in a New York state of mind Friday, April 14 when Spacey joined Joel in a duet at Madison Square Garden that night.

Joel began the song alone on stage. The House of Cards star then joined the musician on stage at the end of the first verse without any formal introduction. Joel briefly welcomed him on stage and without missing a beat, Spacey continued the song and sang the second verse. The pair then finished the six minute song together, switching off lines.

Spacey shared a selfie on Twitter of the two together on stage. He captioned the photo with “On stage at MSG, dayenu. Singing w/ Billy Joel, dayenu. Amazing NY fans, dayenu. All of them together & a selfie?! NY State of Outta My Mind.”

Spacey has performed with Joel before. In 2014 he joined the singer and performed “Piano Man” during the Gershwin Prize for Popular Song.