In the photos taken by TMZ, Bieber is seen pacing the street before sitting with his head in his hands as he talks to someone on the phone.
The site said the pop star seemed "stressed", also noting that it was just before he travelled to visit Baldwin's family.
Fans are speculating about who Bieber was speaking to with many guessing his ex-girlfriend, Selena Gomez.
Hours later, Bieber shared a video to his Instagram of Baldwin giggling and singing before deleting it.
However, we don't think there's any trouble in paradise: the pair were spotted walking the streets of New York that evening, laughing and posing for photos with fans.