If you joined someone on their Live Stream and your Duet stopped abruptly, it means that the duration of the Live Duet is over. 
Every streamer can regulate their Live Duet duration in the Live Settings before and during the Stream. Once the time is up, your Duet is over and the next user in the queue will get the chance to join the streamer on their Duet.