All That Remains released their debut album, Behind Silence and Solitude on March 26, 2002 through Prosthetic Records. The style of the album differs from their future releases, as it is void of clean vocals, and more prominently contains elements of melodic death metal. It was also the band's only release featuring guitarist Chris Bartlett and bassist Josh Stolowitz. Shortly after the album's release, singer Philip Labonte tried out for the vacant vocalist position in Killswitch Engage as mentioned in their DVD World Ablaze. Labonte and Killswitch bassist Mike D'Antonio mentioned he was the runner-up, but current vocalist, Howard Jones got the position.

Their second album, This Darkened Heart was released on March 23, 2004 through Prosthetic Records/Razor & Tie. Produced by Killswitch Engage guitarist Adam Dutkiewicz, the album featured better production compared to its predecessor, as well as the use of clean vocals. Three singles were released from the album in "This Darkened Heart", "Tattered on My Sleeve" and "The Deepest Gray".

Their third album, The Fall of Ideals was released on July 11, 2006 through Prosthetic Records. Once again, the album was produced by Adam Dutkiewicz. The album is also considered to be the band's breakthrough release, as it entered the Billboard 200 charts at number 75, selling almost 13,000 copies in its first week "This Calling" was released as the album's first single. Two music videos were created, with one incorporating footage from Saw III (as it was the lead song from the movie's soundtrack). A music video was made for the album's second single "The Air That I Breathe". The band were also a part of Ozzfest 2006. The song "Six" was featured in Guitar Hero II. On June 20, 2007, it was announced that The Fall of Ideals had surpassed 100,000 sales in the United States A music video for the album's third single "Not Alone" was filmed on July 4 and was released on September 10, 2007.

In 2007, they played at Wacken Open Air in Wacken, Germany.^[

In early 2008, they embarked on a headlining tour with support from Chimaira & Black Tide with Divine Heresy & Light This City splitting the tour's opening slot. Five Finger Death Punch, was originally supposed to play but dropped off before the tour started due to Vocal problems.

They recently resided at Audiohammer Studios to record their latest album, entitled Overcome. Jason Suecof produced the album. It is the first record of the band not to be produced by Adam Dutkiewicz of Killswitch Engage. Currently, recording has finished, and the album awaits its release on September 16, 2008