Haverhill Rovers were founded in 1886, but their beginnings are shrouded in some mystery. The club first started playing at Hamlet Croft in 1913; previously they played at Seven Acres, an area bordered on two sides by the old railway line and Wratting Road. They spent the 1908/09 season in the East Anglian League. Other clubs in that league at that time included Clacton Town and Halstead Town.

At the start of the 1924/25 season Rovers joined the Essex & Suffolk Border League but, after finishing bottom at the end of the 1927/28 season, elected to leave that league. Eight years later, in the 1935/36 season, Rovers rejoined the Border League. In 1946/47, Rovers were runners up behind Colchester Casuals; however, the following year Rovers won not only the championship but also the Knock-Out Cup, beating Brightlingsea United 1-0 in the final.

In 1950/51 Rovers were runners up behind Stowmarket Town. In 1962/63 they won the league for the second time, retaining the title the following year, when they also won the Knock-Out Cup again to record their second “double”.

In 1964 Rovers joined the Eastern Counties League, finishing a creditable sixth in their first season, also winning the League Cup, defeating Stowmarket Town. The following year, Rovers again reached the final of the cup, but lost out to Lowestoft Town. Rovers finished in the bottom three every year from 1970 until 1977. In 1978, under the guidance of former player Roger Staples, Rovers were champions for the first and, so far, only time, pipping Great Yarmouth Town by one point. A crowd of 1,300, a record at the time, packed into Hamlet Croft to see the last game, the decider, against Chatteris Town. Unfortunately, this proved to be a one off, as the following season saw them finish in the lower half of the table.

After this the club made steady progress: 7th in 1989, 14th in 1990 and an excellent 4th in 1991, the first full season under manager Martin Murray. A glut of injuries at the start of 1991/92 meant a final placing of eighth. In 1988 Rovers reached the final of the League Cup for the fourth time, losing to Braintree Town.

In national competitions Rovers, have never reached the first round of the FA Cup, but reached the 6th Round (Quarter Finals) of the FA Vase in 1986/87, losing 3-1 at home to Warrington Town. The attendance on that day was 1,578, which broke the previous record attendance. The club has never won the Suffolk Premier Cup, losing in the final twice, in 1965/66 2-1 to Bury Town and in 1990/91 3-2 to Stowmarket Town. They have lifted the Cambridgeshire Invitation Cup three times. Rovers won the East Anglian Cup in 1990/91, beating Eynesbury Rovers 2-1 at Eynesbury in the final.

Season 1995/96 saw Rovers relegated for the first time in their long history, but this disappointment was sweetened by winning the Suffolk Senior Cup for the first time (at Portman Road) at the end of the 1996/97 season. They were also defeated finalists in the Jewson League Cup, losing 2-0 at Harwich & Parkeston.

1997 saw Rovers win the Suffolk Senior Cup after they defeated Ipswich Wanderers 4-1 at Portman Road. They are also the only team to have lifted the First Division Knock Out Trophy three times (2-1 after extra time against Needham Market in 2001, 2-0 against Kirkley in 2004 and 3-1 against Saffron Walden Town in 2007).

Rovers had an excellent run in the FA Cup in Season 2006/07 included their best result in the past few years when, in the Second Round Qualifying in front of 504 spectators, they defeated Eastbourne Borough 1-0 on 30th September.

Eastbourne were in the Conference South at the time, a huge four divisions above Rovers, but a goal by striker Paul Shaw three minutes into added time at the end of the game was enough to see them through and invoke a roar that was heard all around the town. This was then followed two rounds later with Aldershot Town, who were in the Conference, coming to the Croft in the FA Cup Fourth Qualifying Round. Although Rovers lost 4-0 on the day an amazing 1,730 people packed into the ground to become the biggest gate ever as the team's best FA Cup run to date ended at the final hurdle before the First Round Proper.

The jubilation of the cup run resurfaced at the end of the season when, on the last day, Rovers managed by Steve Taylor secured promotion back to the Premier Division after a ten-year absence with a nail biting 3-2 win over March Town United that saw them finish second in the First Division.

Rovers' time at Hamlet Croft came to an end at the end of 2009/10 with the building of The New Croft, a fantastic brand-new community stadium built to encompass the many teams at several levels that are now a part of the club.

The pitch is currently and professionally maintained with the assistance of Parkers Pitches and in 2020/21 had an irrigation system put in place.