Kempster began his football career with the North Somerset Regional Talent Centre from the age of 11 through to the age of 16, with his final year including scoring two goals in five games for Clevedon under-18s in the Western Counties Floodlight League Premier Division.
He joined Weston in the summer of 2019 as part of the partnership with Weston College, and he went on to score four goals in 16 games for the under-19s during 2019-2020. His form for the youth team saw him rewarded with a first-team debut in the Southern League as a late substitute in the 1-0 win over Walton Casuals in February 2020.
Kempster was involved in the 2020 pre-season with the first-team squad, putting in some promising performances on the pitch which saw him promoted to the first-team.
He joined Southern League Division One South side Willand Rovers on loan for a month in September 2020, which was then extended in October 2020. He made nine appearances and was an unused substitute twice, scoring one goal and assisting one for the Devon side before the season was curtailed.