Semimetals have recently emerged at the forefront of contemporary thermoelectrics research. In this paper, we show that the candidate Dirac semimetal CaAgP forms with a stable CaAg0.9P composition. Intentionally, Ag-deficient CaAg0.9P shows a promising thermoelectric figure of merit, achieving zT = 0.43 at 660 K. This derives from moderate power factors, S2/q = 1.25 mW m-1 K-2 , and a low lattice thermal conductivity, Κlat = 1.2 W m-1 K-1 . The thermoelectric properties of CaAg0.9P are consistent with that of a small bandgap semiconductor. High levels of doping are key to suppressing bipolar transport, enabling promising zT values, despite the low ∼0.2 eV bandgap. Unusually, the onset of bipolar transport coincides with a transition to a more electrically resistive state, indicating a fundamental change in electronic properties at high temperature.