We report the first use of a convolutional neural network for terahertz (THz) imaging combined with a single-pixel camera to achieve high quality hyperspectral THz imaging 10x faster than the current commercial systems, with faster potential only limited by noise. Imaging with a spatial light modulator (SLM) relies on projecting a set of spatial patterns onto an object and recording the transmission with a single-pixel detector, and an optical delay unit (ODU) is used to obtain the terahertz time-of-flight information. A key breakthrough in this work is to synchronize the equipment to not need any time to instruct the ODU to move while projecting the patterns.