Regular monitoring of railway systems is imperative for improving safety and ride quality. To this end, data collection is carried out regularly in the rail industry to document performance and maintenance. The use of machine learning methods in the past recent years has provided opportunities for improved data processing and defect detection and monitoring. Such methods rely on installing instrumentation wayside or collecting data from onboard rolling stock. Using the former approach, only specific locations can be monitored, which could hinder covering a large territory. The latter, however, enables monitoring large sections of track, hence proving far more spatial efficiency. In this paper, we have investigated the feasibility of rail defect detection using deep learning from onboard data. The source of data is acceleration and track geometry collected from onboard railcars. Such an approach allows collecting a large set of data on a regular basis. A long short-term memory (LSTM) architecture is proposed to examine the measured time-series to flag potential track defects. The proposed architecture investigates the characteristics of time-series signatures during a short time (∼ls) and classifies the associated track segment to normal/defect states. Furthermore, a novel automated labeling method is proposed to parse the exception report data (recorded by the maintenance team) and label defects for associated time-series signatures during the training phase. In a pilot study, field data from a revenue service Class I railroad has been used to evaluate the proposed deep learning method. The results show that it is possible to efficiently analyze the data (collected onboard a railcar operated in revenue service) for automated defect detection, with relatively higher accuracy for FRA type I defects.