Emergence timing of a plant, i.e., the time at which the plant is first visible from the surface of the soil, is an important phenotypic event as an indicator of seed quality and plant growth. Uneven emergence timing is associated with lower yield and poor farmer acceptance. The research introduces a novel deep learning based method called EmergeNet with a custom-designed loss function for coleoptile emergence timing detection and tracking its growth from a time-lapse video sequence in presence of cluttered background and extreme variations in illumination. EmergeNet uses a novel ensemble technique that integrates SEResNet, InceptionV3 and VGG19 to detect the coleoptile at its first tiny appearance on the soil surface. Emergence is an important phenotype which not only helps determine the dormancy of seeds for different genotypes in different conditions but also helps determine various aspects of the plant growth at an early stage. To develop and evaluate the algorithm, a benchmark dataset is indispensable. Thus, we introduce and publicly release a benchmark dataset called University of Nebraska-Lincoln Maize Emergence Dataset (UNL-MED). A visible light camera was fitted to capture the top view time-lapse images to form UNL-MED, where imaging started before the emergence and continued until maize seedlings are about 1 inch tall. Experimental evaluation on UNL-MED demonstrates the efficacy of the EmergeNet to detect the emergence timing with 100% accuracy when compared with human perceived groundtruth.