The automated segmentation of the prostate gland from MR images is increasingly used for clinical diagnosis. Since deep learning demonstrates superior performance in computer vision applications, we propose a coarse-to-fine segmentation strategy using ensemble deep convolutional neural networks (DCNNs) to address prostate segmentation in MR images. First, we use registration-based coarse segmentation on pre-processed prostate MR images to define the potential boundary region. We then train four DCNNs as voxel-based classifiers and classify the voxel in the potential region is a prostate voxel when at least three DCNNs made that decision. Finally, we use boundary refinement to eliminate the outliers and smooth the boundary. We evaluated our approach on the MICCAI PROMIS12 challenge dataset and our experimental results verify the effectiveness of the proposed algorithms.