Major depression disorder is one of the diseases with the highest rate of disability and morbidity and is associated with numerous structural and functional differences in neural systems. However, it is difficult to analyze digital medical imaging data without computational intervention. A voxel-wise densely connected convolutional neural network, Three-dimensional Densenet (3D-DenseNet), is proposed to mine the feature differences. In addition, a novel transfer learning method, called Alzheimer’s Disease Neuroimaging Initiative Transfer (ADNI-Transfer), is designed and combined with the proposed 3D-DenseNet. The experimental results on a database that contains 174 subjects, including 99 patients with major depression disorder and 75 healthy controls, show that large changes in brain structures between major depressive disorder patients and healthy controls mainly are located in the regions including superior frontal gyrus, dorsolateral, middle temporal gyrus, middle frontal gyrus, postcentral gyrus, inferior temporal gyrus. In addition, the proposed deep learning network can better extract different features of brain structures between major depressive disorder patients and healthy controls and achieve excellent classification results of major depressive disorder. At the same time, the designed transfer learning method can further improve classification performance. These results verify that our proposed method is feasible and valid for diagnosing and analyzing major depression disorder.