People with diabetes have a higher risk of cognitive impairment than people without diabetes, and recently it is being considered a complication of diabetes mellitus (DM). Because of drastic lifestyle changes in the Mongolian population, diabetes prevalence is increasing rapidly. The rapid increase of diabetes prevalence and its poor control in Mongolia suggest that there might be significant cognitive impairment in the diabetes population. In this case-control study, we compared the Mini-Mental State Examination score to the risk of cognitive impairment, indicating vascular dementia in people with and without diabetes. Upon obtaining their informed consent, each subject was tested with Mini-Mental State Examination. We involved age and gender-matched diabetic (n = 131) and non-diabetic (n = 131) subjects. The mean age was 61.3 \pm 8.5 and 61.0 \pm 8.7 in people with and without diabetes, respectively, and 35.9% of the participants were male. According to study groups, the Mini-Mental State Examination scores were significantly different: 26.1 \pm 3.7 and 27.5 \pm 2.6 for people with and without diabetes, respectively. In logistic regression analysis, age was significantly associated with Mini-Mental State Examination score (Beta coefficient = 1.22; 1.11–1.35, P 0.001) in people without diabetes after adjustments for potential confounders. However, age was not significantly associated with MMSE scores in people with diabetes mellitus. Thus, diabetes duration and poor control may contribute to developing cognitive impairment in people with diabetes. In conclusion, there might be a high prevalence of vascular dementia in people with type 2 diabetes mellitus. However, since Mini-Mental State Examination is sensitive to dementia and not specific to vascular dementia, further studies involving neuroimaging and neurological examination are needed to fully elucidate the link between type 2 diabetes and vascular dementia in dementia in a Mongolian population.