Background: We sought to assess the anxiety and depression scores of pregnant women in Hong Kong during the COVID-19 pandemic and to evaluate the impact of demographic, economic and social factors on these scores. Methods: This was part of an ongoing worldwide cross-sectional study conducted from 22 May 2020 to 28 February 2021. Data were collected through an anonymous web-based survey. The severity of depression and anxiety was assessed using the Patient Health Questionnaire-9 (PHQ-9) score and the General Anxiety Disorder-7 (GAD-7) score, respectively. Results: A total of 361 participants completed both the GAD-7 and PHQ-9 questionnaires. Participants with psychiatric illness reported a significant higher median GAD-7 score (6.00, interquartile range [IQR] 3.00–7.75 vs. 2.00, IQR 0.00–6.00, p = 0.001), while the median PHQ-9 score was also higher but was not statistically significant (6.50, IQR 3.00–11.00 vs. 5.00, IQR 3.00–8.00, p = 0.066). A higher proportion of participants with psychiatric illness reported moderate-severe depression and anxiety (35.7% vs. 16.5%, p = 0.002, 17.8% vs. 3.6%, p 0.001 respectively). Multivariate regression analysis demonstrated that financial difficulty, in education and pregnancy by in-vitro fertilization were associated with a higher PHQ-9 score in pregnant women during the COVID-19 pandemic, while underlying psychiatric illness was associated with a higher GAD-7 score. Support from a partner was demonstrated to be associated with a reduced level of depression and anxiety in pregnancy. Conclusions: Pregnant women with underlying psychiatric illness were more vulnerable during the COVID-19 pandemics than the non-psychiatric counterparts. Partner support is important for alleviating depression and anxiety in pregnancy during the COVID-19 pandemic. Clinical Trial Registration: The study was registered at http://www.clinicaltrials.gov, registration number NCT04377412.