Background: Right ventricular failure (RVF) is a significant cause of morbidity and mortality in patients with a left ventricular assist device (LVAD). This study is aimed to investigate the influence of a pectus excavatum on early and late outcomes, specifically RVF, following LVAD implantation. Methods: A retrospective study was performed, that included patients with a HeartMate 3 LVAD at our tertiary referral center. The Haller index (HI) was calculated using computed tomography (CT) scan to evaluate the chest-wall dimensions. Results: In total, 80 patients (median age 57 years) were included. Two cohorts were identified: 28 patients (35%) with a normal chest wall (HI 2.0) and 52 patients (65%) with pectus excavatum (HI 2.0–3.2), with a mean follow-up time of 28 months. Early (\le30 days) RVF and early acute kidney injury events did not differ between cohorts. Overall survival did not differ between cohorts with a hazard ratio (HR) of 0.47 (95% confidence interval (CI): 0.19–1.19, p = 0.113). Late (>30 days) recurrent readmission for RVF occurred more often in patients with pectus excavatum (p = 0.008). The onset of late RVF started around 18 months after implantation and increased thereafter in the overall study cohort. Conclusions: Pectus excavatum is observed frequently in patients with a LVAD implantation. These patients have an increased rate of readmissions and late RVF. Further investigation is required to explore the extent and severity of chest-wall abnormalities on the risk of RVF.