In recent years, obesity has been widely considered an independent risk factor for diseases/disorders including inflammation, cardiovascular disease, and cancer. Adipocytes separate in diverse types of tissues, playing vital roles in not only homeostasis but also disease progression. Adipose tissue is not only an energy organ but is also an endocrine organ that can communicate with other cells in the microenvironment. In this review, we assess the functions of breast cancer-associated adipose tissue-derived extracellular vesicles (EVs) in the progression of breast cancer including proliferation, metastasis, drug resistance, and immune regulation. A better understanding of the role of EVs in the crosstalk between adipocytes and breast cancer will provide an understanding of the cancer biology and progression, which would further drive improvements of diagnostic strategies as well as therapeutic insights.