Objective Dementia is a common neurological disease that substantially affects public health. A previous study revealed that dementia occurs when the body’s immune system attacks the cells of the brain, indicating that dementia may be similar to autoimmune rheumatic diseases (ARDs). In the current retrospective cohort study, we focused on middle-aged ARD patients (45 years or older) to investigate the association between ARDs in middle-aged people and dementia by using a nationwide population-based database in Taiwan. Method Our study analyzed the medical data of the Taiwanese population from 2001 to 2012, with a follow-up period extending until the end of 2011. We identified middle-aged patients with ARDs by using the Taiwan National Health Insurance Research Database. We selected a comparison cohort from the general population that was randomly frequency-matched by age (in 5-year increments), sex, and index year and further analyzed the dementia risk by using a Cox regression model that considers sex, age, and comorbidities. Results The study enrolled 34,660 middle-aged ARD patients (77% female, mean age = 59.8 years) and 138,640 controls. The risk of developing dementia was 1.18 times higher for middle-aged patients with ARDs compared with patients without ARDs after adjustment for age, sex, and comorbidities. Among the patients with ARDs, the subgroups with rheumatoid arthritis, systemic lupus erythematosus, and Sjögren syndrome (SS) were associated with a significantly higher dementia risk (adjusted hazard ratio [HR] 1.14, 95% confidence index [CI] 1.06–1.32; adjusted HR 1.07, 95% CI 0.86–1.34; adjusted HR 1.46, 95% CI 1.32–1.63, respectively). Furthermore, primary SS and secondary SS patients had the highest risks of dementia among all the ADR subgroups (adjusted HR 1.35, 95% CI 1.18–1.54; adjusted HR 1.67, 95% CI 1.43–1.95 respectively). Conclusion This nationwide retrospective cohort study demonstrated that dementia risk is significantly higher in middle-aged patients with ARDs compared with the general population.