Background: The COVID-19 pandemic has been disseminating fear in the community, which has affected people’s quality of life, especially those with health problems. Health literacy (HL), eHealth literacy (eHEAL), and digital healthy diet literacy (DDL) may have potential impacts on containing the pandemic and its consequences. This study aimed to examine the association between the fear of COVID-19 scale (FCoV-19S) and the health-related quality of life (HRQoL), and to examine the effect modification by HL, eHEAL, and DDL on this association. Methods: A cross-sectional study was conducted in 11 hospitals across Vietnam from 7 April to 31 May 2020. Data were collected on 4348 outpatients, including demographic characteristics, HL, eHEAL, DDL, FCoV-19S, and HRQoL. Multiple linear regression and interaction models were used to explore associations. Results: Patients with higher FCoV-19S scores had lower HRQoL scores (unstandardized coefficient, B = −0.78, p < 0.001). HL (B = 0.20, p < 0.001), eHEAL (B = 0.24, p < 0.001), and DDL (B = 0.20, p < 0.001) were positively associated with higher HRQoL scores. The negative impact of FCoV-19S on HRQoL was significantly attenuated by higher eHEAL score groups (from one standard deviation (SD) below the mean, B = −0.93, p < 0.001; to the mean, B = −0.85, p < 0.001; and one SD above the mean, B = −0.77, p < 0.001); and by higher DDL score groups (from one SD below the mean, B = −0.92, p < 0.001; to the mean, B = −0.82, p < 0.001; and one SD above the mean, B = −0.72, p < 0.001). Conclusions: eHealth literacy and digital healthy diet literacy could help to protect patients’ health-related quality of life from the negative impact of the fear of COVID-19 during the pandemic.