Objective The aim of the study was to study clinical, imaging findings, response patterns, and immune-related adverse events in classical Hodgkin lymphoma (cHL) and non-Hodgkin lymphoma (NHL) patients treated with immune checkpoint inhibitors (ICIs). Methods A retrospective search was performed to identify patients with relapsed/refractory cHL and NHL treated with ICIs from 2015 to 2019. Clinical and laboratory data were collected. Imaging studies were reviewed for treatment response and immune-related adverse events. Results Ten patients with relapsed/refractory cHL (median age, 41 years) and 14 patients with relapsed/refractory NHL (median age, 61 years) were identified. Overall response rate was 70% for cHL patients. None of the NHL patients demonstrated complete or partial response. One case of hyperprogression and one case with atypical response were radiologically detected in cHL patients. Hypothyroidism requiring treatment occurred in 2 (20%) of 10 cHL patients, one of which had imaging correlate. Of 14 NHL patients, 1 (7%) had radiologic evidence of pneumonitis and 1 (7%) had colitis. Conclusions This single-institution observational study demonstrated that overall response rate was higher in patients with cHL undergoing ICI. Immune checkpoint inhibitor therapy has unique response patterns and toxicities in both cHL and NHL patients that radiologists should keep in mind.