The particular problem for House Republicans is this: when Congress is unpopular, voters don’t punish all House incumbents. Instead, they direct their dissatisfaction primarily at majority-party House incumbents. So argue political scientists David Jones and Monika McDermott in their book (see also this article). In the article, Jones finds that a 10-point decrease in approval costs majority-party House incumbents 4 points at the poll. This effect is larger in swing districts and has been getting larger over time, as the parties have polarized.