I had set up my Airport connections to use 18.104.22.168 and 22.214.171.124 as their DNS (those are Google's DNS IPs), however, I hadn't set my ethernet connections to use them. Somehow I managed to realize that it wasn't the router (after restoring to factory defaults), but that it was the DNS. I'm actually still impressed with myself for determining that and not getting carried away with my other possible solutions which included buying a new router and more drastically cancelling Comcast and switching to Verizon. But switching back and forth between Comcast's DNS and Google's DNS proved that for some reason Google's DNS was breaking Hulu for me.
Anyway, it will be interesting to see if the intermittent 2 minute blocks of no internet connectivity followed by a return to connectivity were also DNS-related.
Bye-bye Google DNS.