The Vincent Price episode could scare young kids due to it's spooky theme.
There are also some culturally insensitive and stereotypical depictions of human and puppet characters, including Native Americans, people from the Middle East, and others. Since this is a kid friendly show, this is not meant to be offensive, it played for laughs.
The Alice Cooper episode has some frightening musical numbers, and features Alice Cooper trying to get the muppets to sell their souls to the devil, though mostly comically.
The Liza Minnelli episode features muppets being murdered in a play, nothing too intense, and it has a happy ending.
Nothing much, though the explosions might scare the kids.