Lady Gaga has had the most impact and its not even questionable. She is regarded as the biggest star in the world for a reason. Adele smashed with one album, Rihanna only affects the billboard charts and nothing else, and katy is painfully local. Taylor is starting to branch out though, more to come later on her.
Gaga and Taylor are by far the most impactful of the bunch. People often overlook Taylor's impact because she's not a popstar, but what people don't get is how she changed the country music industry. She opened that world to an entirely new demographic: teenage girls. As for Gaga's impact, all you have to do is look at Gaga's contemporaries, such as Katy and Nicki Minaj.
I think Katy would come after those two, then Rihanna, then Adele.
Adele has made no impact on the industry. No one is changing their looks for her or their sound (let's be real, it's not like that sound would work for anyone other than Adele). Just because she sold a bunch of albums, doesn't mean she has had any impact.