Riddler stuff isn't as good, but It does everything else better than the previous games:
Story - Somehow the Assassins Creed writers came up with a better Batman tale than Paul Dini could after two attempts. I guess they just had more experience with writing for the medium. Closer in tone to the comics and animated series (they actually fix/retcon Rocksteady's awful take on Bane), more interesting characterisation, and full of memorable moments. The Red Hood flashback sequence where Joker meets Harley is the absolute highlight of the series for me.
Combat - Greater enemy variety, OP REC gun removed, no attack cancel into counter, more aggressive enemies, and boss fights that actually fully utilize the combat mechanics.
Boss Fights - This game has them, and they all range from good to great. Playing through on NG+ even enhances some of the early game encounters in unexpected ways. Deathstroke, Copperhead, and the first Bane fight are all amazing. Even side mission bosses like Shiva and Deadshot are more accomplished than any of the boss fights from the Rocksteady games (Mr. Freeze is more of a predator puzzle than a fight).
Stealth - Harder predator rooms that encourage more varied play late game. Remote Claw allows for more creative approaches rather than trivializing stealth encounters as some claim.
Art Design - The snow covered exterior city environments are kind of bland, but interior locations are all among the best in the series (The Royal Hotel actually is the best, both in terms aesthetic and from a gameplay standpoint), and the character designs are fantastic. Origins Batman is one of my all-time favourite Batman designs.
Voice Acting - Kevin Conroy is the definitive Batman, but his performance has been off in all of the Rocksteady games. On the other hand, Roger Craig Smith is great as a younger, angrier Batman. Michael Gough is the series' best Gordon, and Troy Baker is as good a Mark Hamill stand-in as you could ever hope for. His Joker may largely be impersonation, but he completely nails it.
Hell, even the music is better. Christopher Drake's score is amazing. Joker's theme is the most ominous rendition of Carol of the Bells I've ever heard.
https://www.youtube.com/watch?v=Pvkmgi5VjOg
Don't even get me started Cold, Cold Heart. I love Arkham Knight, but Origins DLC alone is superior.