Remember that this movie is part of the Marvel Avengers arc, consisting of Ironman (3 movies), Captain America (2), Thor (2), Hulk, and Avengers (2). This arc also includes two TV series, and also a (so far) unrelated Guardians of the Galaxy movie. Everything is leading to the Infinity Wars movies.
I point this out because while this movie seems like a fine standalone movie for those who have seen none or only a few of the others, the greatest enjoyment will be in seeing it within context. In particular, while Ironman 3 isn't a terribly good movie, it's worth seeing before this movie, as it explains what drives Stark to create Ultron.
With all that said, there are three things that really stand out in this movie.
The first is the Joss Whedon attention to character development. He's got a ton of characters to develop, and the movie does drag a little bit in places while he does it, but he does a phenomenal job. If you like character driven movies, then this is one to see. In particular, the bad guy (Ultron) isn't your generically sinister baddy, but a really interesting character in his own right.
The second is the typical Joss Whedon action sequences. In contrast to crap producers like Michael Bay who likes big explosions and big things crashing into each other, Whedon does creative action. You can see the chess game happening behind the screen, as pieces are gradually removed form the board. Each of the players does something unexpected, which causes the opponent to step up and match the game with something equally unexpected. If you like action driven movies, then this is one to see.
The third is the characteristic Joss Whedon dialog. He starts the dialog using a tried-and-true trope that we've all seen before, and then he either causes the scene to veer in a new direction, or simply interrupts it because it's pointless following through if we know how it's going to end. If you like movies with witty dialog, then this is one to see.
The movie does have problems. Despite my obvious love affair with Joss Whedon, I will admit that some parts didn't quite gel. It contains so much that not everything fits without wrinkles. But at the same time, removing stuff to smooth out the wrinkles would make a lesser movie, so I'm not sure that's a bad thing.
So for you, my gentle reader, I have no recommendation. If you are like me, and are enjoying the the Marvel arc (despite dissatisfaction with a few movies), then of course you are going to see Age of Ultron. The only question is where it fits. I'd place it at #3 in the series, after the first Avengers and first Ironman. In other words, go see it now in the theater with big screen and big sound, rather than simply catching up with it later on bluray, as you did with the Thor movies.
If you know little about the series, then I suspect you'll still enjoy this movie. It may make you want to go back and see some of the others, even Thor. It's still a worthwhile Whedon movie, and I'd always go on a date with Whedon.