First of all, although I tend to agree that they would not have lost to Blue without external interference, it is hard to know if they have enough fire power to actually poof her. She has show to be incredibly durable, with a ship falling on her and all. Of course, it is possible that her power would deplete over time, but still, that would be a long, grueling fight.
Were they to fight Yellow by herself I still think they would be able to win (again, assuming they have the power to poof her). From the short time we saw Yellow fight we can know two things: She can zap gems into being poofed and, differently from Blue, she also fights with physical might - as she sprinted and stomped Steven, so it can be presumed she is not against kicking and punching. She would probably be tougher, but I don't know if her insta poof is unbeatable. a) Steven's shield can probably tank it. We are yet to see anything damage it; b) it can probably be dodged. Once the gems knew she could do that, they would be more careful; c) Steven and presumably Lion - and technically Connie but I wouldn't want her close to Yellow - are immune to Yellow's lightning and could act as shields. So if they overwhelmed her they could have a shot.
Of course, with the two at the same time, if Steven didn't psychic power his way to befriending them, the Crystal Gems would have lost.
Hoooooowever there is something to be considered. When Steven went into the mind world you could see that although the crystal gems' mind versions were all fighting and thinking about the fight, Yellow's and Blue's were not. They didn't appear to be fighting and their toughts were elsewhere. Thus, there is a distinc possibility that at no point the Diamonds really went full power.
Also, best person to last hit the Diamond in the fight is Pearl. Gems seem to be more vulnerable to piercing than anything else (Look at every poofing in the series and most of the times it was due to a sharp object - Hell, Jasper could survive being thrown into a ship's engine and then crashing into earth but a sharp metal stick poofed her good.