Two Outs and the Inning Ends

It had already been a long inning, near the end of a long game when the following happened. One out, runners on first and second. Batter hits a line drive to the second baseman, who catches it. Both runners had run on contact, so the second baseman only had to jog a few feet to get the third out at second base. However, the second baseman thought he made the third out when he caught the ball. Before he touched second base, he casually rolled the ball towards the mound and trotted towards the dugout. Everyone else, including the two runners (who never re-touched) similarly ran to their dugouts with the impression the inning was over.
As this is happening, I look at my indicator which shows only 2 outs. My first thought was I must have missed an out somewhere, but the scorekeeper then told me she only had two outs as well. By now the teams have switched and the pitcher is doing his warmup throws. Finally, the coach is clued in by a parent and he walks over and says, "there were only two outs."
This was a freshman game and no scoreboard. I was working alone. I couldn't immediately think of the applicable rule, so I resorted to "baffle them with bulls*&t". I said the runner abandoned the base when he left the field and entered the dugout, hence he was out.
How would you have called it?