I don't think we have, on balance. Maybe if you go back to the 1920's/30's then we could arguably have had more success back then, but the reasons why we didn't are lost in the mists of time.
Taking the last 35 years (which I'm only choosing as that's the period of time in which I've been aware of football as a concept, and of sheff utd as "my team") I think we're about where we should be. We're not a huge club, we're not a sleeping giant. We're a biggish, well-supported club from a sadly unfashionable and left-behind 2-club city. We've had promotions and cup runs, as well as our share of bad times.
There is an argument that we've had some spectacularly fluky incidents of bad luck. In terms of winning cups, that was always unlikely, but we've been close to finals on a number of occasions, and been undone by some baffling decisions or sheer rotten luck. Seaman's save in 2003, the horrendous officiating at Anfield in the closing minutes in the same season. In the league, we've had the Ched Evans debacle, and most recently the introduction of VAR for a single game in a season without it, when, of course, it screwed us.
In the modern game, any team from outside a select group of maybe 10 actually winning anything is staggeringly rare. If you discount the "Big 6" of Arsenal, Chelsea, Liverpool, Man City, Man Utd and Tottenham, then since 1990 (36 years of trophies) the league cup has been won by Forest, Wednesday, Villa (2), Leicester (2), Blackburn, Middlesborough and Swansea. 9 times in 36 years, so exactly 1 in 4, or 24%.
The FA Cup over the same period, again excluding the big 6, has been won by Everton, Portsmouth, Wigan, Leicester and Palace (all 1 each). 5 out of 36, or roughly 14%.
So from 72 available trophies since 1990, 14 have been won by teams outside that group of 6. 19.5%, call it 1 in 5 to be generous. It doesn't make good reading for clubs like us who are outside the top table.
It's even worse for winning the actual League title. Only Leeds, Blackburn and Leicester have done that in the same time frame.
In total, that puts 23 teams above us in terms of trophies won since 1990, and most of them have only won 1. Of the teams mentioned, I'd argue we are a "bigger" (whatever that means) club than Wigan, Portsmouth, Blackburn and (endless debate aside, speaking in modern terms) Wednesday.
I'm sort of losing track of my original idea here, but I think the conclusion I come to is that, in terms of modern (2025) stature in the game, there are 4 clubs who have achieved demonstrably more than us who we are comparable in size to, or arbitrarily "bigger than." All 4 of them have since spent time in L1 or lower, and none have ever really looked like getting back to the top table.
I think we're about where we should be, at least over the last 35 years, in terms of what we've done. If we'd got lucky like some of these clubs, we might have fluked a cup win, but other than that, we've performed about as we should expect. Whether we should expect more, or deserve more, is a different debate, but I genuinely don't think we have really underachieved, in my lifetime at least.
(FWIW, my 3 votes were:
We haven't underachieved- we've had several promotions and cup semi-finals
The way in which we fail to hang on to good players/persuade them to stay
Too inwardly focused on having those with 'Blades connection' involved with the club)