theres a lot of debate as to what the "best" mp3 player wise, personally I don't like to view things in terms of "good, better, best", because theres always going to be something bigger and better if you dig enough
although there are a few brands that produce mp3/mp4 players that provide similar quality and features to the iPod (afaik nothing is going to provide full 1:1 with the iPod Touch/iPhone though, mostly due to the data abilities and UI capabilities)
I'd suggest you have a look at Archos, SanDisk, Sony (my personal preference), and Creative (the Zen X-Fi was/is probably their most fully featured portable, iirc it had wifi abilities and a few other things, I just remember it was rather expensive the last time I went to buy an mp3 player, so I passed)
I've owned the 3g, 4g, 5g, touch, and 2 nano ipods. I always thought that the click wheel iPods have the best interface, while the Touch is more versatile. The click wheel iPods are probably my favorite MP3 players.
Also own a Sansa Fuze and Zune HD. The OLED on the Zune screen is ridiculously gorgeous. If the idea of a stripped-down iPod touch in a smaller form factor sounds good, you might want to gve the Zune HD a look.
I can wholeheartedly recommend the Sansa Fuze for it's sound quality and format support. And it's cheap and has a MicroSD slot. I auditioned alot of MP3 players before I bought mine, and was willing to spend considerably more than I did, but none of them sounded as good as the Fuze. Strange given it's price but it really is the best sounding IMHO. The UI isn't great, but it gets the job done, and the controls I really do like. Only thing I don't like is battery life, seems to be around 14 hours, which is low in it's class.
There is an 'audiophile' grade mp3 player that is fairly inexpensive (and new to the market to boot), and there have been a ton of positive reviews on it.
You can read about 118 pages of reviews, pros/cons and every type of comment made about the product at head-fi.org. Here's the direct link: Amp3 mp3 player
If you are more into form over function, this player isn't for you. It's a plain box housing (metal) with a stripped down, no-frills user menu. BUT, it's main purpose is to bring the best sound quality to the user...and from the reviews I've read, it does exactly that.
Also, it's paired with a set of high-end Yuin headphones that matches the sound signature of the unit nicely.
I plan on buying one soon, and at $150, it's very easy on the wallet. Whiplashaudio.com is the only distributor of the Amp3 that I know of. Take it into consideration if you are looking for the best sound quality available.
Best all-around inexpensive ultraportable/workout music player:
Sansa Clip+ - great sound quality, small size, large format support (mp3/wmv/ogg/flac), expandable storage, and a built-in clip. But it does nothing but play music.
Player that will give you the best video playback experience:
Zune HD - gives you excellent video quality on the OLED display, plus offers an expansion dock (for extra cost) to output 720p video to an HDTV.
Players highly touted as "the best " in sound quality:
Cowon players (with the exception of the new screenless A2) are highly praised for "the best" audio quality and options for tweaking your audio experience. Now, don't get me wrong: every single player I've listed here has good to great sound quality. But this one is consistently rated at the top. It's also a bit more expensive than other options.
Player that will give you the closest experience to the closed Apple system (if you enjoy that sort of thing):
Zune HD - good overall player with a small list of apps, and a highly-rated software package for managing your music. Includes Zune Marketplace for buying tunes and movies.
Player that "does it all" well, while not being heavy or bulky:
Sansa Fuze - does audio, marginal video, pictures, microSDHC expansion, good battery life, and has the same great sound quality as the Clip+.