First of all, NEVER DELETE IE! (I don't think you can find an option to do so even in 'modify windows components), because it is not just used in showing web pages, a lot of internal files, such as the Help and Support programme uses Internet exploreer controls.
Shifting to other browsers is a very good choice, I personally thinkIE is very very crappy.
Google chrome is fast and simple. We get a larger view of the pageall the tie, there's nothing thats unnecessary, everything is clear and simple. You can perform all necessary functions (save web page, zoom in or out, use in-built download manager, password manager, auto-fill...) easily. Simplicity I would say is the best feature.
However, sometimes it closes abruptly, restarts itself( and may ask us whether or not to replace tabs...), which I find annoying, Leaving that, I would call Chrome the perfect browser.
Fierfox is fast and stable (start-up time is a little more than chrome), easy to use, with tabbed browsing... The most compelling thing is that it suports 'add-ons'. You can freely download add-ons and enhance your browsing experience to a great extent. It is a great gift if you are a web deevloper. Firefox has acquired tremendous popularity with recent trendof going open source.
I really love using Opera- its fast and easy to use, there are a lot of options. What I like the most is that we can save even Dynamic web pages perfectly (And I MEAN perfectly) in a single file (Web Archive).
Safari is good too, but its almost like Chrome except in looks.
What I've done is download all of the lot, coz I'm a web-developer, need to ensure cross-browser compatibilityof my web pages.
Good luck with your new browser!