Browsers don't resolve IP addresses to domain name - getting "Invalid URL" pages

[tankman1989]

For years I've used the IP address to access some sites. The browser connects to the IP address as the domain name is only the pointer to the IP address so using the IP removes the DNS resolution from the step to get to the webpage and if the domain name changes, the site can still be accessed if it is still at the same location (IP address).

I tried about 20 sites and couldn't get to one by using the IP address except for Google, Ask and Yahoo (and only a few IP's worked for them). I used Windows, Linux and Andriod and tried 4 browsers on both Linux and Windows.

To find the IP address I used a few commands in Windows and Linux to resolve such as ping, tracerout, pingpath, dig, and a few others. I also used 4-6 website/tools to find the IP addresses linked to the domains. Almost all had the same IP addresses across all the domain names, some had a few results on the web tools and I think that is because they had multiple servers.

I took screen shots of some of the sites, 4 browsers per pic of the invalid URL, 4 pages with the web tool search results, shots of both windows and linux command line queries - all showing the addresses for the domain.

So what is very odd is that the only way I am able to access sites is by using the domain name.? For a domain to work, it must resolve the name to the IP address (which is what the browser actually uses to reach the site but then displays the domain if it has one), so why am I getting different results here?

 

[FrgMstr]

So what is very odd is that the only way I am able to access sites is by using the domain name.? For a domain to work, it must resolve the name to the IP address (which is what the browser actually uses to reach the site but then displays the domain if it has one), so why am I getting different results here?

What you are likely seeing is a security technique that is used to combat DDOS attacks. We do this with HardOCP now. The actual IP of our site sits behind CloudFlare and cannot be accessed directly. Not sure if that is exactly what you are seeing or not.

 

[rma]

the web server responds to what domain you resolve, therefore you end up with the IIS7 web page, that's normal behaviour.
whay are you using only IP's?

 

[Nicklebon]

A single IP can host many websites and often times does. The web server needs to know which one to serve up. This is nothing new.

 

[SJConsultant]

A number of things will prevent the IP from working, DDOS mitigation services, load balancers, SSL accelerators, multiple websites hosted using a single IP, reverse proxy servers, etc.

Use the proper fully qualified domain name.

 

[tankman1989]

the web server responds to what domain you resolve, therefore you end up with the IIS7 web page, that's normal behaviour.
whay are you using only IP's?

I was doing a test of single domain/IP address and I had the IP on my clipboard so I pasted it in the browser. The message I got was one I had never seen before so I tried another browser, and got a similar but different format message. Then I checked the IP on some websites (IP checker/whois and such) ping, etc. Had the correct IP. I tried it for a few big companies and they all did the same thing I tried 11 (very large) companies and got the same thing except for the 3 search engines.

 

[mwarps]

You have two really good trolling threads on here.. Either you are SHOCKINGLY unaware of how the internet works, you have suddenly stopped taking your meds, or you are actually the new Red..whateverhisnameishesblockedanyway.

"virtual host" - google it.

 

[tankman1989]

You have two really good trolling threads on here.. Either you are SHOCKINGLY unaware of how the internet works, you have suddenly stopped taking your meds, or you are actually the new Red..whateverhisnameishesblockedanyway.

"virtual host" - google it.

Could you explain what "SHOCKINGLY unaware of how the Internet works" is, meaning what should be known that "isn't" which would qualify as shocking - which I would assume means that this is "universal household knowledge" like a "household name". What info is so shocking that I didn't "know".

What is the last part that you posted "whateverhisnameishesblockedanyway"

Yeah, virtual hosts are all good and fine and I understand what they are but they usually aren't used where high volume sites are involved. That would be like using one IP for Google, yahoo, Microsoft and Apple on a "virtual host" - just because "it's possible". So all traffic has to go through a routing system to get to the correct site, instead of using a proper uniquie IP so no extra routing (NAT) is needed.

The sites I chose were all heavy volume/traffic sites which I expected to have their own uniquie IP, which I didn't find any other domains linked to their IP for all but 2 of the domains I searched.

 

[goodcooper]

Nicklebon and SJConsultant already answered your question, re-read them and research the terms and technologies they mentioned

 

[mwarps]

Could you explain what "SHOCKINGLY unaware of how the Internet works" is, meaning what should be known that "isn't" which would qualify as shocking - which I would assume means that this is "universal household knowledge" like a "household name". What info is so shocking that I didn't "know".

How the internet works: Open web browser, enter "google.com" into address bar, hit enter. The web browser does its thing. Done. Period. This is pretty much 'universal household knowledge'. - my 90 year old grandmother can do it.

IP addresses, dig, nslookup - sure as shit not household knowledge. Yes, you feel special that you know how to do something 5% of the world knows how to do.

Just for a historical overview, HTTP 1.1 requires the hostname to be provided in the request.
It came out in 1997.

Yeah, virtual hosts are all good and fine and I understand what they are but they usually aren't used where high volume sites are involved. That would be like using one IP for Google, yahoo, Microsoft and Apple on a "virtual host" - just because "it's possible". So all traffic has to go through a routing system to get to the correct site, instead of using a proper uniquie IP so no extra routing (NAT) is needed.

None of it uses NAT. None of it uses routing. A "proper unique" IP address is a pathological construct from your head.

Virtual hosting is how most if not all websites are run. Apache has had it for more than ten years. nginx was born with it. I'm not sure when IIS came out, but it's probably been there for just as long as apache.

Even if there is no other host, the webserver will throw you at the default virtual host.
It's not because "it's possible" - it's because it allows nearly infinite flexibility of management of the back end. With virtual hosting and/or loadbalancing, I can do anything. I can choose which back end to use (essentially 75% of the application delivery stack), where to send the request, whether to drop the request .. limitless possibilities if you understand how things actually work.

Just so you know, Virtual hosting isn't the only thing this pathological escapade into `dig` breaks. All front end caching requires URLs, loadbalancing can use URLs, HTTP/1.1 and over and a host of other things that I am not even aware of require and/or use URLs.

The sites I chose were all heavy volume/traffic sites which I expected to have their own uniquie IP, which I didn't find any other domains linked to their IP for all but 2 of the domains I searched.

Never, EVER do you want a high volume site with one IP address. EVER. This is suicidal for a business in the world today. Unless it's anycasted and/or loadbalanced, NEVER EVER, EVER, EVER have one IP address as a front end.