Delay when accessing sites on my server

B

bozrdang

Weaksauce
Joined
Aug 3, 2004
Messages
89
I run an unraid server at home which runs a bunch of Docker containers. I recently set up Letsencrypt and nginx as a reverse proxy so that I can securely access some of my containers (Emby, Radarr, Sonarr, NZBget, Nextcloud and a few others). I purchased a domain through Namecheap and use their DDNS service to deal with my residential dyanmic IP. I use subdomains to access my Docker sites (ex. emby.myserver.com)


I have everything working but I experience a 20-25 second delay sometimes when accessing any of those sites via their domain. When I type the address and hit enter absolutely nothing will happen for 20-25 seconds. The page doesn't change at all. As soon as the the first element of the page I am trying to go to appears, the rest of the page loads as quickly as I'd expect. I experience this no matter if I am trying from another computer on my LAN or a remote computer over the internet. Sometimes the sites load just fine. It seems totally random.

When I try to access via IP address and port I do not experience and delays. I assume this is some type of DNS issue but frankly, I don't know much about networking. I followed a tutorial to set up the reverse proxy.

Where can I start to troubleshoot this and try to track down the issue?
 
That's a long time to get a response. Potentially the DNS you are using right now. Still really odd that it would take that long...

Why don't you resolve the domains locally without going out to the internet just to come back into your own network? Modify your hosts file to point to the local IP address. See if you experience the same issue.
Edit: here's a really good guide if you need one: https://www.inmotionhosting.com/support/website/how-to/modifying-your-hosts-file
 
Because ultimately I need things to work right over the internet. If it weren't for that, I wouldn't have bothered with any of this and just used the local IP.
 
bozrdang said:
Because ultimately I need things to work right over the internet. If it weren't for that, I wouldn't have bothered with any of this and just used the local IP.
Click to expand...
The first step to ruling things out is to set it up locally as I've described.
 
I added the following lines to my host file on my laptop:

emby.myserver.com 192.168.1.42:8096
nzbget.myserver.com 192.168.4.42:6789
sonarr.myserver.com 192.168.1.42:8989

I then rebooted my laptop (couldn't remember if I needed to or not). I still sometimes have the delay if I type any of them in my browser (chrome always appends https:// to the beginning if that matters.)
 
Last edited:
bozrdang said:
I added the following lines to my host file on my laptop:

emby.myserver.com 192.168.1.42:8096
nzbget.myserver.com 192.168.4.42:6789
sonarr.myserver.com 192.168.1.42:8989

I then rebooted my laptop (couldn't remember if I needed to or not). I still sometimes have the delay if I type any of them in my browser (chrome always appends https:// to the beginning if that matters.)
Click to expand...
That about browsers other than Chrome?
 
I just tried Edge and get the delay with it also. Again, not always, but sometimes. One thing though. I went and edited the host file again so that sonarr.myserver.com was directed to the IP and pot for NZBget. I then rebooted and typed sonarr.myserver.com into both Chrome and Edge and it still took me to Sonarr. Even after clearing the browser cache.
 
bozrdang said:
I just tried Edge and get the delay with it also. Again, not always, but sometimes. One thing though. I went and edited the host file again so that sonarr.myserver.com was directed to the IP and pot for NZBget. I then rebooted and typed sonarr.myserver.com into both Chrome and Edge and it still took me to Sonarr. Even after clearing the browser cache.
Click to expand...
Okay. We know the issue happens on Windows machines, whether using external DNS or bypassing external DNS. Browser doesn't matter, so now we want to determine if it's specific to the operating system you're using on your PCs, or if it is the server causing this. Since you already said that accessing it via the IP address doesn't experience these issues, it's looking like it isn't the server that's the issue. What happens when you access it from a phone? Have a Linux machine you can access it from?
 
Don't have a Linux computer but the problem also happens on my Android phone. Another odd thing about this is that sometimes (like right now) it doesn't do it at all. No matter how many times I navigate to all my sites they load as expected. It's rare that it doesn't do it at all though. Usually I'll experience it at least once out of every 5 tries. So darn weird!
 
bozrdang said:
Don't have a Linux computer but the problem also happens on my Android phone. Another odd thing about this is that sometimes (like right now) it doesn't do it at all. No matter how many times I navigate to all my sites they load as expected. It's rare that it doesn't do it at all though. Usually I'll experience it at least once out of every 5 tries. So darn weird!
Click to expand...
Okay. So, no matter what device you access it from, as long as you access from the domain you experience the issue. But, if you access from the IP address you don't experience the issue. Have you done any troubleshooting on the server? Restarted it to see if anything changes? What guide did you follow to set it up? What parts of the guide did you do different? Have you completely started from scratch and see if the same thing happens?
 
Yes, I have rebooted the server. In fact I just rebuilt the server with all new hardware. I started out using CyanLabs guide (http://tinyurl.com/ydg8n5ph) and couldn't get things working. So I deleted and reinstalled and used the Linuxserver.io guide (http://tinyurl.com/y7xxtw6y) as it was their Docker package I was using anyhow and got things working. In trying to solve the delay issue I ended up talking with an experienced nginx user on the Emby forums who spent quite a bit of time helping me and changed a bunch of things with no luck.
 
bozrdang said:
Yes, I have rebooted the server. In fact I just rebuilt the server with all new hardware. I started out using CyanLabs guide (http://tinyurl.com/ydg8n5ph) and couldn't get things working. So I deleted and reinstalled and used the Linuxserver.io guide (http://tinyurl.com/y7xxtw6y) as it was their Docker package I was using anyhow and got things working. In trying to solve the delay issue I ended up talking with an experienced nginx user on the Emby forums who spent quite a bit of time helping me and changed a bunch of things with no luck.
Click to expand...
Very peculiar indeed. That's good information to know. Especially that someone else has already looked at it. Have you tried the exact same steps on another machine to see if you get the same results?
If I get some time this week and you haven't made any progress, I'll follow the instructions and see what I get
 
No I haven't tried another machine or setup. I don't have an easy way to do that at the moment.
 
Namecheap is my domain provider. I just looked and the name server is set to Namecheap Basic DNS.

The other options are: Namecheap Web Hosting DNS and Custom
 
bozrdang said:
Namecheap is my domain provider. I just looked and the name server is set to Namecheap Basic DNS.

The other options are: Namecheap Web Hosting DNS and Custom
Click to expand...
Remember that according to the testing done, it happens whether he uses external DNS or bypasses it with hosts file mod. So, the external DNS has been ruled out.
 
FNtastic said:
Remember that according to the testing done, it happens whether he uses external DNS or bypasses it with hosts file mod. So, the external DNS has been ruled out.
Click to expand...

Except that above I said:

" I went and edited the host file again so that sonarr.myserver.com was directed to the IP and pot for NZBget. I then rebooted and typed sonarr.myserver.com into both Chrome and Edge and it still took me to Sonarr."

So it didn't seem like the host file mattered.
 
bozrdang said:
Except that above I said:

" I went and edited the host file again so that sonarr.myserver.com was directed to the IP and pot for NZBget. I then rebooted and typed sonarr.myserver.com into both Chrome and Edge and it still took me to Sonarr."

So it didn't seem like the host file mattered.
Click to expand...
I went back and reread again. Is it taking you to the wrong URL? Your statement that it's bringing you to Sonarr is vague. Please include the URL it directs you to.
Try doing the hosts file mod without the ports at the end of the IP address. If you need to, add the ports to the end of the URL when you type it in your browser. As a friendly reminder, be sure you clear your DNS cache after making the changes.
As a side note, Namecheap DNS is completely capable at pushing it your records. If you are experiencing a delay, it's typically related to the DNS that you have your devices using. However, you have already stated that you've tried outside your network, and we would assume those devices are using different DNS than you are on your network. As a test, you could update your local DNS to another provider, but based on everything we have so far, this doesn't appear to be DNS related, and I wouldn't recommend spending time there until you completely replicate on another server with different domains.
 
I'm not sue how much you know about Unraid and/or Docker, but Unraid is a NAS software that has Docker built in. With it's configuration, all the Docker containers share the same IP as the host (Unraid) so the port is crucial for directing to the proper container. Three of the Docker containers I run are Emby, NZBget and Sonarr. In my case Unraid's IP is 192.168.1.42. So Emby's is 192.168.1.42:8096, NZBget's is 192.168.1.42:6789 and Sonarr's is 192.168.1.42:8989. So I need to include the port to direct to the proper site/container.

After I added the info to the host file and still experienced the delays I wondered how to know that it actually using the host file. So as a test I made one line read as follows:

sonarr.myserver.com 192.168.1.42:6789

So I purposely pointed the domain for Sonarr (sonarr.myserver.com) to NZBget's IP and port. If the host file was directing the traffic shouldn't I have gone to NZBget's page by going to sonarr.myserver.com? But I didn't I was still taken to Sonarr's page. So I am assuming that, for whatever reason, the host file was rendered moot.

I am not sure how to clear my DNS cache. I'll have to look that up. What do you mean by "completely replicate on another server with different domains"? You're saying to get a different domain for testing?
 
bozrdang said:
I'm not sue how much you know about Unraid and/or Docker, but Unraid is a NAS software that has Docker built in. With it's configuration, all the Docker containers share the same IP as the host (Unraid) so the port is crucial for directing to the proper container. Three of the Docker containers I run are Emby, NZBget and Sonarr. In my case Unraid's IP is 192.168.1.42. So Emby's is 192.168.1.42:8096, NZBget's is 192.168.1.42:6789 and Sonarr's is 192.168.1.42:8989. So I need to include the port to direct to the proper site/container.

After I added the info to the host file and still experienced the delays I wondered how to know that it actually using the host file. So as a test I made one line read as follows:

sonarr.myserver.com 192.168.1.42:6789

So I purposely pointed the domain for Sonarr (sonarr.myserver.com) to NZBget's IP and port. If the host file was directing the traffic shouldn't I have gone to NZBget's page by going to sonarr.myserver.com? But I didn't I was still taken to Sonarr's page. So I am assuming that, for whatever reason, the host file was rendered moot.

I am not sure how to clear my DNS cache. I'll have to look that up. What do you mean by "completely replicate on another server with different domains"? You're saying to get a different domain for testing?
Click to expand...
https://stackoverflow.com/questions/8652948/using-port-number-in-windows-host-file
Like I said, type them in your domain in the browser URL if you need to use ports. The service you're running doesn't matter. How you're trying to access it is what matters. Over a specific port. I'm familiar with how docker and ports work. Please put the port at the end of the domain in your browser URL, NOT in the hosts file. The hosts file does not work that way, as explained in the link that I put here for you.

How to flush your local DNS cache: https://www.inmotionhosting.com/sup...ver-changes/how-do-i-clear-my-local-dns-cache

To test, ping sonarr.myserver.com after you make the changes and flush the local DNS cache on your machine.

What I mean by completely replicating on another machine is to follow the guide from scratch on another server. This will help you isolate if you need to start looking at hardware, start from scratch on your existing server, etc.
 
Last edited:
I changed my host file so it now reads:

emby.myserver.com 192.168.1.42
nzbget.myserver.com 192.168.1.42
sonarr.myserver.com 192.68.1.42

I then rebooted and flushed my DNS cache. If I type emby.myserver.com:8096, nzbget.myserver.com:6789 or sonarr.myserver.com:8989 into a browser I get "This page can't be reached". However, if I forward their ports in my router, then they work fine.

If I ping sonarr.myserver.com it gets through, but it shows my WAN address.
 
bozrdang said:
I changed my host file so it now reads:

emby.myserver.com 192.168.1.42
nzbget.myserver.com 192.168.1.42
sonarr.myserver.com 192.68.1.42

I then rebooted and flushed my DNS cache. If I type emby.myserver.com:8096, nzbget.myserver.com:6789 or sonarr.myserver.com:8989 into a browser I get "This page can't be reached". However, if I forward their ports in my router, then they work fine.

If I ping sonarr.myserver.com it gets through, but it shows my WAN address.
Click to expand...
Okay. That's good info.
You should have already forwarded, or opened the ports, for the internet traffic to reach the server. So, leave it that way. It could be related, or not.
With it in this configuration, did you experience the delay?
Remove the hosts file mod and flush DNS cache and try to access each. Do you experience the delay?
 
FNtastic said:
Okay. That's good info.
You should have already forwarded, or opened the ports, for the internet traffic to reach the server. So, leave it that way. It could be related, or not.
With it in this configuration, did you experience the delay?
Remove the hosts file mod and flush DNS cache and try to access each. Do you experience the delay?
Click to expand...

I'm confused. I thought the whole point running a reverse proxy was so that all traffic came in over https and that you could close all your other ports?
 
bozrdang said:
I'm confused. I thought the whole point running a reverse proxy was so that all traffic came in over https and that you could close all your other ports?
Click to expand...
Please run the tests. I'll reread your guides while you do that just to make sure what the intended operation is, so we can move forward with that after testing
 
After reading the guide again, I can only assume you deviated from it. They have
Code: 
listen 443 ssl;
    server_name nextcloud.server.com;
Can you please include your nginx config for reference?
 
I used the open port method for years. It worked fine. I set up Letsencrypt and nginx to close those ports and have a more secure server.
FNtastic said:
Okay. That's good info.
You should have already forwarded, or opened the ports, for the internet traffic to reach the server. So, leave it that way. It could be related, or not.
With it in this configuration, did you experience the delay?
Remove the hosts file mod and flush DNS cache and try to access each. Do you experience the delay?
Click to expand...

Yes. I still get the delay after doing this.
 
FNtastic said:
Afteeading the guide again, I can only assume you deviated from it. They have
Code: 
listen 443 ssl;
    server_name nextcloud.server.com;
Can you please include your nginx config for reference?
Click to expand...

Do you want the actual nginx.conf or the nextcloud file in the site-conf folder? The excerpt you quoted is from the nextcloud file, not the nginx.conf.
 
bozrdang said:
I used the open port method for years. It worked fine. I set up Letsencrypt and nginx to close those ports and have a more secure server.


Yes. I still get the delay after doing this.
Click to expand...
Something really funky is going on with your setup. Out of curiosity, what type of router do you have? Have you restarted it as part of your troubleshooting yet? I've definitely seen behavior like you're describing due to a router "freaking out" and erroneously, and intermittently, blocking traffic it shouldn't...
bozrdang said:
Do you want the actual nginx.conf or the nextcloud file in the site-conf folder? The excerpt you quoted is from the nextcloud file, not the nginx.conf.
Click to expand...
The nginx config for each subdomain you have.
 
My router is pfsense running on an old computer. I have rebooted it numerous tomes during this ordeal. And remember, while you are suggesting running the same server setup on on a different system, I have essentially done that. I rebuilt my server this past weekend with new mobo, cpu, ram and psu. Only kept the hard drives .

I have quite a few subdomains so I'll just post the three we've been discussing to start:

Emby:
Code: 
server {
    listen 443 ssl http2;
    server_name emby.myserver.com;
   
    include /config/nginx/ssl.conf;
   
    location / {
            proxy_pass http://192.168.1.42:8096;
        proxy_hide_header X-Powered-By;
        proxy_set_header Range $http_range;
        proxy_set_header If-Range $http_if_range;
        proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        ## WEBSOCKET SETTINGS ##
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $http_connection;

    }
}


NZBget:
Code: 
# make sure that your dns has a cname set for nzbget
# to enable password access, uncomment the two auth_basic lines

server {
    listen 443 ssl;

    server_name nzbget.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
#        auth_basic "Restricted";
#        auth_basic_user_file /config/nginx/.htpasswd;
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_nzbget nzbget;
        proxy_pass http://$upstream_nzbget:6789;
    }
}


Sonarr:
Code: 
# make sure that your dns has a cname set for sonarr and that your sonarr container is not using a base url
# to enable password access, uncomment the two auth_basic lines

server {
    listen 443 ssl;

    server_name sonarr.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
#        auth_basic "Restricted";
#        auth_basic_user_file /config/nginx/.htpasswd;
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_sonarr sonarr;
        proxy_pass http://$upstream_sonarr:8989;
    }
}
 
I added ipv6=off to the resolver line on both my NZBget and Sonarr configs. The Emby config doesn't utilize the resolver. Still have the delay on all three sites.
 
bozrdang said:
I added ipv6=off to the resolver line on both my NZBget and Sonarr configs. The Emby config doesn't utilize the resolver. Still have the delay on all three sites.
Click to expand...
Did you restart nginx after making the change? Also, take a look at that open issue in my last post (which looks like we posted at the same time)
 
FNtastic said:
Is it possible your issue is related to this open issue on the official docker github? This looks awfully close to what you've described as your setup and the behavior being experienced: https://github.com/docker/for-linux/issues/169
Click to expand...

A lot of that is over my head. Not sure what port mapping is in nginx. I know there is supposed to be a way in Unraid to give each container it's own IP but I'm not sure how to set that up. It would eliminate the need to specify ports if that's what they are talking about.
 
bozrdang said:
A lot of that is over my head. Not sure what port mapping is in nginx. I know there is supposed to be a way in Unraid to give each container it's own IP but I'm not sure how to set that up. It would eliminate the need to specify ports if that's what they are talking about.
Click to expand...
Are you running nginx in a docker container?
 
bozrdang said:
Yes.
Click to expand...
That bug report is specific to running nginx within a docker container with port forwarding (mapping) setup. The same conditions you are using. Try running nginx directly on the server (outside of docker). Or, live with the delay and wait for someone to address that open issue on github.
 
I appreciate the help! Maybe it's a bug, but many users are using it the same way I am with nginx in a Docker. A few even report that they are running production websites this way. That bug report does not mean it's an actual bug. IN fact it hasn't even been labeled a bug yet. I'll keep playing around with it and try to find the solution. Thanks again.
 
bozrdang said:
I appreciate the help! Maybe it's a bug, but many users are using it the same way I am with nginx in a Docker. A few even report that they are running production websites this way. That bug report does not mean it's an actual bug. IN fact it hasn't even been labeled a bug yet. I'll keep playing around with it and try to find the solution. Thanks again.
Click to expand...
It's reported as a bug by that user, but that's just playing semantics. The reason that is the best information moving forward is that you, another person you mentioned, and now us together have determined that by all accounts everything is configured properly. I agree that there are certainly still other possibilities. I encourage you to continue troubleshooting. Report your experience to docker if you can determine something. A couple things that might help: try different docker versions with different nginx versions, of course like I recommended before try nginx outside of docker, and you could always still try another server or starting from scratch again.
I'm really interested in hearing the results. Please share whatever you find or try, and if you experience different behaviors. I'm going to watch this thread for updates. And best of luck!
 
You must log in or register to reply here.
Back
Top