Jump to content
Storyist Forums

Forum slowdowns


codemer

Recommended Posts

Has anyone else noticed that the forums seem to slow to a crawl right about the time that Google shows up in the active users list? I wonder if there's a way to make them crawl with a little less aggression.

 

IF

Hadn't linked it to Google, but I have noted that in the middle of the day the forums make molasses look like cheetahs (there's a mixed metaphor for you!).

M

Link to comment
Share on other sites

Hadn't linked it to Google, but I have noted that in the middle of the day the forums make molasses look like cheetahs (there's a mixed metaphor for you!).

M

 

I've seen this too.

 

I've contacted support a couple of times and the answer is usually like this one: "We received no other complaints about this issue from any other users on the server. When we logged in the server load was fine and there were no issues with any locked queries."

 

Lets use this thread to document the instances of the problem.

 

-Steve

Link to comment
Share on other sites

I didn't tie it to Google either but...

It took me roughly a minute to both pull up the forum and then log on. Google was there. I did it just now in under 4 seconds. No Google. Still, it could be a coincidence. I'll have to check again adjusting for general line traffic (there's always less in the wee hours).

-Thoth.

Link to comment
Share on other sites

I didn't tie it to Google either but...

It took me roughly a minute to both pull up the forum and then log on. Google was there. I did it just now in under 4 seconds. No Google. Still, it could be a coincidence. I'll have to check again adjusting for general line traffic (there's always less in the wee hours).

-Thoth.

Just tried it again with Google there: 4 seconds.

They might not be the problem.

-Thoth.

Link to comment
Share on other sites

It just took me 50 seconds just to get to the announcements thread. (Google present.) When I clicked on Storyist Forums to get out I got this after 60 seconds:

 

Safari can’t open the page.

Safari could not open the page “http://storyist.invisionzone.com/index.php?act=idx” because the server is not responding.

 

This pretty much cinches it. It's them. But exactly how is it them?

 

-Thoth.

Link to comment
Share on other sites

I had the same experience. The forums had been loading quickly, but just now the page took forever. When it finally loaded, Google was listed. I couldn't load your most recent post at all, Thoth. I clicked on stop, waited five minutes, came back and tried again—no Google, page loaded quickly. In fact, the forums fulfilled my previous canceled request and loaded this thread without asking.

 

It is weird. Could the indexing slow things down? That's basically what Google does, right?

 

Until Isaac mentioned it, I thought the slowdowns were caused either by heavy midday traffic or by one person requesting a thread that someone else was working on. But so far, the evidence does seem to point to Google.

M

Link to comment
Share on other sites

It is weird. Could the indexing slow things down? That's basically what Google does, right?

Its spiders grab every page but it does its indexing at their end. Still, you're right: what Google does could be slowing us down. But why does it keep popping up so often? I've seen it morning, noon and night, on and off.

 

Until Isaac mentioned it, I thought the slowdowns were caused either by heavy midday traffic or by one person requesting a thread that someone else was working on.

Same here.

 

But so far, the evidence does seem to point to Google.

There is another possibility. My health insurance company's web site is always warning us that on such-and-such a date they'll be taking "some" of the servers down, and that will slow things. Perhaps the tech people for Invision Zone are doing something similar. It doesn't have to be a whole server. It could be a control unit or a modem rack or whatever. It could even be a load balancing problem in the software. I'm just saying that it's still possible it's Invision Zone, not Google.

 

-Thoth.

Link to comment
Share on other sites

Its spiders grab every page but it does its indexing at their end. Still, you're right: what Google does could be slowing us down. But why does it keep popping up so often? I've seen it morning, noon and night, on and off.

 

Here's what I know about spiders. Many can detect volatile web sites like blogs and forums, and will scan them more aggressively to keep its indexes up to date. I've seen a little of this on my own blog. When I updated it more regularly, it was more likely to get scanned in successive days. Normally it's scanned much less often (once a week or once a month by most search engines).

 

It's not just Google, though. I've seen the MSN spiders behave similarly.

 

Now, here's where success has its problems. The forums get enough posts to warrant regular scanning. Yet, as we post more content, it will take the spiders longer and longer to scan the forums. The search engines may direct more spider to the forums as the content grows so they can continue to scan the forums in a timely manner.

 

How do we verify this? The way I did it on my site was just to look at the web logs. I found MSN to be more aggressive than Google in spidering my site. Scans were closer together, and they tended to hit my site from multiple IP addresses instead of one. I never see msn in the active users list, so I suppose the forum authors probably don't care about it enough to add it, but I'll bet they're still scanning the forums.

 

This is just based on what I've seen on my own web site. There are also spiders for yahoo, altavista, webcrawler, and about a gazillion other search engines that show up in my web logs from time to time. This is a popular site, so I'd analyze the log files and make sure there is enough capacity to handle multiple spiders hitting at once.

 

IF

Link to comment
Share on other sites

Here's what I know about spiders. Many can detect volatile web sites like blogs and forums, and will scan them more aggressively to keep its indexes up to date. I've seen a little of this on my own blog. When I updated it more regularly, it was more likely to get scanned in successive days. Normally it's scanned much less often (once a week or once a month by most search engines).

 

It's not just Google, though. I've seen the MSN spiders behave similarly.

 

Now, here's where success has its problems. The forums get enough posts to warrant regular scanning. Yet, as we post more content, it will take the spiders longer and longer to scan the forums. The search engines may direct more spider to the forums as the content grows so they can continue to scan the forums in a timely manner.

 

How do we verify this? The way I did it on my site was just to look at the web logs. I found MSN to be more aggressive than Google in spidering my site. Scans were closer together, and they tended to hit my site from multiple IP addresses instead of one. I never see msn in the active users list, so I suppose the forum authors probably don't care about it enough to add it, but I'll bet they're still scanning the forums.

 

This is just based on what I've seen on my own web site. There are also spiders for yahoo, altavista, webcrawler, and about a gazillion other search engines that show up in my web logs from time to time. This is a popular site, so I'd analyze the log files and make sure there is enough capacity to handle multiple spiders hitting at once.

 

IF

I did just experience a slowdown, however, although Google didn't seem to be around.

M

Link to comment
Share on other sites

Here's what I know about spiders. Many can detect volatile web sites like blogs and forums, and will scan them more aggressively to keep its indexes up to date. I've seen a little of this on my own blog. When I updated it more regularly, it was more likely to get scanned in successive days. Normally it's scanned much less often (once a week or once a month by most search engines).

 

The forum ought to be able to handle the traffic. I'd put my money on the general server load (there are other clients on this server).

 

-Steve

Link to comment
Share on other sites

The forum ought to be able to handle the traffic. I'd put my money on the general server load (there are other clients on this server).

I think I'll hold onto my money, thank you, but someone should take a look at those server logs. If the general server load is the culprit then that should be verifiable. But then what? It's not like we can ask the other Invision Zone clients to leave. And I'm pretty sure Invision Zone will be sticking to their own capacity expansion schedule since "We received no other complaints about this issue from any other users on the server. When we logged in the server load was fine and there were no issues with any locked queries."

 

-Thoth

Link to comment
Share on other sites

If the general server load is the culprit then that should be verifiable. But then what? It's not like we can ask the other Invision Zone clients to leave. And I'm pretty sure Invision Zone will be sticking to their own capacity expansion schedule since "no one has complained."

-Thoth.

 

We can document the problems and asked to be moved to another node.

 

-Steve

Link to comment
Share on other sites

Maybe, though they seem to be able to run their corporate board (which is significantly larger than this one) with no trouble.

 

I'd be willing to bet that their corporate board is on a much less loaded, if not dedicated, server.

 

>host storyist.invisionzone.com

storyist.invisionzone.com has address 208.67.212.53

>host forums.invisionzone.com

forums.invisionzone.com has address 208.67.212.58

 

Ask to have all the porn people moved to their own servers. That's what a bunch of us DreamHost customers did. :D

 

IF

Link to comment
Share on other sites

We can document the problems and asked to be moved to another node.

Do you want this documentation in any particular format? E.g., Server Unavailable at 2pm; 5 minutes to long on at 3am, Google present; Had to Force Quit at 4am. That sort of thing?

-Thoth.

Link to comment
Share on other sites

I'd be willing to bet that their corporate board is on a much less loaded, if not dedicated, server.

 

>host storyist.invisionzone.com

storyist.invisionzone.com has address 208.67.212.53

>host forums.invisionzone.com

forums.invisionzone.com has address 208.67.212.58

 

Ask to have all the porn people moved to their own servers. That's what a bunch of us DreamHost customers did. :D

 

IF

I, for one, wouldn't be at all surprised. It is, after all, the site they use to sell their service. Still, it is technically possible for device 53 and device 58 to be part of the same physical machine or virtually share the same I/O channels. Then again, they could be running a huge server farm where 58 is actually several physical machines.

 

So where does this speculation get us? Moving to another node could make things worse. Would they even tell is the loads on virtual servers? Assuming they use them. Does Steve's contract limit foreign load capacity on a physical machine? My head hurts.

-Thoth.

Link to comment
Share on other sites

I, for one, wouldn't be at all surprised. It is, after all, the site they use to sell their service. Still, it is technically possible for device 53 and device 58 to be part of the same physical machine or virtually share the same I/O channels. Then again, they could be running a huge server farm where 58 is actually several physical machines.

 

I would imagine that 53 and 58 are different machines because you gain no technological benefit from putting multiple IP addresses on a single server for a single application. If they were divvying up the IPs for SQL, Web, etc, it might make sense, but honestly, the SQL server should be behind the DMZ. Some people like having a static IP so the reverse DNS resolves to their trademarked domain, but that does not appear to be the case here:

 

>host 208.67.212.53

Host 53.212.67.208.in-addr.arpa not found: 2(SERVFAIL)

>host 208.67.212.58

Host 58.212.67.208.in-addr.arpa not found: 2(SERVFAIL)

 

So where does this speculation get us? Moving to another node could make things worse. Would they even tell is the loads on virtual servers? Assuming they use them. Does Steve's contract limit foreign load capacity on a physical machine? My head hurts.

 

Speculation helps users to vent their frustrations (or breeds malcontent, but I don't think we have those types in this crowd).

 

I checked out how much these guys charge for hosting, though, and I think they should fix this problem. I'm only paying $10/month for my web service, and rarely see slowdowns like I see here (lower frequency, less severity).

 

IF

Link to comment
Share on other sites

I would imagine that 53 and 58 are different machines because you gain no technological benefit from putting multiple IP addresses on a single server for a single application.

Unless they're running as multiple virtual machines. It's all in the architecture...

 

If they were divvying up the IPs for SQL, Web, etc, it might make sense, but honestly, the SQL server should be behind the DMZ. Some people like having a static IP so the reverse DNS resolves to their trademarked domain, but that does not appear to be the case here:

 

>host 208.67.212.53

Host 53.212.67.208.in-addr.arpa not found: 2(SERVFAIL)

>host 208.67.212.58

Host 58.212.67.208.in-addr.arpa not found: 2(SERVFAIL)

...But I see your point.

 

Speculation helps users to vent their frustrations...

And it's fun too!

 

I checked out how much these guys charge for hosting, though, and I think they should fix this problem. I'm only paying $10/month for my web service, and rarely see slowdowns like I see here (lower frequency, less severity).

I applaud your research, sir. Just how much do they charge for their service?

 

Curious.

-Thoth.

Link to comment
Share on other sites

Unless they're running as multiple virtual machines. It's all in the architecture...

 

I do a LOT with virtual machines, and I can tell you it does not make sense to bust web hosting into virtual machines unless the clients are paying for it. While there are some discount virtual hosts out there, you're probably looking at starting around $25/month for anything halfway decent.

 

I can see maybe putting the web host under a single virtual machine for security reasons, but it consumes too many resources to use multiple virtual machines on a single host for non-dedicated hosting.

 

Now, this is my opinion, and does not mean that some crazy fool would not go against reason and do this anyway.

 

I applaud your research, sir. Just how much do they charge for their service?

 

They had plans starting around $10/month, but with a fraction of the storage (~3/10000) and bandwidth (~8/10000) that I get from my $10/month plan. I'm not exaggerating. For the trade off in space and bandwidth, I would expect beefier servers that do not get bogged down. I do have to confess, though, that my provider rewards you for being a loyal customer, and my service keeps improving with time, even if there have been a few disasters along the way (like that time they accidentally charged half their customers for an extra year of service. Oops!).

 

IF

Link to comment
Share on other sites

I do a LOT with virtual machines, and I can tell you it does not make sense to bust web hosting into virtual machines unless the clients are paying for it. While there are some discount virtual hosts out there, you're probably looking at starting around $25/month for anything halfway decent.

 

I can see maybe putting the web host under a single virtual machine for security reasons, but it consumes too many resources to use multiple virtual machines on a single host for non-dedicated hosting.

 

Now, this is my opinion, and does not mean that some crazy fool would not go against reason and do this anyway.

Calm down, Beagle Bunny. I'll take your word on the current VM pricing schemes (for application VM & process VM?) and their quality. I can only say that there are plenty of "crazy fools" out there that reason differently than you. Perhaps their priorities are a wee bit different then yours. Who knows? (The crazy fools!)

 

They had plans starting around $10/month, but with a fraction of the storage (~3/10000) and bandwidth (~8/10000) that I get from my $10/month plan. I'm not exaggerating. For the trade off in space and bandwidth, I would expect beefier servers that do not get bogged down. I do have to confess, though, that my provider rewards you for being a loyal customer, and my service keeps improving with time, even if there have been a few disasters along the way (like that time they accidentally charged half their customers for an extra year of service. Oops!).

"Oops!" indeed.

-Thoth.

Link to comment
Share on other sites

Calm down, Beagle Bunny. I'll take your word on the current VM pricing schemes (for application VM & process VM?) and their quality. I can only say that there are plenty of "crazy fools" out there that reason differently than you. Perhaps their priorities are a wee bit different then yours. Who knows? (The crazy fools!)

 

 

"Oops!" indeed.

-Thoth.

Google was there just now when I came back to check for messages, so I clicked on this post to see if it would cause a slowdown. It didn't.

Results mixed earlier, too.

M

Link to comment
Share on other sites

Google was there just now when I came back to check for messages, so I clicked on this post to see if it would cause a slowdown. It didn't.

 

It can be so hard to tell because they will stay on the active user list for several minutes past when they finish scanning. It's probably some combination of factors, though. I think Google may be the straw that broke the camels back, so to speak, when we see the forums slow down.

 

IF

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...