info Autoshite technical help. Ask questions about how the site works here.

[SiC]

Interesting and good feedback.

A few days ago I switched the server down a plan. The idea was to save €30 pcm - so not an inconsiderable amount.

Some may have noticed when I did this as I put a header announcement up for the 5 mins I did it. However I didn't want to leave anything permanent to gauge properly if there was a slow down. I knew there would be a slight slow down but that often can be small enough that people won't even realise. Evidently at times it's too much.

So that means going back to a bigger allocation of resources (more cores and more RAM). This brings it back to the €95 to €100pcm monthly fee (versus the €70pcm on this server). 

That just seems extraordinarily expensive for a 4 core, 16GB ram and 300GB storage setup. ('Unlimited' data on a 400Mbps pipe)

While out of flavour nowadays, I am tempted to move to a old fashioned dedicated server. Intel Xeon E3 1220 4 core with 32GB ram, 2x1TB SSD for €32.99pcm is much more reasonable. Disadvantage is that you don't get the benefits of cloud virtualization. i.e. I'd have to setup monitoring and keep an eye for hardware failures to then put support tickets in. Something that is outsourced with cloud and with a VPS, means I just need to keep the OS+services healthy. Rebuilding is also a lot more involved as no easy snapshotting like with a VPS. Likewise moving up to more performant server is not a 5 mins job with a dedicated server either like it is with a VPS.

But it would be a third less on the bill...

Or half the bill with higher performance that should have plenty of growth...

Cheaper the hosting, the more sustainable this site is with donations going further. But that needs to be balanced with admin overhead (we all have only so much time) and robustness of services. 

[loserone]

I am not sure that funding donations are an issue, are they? Seems like when the pot opens it fills pretty quickly?

[High Jetter]

€30 pm. Between all of us?

I'm tight, but that's nothing

[IronStar]

39 minutes ago, SiC said:

Interesting and good feedback.

A few days ago I switched the server down a plan. The idea was to save €30 pcm - so not an inconsiderable amount.

Some may have noticed when I did this as I put a header announcement up for the 5 mins I did it. However I didn't want to leave anything permanent to gauge properly if there was a slow down. I knew there would be a slight slow down but that often can be small enough that people won't even realise. Evidently at times it's too much.

So that means going back to a bigger allocation of resources (more cores and more RAM). This brings it back to the €95 to €100pcm monthly fee (versus the €70pcm on this server). 

That just seems extraordinarily expensive for a 4 core, 16GB ram and 300GB storage setup. ('Unlimited' data on a 400Mbps pipe)

While out of flavour nowadays, I am tempted to move to a old fashioned dedicated server. Intel Xeon E3 1220 4 core with 32GB ram, 2x1TB SSD for €32.99pcm is much more reasonable. Disadvantage is that you don't get the benefits of cloud virtualization. i.e. I'd have to setup monitoring and keep an eye for hardware failures to then put support tickets in. Something that is outsourced with cloud and with a VPS, means I just need to keep the OS+services healthy. Rebuilding is also a lot more involved as no easy snapshotting like with a VPS. Likewise moving up to more performant server is not a 5 mins job with a dedicated server either like it is with a VPS.

But it would be a third less on the bill...

Or half the bill with higher performance that should have plenty of growth...

Cheaper the hosting, the more sustainable this site is with donations going further. But that needs to be balanced with admin overhead (we all have only so much time) and robustness of services. 

Out of curiosity- where are we hosted?
That does sound quite expensive for what it is. Not major cloud providers expensive (hello AWS, yes, your pricing is still insane unless you’re a large business), but not Hetzner VPS cheapness either.

[SiC]

Out of curiosity- where are we hosted?
That does sound quite expensive for what it is. Not major cloud providers expensive (hello AWS, yes, your pricing is still insane unless you’re a large business), but not Hetzner VPS cheapness either.

Scaleway

The issue with most vps providers is they don't offer much storage with their packages. We're storage heavy (loads of images) but processing low in comparison.

[SiC]

Mega slow this morning. I'll switch it back up a plan later. 

[bunglebus]

It's been very laggy for me for several days now, I managed to do several duplicate posts too as it didn't seem like the "post" button had done anything, so I tapped it again.

It has been very good for several weeks, so whatever you you were up to worked.

I feel a bit like one of those annoying car owners that doesn't have a clue how their vehicle functions, just that it isn't quite right.

[IronStar]

45 minutes ago, SiC said:

 

Scaleway

The issue with most vps providers is they don't offer much storage with their packages. We're storage heavy (loads of images) but processing low in comparison.

 

Hetzner does 16vcpu / 32GB RAM / 320GB NVME / 20TB traffic for 39€ / month (including VAT). A bit under 47€/month with backups enabled. Extra storage available as an attached volume at 5.28€ per 100GB.

They really do crazy value for money, I don’t get their profit margins. 😄

[SiC]

45 minutes ago, IronStar said:

Hetzner does 16vcpu / 32GB RAM / 320GB NVME / 20TB traffic for 39€ / month (including VAT). A bit under 47€/month with backups enabled. Extra storage available as an attached volume at 5.28€ per 100GB.

They really do crazy value for money, I don’t get their profit margins. 😄

Problem is that 320GB isn't much to grow into. Attached storage is fine except the current allocations will make it a pain. It's about 50GB for the main boot image + software+config+etc, database and search index. Then 217GB of content. While that would all fit on one for now (but not a huge margin for growth after filesystem losses are taken into account), if that needs to migrate to attached storage then it'll be easiest to migrate that whole content storage across. Otherwise mounting half and half will be a pain in the arse.

Other option is to go for a straight Hetzner dedicated box. For similar money you can get dual 1TB nvme (so RAID 1). A mate runs a forum on one of their boxes and has found it fine. 

However I have heard (and seen plenty of reviews to back up the point) that they've been killing accounts suddenly with no warning for unidentified reasons. Whether they dislike your ID suddenly, don't like the traffic levels from your box (despite unlimited) or even simply because someone made a spurious abuse complaint. This has happened to individuals with accounts for 5-10yrs too. Doesn't give much confidence in their services nowadays. 

Flip side is that Scaleway isn't that much dissimilar pricewise really to Hetzner. More expensive but not massively more. 

[SiC]

Switched back up to the higher plan. All being well, performance should be back to how it was the other week.

[Split_Pin]

Thanks Si I knew you'd be on this. Happy to chuck some money in the covers when they open again.

[IronStar]

2 hours ago, SiC said:

Problem is that 320GB isn't much to grow into. Attached storage is fine except the current allocations will make it a pain. It's about 50GB for the main boot image + software+config+etc, database and search index. Then 217GB of content. While that would all fit on one for now (but not a huge margin for growth after filesystem losses are taken into account), if that needs to migrate to attached storage then it'll be easiest to migrate that whole content storage across. Otherwise mounting half and half will be a pain in the arse.

Other option is to go for a straight Hetzner dedicated box. For similar money you can get dual 1TB nvme (so RAID 1). A mate runs a forum on one of their boxes and has found it fine. 

However I have heard (and seen plenty of reviews to back up the point) that they've been killing accounts suddenly with no warning for unidentified reasons. Whether they dislike your ID suddenly, don't like the traffic levels from your box (despite unlimited) or even simply because someone made a spurious abuse complaint. This has happened to individuals with accounts for 5-10yrs too. Doesn't give much confidence in their services nowadays. 

Flip side is that Scaleway isn't that much dissimilar pricewise really to Hetzner. More expensive but not massively more. 

I agree, 320GB is really not much in this day and age.  

I somehow missed the random terminations though. I've been running a dedicated server with them for 6 or so years now, and never had any issues other than HDD (yes, I know, I really need to get rid of that ancient server) failing, but they sorted it within 30 minutes of me complaining that it's broken. Managing a dedicated is a bit of a faff, but it's not much worse than a VPS if you add Proxmox on top, especially if you're running a single instance on it. Attach an external storage for backups, and everything should be fine.

 

Unrelated to this, is it just me or it's still slow even after you got the instance back up to the bigger one? It was working well for a bit, then slowed down significantly again.

[SiC]

23 minutes ago, IronStar said:

Unrelated to this, is it just me or it's still slow even after you got the instance back up to the bigger one? It was working well for a bit, then slowed down significantly again.

I kicked off a search index rebuild earlier, so that might have pushed the load up a bit. Right now the load looks pretty low again now. 

[loserone]

2 hours ago, Split_Pin said:

Thanks Si I knew you'd be on this. Happy to chuck some money in the covers when they open again.

It's open the now, if someone could let SP know

[St.Jude]

3 hours ago, SiC said:

Switched back up to the higher plan. All being well, performance should be back to how it was the other week.

It’s still noticeably slow on page load for me.

[High Jetter]

6 minutes ago, St.Jude said:

It’s still noticeably slow on page load for me.

For me too but not every time, strangely. There doesn't seem to be any pattern that I can make out.

[Cookiesouwest]

37 minutes ago, High Jetter said:

For me too but not every time, strangely. There doesn't seem to be any pattern that I can make out.

Snap

[Markeh]

Yeah I'm noticing slowness too - it does seem to be more often than not.

[SiC]

I'm at work so don't really have much time to look at it atm unfortunately. But I'm seeing it going from low periods of barely any activity to it shooting right up inexplicably. Nothing obvious why except a lot of database process activity happening.

[GrumpiusMaximus]

Raid 0 that bastard and live on the edge.  You know you want to.

[St.Jude]

So noticing strange behaviour.

If I go to a page like this, and then I go back it’s slow loading. Then it’ll say page can’t be displayed, or I get this message:

Refreshing the page solves it.

[Scruffy Bodger]

Very hit and miss for me today, earlier on very slow indeed, then sped up gain, now slow again. Taking 10 to 15 seconds to load some pages.

[IronStar]

 

Some PHP script is probably not playing nice, and is repeatedly starving server of resources.

[SiC]

I've not had any time to diagnose this but I've disabled forum access to guests and the load has dropped right off. Which suggests some search engine crawler is hammering the site. 

Probably some front end caching system for not logged in guests would ease the load a lot. Just Invision is a bit shit for having anything like that in front of it and tends to have issues if not configured perfectly. 

[SiC]

CPU load is ridiculously low with guest access disabled. Which suggests that it is guest users that are hammering the site. I suspect it's not actually humans but more likely search engine crawler bots. 

I'm going to re-enable guest access again as it's important that lurkers can see the forum. But might mean things slow down again. If it does then it pretty much points to that fact. 

I'll have to figure a solution to get visibility for guests that doesn't hammer the server (likely some form of caching)

[SiC]

And boom, it's gone chronically slow again. *sigh* I'll have to disable guest access again for now. 

[Zelandeth]

2 hours ago, SiC said:

...Just Invision is a bit shit. 

Edited for accuracy.

Edit: Also for reference, that took the best part of a minute to post.

[IronStar]

45 minutes ago, SiC said:

CPU load is ridiculously low with guest access disabled. Which suggests that it is guest users that are hammering the site. I suspect it's not actually humans but more likely search engine crawler bots. 

I'm going to re-enable guest access again as it's important that lurkers can see the forum. But might mean things slow down again. If it does then it pretty much points to that fact. 

I'll have to figure a solution to get visibility for guests that doesn't hammer the server (likely some form of caching)

Are we using rate limiting on the Cloudflare side? That may do the trick if we do, set to something sensible like 250-300 requests / minute per IP..

I assume that all the other speedups and image optimizations are already enabled? 
 

Edit: I make sure that shit stays online for a living, if you think there’s anything I could do to help/suggest/anything else feel free to shoot me a PM.

[SiC]

7 minutes ago, IronStar said:

Are we using rate limiting on the Cloudflare side? That may do the trick if we do, set to something sensible like 250-300 requests / minute per IP..

I assume that all the other speedups and image optimizations are already enabled? 

When I took over, all Cloudflare services was disabled and still currently disabled. It used to be enabled a few years back but I'm not sure why it was disabled. I have vague memories there was issues with cache not being flushed properly when posts were made.

Enabling their caching is something I have been considered ...however there are plenty tails of woe on the Invision forum with using it. Some find it great and no issue, while some have reoccuring issues. Invision official line on it seems to be "Its not supported".

The code smell around Invisions products is pretty stinky. Have I ever said?

23 minutes ago, Zelandeth said:

2 hours ago, SiC said:

...Just Invision is a bit shit. 

Edited for accuracy.

Edit: Also for reference, that took the best part of a minute to post.

🙃

 

Tonights job was to have a poke around Cloudflare and give it a go.

[IronStar]

2 minutes ago, SiC said:

When I took over, all Cloudflare services was disabled and still currently disabled. It used to be enabled a few years back but I'm not sure why it was disabled. I have vague memories there was issues with cache not being flushed properly when posts were made.

Enabling their caching is something I have been considered ...however there are plenty tails of woe on the Invision forum with using it. Some find it great and no issue, while some have reoccuring issues. Invision official line on it seems to be "Its not supported".

The code smell around Invisions products is pretty stinky. Have I ever said?

🙃

 

Tonights job was to have a poke around Cloudflare and give it a go.

Do a no query string cache and enable all the optimizations. Http2, quick, brotli, all the nice stuff. They really significantly beefed up their free tier offerings in the latest Cloudflare days event.