[Resolved] API delays and drop in project container availability

We are investigating a degradation in api performance. Site performance may be slow and cause data requests to time out. You can receive updates by going to status.glitch.com.

2 Likes

Glitch really needs to sort out these Glitches! I’m so funny! :joy:

4 Likes

@tasha What is actually going on? Why is this happening? Why are the logins not working and the terminal?

@glitch_support

Decrease in project container availability that is causing project apps to load forever and occasionally return a 504 Gateway Time-out.

image

Keep an eye on status.glitch.com for more updates!

5 Likes

Can confirm this. It’s not as bad, but it is a bit annoying
status.glitch.com for more info btw

1 Like

Seems to be coming back up, although I can’t totally trust it.

1 Like

Might just come straight back down again like usual :joy:

1 Like

My projects are loading endlessly…

And it seems to back. I’ll mark this as solved if it stays at 100% for the next few minutes.

Guess I was right after all. We have a massive drop in container availability and a huge spike in the API response time. @glitch_support!

I can’t access my project right now :frowning:

https://support.glitch.com/t/project-container-availability-30-06-2020/25183/7?u=khalby786

@EddiesTech ^^^^


(Yes that is ‘Best hold music’ I am listening to as you can see from my tabs :joy:)

Glitch status updates continued on

Thanks @khalby786 and @EddiesTech. I was just notified by the team.

We’re bootstrapping now, and getting things sorted. So we are on the case!

One thing to note is one of our monitoring services went down recently, and this is making it more tricky to observe the overall affects of the API slowness for users. For this reason, please feel free to continue posting any issues you are seeing and think may be related to this incident in this. thread. That way, we can aggregate the info and do a better job identifying how this is affecting users.

2 Likes

my project’s are about to shutdown and there is lag …

1 Like

My project is now loading all fine! :slight_smile: Expect more issues though :expressionless:

1 Like

One thing I am noticing from support requests that were sent 4-2 hours ago is that terminal was not loading. I checked one of my test projects and was able to bring terminal up. Are folks here still seeing terminal issues?

@tasha seems to be fine now. :slight_smile:

@tasha Actually, no. I think it’s recently made projects, cause the one I made today shows the error;


However another is fine

Eddie

the same problem I hope this fixes as soon as possible

https://status.glitch.com/
site is not work? just a white screen

oh well he’s back :slight_smile:

cross-posting from status.glitch.com:

We have received reports that this may also be affecting account creation/login. If clearing your browser cache does not fix the issue, it is most likely incident-related and will be resolved once everything is back up and running.

1 Like

@tasha Maybe the terminal is down on specific containers/ips? Cause one of my projects can run it and one can’t

its been a day and I can’t acces glitch

This still happening, why sometime in the month, api gets drunk, and decide to sleep, and is Amazon responsible for this, since they provide the SSL Certificates, and I assume they host all glitch projects?

@HeemPlayz Yes, AWS hosts the projects. But, their status page shows no issues so I think this an error with Glitch, not them

Well, you are right about this.

1 Like

Glitch has to keep up and fix these spikes! Its messing with the performance! I can’t afford for my bot to go offline for a day and loose 3 servers my bot is in!

2 Likes

Latency was really slow earlier. I wonder what has been causing this.

Very frustrating how frequently glitch has issues, I just boosted my apps thinking it would help
It didn’t
$96 down the drain

2 Likes

@glitch_support @tasha this is pretty urgent now. Projects won’t load at all…

Eddie

@glitch_support Are you fixing it? Is that why it now comes up with maintenance?

stuck in Maintenance loop

1 Like

Update:

We want to test if reducing traffic will help, which involves temporarily shutting down some projects to see if the rest respond better. We’re monitoring results of that and will slowly bring the rest of the projects back as we keep investigating.

4 Likes

I’m wondering if this has to do with how glitch may have not been designed to handle so much people. The amount of people using glitch has increased significantly(I think) since boosts arrived and maybe the glitch team might have not planned to have so much people using glitch at this point.

1 Like

@tasha Do you have a time-frame for how long this may take? I mean, at least you’re trying something, but some people rely on the uptime of this service. For example, Discord Bots or they have code to check they got right, like me

1 Like

If you are just testing you can run it locally. For my big project I keep a copy on my computer and have a specialized system for uploading to glitch.

Good idea. May set that up now

Questions like this are difficult for support to answer because we run the risk of broadcasting info that has the potential to be inaccurate. There are still too many variables at play.

Here is what I can say:

  • We expect to be scaling traffic back up within 20 minutes, but this does not guarantee that it will fully resolve the issue.
4 Likes

Things should be coming back up now. We are monitoring our systems.

4 Likes

@tasha the google sign in still isn’t working

My projects are still taking incredibly long to load
Now completely down

@tasha @glitch_support project availability just dropped down to 0

There’s a reason that they have an incident open. Chill out guys, it is a free service after all.

1 Like

Not for everyone

1 Like

Hey all. We are looking into the current drop in api performance.

2 Likes

@tasha, project app pages are returning a 504 Gateway Time-out now!

these are the consequences of the outage. Read the OP


Container keeps on spiking!

2 Likes

My websites seem to be working right now.

1 Like

@tasha @glitch_support We seem to have gone into Maintenance mode again. I am in a loop. status.glitch.com says you are looking into the previous issues, so I assume this is you testing things again?

1 Like

My site just went into Maintenance mode too.

1 Like

ive kinda lost respect for glitch, the past 4 days has been full of downtime, I get on to work on projects and i cant even see what i am doing. 4 days straight!!! and when it does load css and images sometimes dont load.

You see, I don’t pay. But if I was paying, I would lost respect and cancelled a week ago when the downtime started!

YES! and one of the projects I am working on is being payed for

@glitch_support Refund or money off next month maybe?

2 Likes

This is so beautiful what im seeing right now!

1 Like

IMG_20200527_223340

2 Likes

Can we possibly have a self-hosted version of glitch. One of the main reason I use glitch is to code and be able to switch computers.

3 Likes

Github?
Maybe git in general.
Repl.it?

I pefer not to install my full blown dev essentials on every computer I use

Then repl.it is your best bet.
I normally only use 3-4 computers so I’ve got the essentials on all those.

We are looking into current API latency.

5 Likes

Unfortunately we had to temporarily block traffic to 50% our running projects, but things are looking healthy now. We’re still monitoring. For updates see https://glitch.statuspage.io

2 Likes

Don’t pay extra for the ability to add a domain to your status page if you don’t use it :joy: https://status.glitch.com - Good to hear you are working on it @mads

Hello @tasha i think that right now glitch has a mega outage, i just talk with others my friend and they are reporting that all bots are offline you should put the status of the website as Degraded perfomance.

thank you

if you have the same issue click the :heart:

5 Likes

@tasha Its been 3 days and I still can’t use glitch

As @mads says, they are working on it. Everybody, please be patient, especially if your projects aren’t boosted! :slight_smile:

2 Likes

If I’m getting a black screen with the following message, is it related to this issue?

Failed to compile.
Error in Invalid or unexpected token
@ multi main

First I thought it was code issue, but I haven’t changed the code for a month and it always worked well until this past weekend

I doubt it is. Please check your code.

Hello!

This incident has now been resolved. If you continue to see issues, please try clearing your browser cache and then accessing Glitch again.

If you have projects that are suspended for Preparing Error or are not loading otherwise, please send a message to [email protected] so we can help you out!

4 Likes

@glitch_support @tasha I don’t think this has been resolved - I can’t even open the editor (says ‘There was an error starting the editor. Maybe try to reload?’), people can’t log in at all and it all seems to come down to the API (e.g. I see ‘waiting for api.glitch.com’ when try to load the editor). Not sure if the terminal ever got working. What is this API Uptime?
image
I understand you have a incident logged on status.glitch.com, but not much on the forum apart from a banner.
Projects are now fine but editor/login or anything to do with the API are dreadful!
I can understand why people are getting impatient now, especially if they are paying for boosted plans, that should be on separate IPs to all the other projects anyway (e.g. to stop them getting banned by Discord due to spammy free bots). I understand you are trying all you can to fix this, but if we could have some clarification on what’s going on now (e.g. why you thought it was resolved and it’s not, what’s different and why it’s not affecting project hosting like before), that may be nice to know.

Eddie

1 Like

I also just noticed that when you go on to api.glitch.com it seems to be taking a very long time to load (usually it throws back ‘Yo!’). Is the API working at all?
Edit: I am right, API Uptime has gone all the way to 0.
image
Mind you, I just go the ‘yo’ after a short while:

Try to help @glitch_support by putting all the issues into one place. I got a 504 error when trying the load the API from my browser:


@tasha

Hello @EddiesTech glitch is aware of all his problem you can find a notification of it in the forum and especially in the Status https://status.glitch.com/ be patient they will solve everything as always

@Sonostrano20