Slowness

Nash Kabbara's Avatar

Nash Kabbara

06 Jun, 2009 04:58 PM

I've experiencing a lit of slowness since I've signed up.

Am I the only one?

  1. 1 Posted by Kyle Neath (Git... on 06 Jun, 2009 08:46 PM

    Kyle Neath (GitHub Staff) 's Avatar

    Do you have any specific pages or times of day that this is happening in particular? Also, where are you connecting from (geographically)?

    I haven't noticed any slowness personally, but we can try and see what might be causing it for you.

    Kyle

  2. 2 Posted by Nash Kabbara on 06 Jun, 2009 08:52 PM

    Nash Kabbara's Avatar

    Hi Kyle,

      I'm located in Houston, TX. When I do updates to my documents or refresh the
    dashboard, it hangs for about 4 seconds. Not always. I would say 40% of the
    time.

      Thanks.

    -Nash

  3. 3 Posted by rick on 07 Jun, 2009 06:58 PM

    rick's Avatar

    I'm seeing something too, but it's very random and not database related. I'm looking into it.

  4. 4 Posted by peter on 08 Jun, 2009 06:08 PM

    peter's Avatar

    Rick,

    We've been having the intermittent slowness issue since last week. Gets pretty slow at times, taking over 2 minutes to load a ticket.

  5. 5 Posted by peter on 10 Jun, 2009 04:34 PM

    peter's Avatar

    Any word on this?

    It's gotten better but still having the occasional slow down.

  6. 6 Posted by mike on 10 Jun, 2009 04:36 PM

    mike's Avatar

    Noticing this from my location, as well. Tickets are slow to load, assets slow to process...

  7. 7 Posted by rick on 10 Jun, 2009 06:07 PM

    rick's Avatar

    We're looking into this, but it's very random. Traces show that most of the time it's fast, and a few requests here and there are real slow. We're having issues pinpointing the actual cause of the slowness. For instance, NewRelic reports it happening in outside of the database during view rendering. That's a little strange, it's sounding like a hardware issue.

    Mike: it sounds like you're talking about job queue slowness. That could happen if there are a flood of new jobs to handle. I keep an eye on the queue and it's usually empty.

  8. 8 Posted by rick on 18 Jun, 2009 04:27 PM

    rick's Avatar

    Wow, sorry about the delay -- we've been going back and forth with our host on this. Application traces didn't point to anything code-related. I pushed out a few tweaks (queueing sphinx index updates in DJ, memcaching some stuff, etc) and none of it seemed to fix the issue.

    Last night, one of the Engine Yard techs found that we have an extra DJ job runner on a server, causing it to swap. We think stuff should be a lot smoother now.

  9. 9 Posted by peter on 18 Jun, 2009 04:54 PM

    peter's Avatar

    Thanks for the update, Rick. I don't recall seeing any slow issues yet today, good sign.

  10. 10 Posted by Nash Kabbara on 18 Jun, 2009 04:55 PM

    Nash Kabbara's Avatar

    Thanks. Will keep an eye.

    -Nash

  11. 11 Posted by peter on 23 Oct, 2009 07:32 PM

    peter's Avatar

    Hey guys,

    This was getting progressively worse for a couple weeks, and completely stopped working for about 20 minutes just recently.

    I saw your blog post that there was also a big issue overnight http://hoth.entp.com/2009/10/23/tender-slowdown but this has apparently not yet been fixed?

  12. 12 Posted by rick on 23 Oct, 2009 08:13 PM

    rick's Avatar

    Same problems, different cause. The problems aren't as severe as last night's. We're just seeing increased activity. We're looking at completely offloading the email processing from the front-end app servers in the short term and upgrading the hardware.

  13. 13 Posted by rick on 23 Oct, 2009 08:13 PM

    rick's Avatar

    Same problems, different cause. The problems aren't as severe as last night's. We're just seeing increased activity. We're looking at completely offloading the email processing from the front-end app servers in the short term and upgrading the hardware.

  14. 14 Posted by wbenedetto on 26 Oct, 2009 10:52 PM

    wbenedetto's Avatar

    We have been experiencing agonizing slowness for about a week or so, and it's happening right now. Pages take 60 to 90 seconds to resolve. When they don't resolve, they don't time out -- instead we get an error notice from Tender. It's not just our app (help.allaccess.com), but help.tenderapp.com as well.

    FYI, It's 3:45 PM PST.

  15. 15 Posted by Will on 26 Oct, 2009 10:55 PM

    Will's Avatar

    Hey wbenedetto,

    It's a systemwide issue where we're being hit with heavy response times -- http://hoth.entp.com/2009/10/26/tender-status

    We're still working to resolve the issue and sincerely apologize about the inconvenience. We hope to have everything smoothed out as soon as possible.

  16. Support Staff 16 Posted by Courtenay on 26 Oct, 2009 10:59 PM

    Courtenay's Avatar

    FYI, I just checked, 80% of requests to your tender are within 440ms. There
    are a small number of requests that take longer.

    @@@

    Percentage of the requests served within a certain time (ms)
      50% 167
      66% 194
      75% 216
      80% 440
      90% 2535

    @@@

    On Mon, Oct 26, 2009 at 3:55 PM, Will <
    [email blocked]<tender%[email blocked]>
    > wrote:

  17. Nicole closed this discussion on 04 Dec, 2009 09:45 PM.

Discussions are closed to public comments.
If you need help with Tender please start a new discussion.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac