Buffer security breach has been resolved – here is what you need to know

Update: This article was originally titled “Buffer has been hacked – here is what’s going on”. The hacking incident happened yesterday (Saturday) and below is a recap of everything that happened. Please ask us any questions you have in the comments below.

If you’re reading this, the most important section for you is Update 7.

We’ve discovered the source of the breach and closed the vulnerability. Keep reading for the full story.

Update 9: We’ve discovered the exact details of how the Twitter and Facebook access tokens were obtained to send spam posts.

Update 10: We’ve uncovered this weekend that the hackers also gained access to our code.

I wanted to post a quick update and apologize for the awful experience we’ve caused many of you on your weekend. Buffer was hacked around 2 hours ago, and many of you may have experienced spam posts sent from you via Buffer. I can only understand how angry and disappointed you must be right now.

Not everyone who has signed up for Buffer has been affected, but you may want to check on your accounts. We’re working hard to fix this problem right now and we’re expecting to have everything back to normal shortly.

We’re posting continual updates on the Buffer Facebook page and the Buffer Twitter page to keep you in the loop on everything.

The best steps for you to take right now and important information for you:

  • Remove any postings from your Facebook page or Twitter page that look like spam
  • Keep an eye on Buffer’s Twitter page and Facebook page
  • Your Buffer passwords are not affected
  • No billing or payment information was affected or exposed
  • Update: All FB posts are being posted normally again and no more spam postings will occur.

I am incredibly sorry this has happened and affected you and your company. We’re working around the clock right now to get this resolved and we’ll continue to post updates on Facebook and Twitter.

If you have any questions at all, please ask in the comments below or email us hello@bufferapp.com. Understandably, a lot of people have emailed us, so we might take a short while to get back to everyone, but we will respond to every single email.

We’re going to update this article as we’ve got more news to share for you!

Update: 1pm PST

All hidden Facebook posts are now shown again. There is a chance that some spam postings are now live again, please check on your Facebook accounts and delete them.

No more spam updates should occur at this point, as all posting has been disabled. All hidden posts however should show again.

We’re keeping you posted on everything!

Update 2: 3pm PST

We’ve increased security for how store Twitter tokens and deployed a fix.

You can login with Twitter again. You will have to reconnect all your Twitter accounts in Buffer. Here is how to reconnect them.

You can now send Tweets via Buffer again.

Update 3: 5:30pm PST

We’re currently working directly with Facebook and AWS to get this all sorted out. It looks like we are making our way closer to a full recovery. Twitter (see Update 2 above) should be working again 100%.

About your Facebook posts: Currently it’s  not possible to connect or post to FB with Buffer. We hope to have this working again real soon for you and I greatly apologize for the hassles this might have created.

Update 4: 8pm PST – All posting is working again!

We’ve greatly increased the security of how we handle all social messages being posted and everything is back to normal. Please try signing into your Buffer account from http://bufferapp.com instead of the mobile apps for now.

For your Facebook account:

If you had Facebook posts via Buffer scheduled during the outage, they will likely appear as “failed” in your Buffer queue. You can just hit “retry in Buffer” and they should then be scheduled normally and go out as expected again.

For your Twitter account: You will have to reconnect all your Twitter accounts in order to start posting again. Here is how you can reconnect your Twitter account.

We’re also going to publish an in-depth post about what the spammers got access to and what we did to fix it. In short, we encrypted all access tokens for Twitter and Facebook and also added other security measurements to make everything much more bullet proof. More on this in a coming post!

 

Update 5: 9:00am PST Sunday, 27th of October

We have monitored all behaviour overnight and everything has remained normal. All posts to Facebook and Twitter via Buffer should be going out normally. For Twitter you will have to reconnect your accounts from the web dashboard.

We have greatly increased security of how we are posting to Twitter and Facebook and have confidence to cover the security holes the hackers have used to break into our system.

What’s next: We’re working with several security experts on tracking down exactly how it was possible for the spammers to get into our system. We’re making good progress on this, this morning. What will follow is that we’re going to publish an in-depth update on the impact of the hack and everything we know about how it happened.

Update 6: 3:00pm PST Sunday, 27th of October – Extent of the impact

We’re now able to recover further insights as to what has happened exactly:

As soon as we noticed the issue, we disabled all postings to prevent more spam from going out.

In terms of exact numbers, Facebook confirmed with us that 30,000 Buffer users who had a Facebook page connected (out of  476,343 total connected pages to Buffer) were affected and had spam posted on their behalf. This means that 6.3% of Buffer users on Facebook were impacted by this.

Since then we’ve taken key security measures: we have added encryption of OAuth access tokens and we have changed all API calls to use an added security parameter.

Service has resumed with increased security since the incidents. You can now head into the Buffer dashboard and use Buffer again as normal.

Update 7: 8:00pm PST Sunday, 27th of October – All Twitter accounts need to be reconnected

We’ve taken further security measures and as a result all Twitter accounts will have to be reconnected. Even if you’ve already done so, you will have to reconnect your account one last time.

I greatly apologize for having you do this again, but we want to make sure that we are on the safe side with this.

Go to the web dashboard now to reconnect all your Twitter accounts.

Update 8: 3:00pm PST Monday, 28th of October – The origin of the breach has been identified

Here is an important note from our CTO Sunil, who is leading the technical investigation on this issue:

We’ve learned how the hackers breached our system on Saturday. We’ve worked with our partners to trace back their steps and we’ve closed the vulnerability. This is a big relief, as understanding how the hack occurred was the biggest worry in our eyes. Here’s what we know.:

  • The hackers were able to steal some of our Facebook and Twitter access tokens from our users. We have confirmed that the hackers were not able to get access to any passwords, billing information or other user information other than specifically the Twitter and Facebook access tokens.
  • We have since invalidated all Twitter access tokens. We’ve added encryption for all Twitter access tokens.
  • For Facebook API calls we are now using an extra security parameter to make all tokens more secure.
  • With these improvements your Twitter and Facebook accounts are not at risk anymore. Attackers will not be able to use this method to send spam anymore.
  • The method which left our data vulnerable is now locked and secure.

My apologies for the disruptions this has caused you and your company. Please feel free to ask me any questions about this below. We are going to add more detailed updates about this as we uncover more. Expect a detailed report!

Update 9: 2:00pm PST Tuesday, 29th of October – How were Twitter and Facebook access tokens obtained to spam?

As of today, we’ve learnt some important, new information about how the hackers were able to get access to the Buffer database and steal the API tokens for Twitter and Facebook that were used to post spam on our users behalf.

The backdoor that was created through one of our partners, MongoHQ who are managing our database. MongoHQ, who have been incredibly responsible and responsive regarding this also just released an update about the security breach on their blog.

In short, the MongoHQ password of one of MongoHQ’s employees was stolen. That way the hackers logged into the main admin dashboard of MongoHQ and were able to use the “impersonate” feature to see all of Buffer’s database information. Through that, they wrote a script to steal our social access tokens and post spam messages on behalf of our users.

From their blog post:

“On October 28, our operations team detected unauthorized access to an internal, employee-facing support application.

We immediately responded to this event, by shutting down our employee support applications and beginning an investigation which quickly isolated the improperly secured account. We have determined that the unauthorized access was enabled by a credential that had been shared with a compromised personal account.

No internal application was made available to our team before a team-wide credential reset and audit.

Users of our support application have access to account information, including lists of databases, email addresses, and bcrypt-hashed user credentials.

Our support tool includes an “impersonate” feature that enables MongoHQ employees to access our primary web UI as if they were a logged in customer, for use in troubleshooting customer problems. This feature was used with a small number of customer web UI accounts. Our primary web UI allows customers to browse data and manage their databases. We are contacting affected customers directly.”

We have full trust in MongoHQ that they have closed the security hole and are also very grateful about their fast update and the company helping us clear up all confusions in connection to the breach.

I want to be clear that this is still our fault. If access tokens were encrypted (which they are now) then this would have been avoided. In addition, MongoHQ have provided great insights and have much more logging in place than we have ourselves. We’re also increasing logging significantly as a result.

Update 10: 1:00pm PST Monday, 4th of November – Hackers also gained access to Buffer’s code via GitHub

We’ve recently (more precisely on Friday night) learned some important new information that I want to share with you.

We’ve been continuing investigation since the initial spam attack. We’ve learned that in addition to extracting access tokens through our database provider MongoHQ, the hackers were also able to get unauthorized access to our Buffer GitHub account. We discovered this by viewing the Security History of one of our team members whose account was used to access the code. The logs showed logins and current sessions, which we revoked immediately. With access to the code they extracted our consumer key and secret to post spam to Twitter.

We suspect that the way this happened could have been through the Adobe leaked passwords or some other potential leak where a large number of passwords were stolen. We have no way of proving this.

After learning about this, we secured the codebase and then reset our Twitter consumer key and secret once more. It seems clear by now that Buffer was specifically targeted for this spam attack.

The whole Buffer team has changed their passwords and enabled 2-step login for as many services as allow it. (Google, AWS, Twitter, Facebook, Github).

We are going to publish more info on this as we learn more.

Please ask us any questions in the comments below or if you have any questions just email us hello@bufferapp.com.

We’re keeping you posted about all new updates from here!

  • Henri Deschamps

    You folks are awesome in every way, including when the chips are down. Remain Calm & Buffer On.

    • LeoWid

      thanks so much Henri for believing in us here, hope to get an update out soon!

      • Henri Deschamps

        Seems to be back up, I just went to one of our pages and the buffered posts which had disappeared and left the page pretty much a blank slate are back on there. That’s a serious relief :-)

        • LeoWid

          Hey Henri, yes, it should be largely resolved and back to normal now! We’re waiting on the final confirmation and will send an email to everyone!

      • http://www.techgyd.com/ Saurabh Saha

        You Rock Guys. We have full trust on you :-)

    • http://www.cazoomi.com/ Clint Wilson

      Great phrase Henri I think I will steal it:) “Buffer On” – awesome:)

      ~Clint
      @cazoomi

    • Peter Kirwan

      Agreed. Thanks for reacting fast and you have my sympathies as your weekend is undoubtedly worse than ours. p.s. can’t see any spam on my accounts but double checking.

      • LeoWid

        Hey Peter, thanks for jumping in here and so glad that you weren’t affected!

    • mouselink

      Agreed. Buffer is the best.

  • http://about.me/Otir Otir

    It is nice indeed that you keep us updated so quickly.

    • LeoWid

      Thanks Otir, this means a lot!

  • Andrew Burns

    It would be great to have a debriefing so we understand exactly what happened.

    • LeoWid

      Hey Andrew, yes, we’re working hard right now to put something together about exact details.

      • Andrew Burns

        Perfect… please get it handled. I just didn’t want a quick ‘it is fixed now’ message.

  • http://www.allinbound.ro/ Alin Vlad

    Hang in tight, guys! This happens to the best of us. For me, as a Social Media Marketing specialist, your reaction can already be classified as one the best Crysis Management examples. :)

    • LeoWid

      Hey Alin, thanks so much for your kind understanding and encouragement. We’re trying everything we can as we keep people in the loop, more updates to come soon. Our engineers are working hard right now to patch everything up!

    • http://hipcider.com/ tdhurst

      Yeah, no doubt there.

  • http://www.phildrysdale.com Phil Drysdale

    Can happen to even the best of us, don’t worry about it!

    If there was ever a team I’d want working on something this important to me it would be the buffer team!

    • LeoWid

      Thanks for the encouragement and kind words Phil. Hope to get an update with more info and insights out real soon.

  • TheTechieGuy

    great communication guys ! maybe just some reassurance for those that don’t know that as your system uses oAuth it means that you don’t keep social network password so these are not compromised.
    Good luck with the patching and thanks for keeping us in formed.

  • rdouma

    Yes, my first reaction when these spam posts appeared on my pages was shock and frustration. Great to see that you don’t beat around the bush and just say you’re sorry for the unpleasant experience and say you’re working on it. Good luck and thanks for the update.

  • iwaffles

    Well said Joel and best of luck today!

    • carokopp

      Thanks, friend!

  • Rabbi Ruth Adar

    Still a hardcore Buffer fan. You folks are sterling.

  • Ben Newton

    #GetWellSoonBuffer

  • http://beeminder.com Daniel Reeves

    Bright side: you were a juicy enough target for the bad guys to bother with! :)
    We’ve had multiple “crashes of ineptitude” as we call them, and always come out even stronger.

    Suggestion: Write another blog post where you document everything, even the technical details. That’s what we’ve done, pushing the technical details to a “nerds only” section: blog.beeminder.com/mongo

    I was super impressed with your ultra-transparent blog post about your progress since launching, even including revenue numbers. I’ve pointed many people to that who are surprised that we’re willing to be so transparent our financials. So thanks for that, and for being generally awesome!

  • http://lovesagame.com/ Eddie Corbano

    My Facebook post of two month were deleted. Did you guys do that or was it Facebook?

    • carokopp

      Were they sent from Buffer? They will reappear. :)

  • Kurn

    Thanks so much for all the information. You guys are doing a great job in keeping us informed. Sorry to hear this happened, but I know you guys will get things fixed up soon. :)

    • carokopp

      You’re so kind, thank you so much!

  • http://www.twitter.com/amritachandra Amrita Chandra

    Rooting for you guys. I didn’t get hacked because I hadn’t yet connected my Facebook account, but appreciate the heads up and updates.

    • LeoWid

      Hey Amrita, thanks for the kind encouragement!

  • http://www.jamiemchale.com/ Jamie McHale

    Good luck with the fix. Really excellent communication about what’s going on.

    • LeoWid

      Thanks Jamie, we’ll continue to update!

  • Günter Exel

    You’re not only doing a great job at Buffer every day – it’s also good to see how you deal with unforeseen problems. Heads up, Leo and Joel!

    • LeoWid

      Hey Guenther, thanks so much, talk about an unforeseen Saturday!

    • http://www.travelworldonline.de/ Monika Fuchs

      I totally agree with you, Günter. What a great way to handle a situation like this.

  • Jan Miřacký

    Guys, thank you for this approach and quick solving and communicating it. Fingers crossed so it doesn’t affect you as well – you are doing a great job and as we all know $#!t happens…

    • LeoWid

      Thanks so much for the kind encouragement and understanding Jan! Working hard to get this fixed.

  • http://www.wordsfyi.com/ Marge Brown

    Proof positive that full transparency and openness is the only way to go when situations like this occur. Kudos to Buffer. I am not currently a user but will seriously look at your solution, now.

    • http://www.organiclifeproducts.com/ Montina Portis

      Hi, Marge, my husband and I both use it for all of our social media updates and for offline biz clients too. We’ve been paying clients for nearly a year and it is one of the best investments we have made.

    • http://www.killfive.com/ Jessica Bosari

      Go for it! I’ve been using buffer as a free account for several months and just upgraded to paid because it has been such a useful tool. This situation, Buffer’s response, and the ultra cool, level-headed response of Buffer’s clients all make me feel pretty darn good to be a part of it!

  • Peter Wallhead

    Nice work on the quick updates guys, good luck for getting it sorted!

  • Andrew Mastrandonas

    Great job guys!

  • Lorraine C. Ladish

    Thank you for the fast action taken! Hacking will always happen. You are as usual providing the best service you could under the circumstances.

    • LeoWid

      Thanks so much for the kind words and understanding Lorraine, we hope to get this fixed asap!

  • http://ajjoshi.com/ AJ Joshi

    Good luck Joel, hope it’s resolved quickly without too much work, happens to the best of us, you deliver an amazing service and those that support will understand

    Oh and by the way, you handled it perfectly, no bs, honest and straight to the point while quickly fixing the issue.

    I commend you and your company as always and will continue on supporting away…

    Don’t let it stress you! We’re all with you on this.

    • http://ajjoshi.com/ AJ Joshi

      ….and Leo Of course :) both dealing with it so well that others should take a leaf out of your book.

  • pravil

    Bad things happen. But the way you reacted to the situation and the way you guys trying to get the situation under control is really appreciated. Transparency and promptness is the key.

    Kudos the Team Buffer!

    Life has to move on. Let us learn from the mistakes and keep Buffering!

    Cheers!
    Pravil.

  • Ben Werdmuller

    You’re handling this brilliantly. Awful that it happened, but this is a textbook example of how to deal with this kind of crisis. Good luck!

  • 1jour1son

    This is what I call a Fuc***g Crisis Management, well done guys !

  • Bassim

    Do I need to revoke Buffer from Twitter?

    • http://niel.delarouviere.com NielDLR

      Hey Bassim,

      you’ll have to reconnect Twitter. Go to Settings -> Reconnect on the Buffer site and it should all work fine again.

  • Monstruonauta

    Thank you!

  • Planet Earth

    We are still friends :))

    • LeoWid

      Absolutely love this photo, thanks for still being friends with us! :)

  • katelindsaylifecoach

    you guys for real have the very best customer service ever. Someone just wrote a blog post about it even and in my comments I totally agreed and shared my own story. Thanks for doing what you do the way you do it

  • Guy

    Incredibly well handled – big appreciation from a loyal customer.

    • http://jonnyrowntree.com/ Jonny Rowntree

      I second that thought.

      • LeoWid

        thanks so much for the kind response guys, I really appreciate the encouragement, hope be back on our feet real soon!

  • http://www.evolveyourweddingbusiness.com/ Heidi Thompson

    As I would have expected, you guys handled this exceptionally well :) I noticed a Facebook page post failed so I went to reconnect the page to Buffer and it just goes through this endless loop of starting to load the Facebook URL and then starting to load the Buffer Oauth URL. I’ll give it a try later but just wanted to let you know in case it was posing problems for others.

    • http://niel.delarouviere.com NielDLR

      Hey Heidi,

      Facebook posting should be working again now. Let me know how it goes and if there’s any problems!

      Sorry again for all the trouble (and thanks for the kind words there!)

  • http://www.organiclifeproducts.com/ Montina Portis

    Thank you SO much for keeping us updated! Way to take the lead and keep your clients informed!

  • Jamie Smyth

    I spy Skitch

  • lacouvee

    Thanks for your transparency in keeping us informed and best of luck in figuring out what happened. Here’s another reason to NOT uncheck the box for any social media platform that says “receive periodic updates from the company”. Even though I have an account, I don’t use Buffer and would likely have never known about the hack.

  • CurvyGirlHealth

    Good luck guys! I’m still a huge fan!

  • http://www.pinchmeliving.com/ Bernadette @ PinchMeLiving.com

    Great job on lightning fast response to this and open communication! Awesome work.

  • Steve S. Ryan, Ph.D.

    Hang in there guys.

  • Zach

    This unfortunate situation has only made me like/respect Buffer all the more.

    Adversity doesn’t create character — it reveals it!

    • LeoWid

      Thanks so much for these kind words Zach, it means a lot that you’re still putting your trust in use, we hope be up and running asap again!

  • Tembrooke

    Thanks for the quick response & clear instructions on what steps we need to take! Hang in there.

    • LeoWid

      thanks so much for the encouragement, we hope to be back to normal real soon!

  • http://www.virinzy.workpress.com/ Virinzy jinapron

    wOW^

  • Kathryn M. Crane

    No skin off my back! I was just jealous that some of my friends got to lose 30 pounds, and I only got to lose 8. If this is the worst thing that happens this weekend, then I am golden.

    • LeoWid

      haha, that is hilarious, it’s very humbling that you are taking this with such humour Kathryn when it was all our fault to create this mess!

      We hope to have things up and running again for you real soon!

  • hongjun

    Keep it up!

  • http://www.almostsavvy.com Irene Koehler

    You all are doing a phenomenal job. Being open, transparent and communicate with your users as if they are intelligent people – it wins every time.

    • LeoWid

      thanks so much Irene, really appreciate this!

  • sMorac

    Great job with the updates, transparency FTW! Good luck to sort this out.

    • LeoWid

      so glad that this was helpful! We’re almost through the worst now!

      • hongjun

        Glad to hear. Looks like topping up my buffer is not far away.

  • Guest

    Thanks

  • m legrand

    wow, amazing response. Almost thought the alert message was a hack because what company is actually so forthright when something goes wrong?! Great integrity just won you a new customer!

  • Darryl C. Treadwell

    Outstanding communication and support Buffer!!! I wish more businesses had your integrity and loyalty to their clients!

    • LeoWid

      thanks so much Darryl!

  • http://gregkocis.com/ Greg Kocis

    Should we still do this if there wasn’t any spam on our accounts ?

    • LeoWid

      Hi Greg, if there was no spam on your account, all you have to do is reconnect your Twitter account and that’s it!

  • Dave Powell

    Sorry to hear you guys got hacked but love the transparent way you guys are handling it. My blog was hacked once which made me follow up with a ‘Dear Hackers: You Suck’ post. Keep up the great work Buffers. I love you guys…

    • LeoWid

      thanks so much for the encouragment Dave! Yes, we’re working around the clock now to trace back where the hackers came from.

  • AM2

    Like always Buffer, you are awesome. Sorry you’ve had to deal with all of this today! Thanks for the fix. Thankfully, I didn’t have any issues with all of my accounts.

    • LeoWid

      so glad we could get this sorted again for you!

  • Rebekah

    Your prompt updates and contact with us during this stressful time give me great confidence in you and your business. Thanks for keeping us in the loop and working so hard to resolve this issue.

    • LeoWid

      Hi Rebekah, thanks so much for the kind encouragement here, this means a lot, we’ll continue to post updates here!

  • Gareth Ellis

    Liking how you guys have been upfront about this right off the bat and dealing with it. Honesty is always the best policy!

    • LeoWid

      Hey Gareth, thanks so much for the kind words and the encouragement, yes, I think being upfront about this is really key.

  • http://janetfouts.com Janet Fouts

    WOW you guys really handled this gracefully. Thanks for being ON it and letting us all know so quickly. I was just writing up a blog post about the mobile app based on the email you sent out earlier. I’ll wait until Tues or Weds to post so things have time to settle down. We <3 Buffer!

    • LeoWid

      Hi Janet, thanks so much for the kind words and for sticking with us! Yes, Tue/Weds everything should be running smoothly again!

  • theirmind

    Thank you for your efforts.

    • LeoWid

      thank you for hanging in there!

  • Jigme Datse Yli-Rasku

    You guys totally rock. From what I can see it looks like I wasn’t affected. Other than the things you did to mitigate the damage. Cleaning that up now. Great job guys.

    • LeoWid

      HI Jigme, thanks so much for the kind words and encouragement, so glad that you weren’t affected!

      • Jigme Datse Yli-Rasku

        Sounds like you saw the vector they got through, and have since closed that hole. Wonderous indeed.

  • http://www.advancedtele.com ballparkbob

    We do text message marketing and our site at 84444.com was attacked by a hacker 2 years ago. We had to make the same apology. No matter how hard you try, it is difficult to have a foolproof system to stop a savvy hacker.

    • LeoWid

      Oh wow, that makes sense, I can only see that this must have caused you a similar pain. It looks like luckily the worst is behind us now!

  • The Dork

    Um, mine’s been working fine all day.

    • LeoWid

      so glad you weren’t affected!

  • http://www.liewcf.com/ LiewCF

    Thanks for the quick response. Do we need to reconnect Google+?

    • LeoWid

      Hi Liew, nothing needs to be done for G+, it wasn’t compromised!

  • http://about.me/trapolino Christina Trapolino

    Dear Rest of The Internet:

    This is how you deal with a crisis.

    Thanks, Buffer!

    Love,
    Christina

    • LeoWid

      thanks so much for the kind words and encouragement Christina, it’s amazing to have you on board!

  • http://www.socialpositives.com/ Mohammed Anzil

    I think now everything is okay. You rock guys. Come on…………. We are friends ever……

    • LeoWid

      thanks Mohammed!

  • http://www.buildandbalance.com/ Michael_N

    Always been a fan; always will be a fan. Thanks for handling this issue so quickly. I just one little FB glitch in my account and it’s resolved now. To the bad guys: “Go ahead and mess with my Buffer. You’ll just make it tougher.”

    • LeoWid

      Hi Michael, thanks so much for the kind responses and encouragement with this, indeed, we’ll do everything we can to make this bullet proof now!

  • http://primaltoad.com/ Todd Dosenberry

    Role models at their best. Keep it up!

    • LeoWid

      thanks so much Todd!

  • Marcos

    How is possible to recover the Buffer Pro subscription?

    • LeoWid

      Hi Marcos, yes, that’s definitely possible and we’d love to do that, could you drop us an email hello@bufferapp.com? Would love to sort you out asap!

      • Guest

        I sent it

      • Marcos

        I send it

  • https://www.facebook.com/DakinAssociates Shaun Dakin

    Great example of “crisis management” leadership. Bravo !

    • LeoWid

      thanks so much Shaun, it’s amazing to have you on board!

  • LeaderSource

    Thanks for your fast response and for your great communication (as always).

    • LeoWid

      So glad we could keep you in the loop on this and we’ll continue to post updates here!

  • Belinda Smith

    Well done, gang. Well done.

    • LeoWid

      thanks so much for the encouragement Belinda, everything should be working normally again now! We’ll keep you posted here if anything comes up!

  • http://vatark.tel/ Robert Killington

    Well done! You handled this very well. Thank you for keeping us informed of what was going on. Others can learn from your example.

    • LeoWid

      Thanks so much for sticking with us through this Robert, it means a lot!

  • Sascha

    Great performance, perfect information. If all sites would always react as you do… But most don’t. So buffer is perfect, as always :)

    • http://niel.delarouviere.com NielDLR

      Hey Sascha,

      thanks so much for the kind words. Really appreciate the support. We do this for people like you!

  • Digit Panda

    Good Job with the updates. I applaud you guys for informing us right away. :)

    • http://niel.delarouviere.com NielDLR

      Thanks Digit Panda! Really appreciate the support!

  • http://www.justpractising.com Su Butcher

    Hi folks, great work.

    Have there been any issues with posting to Linkedin or Google Plus?

    • http://niel.delarouviere.com NielDLR

      Hey Su,

      there should be no issues posting to LinkedIn or Google+. Only Facebook and Twitter have been affected.

  • http://www.nuxnix.com nuxnix

    Beautifully well done and Open with a capital O. Nice work.

  • http://searchbuzz.co/ SearchBuzz

    Very glad you got this sorted out so quickly.

  • Jim Hopkins

    Regardless of root causes, your transparency — refreshing and damn near unique — strengthens my enthusiasm for this product. Look forward to learning more. Thanks.

  • Alex D’Ambra

    I’ve reconnected Twitter account but buffered tweets still not going through?

    • http://niel.delarouviere.com NielDLR

      Hey Alex,

      oh darn! That’s no good! Are you still experiencing problems? If so send an email to hello@bufferapp.com and we’ll help you out!

      • http://i-am.addic7ed.com honeybunny

        I have sent an email earlier this morning and I still haven’t received any reply. Posting on my FB pages is not working since last night. I need help solving this, please.

  • http://i-am.addic7ed.com honeybunny

    I am sorry, but posting to my FB pages is not working. I tried reconnecting and it didn’t help. Any other suggestions, please?

  • SydesJokes

    Buffers reaction was fast and was great to see how the team reacted to the attack keeping customers informed of the issue. I was not affected but gave me the opportunity to check my accounts to see if there were any spam posts. I appreciate the honesty here and this has strengthened my commitment to use this excellent service. Keep up the great work :)

  • raphaelhunold

    So no need for password resetting or is it better anyway ?

    • http://niel.delarouviere.com NielDLR

      Hey,

      luckily there’s no need to reset passwords.

  • Julia Harris

    Thanks for sorting so quickly and keeping people updated. I <3 Buffer!!

  • TheBravoBlonde

    My buffer does not seem to be posting to my twitter. I’ve been testing with the share now feature. The posts are vanishing but not showing up on twitter.

  • http://evanvolgas.com/ Evan Volgas

    There’s something to be said for the way you conduct yourself when you screw something up. You guys handled this like rockstars. Wish more companies would follow your example. Sucks that it happened, but it could happen to anyone and you guys did a great job handling it. Cheers.

    • http://niel.delarouviere.com NielDLR

      Hey Evan,

      wow. Thanks so much for the kind words. Helps us through the tough times.

  • http://www.thadthoughts.com/ Thad Puckett

    Thanks for keeping everyone updated so quickly. Speaks volumes! I look forward to reading your in-depth post about the cause of the problem.

    • http://niel.delarouviere.com NielDLR

      Hey Thad,

      thanks for the support and understanding. I’m just as curious as you are and we’ll have a post out as soon as possible!

  • http://charliesaidthat.com/digital/ charliesaidthat

    Well handled.

    Nice to see the clear, transparent updates. Honesty in communications goes a long way. Good job guys.

    • http://niel.delarouviere.com NielDLR

      Hey Charlie,

      thanks so much for the kind words. Really means a lot!

  • Atlanta Kings

    I use Buffer for automated marketing to multiple Twitter accounts. I rarely log in to the web site nor do I check your blog or Twitter. 19 hours passed before I noticed something was up with my automated tweets and the only reason I noticed it was by seeing a drastic dip in traffic via Google Analytics. It’s great that you got everything up but it would have been nice to get a heads up e-mail telling me that I have to reconnect ALL my Twitter accounts (which is a pain, by the way).

    • http://www.technetafrica.com/ Jason Malan

      Ditto. I’m confused why people people are so impressed with Buffer. I now have to go change the passwords to every account I added to Buffer. Sure, S**t happens. But there are always consequences. This isn’t a confidence inspiring experience

      • http://niel.delarouviere.com NielDLR

        Hey Jason,

        terribly sorry for the mess up. I know it must suck. However, you need not have to change your passwords. That info was not compromised. You’ll only have to reconnect them (Settings -> Reconnect).

        • http://www.technetafrica.com/ Jason Malan

          Acknowledged. I appreciate the apology. But as @atlantakings:disqus said, you should have emailed us when this happened.

          • http://WRRY.me/ William R. Reynolds Young

            I received an email only 1 hour after the hack. Then when service was reactivated I received the emails of the failed tweets to reconnect accounts.

          • http://www.technetafrica.com/ Jason Malan

            I had no spam posted to any of my connected accounts. Maybe that’s why I didn’t receive an email. Would still have liked to have been contacted though.

          • http://www.jaylogan.com/ Jay Logan

            Same here. I didn’t get any e-mails. My account wasn’t hacked yet I still had to reconnect them all in order for my service to resume. So if the decision was to only notify hacked accounts then that wasn’t right at all.

          • http://WRRY.me/ William R. Reynolds Young

            Jason, I would check your email settings – None of my accounts were compromised however I received the email on all three of the Buffer accounts I manage.

            Look for an email from Hello@Buffer.com at around 4:446 pm Eastern Time on Oct 26th with the subject – “Buffer has been hacked – here is what’s going on”

          • http://www.thegraphicmac.com/ JimD

            I got my first email immediately after it happened.

          • http://www.technetafrica.com/ Jason Malan

            It seems like a lot of users got emailed. Was spam posted to any of your accounts?

          • http://www.thegraphicmac.com/ JimD

            Yes. I had a few friends and clients notify me about the same time I received the email from Buffer.

          • Ela Bednarek

            I got emails too, even though my accounts were not affected.

          • Jessie Wood

            I got emails from both of my Buffer accounts on Saturday afternoon and neither of them were victims of spam. Perhaps your email got caught in the spam filter? I believe they emailed everyone.

  • Ben Buxton

    Great job on handling this to the crew at Buffer. You guys were on top of it, but also fully transparent..you earned my trust.

    • LeoWid

      Hey Ben, thanks so much for the kind words and the trust, this means a lot!

  • http://www.skybluweb.co.uk/ Mat Durham

    What started as a nasty episode appears to have strengthened the loyalty of your customers and bagged you a shedload of free PR. Brilliantly handled.

    • LeoWid

      Hey Mat, way to look at the bright side of this, thanks so much for believing in us!

  • netizenpros

    BufferApp has GREAT CUSTOMER SERVICE!!! Let this be a lesson to other companies. Bad communication will sink a business faster than the titanic. Thanks for handling this issue in a prompt and professional way.

    • LeoWid

      thanks so much for the kind words! So glad we could handle in an appropriate way here. :)

  • John Chapman

    Seems only female accounts were used in my case. Each a single post promoting a fruit diet which ‘worked for me’.

    • LeoWid

      Hey John, oh that’s very interesting! Will see if that can help us at all trace back the spammers.

  • http://www.paigeworthy.com paigeworthy

    You guys rule.

    • LeoWid

      Hey Paige, thanks so much for stopping by, it’s so great to have you on board! :)

  • Rene

    You guys are great, love the way you handled the situation. Lots of professionalism. Glad to be able to use Buffer again!

    • LeoWid

      Hey Rene, thanks so much for the kind words and encouragement! Yes, all should be back to normal!

  • Tony Parkin

    Sh*t happens! It’s how you clean up that counts. Great job on transparency, honesty and hard work to restore things. Have always been impressed with your customer relations… now doubly so. Sympathy for the problem, and support for you solution.

    • LeoWid

      Hey Tony, thanks so much for the kind understanding here! Indeed, it was a crazy day, everything should be back to normal now!

      • Tony Parkin

        Had a glitch with my LinkedIn association too… solved after fiddling about by deleting and reconnecting manually.

        Hope you are giving the team a post-hack party? Fine to have inquest etc… but also the worst times are the best times to recognise the work & commitment from the team!

  • http://www.marketinginbocconi.com/ Dario Pagnoni

    In my case I didn’t find any spam post in the 21 social accounts I manage with buffer. Is it possible that I was lucky, or should I check them once again?
    Anyway, tweets were blocked and I had to reconnect all my twitter accounts.
    Good job, guys ;-)

    • LeoWid

      Hi Dario, great to hear from you! Yes, so only a limited amount of accounts should have been affected, if you didn’t see anything strange, then you are all good! Yes, your Twitter accounts will have had to reconnect!

      Let me know if you run into any more questions that I could answer!

  • Anca Dumitru

    Security breaches happen all the time. I wasn’t among those affected. And even if I were, I would have trusted that you’d do your utmost to mitigate the damage. I kept an eye on the way you guys handled the situation and all I can say is kudos to the Buffer team for its sense of urgency and transparency!

    • LeoWid

      Hey Anca, great to hear from you and thanks so much for the kind words. You’re right, these have happened frequently in the past and so glad we could keep you in the loop on everything! It looks like we’re over the hill.

  • http://vernonchan.com Vernon Chan

    Good job for sorting it out quickly guys.

    • LeoWid

      thanks for the kind words Vernon!

  • Sean Cobb

    I upgrade to the awesome plan because of how well Buffer handled this situation. Keep up the transparency.

    • LeoWid

      wow, that is incredibly humbling, thanks so much for believing in us in this tough period Sean!

      • Sean Cobb

        I love your service and you have one of the best blogs in the business. I think you harnessed the incident by the transparency and you really gained more of my respect. I don’t see why you couldn’t gain more awesome subscribers from this episode.

  • Charles Terry

    I for one have to say that I am seriously disappointed. In a world where hacking happens all the time to companies such as yours, you should have been better prepared to fall to pieces and completely take forever to find a fix thus resulting in our account being compromised for an incredible amount of time allowing the spammers to send all kinds of porn and gambling ads for casinos in eastern Europe to all my friends and colleagues. How dare you actually fix the problem in such a timely manner, then actually have the nerve to post a blog message saying how not only you fixed the problem, but have strengthen the security of Buffer and are working with security experts on finding out what happened to prevent future attacks!?! And you continue to go on treating us like adults by providing future updates and an in-depth update on the incident to continue to encourage future confidence in this company! My God man, you make me want to throw up! Buffer has completely let me down and if this sort of customer service continues, I will have no choice but to continue paying for this service with my hard earned money and tell all my associates how terrible this service is. When we live in a world where not even our government can function properly, what kind of example are you setting? Where’s the panic?! Where’s the stress for us users to completely frustrated by a lack of communication and responsibility from the products and services we use and rely upon?! I’m physically sick from how this whole thing was handled and I ask this to the entire buffer team… How do you all sleep at night! Good day sir…

    • LeoWid

      Charles, wow, what a comment. Thanks so much for your kind words and encouragement. And it makes me really happy that you will continue to spend your hard earned money with us.

      We’ll do everything we can to keep you in the loop on this as we learn more!

      • Charles Terry

        Wow! A speedy response that shows you just read what you want to see! I can’t believe this company! I guess I’ll just have to send you all a nice fruit basket this christmas just to show you how upset I am with this kind of treatment! Unbelievable! :)

        • http://soundreview.net/ Johnny

          I can’t decide whether what your trying to say is good bad, or sarcastic..

          • Charles Terry

            Buffer is awesome. I’m being sarcastic….. Or am I???

          • http://soundreview.net/ Johnny

            I can’t tell because text doesn’t have expression unless you write it out ‘wink wink’

          • http://www.facebook.com/profile.php?id=1026609730 Jim Balter

            Johnny, there are numerous clues.

          • http://soundreview.net/ Johnny

            For the good or bad?

          • http://soundreview.net/ Johnny

            Wait so everything was alright yesterday apparently and all sorted, now I have to reconnect?

          • LeoWid

            Hi Johnny, yes indeed, you’ll have to reconnect your Twitter accounts! So sorry for the hassles!

  • Ashley Wagner

    I don’t think I have ever seen a company handle a crisis situation like this any better than you guys have! Mad props!

    • LeoWid

      wow, thanks so much for the kind words Ashley, really appreciate your support! :)

  • Harvey Brofman

    Way to handle this Leo and the Buffer Team. Just another example of why I hold all of my other startup investment companies to the transparency, service, and reporting standard you guys set by the way you do business. Thank you. Proud to be an early investor, love the work you are doing.

    • LeoWid

      Hey Harvey, thanks so much for the kind words, it means a lot coming from you especially. We’ll keep you posted on everything with further updates!

  • http://www.the7thMatrix.com/ Roderick T Faulkner

    I concur with everyone else. Your response to this situation has been nothing short of stellar!

    • LeoWid

      Hey Roderick, thanks so much for the kind encouragement! :)

  • Stefan Förster

    Thak’s

  • http://basus.me/ Shrutarshi Basu

    Hi,
    Thanks for the quick response and getting things up and running quickly. However, as a technical user, I would like to know the details of the compromise (what was the attack vector and what information was leaked) and what security measures are currently in place. While I acknowledge that you guys have done a good job, security is hard and I would like to make my own assessment of whether or not your measures are sufficient for my needs.

    Thanks again, and good luck!

    • LeoWid

      Hi Shrutarshi, great to have you stop by and you are 100% right. We currently have all of our tech team on figuring exactly that out. What was the full extent of the attack and how it all was possible to even happen.

      We plan to do a detailed technical updates on everything we’ve learnt asap then, you’re absolutely right that we need to get this all straigthened out!

  • http://www.davestravelpages.com/ Dave Briggs

    The fast and transparent response does you credit. I had only just started using Buffer, and was thankfully unaffected.

    • LeoWid

      Hi Dave, so amazing to have you on board and so glad you weren’t affected! Thanks for the kind words!

  • http://www.sociallysorted.com.au/ Donna Moritz

    Super impressed with how you guys handled this – but then again it was what I would have expected from you as you are so committed to providing a quality service and a great experience – I am sure this will be one of those grateful it happened + good learning experience things…. and all the talk I have heard online has been nothing but positive about Buffer :o)

    • LeoWid

      Hey Donna, wow, thanks so much for the kind words and for sticking with us through this. You’re right, trying to see this definitely as a positive learning experience!

  • http://www.geeknik.com/ geeknik

    I’m curious who the “security experts” are.

    • LeoWid

      Currently we’re working with our friends from Feedly, who have a lot of experience fortunately! (check them out at http://feedly.com )

      • http://www.geeknik.com/ geeknik

        I know who Feedly is, I use them for RSS feeds and they aren’t even close to who I think of when I hear the words “security experts”. Hindsight is 20/20, but Buffer should’ve hired me when I sent in my resume, might have been able to keep this from happening. ;)

  • http://www.CharlesSpecht.com/ Charles Specht

    you guys are awesome

    • LeoWid

      thanks so much for the kind words Charles!

  • Nick

    Hey guys, I just signed up yesterday and understand there were some issues but when i click on “Connect it now” for facebook, it just hangs on this page: “https://bufferapp.com/oauth/facebook/page/448687428506527″ and goes nowhere. Any idea how i can connect it to my account and start using Buffer?

    • LeoWid

      Hey Nick, thanks for the heads up on this and so sorry for the hassles here! Is this happening on the Android or iPhone app?

      • Nick

        Hey Leo, thanks for getting back to me. This is just through the website, but i did just download the android app and I am also have issues there linking it to my business page.

        • LeoWid

          Hi Nick, so sorry for the hassles with this, do you mind sending me a quick email to hello@bufferapp.com with the email you’ve registered with us? Keen to get things working well for you as soon as possible!

      • Nick

        Hey Leo, this is happening on the website. Seems to be just happening with my business page, my person page connects fine.

  • http://www.wordsfromnerds.com mtk67

    Outstanding job of communicating with your customers and taking swift action. Sadly, site attacks are getting to be commonplace these days. And, more often than not, companies try to hide the fact they were affected and wait days, if not longer, before saying anything. To have sent a note an hour after being attacked, to post regular updates throughout the weekend, to take the precautionary measures that you did, and then to quickly get us back up and running speaks volumes about Buffer to me. You’ve affirmed my decision to use your service with great customer service like this. Kudos.

    Now, just never let it happen again. :)

    • LeoWid

      wow, thanks for the kind words of encouragement here, really amazing to have you on board! :)

  • Sporapp

    Still i cant share anything to twitter on bufferapp. when i click share now it doesnt work

    • LeoWid

      Hey there,

      Great to hear from you and yes, you’ll have to reauthorize your Twitter account and that’ll have to happen on the web dashboard (http://bufferapp.com ) it won’t quite work on the iPhone right now! To reauthorize on the web, just follow this guide: http://open.bufferapp.com/wp-content/uploads/2013/10/reconnect.png

      So sorry for the hassles! Let me know if that helps at all!

      Keen to get things working well for you as soon as possible! – Leo

  • https://www.facebook.com/TheJuliaClarkOrganization Julia Clark

    Wow, I have not noticed a glitch at all. The hack did not effect me. Is there anything I should do like sign in and out or something? Everything works fine. I just posted a buffer to test it.

    • LeoWid

      Hi Julia, great to hear from you and if you weren’t affected you won’t have to do anything apart from reconnecting your Twitter account!

  • Cdog

    All of my “failed” tweets were gone when I re-authorized my Twitter account. Where did they go and how can I recover them? And, why didn’t we get an email letting us know what was going on. This impacts my business.

    • LeoWid

      Hi there, so sorry for the hassles we’ve caused here. Could you try logging in at http://bufferapp.com and reconnect your Twitter account one last time! It should work now!

  • Jamin Guy

    Really awesome triage of this situation. You guys rock!

    • LeoWid

      thanks so much Jamin! :)

  • piotry

    I did, am and will always love you :3

    • LeoWid

      thanks! :)

  • Rebecca Ishibashi

    thanks for giving us details every step of the way!

    • LeoWid

      thanks for helping us through this Rebecca!

  • Biswajit Dutta Baruah

    The twitter reconnect is not happening for me. :-(

    • LeoWid

      Hi Biswaijt, thanks for the heads up, yes, we unfortunately had some issues here. Could you try reconnecting one more time!

      • Biswajit Dutta Baruah

        Hey Leo, its not happening man. I tried 3 times.

        • LeoWid

          Hi Biswajit, so sorry for the continued troubles here. A quick question, can you make sure to be logged into the Twitter account you try to reconnect in a separate window before you reconnect in Buffer? Keen to fix this asap!

          • Biswajit Dutta Baruah

            I did that too. Still unable to connect. Says page missing at api.twitter.com Hope the screenshots help.

          • LeoWid

            Hmm, could you try and send the screenshot again? Somehow it looks like it didn’t come through!

          • B D Baruah

            The comments are back. Buffer’s back and connecting to twitter well. What else can I want? Good job done!

          • Biswajit Dutta Baruah

            Hey Leo. Why did the comments disappear? Is there a bug in Disqus?? :-D

          • LeoWid

            It’s back now! Ha, this was really strange, not sure what is going on here!

            Another suggestion, could you try just removing the Twitter account in Buffer altogether and then reconnecting it again?

          • Biswajit Dutta Baruah

            Ok I will do that. Just tell me how to back up first.

          • Biswajit Dutta Baruah

            It surely fits “The case of the Disappearing Comments”.

            Anyway just wanted you to know that even at ohours.org, twitter is throwing up the same page on attempting to authenticate. So the problem may be just a change in the api call. I hope this helps!!

            And do something about the comments. I believe you are pretty clever and can certainly do something about it. ;-)

  • wagthedoguk

    Thanks so much for being fantastic in regards to letting us all know what happen and your quick response to fix it! You basically buffed it up and we’re already to rumble!

    • LeoWid

      Wow, thanks so much for the kind words of encouragement and yes, glad we could go back to rumbling, err, Buffering! :)

  • Tadej Stanič

    Your customer support should be nominated for a Nobel prize :D Don’t let your amazing work be harmed by stupid guys. Wish all the best!

    • LeoWid

      Wow, thanks so much for the kind words Tadej, this is amazing to have you on board!

  • http://www.patriciamellin.com/ Patricia Mellin

    Thanks a lot for all information and the amazing job you have done. Buffer is the best.

    • LeoWid

      thanks so much Patricia!

  • Kobus Beets

    Good job Buffer team… The fact that you respond so quickly to your clients shows what kind of team you are, totally awesome. Here at kitchen [dot] net we enjoy using your product and service. As a software and web developer I know what it’s like being hacked and the hours that needs to go into it to get everything back into place. Keep up the good work developers. ;)

    • LeoWid

      Hi Kobus, thanks so much for the kind words and encouragement here, I can’t tell you enough how grateful we are to have you on board!

  • whiskers75

    Great job in recovering from the hack – I know it’s hard when things like this happen (my stuff’s got hacked in the past) and you handled it well. :)

    • LeoWid

      thanks so much for the kind words, so glad to have you on board!

  • mbrambila

    Thank u for increasing security.
    However, it would be so nice from it side to increase the buffer posts freely–as a compensation.
    A gift.
    Thanks!

    • LeoWid

      Thanks so much for the heads up on this and yes, we’re definitely pondering compensation right now!

  • Jake Criss

    Dear Buffer, You Rock The Web! Special note to Leo, well, you ROCK everything!

    • LeoWid

      Jake, thanks so much for the kind words and for sticking with us through this!

  • http://www.ronvanpeursem.com/ Ron VanPeursem

    Good job, Buffer! Good job, Joel! Good job, Leo! Taking responsibility; taking action; and being apologetic. What else could we ask for, man? Excellent!!!

    • LeoWid

      Ron, thanks so much for the encouragement and for cheering us on here! We hope to be back on track with everything now!

  • http://www.canadawebservices.com/ Steve

    Thanks for the update! Hope you guys will take care additional Security steps in order to save it from future hacks!

    • LeoWid

      Hi Steve, great to hear from you and yes, that’s correct, we’re doing everything right now to increase security from our end!

  • http://www.simplehrguide.com Luke

    I am really sorry, but reconnecting does not work for me… :-( The message is … it is nice your “” profile is reconnected…. but it is empty….

    • LeoWid

      Hey Luke, so sorry for the hassles on this! That’s no good at all. Is there any chance you can try logging into the Twitter account you want to reconnect in a separate window and have that open and then go back to the Buffer dashboard and hit the reconnect button one more time?

      So sorry for the confusion, let me know if the above works at all!

      • http://www.simplehrguide.com Luke

        it worked :-) thanks a lot and i wish you no more attacks! :-)

  • http://communityvillage.us/ Glenn Robinson

    thx! I’m reconnected now.

    • LeoWid

      Awesome, so glad this works!

  • darkdomino

    I’m sorry but you’re not going to get a cheerful “Awww” post from me. I’m very disturbed by this. I don’t see how you can expect us to trust and use your product when you can’t keep our information safe. Security should have been your top priority… People’s livelihoods are at stake here. Very not cool, guys. Very not cool.

    • James Mayes

      If you only work with providers who are 100% secure on your data, you better stop working with the internet right about now. It happens to everyone, at some point. What counts is how it’s handled.

      • LeoWid

        thanks for this James!

    • LeoWid

      Hey there, thanks so much for the heads up on this and you’re absolutely right. This was a huge mistake on our side and we’ve messed up big time. We’re doing everything to increase security on all our servers and will do everything we can to improve the experience from here onward.

      Let me know if I can answer any more questions directly for you here!

  • Kyle James Frazer

    Excellent customer service from Buffer. I was out over the weekend and just came in to read it all, impressive stuff.

    • LeoWid

      Thanks so much for the kind words Kyle!

  • Keith Neilson

    I don’t remember ever signing up with Buffer, If you have data of mine delete it please.

    • LeoWid

      Hey Keith, absolutely, I’ll check on this right away!

  • Desmond Brown

    Thank you guys for your honest and open communication about the topic. I am staying with you!

    • LeoWid

      Hey Desmond, thanks so much for the kind heads up on this!

  • http://www.webhostingsecretrevealed.net/ Jerry Low

    Remember what Thomas Wayne told little Bruce?

    “Why do we fall? So that we can learn to pick ourselves up.”

    Thanks for reacting fast and handling well on this. You guys rock!

    • LeoWid

      Hey Jerry, wow, I love that quote, thanks for the heads up!

  • Stefan van der Valk

    Guys, shit happens. But you did a terrific job in handling the problems.
    You are awesome. My compliments!

    • LeoWid

      Hey Stefan, really awesome to have you on board and in our corner!

  • us0r

    So what was the hack?

    • LeoWid

      We’re going to make a detailed announcement on this really soon and will publish another update!

  • http://intranetfuture.com/ Jonathan Pollinger

    Well done Joel and the team! Superb example of how to react swiftly and communicate openly, honestly and regularly in a crisis. Great to see Buffer back to normal. Buffer on!

    • LeoWid

      Hey Jonathan, thanks so much for the kind words of encouragement!

  • http://presspadapp.com/ PressPad

    Hi guys,
    We’re all delighted here in @PressPadApp how informative you were (actually you are) during this incident, so we did not have to worry at all. You’re one of our most valuable assets.

    Keep up fantastic work.

    • LeoWid

      Hey guys, thanks so much for the kind words and for being on board, so glad that you are putting our trust in us despite this big mistake!

  • Sarah Arrow

    Hey guys, I’m so impressed by how fast you acted upon this, and how you kept all the important things like our passwords and financial data safe. I know it must hurt, but I think you did a pretty impressive job in the face of what could have been a huge disaster. Kudos to you and your team.

    • LeoWid

      Hey Sarah, thanks so much for the kind words and yes, we’ve definitely tried to take special care of these details!

  • Jay Dixit

    Well handled.

    • LeoWid

      Thanks so much Jay!

  • fiveadayco

    It is times like this when you realise there are some very decent people behind your favourite services.

    • LeoWid

      Wow, this really means a lot thanks so much!

  • http://www.warptest.com/ Jonathan Ross

    A huge thank you for the professional way in which you handled this and huge props to whoever on the Buffer team spent their weekend resolving this. You guys are superstars!

  • thibault sarlat

    Keep on the good work. I like your transparancy.

  • Debbie Young

    Really impressed with how quickly and professionally you guys let us know about this incident. You clearly had it all under control straight away – very impressive. It certainly won’t put me of using Buffer, which I find a really useful app. Thank you. :)

  • http://felixrelationshipmarketing.com/ Juan Felix

    Thanks a lot for keeping us all up to date! The way you handled this is a great example for every other company wondering how to be a likable social brand! You guys still rock!

    • Drauch

      Awe come on. Unfortunately, Everyone gets hacked these days. The real test is how they respond and deal with their customers. I would say the consequences of the hack were minimal and the Buffer response was exceptional!

  • Michael de Groot

    Great communication and great recovery, well done to all of you for doing an awesome job and keeping your customers informed. Reputation in tact, no lasting damage I hope for you guys and I’m sure all customers stayed with you. Success and have a bit of a rest when you can!

  • http://lifestoogood.net/ Alan | Life’s Too Good

    Hey Guys,

    just wanted to echo the overriding sentiment of these comments but wanted to say so anyway – great job handling this & great communications throughout. I know how gut0wrenching being hacked can be and with such a huge customer base, you’ve handled this fantastically well,

    very best wishes,
    Alan

  • Lynn Serafinn

    Thanks for acting upon this rapidly! The impact on my blog traffic was minimal as a result.

  • http://www.davedelaney.me/ Dave Delaney

    Great job getting everything back up and running. Your speed and honesty through the incident was appreciated. Keep up the great work.

  • http://cliffpro.com/ Cliff Huizenga

    Thank you for your transparency and honesty with this incident. It’s nice to see a company take responsibility and quick action when something like this happens. Keep up the great work and remember: you still have us on your side.

  • Malinko Home Care

    Its a rite of passage surely? You are big enough to hack now :-) Thanks for sorting it all out so quickly. Communication was brilliant as ever. These things are pretty much inevitable, its how you responded when it did that marks you out as a brilliant company.

  • Shaun Heath

    Well handled, thanks. Great level of communication. Nobody likes to deal with problems, but you’ve done so admirably.

  • http://about.me/martingomez Martin Alejandro

    You guys are awesome. Shame on the hacker. I totally support you.

  • Women Art Dealers

    I was still asked to reconnect for a 3rd time today Monday 28th Oct at ca. noon London, UK time & it seems tweets buffered to take place during the past 12 hours have not gone out unfortunately

  • Carla Reilly

    Thank you for getting to the bottom of this so quickly. Hacking happens. We live in a digital world. It’s great that you are so transparent and that you are doing what needs to be done to rectify the situation and put security in place to stop it from happening again.

  • Stephen Bradley

    Still love Buffer Joel. Shit happens man.

  • http://www.adaptationproject.co.uk/ Amy Manuel

    I wasn’t affected as much as others must have been but I just want to say that I really appreciate the way you guys have handled this. Your updates have been spot on and have convinced me to continue using Buffer!

  • Susana

    Great job,well done! I love this app and your team has done a very good job in this crisis situation. In Spain I didn´t have any problem with the hacker.Great job,
    I will continue using Buffer because I like to work with serious people.
    Goog luck!

  • http://www.magicmarketingsystems.com/ Gary J P Hewett

    Kodus for transparency and quick communication throughout this ordeal.

    One of two things happen to a customer base when an incident like this occurs . Either they leave you in droves because they no longer trust you OR they stand beside you as proud partners knowing that since you’ve got their back they have yours.

    Joel – I’ve got your back :)

    I’m glad this was detected and handled with incredible speed and precision. (For those of you that don’t develop software you have no idea how complex managing a situation like this really is).

  • http://lifeinahouseoftestosterone.com/ Kim Miller

    You guys did an amazing job getting everything back up and running!!! I agree with Henri’s comment – “Remain Calm & Buffer On”

    • LeoWid

      Hi Kim, thanks so much and yes, we’ll Buffer On! Really amazing to have you on board! :)

  • http://www.wandsandworlds.com/blog1 SheilaRuth

    Thank you so much for being on top of this and working so hard on the weekend to get everything fixed and preventing further damage.

    • LeoWid

      Hi Sheila, thanks so much for your kind words of encouragement and yes, we’re working hard to prevent any further issues like this!

  • http://www.EGAFutura.com Juan Manuel Garrido

    Keep Calm & Buffer On :)

    • LeoWid

      Indeed we will, thanks for the kind encouragement Juan!

  • http://www.techtiplib.com/ Dao Vu

    What Bufferapp can do?

  • http://spacebarpress.com/ Julia Borgini

    Thanks for the prompt response & reaction! I knew all of you at Buffer were great, but this just proves it. =)

    • LeoWid

      Wow, thanks for the kind words Julia!

  • SmplProdBlog

    I want to thank you for taking responsibility for what happened. It would have been very easy just to blame the hackers. It is rare to see people own up and fix a problem. THANK YOU!

    • LeoWid

      Thanks so much for your kind words of encouragement. Yes, this was definitely our fault and we’re doing everything to prevent it in the future!

  • Roberta Oyakawa

    Your response is a master class in crisis management. Other companies need to observe and take note. Thank you … Buffer On!

    • LeoWid

      Thanks so much Roberta, Buffer on we will!

  • http://cwbuecheler.com/ Christopher Buecheler

    Thanks for all the updates, guys. I wasn’t one of the ones affected (lucky me!) but I appreciated your professionalism over the weekend. Really like the service and will be happy to stick with it.

    • LeoWid

      Hey Christopher, thanks so much for the kind words and for helping us through this!

  • http://www.cyclemania.ca Les Faber

    Totally transparency. Total professionalism. There are a lot companies that could learn a few things from you.

    • LeoWid

      This means a lot, thanks so much Les!

  • Spark Marketer

    We really do appreciate the updates. While it did cause me to have to go back and do some more work to re-queue my tweets, your timely updates made the process as smooth as it could possibly be. There are truly greater tragedies in this world. Thanks for your transparency!

    • LeoWid

      Thanks so much for the kind words and yes, so sorry for the troubles with the requeuing of your posts! If you reconnect your Twitter accounts one last time, it should all work!

  • http://www.primal-media.co.uk/ Rob Brideson

    Well done to all at buffer for your swift reaction to the problems that you have faced over the weekend. Open, honest and frank accounts of what has gone on and what was needed to be done to avert a catastrophe were forthcoming incredibly fast. If only other companies did the same, we would be very happy campers indeed. Needless to say I will be staying with buffer app as I know you guys have my six. Thank you

    • LeoWid

      Hi Rob, wow, thanks so much for the kind words of encouragement and so amazing to have you on board and put your trust in us after this big mistake we made.

  • http://www.ghostbloggermarie.com/ GhostbloggerMarie

    Your response to this unforeseen invasion has been commendable. Buffer is as much a victim as we, your customers. On the bright side, CONGRATULATIONS on being important enough to be hacked! :0) Love you guys

    • LeoWid

      Marie, thanks so much for the kind encouragement, I’m truly grateful to have you on board!

  • Anonymous

    So, how exactly did the attackers gain access?

    • LeoWid

      Hey there, that’s a great question! We’re completing our post-mortem analysis right now and hope to get this all ironed out asap and will publish a technical analysis after this!

  • Bryan Wachs

    I have not been able to use Buffer completely yet, as we are growing and watching our p’s and q’s. Just the way you handled this gives me confidence in your management. Kudos to you all!

    • LeoWid

      Wow, thanks so much for the kind words Bryan, it’s really amazing to have you in our corner!

  • http://giveawaygator.com/ SDgator

    The small interruption was nothing. Buffer will always be my social media tool of choice. Buffer 1 – hackers 0

    • LeoWid

      wow, thanks so much! :)

  • Guest

    You guys are killing me. I reconnected my account yesterday and to my surprise all my posts from today were not posted.

    • LeoWid

      So sorry for the hassles with this, that’s really bad on our side. We’ve added some extra security layers so you’ll have to reconnect all Twitter accounts one last time if you just go to http://bufferapp.com I hope this might help!

  • http://www.squidoo.com/lensmasters/flycatcher flycatcher

    The way you’ve handled this is admirable, guys. I’m upgrading to Awesome.

    • LeoWid

      wow, thanks so much!

  • daviddaba7

    I echo all the comments here. Every company can be hacked in some way, shape or form. It is how the company handles it and the Buffer Team rocked it. It all comes down to communication and it was spot on! I keep saying to myself, WHY haven’t I found you sooner but I move forward.

    Keep up the amazing work! Rock on!

    Dave

    • LeoWid

      Wow, thank you so so much Dave, this is an incredible comment, thanks for sticking with us!

  • Atlanta Kings

    So after I vent about NOT being notified the first time it was necessary to reconnect all my Twitter accounts in order for posting to resume you guys have done something on your end that now requires me to re-reconnect my accounts. 11 more hours have passed (after an initial 17 hours of no postings for me) and I’m thinking all is well in the land of Buffer but it’s not. Thousands of hits lost.

    I really don’t get why all these folks are praising how you handled the hacking. “Their service was hacked but then they fixed it.” NEWSFLASH: That’s what they are supposed to do. Especially if you are paying for it. That’s great that you guys got an e-mail about it but I didn’t. Don’t know why. It’s not in SPAM, it’s no where.

    I’ve decided to no longer use Buffer due to this. If you don’t care enough about me to let me know when my account isn’t posting tweets (which is ALL I use this for) then what am I paying for? Pile this on top of the lack of development on additional networks like Tumblr as well as the inaccurate analytics I really just can’t justify this expense.

    • LeoWid

      Hi there, I greatly apologize that we’ve been so negligent in your case here and haven’t sent an email on this notifying you throughout the hassles.

      I can only understand how much of a pain this must have been for you, I’m deeply sorry.

      I can completely understand that you don’t want to use Buffer anymore and if you want to let me know your email I can also go in and refund you the last payment asap, at least as a small token of apology.

      Again, my sincerest apologies and please let me know if there are any more questions I can answer here!

  • Kelsey Jones

    Thanks for the fast updates on this and the updates by email! I appreciated knowing what was going on. Honesty always gains my trust!

    • LeoWid

      Hey Kelsey, thanks so much for the kind words and for putting your trust in us, this is incredibly generous!

  • Meredith Gould

    The way you’ve handled this with fans, followers, friends, and clients is exemplary. I plan to use (in a good way!) you as a case study with my clients.

    • LeoWid

      Thanks so much for the kind words Meredith and for helping us through these tough times!

  • John Chapman

    If you are using multiple accounts in buffer your instructions for reconnecting accounts need a crucial extra step – Make sure you are logged into Twitter as the account you want to re-connect – Without doing that you are locked in an endless loop of reconnect notices.

    • LeoWid

      Hi John, great to have you stop by and thanks so much for pointing this out! upvoted the comment, being logged into the right Twitter account on Twitter is crucial!

  • http://www.biogs.com/ Danny Rosenbaum

    I really appreciate you openness. I will happily keep using Buffer!

    • LeoWid

      thanks so much for the kind words Danny!

  • http://www.charitykountz.com/ Charity Kountz

    I’m glad to see this. I’m really glad to see that my account was not affected nor were my clients. I’ll be monitoring moving forward but all’s clear here so far. Thanks for taking such a proactive stance. Hacks happen to the best of us and your actions have been A+ in terms of crisis resolution and communication about the issue. Kudos Buffer team!

    • LeoWid

      Hi Charity, thanks so much for the kind words and so glad that you and your clients weren’t affected.

      I can’t tell you enough how grateful I am that you are keeping up your trust in use, we’ll continue to post updates here.

  • Anthea Kawakib

    buffer is so wonderful – every company should be run like this.

    • LeoWid

      Wow, thanks so much Anthea, really appreciate the kind words!

  • Allen Mireles

    Impressed with how you have handled this. Thanks for the ongoing updates.

    • LeoWid

      Thanks so much Allen, really appreciate this!

  • OIRMS

    No problemo Team Buffer, hackers suck! However, I DO appreciate that you guys jumped right on this, are genuinely sorry, and kept us updated. Can’t ask much more than that.

    • LeoWid

      really appreciate the encouragement, we’ll continue to post further updates here!

  • http://www.thekcagent.com/ Bryan Tobiason

    Sometimes poop happens! I commend you guys for sending out update emails and letting us know you’re on top of the issue. I’m also certain you’ll use this experience to further improve your security which is never perfect in our advancing world. Thanks again & I’ll continue happily using Buffer!

    • LeoWid

      Hi Bryan, really appreciate your kind words here!

  • https://bmtmedia.org/ turner_bethany

    These types of situations are difficult, but I have been impressed with how the Buffer team has handled it. Thank you for keeping us well-informed throughout the whole process.

    • LeoWid

      Hey Bethany, thanks so much for the kind words of encouragement!

  • eryn o.

    Thank you for your candid and communicative handling of this situation. It sucks a lot when these things happen, but they DO happen, and how a company responds says a lot about them. Other companies could learn a lot from this example, and I hope they do.

    • LeoWid

      Hey Eryn, thanks so much for the kind words of encouragement, so glad that we could keep you in the loop! More updates to come!

  • MJ Schiller

    I am incredibly impressed by how well this was handled, both in a timely manner and with open, caring communication with users. Thank you. This is why I will continue to be a Buffer fan!

    • LeoWid

      Wow, this really means a lot, thanks so much, we’ll work hard to continue on with the update and fixes!

  • Luc Jallois

    Congrats for your crisis handling and your five stars transparency and communication. You guys rock! ❤️

    • LeoWid

      Hi Luc, thanks so much for the kind words of encouragement here!

  • WHAKi

    Excellent work this weekend. Thank you for all your updates!

    • LeoWid

      thanks so much, we’ll keep posting updates here!

  • Nazila Alasti

    It would have been easy to lose me as a customer, as I only recently began. But you folks did a masterful job of turning adversity into lemonade! Good job and great customer service.

    • LeoWid

      Hi Nazila, thanks so much for the kind words of encouragement, so glad we could keep you in the loop on everything!

  • http://about.me/josephmanna Joseph Manna

    Great post, crappy situation. Thanks for keeping us informed to how you are responding and mitigating these risks.

    • LeoWid

      Hi Joseph,

      Thanks so much man, it means a lot coming from you, glad we could keep you in the loop!

  • David Davies

    I’ve had to reconnect twice. Hope it’s going to work again this time. Annoying, as I’m still trying to tweet Saturday’s news :(

    • LeoWid

      Hi David,

      so sorry for the hassles on this! That’s no good at all. Is there any chance you can try logging into the Twitter account you want to reconnect in a separate window and have that open and then go back to the Buffer dashboard and hit the reconnect button one more time?

      So sorry for the confusion, let me know if the above works at all!

      • David Davies

        Hi Leo

        Looks like the second reconnect worked. Fingers crossed.

        David

  • Cathryn Wellner

    Thanks for your hard work – sorry you had to go through all the hoops.

    One question – in the second Twitter update, we have to agree Buffer can update our Twitter profile. Did I misinterpret?

    • LeoWid

      Hi Cathryn, thanks so much for the kind and encouraging words.

      Yes, that’s correct, you will have to agree that Buffer can update your Twitter profile! Let me know if you run into any issues with this!

      • Cathryn Wellner

        Thanks for the quick reply, Leo. I must be missing the implication. Why would I let anyone else update my profile?

        • LeoWid

          Hi Cathryn, oh right yes, that makes sense. So if you want to schedule updates with Buffer, then we’ll have to have access to post on your behalf. So sorry for the confusion, hope that makes sense!

          • Cathryn Wellner

            Thanks for your patience, Leo – Buffer’s terrific.

  • http://www.Marisashadrick.com/ Marisa Shadrick

    Things happen. I’m glad you guys are on top of it, and I appreciate the notifications. Don’t stress, life is too short.

    • LeoWid

      Thanks so much for the kind and encouraging words Marisa!

  • 1fiftyfive

    i keep reconnecting and it says it’s reconnected but all my posts fail. help!!!!!!

    • LeoWid

      Hey there, so sorry for the hassles on this! That’s no good at all. Is there any chance you can try logging into the Twitter account you want to reconnect in a separate window and have that open and then go back to the Buffer dashboard and hit the reconnect button one more time?

      So sorry for the confusion, let me know if the above works at all!

  • pulsepointdesign

    I’ve never been so impressed with a company’s behavior in the midst of crisis. Through everything your communication has been honest, thorough, and quick to address the issues. Your sincerity and transparency through this process is a model of exactly how to handle a bad situation. I’m cheering for you!!

    • LeoWid

      Wow, thank you so much for the kind words of encouragement, it’s amazing to have you on board cheering for us!

  • Atlanta Kings

    Switched to Sendible. Hands down it’s better for my needs plus it supports Tumblr, Delicious, WordPress, etc. Loving it so far. (Also it’s cheaper. Go figure.)

    • LeoWid

      Hey there, absolutely understood, I think Sendible is a great solution! If we can help at all with any merging over of your old accounts please let me know!

      Again, I greatly apologize for the hassles here!

  • http://www.dionnekasianlew.com/ Dionne Kasian-Lew

    Like everyone else really loved the way you guys communicated this – a great case study in crisis management – kudos kudos kudos

    • LeoWid

      Hey Dionne, thanks so much for the kind heads up and for helping us through these tough times!

  • http://www.sweetnessorganic.com/ Sweetness Organic

    Thanks for your #hardwork :)

  • LS25 Web Design

    What is going on? I reconnected my twitter accounts yesterday and successfully published tweets from buffer but I have just checked today after engagement with my business was slow today and I find all my tweets are failing again.

    • LeoWid

      Hey there,

      So sorry for the hassles on this! That’s no good at all. Is there any chance you can try logging into the Twitter account you want to reconnect in a separate window and have that open and then go back to the Buffer dashboard and hit the reconnect button one more time?

      So sorry for the confusion, let me know if the above works at all!

      • LS25 Web Design

        Thanks for the prompt response Leo. You must be very busy at the moment. I have reconnected my Twitter accounts and it seems to be working again now. I will monitor it tomorrow and let you know if there is still a problem.

        Again, thanks for the prompt follow up. It’s much appreciated.

        • LeoWid

          Awesome, so glad that this is working now! Please let me know if you run into any more issues!

  • http://www.lucrazon.com/blog Alice Ly | Lucrazon

    Great to hear that everything is fixed! Keep up the great work!

    • LeoWid

      Hey Alice, thanks so much for the kind support!

  • cindykendall

    I really appreciate Buffer’s communications via many channels during this breach. Nice job keeping customers informed and working through the process. Thank you for the honesty, transparency, and assistance.

    • LeoWid

      You rock Cindy, thanks a huge amount for your kind words and encouragement!

  • Tom James

    As a company, you are inspirational.

    • LeoWid

      Thanks so much for the kind shout Tom!

  • Fabio Federici

    Even though I’m not a regular user of Buffer I would like to tell you that I am truly impressed with your crisis management. Every company should take you as an example for how to handle such a critical situation.

    Best regards from Switzerland and keep going,

    Fabio.

    • LeoWid

      Hi Fabio, thanks so much, I’m glad that we could handle the crisis in a good way. We’re also going to continue to post updates on this!

  • http://www.sarahmillar.com Sarah Millar

    Thank you for the wonderful communication and proactive communication you’ve had with all users since this happened. This makes me love Buffer even more!

    • LeoWid

      Wow, this is really amazing, thank you so much for the encouragement Sarah, so glad to have you on board with us!

  • Modify Watches

    Way to go team! Great effort in finding the issue, identifying, communicating. Just want to echo everyone else’s kudos. Bad things will happen, it’s how you react and own the issue that counts. Y’all get A+’s

    • LeoWid

      Thanks so much, really appreciate the kind words! :)

  • Andrew Mastrandonas

    Best job I’ve seen ever in responding to an incident, Joel, Leo, and team! Problems do happen that you cannot control but you guys did a GREAT job keeping people updated.

    • LeoWid

      Thanks so much Andrew, it means a lot to hear that you approve of the way we handled things, thanks so much!

  • http://twitter.com/guy Guy Malachi

    You guys handled the situation very well. My biggest concern with the hack is that apparently you can remove all my Facebook posts made through Buffer. How do I know that posts won’t disappear in the future?

    • LeoWid

      Hi Guy, thanks so much for the kind words. Yes, that was actually a measure taken on our behalf and I believe that this was likely not the best way to have handled this, I think we aren’t going to hide all FB posts in the future, but I greatly apologize that we’ve jeopardized you and your company through this!

      • http://twitter.com/guy Guy Malachi

        Thanks for the reply, way to keep on top of things. Keep up the good work

  • Conley Milam

    Great response time.

    • LeoWid

      Thanks so much Conley!

  • http://www.2ConnectWithGreg.com/ ConnectWith Greg

    Very classy and handled honestly and professionally. I much appreciate your quick response and updates to let us know what’s happening.

    • LeoWid

      Thanks so much Greg!

  • http://www.chipdizard.com/ Chip Dizard

    I heard about the attack and I got a tweet from a friend, but I had no doubt that the Buffer team was on it. This is truly a case study in how to handle a hack. I think I will blog about this incident next. Cheers guys, and I will continue to use your service.

    • LeoWid

      Hi Chip, thanks so much for the kind words and for believing in us through these tough times, this really means a lot!

  • Laura Simms

    I was surprised to check my Buffer on Monday morning and see that posts relating to a launch were not sent this weekend. Wish I had known sooner so I could have posted manually and not lost those marketing days. I love Buffer, but am disappointed I wasn’t notified about this sooner. I know some users have reported receiving emails right away, but I have checked all folders, including spam and trash, and I did not get notice.

    • LeoWid

      Hi Laura, great to hear from you and I greatly apologize for our mistake in communicating with you! I’m not sure why you weren’t on our email list to receive this update, but this is definitely fault. Is there any chance I could help you with any remaining questions right now? Keen to get things working well for you as soon as possible!

      • Laura Simms

        Thanks for the reply, Leo. I’m all up to speed. Thanks!

  • http://kylemjones.com/ Kyle Jones

    You were COMPLETELY transparent. You were COMPLETELY upfront. You did an AMAZING job at communicating with your customers. HONESTLY….all I can say is “KUDOS!”

    • LeoWid

      Kyle, wow, thanks so much for the kind words and encouragement here, this really means a lot!

      • http://kylemjones.com/ Kyle Jones

        Very welcome.

  • N. Horsthuis

    Thanks for letting us know. We had a lot of users respond to the post on our FB page with about 350,000 fans. We have seen a spike in unlikes from the page
    :(
    My immediate reaction was to delete the post, change passwords and remove apps. Thankfully my mind is at rest after reading this post.

    I hope this doesn’t happen again. Still an awesome tool and I hope this doesn’t effect your growth.

    • LeoWid

      Hi there, I greatly apologize for the hassles we’ve created here for you and your big Facebook page, that’s really bad.

      I also greatly apologize that we didn’t send you an email, really not sure how we didn’t get you onto our list!

      Yes, we’re trying everything not to have this happen again in the future and I hope we can have this working again for you in no time.

  • Lucas

    Some of you might know that Korean language has different levels of language : one of the these higher level of politeness and respect is for the grandparents AND the customer. This is a concept that is not very common in our western countries. With this in mind I can tell you that I am just amazed by :
    - the respect of Bufferapp team for their customers
    - the total transparency and the positive way this “crisis” has been handled
    - the fact that Leo takes personally time to reply each comment…
    Just one word : Bravo !

    • LeoWid

      Hi Lucas, thanks so much for stopping by here and I hadn’t heard about how the Korean language works before you mentioned it, that’s super interesting!

      I’m so glad we could keep you in the loop on everything and we’ll continue to post updates on the blog with further things we uncover!

  • gmlevinmd123

    Nice and credible reaction to a surprise breach. . Thanks

    • LeoWid

      It caught us by surprise indeed!

  • Marianne

    I’m very impressed with the way you guys have handled this. Maybe you could help the US govt deal with healthcare.gov!!

    • LeoWid

      thanks so much Marianne, so glad we could keep you in the loop on this!

  • REMIBRASIL

    I liked the beginning Buffer and decided to pay to post my link but today I decided to suspend the subscription because ‘do more’ to insert the links before you could double that. then I pay to do if I can not enter the doubles? GOODBYE

    • LeoWid

      Hi there, so sorry for the hassles that we’ve created here. That makes a lot of sense, I greatly apologize for this bad experience. Just to quickly clarify, you mentioned “doubles” do you mean adding 2 links to a social update? Keen to get things working well for you as soon as possible!

      • REMIBRASIL

        will double in the same way the various links at the same time. early work was putting them together now also 3-4 and ‘limited to only one, so for me it makes no sense to pay. if it will be ‘reset this option lighthouse’ of the new paid subscription. thank you for answer

  • frenat

    This was crazy good service and communication. Crazy good.

    • LeoWid

      thanks so much for the kind words, so glad that we could keep you up to date, more to come!

  • einfal

    Thanks for the update @LeoWid:disqus . This is most concerning to learn of. I had noticed i had to reauthorize buffer, i did not realize until i received the email, why!

  • bmhansen

    You are all doing great work. Keep it up and keep smiling.

    • LeoWid

      Really appreciate that, thanks so much for the kind words. :)

  • http://barca.daa.jp daisuke

    Hackers did not steal to souls of Buffer :^) ANIMO Joel, teams!

    • LeoWid

      Indeed, that did not! Thanks so much for the kind words!

  • http://akutidakjugakau.blogspot.com/ Ardhiansyam

    Google+ post from ifttt failed, any idea?

    • LeoWid

      Hey there! G+ posts are currently not working, but should work again in a few hours, could you check back then? So sorry for the hassles.

      • http://akutidakjugakau.blogspot.com/ Ardhiansyam

        Thank you for your great work! I’ll check later :)

  • http://refinch.com Bob Finch

    I must say that you guys really know how to handle a crisis from a communication standpoint. Kudos!

    • LeoWid

      Thanks so much Bob, really means a lot to hear that from you!

  • Alexis S.

    Thank for the last update.
    Since lot of us are web dev or startup entrepreneurs, would you be nice enough to share a more technical explanation on how the hack was possible.
    We would learn a lot for sure and it would protect our business too.
    Thanks

    • LeoWid

      Hey Alexis, great point and yes, we’ll definitely come out with a more detailed explanation on this!

  • http://chrishumboldt.com/ Chris Humboldt

    Glad you guys were this open and honest about it. Transparency is the best course.

    • LeoWid

      thanks so much Chris!

  • Paul Kuijs

    You have done a great job in resolving the issue! But what I appreciate most is your transparancy and openess on the matter and the way and frequency you have been communicating about it! Keep up the good work!

    • LeoWid

      Thanks so much for the kind words Paul, so glad we could keep you in the loop on everything here!

  • Conley Milam

    Does Buffer have the ability to post to multiple Facebook groups?

    • LeoWid

      Hi Conley, great to see you here and currently you can’t quite post to multiple FB groups, FB groups is a feature we hope to have soon though!

      • Conley Milam

        Thanks Leo, this would be a great feature.

  • jonaswagner

    I’d appreciate it if you’d release the technical details about the incident. That might restore some of the trust in you.

    • LeoWid

      Hey Jonas, great to see you here and absolutely! We’re working on a big report right now to go into a lot more technical details on this. Keen to put more info about this out soon!

  • msx

    Bad things happens, that’s it. Don’t look back guys.

    • LeoWid

      Great outlook, exactly the perspective we’re trying to take here!

  • http://joshlsullivan.com/ Josh Sullivan

    Excellent work. Thank you.

    • LeoWid

      thanks so much Josh!

  • Patrick Haley

    It could be worse. At least they won’t be coming in through the same back door.

    • LeoWid

      Hey Patrick, totally agree, it’s a big relief that this is closed now!

  • http://www.thetolkienist.com/ Marcel Aubron-Bülles

    I’ve been using Buffer as a free service for quite some time now but have been considering options on paid services for what I would like to do on social media. Although any hacks aren’t fun to get along with your way of keeping users up-to-date and showing your concern on all issues has been quite a ‘thumbs up’ to me. Others might not have gone your route… So all things considered I’ll probably become a paying customer at the start of next year.

    Keep some extra security tokens for the billing info, though ;)

    • LeoWid

      Hey Marcel, wow, thanks so much for the kind words and for still sticking with us through this difficult time. It really means a lot to have you on board!

      And yes, lots more security added there!

  • Kevin Brinkley

    I am glad you’re being careful but…PLEASE EMAIL ME WHEN IT’S NOT WORKING! Buffer has been so reliable that I don’t go check to see if it’s actually posting. Today I checked and several feeds are empty for the last four days. Please let us know when it’s down or there is action required on our part. Otherwise, well done.

    • sunils34

      Hi Kevin! I’m so sorry that you logged in today and saw a bunch of failed posts. That’s not good.

      We have an email setting that you can turn on which will email you every time a post fails for some reason. If you turn this on, we’ll email you when we’re not working :). https://bufferapp.com/app/account/email

      Hope that helps!
      Sunil,
      Buffer CTO

  • http://about.me/arturogarrido arturogarrido

    Thanks for the transparency and your quick reaction to this unfortunate events.

    After reading how the hackers stole the access tokens from your users, I have a question: did you store your FB & Twitter API keys in the MongoHQ database?

    If I’m not wrong, even in the case that a hacker steal the users access tokens, the hacker still needs your API keys in order to make valid requests to FB and Twitter. One of the first lessons I learned working with third party APIs is to store the app API keys in a secure and separate place from the users access tokens.

    I’m confident that these events will make Buffer (and MongoHQ) more secure and robust. Kudos to the team.

    Arturo

    • sunils34

      Hi Arturo, Thanks for asking these questions for us to clarify! We do not store our FB and Twitter API keys in our MongoHQ db. They’re located in a separate place.

      With how we originally set up our apps, the access tokens did not need the API keys to make a valid API request. Which is why the hacker was able to spew spam. We’ve changed our Facebook app to require an appsecret_proof, which checks the validity of the access token with our API key. Thus stopping the spammer from making valid requests. Unfortunately at this time, Twitter doesn’t not have an equivalent of this extra parameter. This is why we had to invalidate all Twitter tokens and encrypt them ourselves.

      Let me know if you have any more questions about this!
      Sunil
      Buffer CTO

      • http://about.me/arturogarrido arturogarrido

        Sunil, thanks for your explanation.

        Interesting what you said about Twitter, because I´m pretty sure that you need your API Keys (Consumer Key and Consumer Secret) and the user Access Token to sign and make an authenticated request to the Twitter API.

        Just as a quick exercise, using Twitter OAuth tool I just tried to use my access tokens generated for one app with another app and as expected i received an error: {“message”:”Could not authenticate you”,”code”:32}

        Maybe I could be wrong and maybe I was too technical. I’m not questioning or attacking. Please take my comments as well intentioned from someone who just wants to make sure about this stuff so we can prevent this to happen to other devs too.

        Thanks again,

        Arturo

        PS: Is the backend engineer position still open? =)

  • http://www.commentluv.com/ Andy Bailey

    Thanks for keeping us informed, I really get the urgency in how buffer tracked and finally fixed this.

    I feel confident that I’ll continue to pay for this service.

    Good job with your transparency and honesty!

    • LeoWid

      Thanks so much Andy, that really means a lot!

  • http://notdrinkingcoffee.com/ Lynn Long

    Your customer service has always been tops. And now, after seeing your prompt and honest response to the problem, I’m a bigger fan. Thank you for providing us with real, meaningful updates. As @henri_deschamps:disqus said, “Buffer on.”

    • LeoWid

      Hey Lynn, thanks so much for stopping by and I really appreciate the kind words!

  • Williams Martinez 

    Awesome! You guys are awesome! An example of transparency, this is how trust is build. Rock on!

    • LeoWid

      Thanks so much Williams, that’s so kind of you!

  • Tom

    Probably why it’s best to manage your own databases, at least your in control of your own passwords then!

    • LeoWid

      Wow, thanks Tom, appreciate the kind words!

      • Tom

        Thanks the sarcasm, if it was sarcasm or seeing as this is a similar reply you gave to all posts, maybe you didn’t read them before replying. This was not a compliment. It was a huge failure that could have been prevented by knowing exactly where all your DB passwords are and keeping all ways in inside your own company.

        • LeoWid

          Whoa, you’re right, totally misread this, my mistake! just updated the previous comment.

  • Jim Tompkins

    All appeared calm on the surface, but you guys must have been paddling like crazy beneath , Thanks (-:

    • http://joel.is/ Joel Gascoigne

      Haha, that’s for sure Jim! Thanks for the kind words :)

  • http://craighunkele.com/ Craig Hunkele

    I think it’s awesome that the initiative was taken to keep us posted on what and how everything happened. The manner in which the situation was handled only added to the already positive image of the Buffer Team.

    • LeoWid

      Hey Craig, thanks so much for the kind encouragement, it really means a lot!

  • http://www.carlconrad.net/ Carl Conrad

    I’m impressed by this level of transparency. A lesson for many of us.

    • http://joel.is/ Joel Gascoigne

      Thanks Carl! Glad you mention that. It inspires us to keep pushing further with transparency.

  • Rooster

    Joel, and the whole buffer team.
    Your postings, the transparency and frankness, the acceptance of responsibility, the speed to resolution are wonderful. A true model for many to emulate. This type of response increase everyones trust in you, your product/service and gains you more fans and customers. Thank you and everyone involved at Buffer and at MongoHQ for the work in solving this.
    John

    • http://joel.is/ Joel Gascoigne

      Thank you so much for taking the time to say this, it’s really appreciated. We’re lucky you and others are so forgiving. We’re doing lots to improve our security now.

  • http://johnconway.co/ John Conway

    Amazing that getting hacked could make me more appreciative of a company, but your response has been genuine class. Nicely handled.

    • LeoWid

      Hey John, thanks so much for the kind words and for being on board! :)

  • http://ww.projectaddapp.com/ Kouris Kalligas

    Buffer team you are absolutely amazing! My account was not among the ones which was hacked but I have been following all updates & most comments. I think your response to this must become a case study for customer service seminars & students! You have replied to every single comment and this is awesome! I think you came even stronger after this unfortunate incident!

    • LeoWid

      Wow, thanks so much for the kind encouragement Kouris!

  • molipier

    Don’t worry guys! You are great.
    I have only a little problem, i can’t reconnect my twitter account… :(

    • LeoWid

      Hey there, so sorry for the troubles on this! Could you let me know what the error is you’re seeing here? Keen to fix asap!

      • molipier

        Hi Leo, no problem, now it seems it’s working! Thanks! :)

        • LeoWid

          Awesome, that’s so great to hear!

  • Angie Maxhimer

    I still cannot reconnect my accounts.

    • LeoWid

      Hey Angie, so sorry for the hassles here! Are you using the Buffer webdsahboard to reconnect? If so, could you let me know which error you’re seeing?

  • Brian Pensinger

    This is the precise way a company should handle this sort of thing. Well done Buffer!

    • LeoWid

      Hey Brian, thanks so much for the kind words, really amazing to have you on board, glad we could handle it in a good way for you!

  • Larisa Kudisheva

    I was lucky and my account was not effected by the hackers.

    • LeoWid

      hi Larisa, really glad you weren’t affected here!

  • nicktalwar

    i am still getting complaints from people that i am spamming their newsfeed even though i disconnected my buffer account from twitter and deleted the app privileges from facebook. can someone please help?

    • LeoWid

      Hi Nick, whoa, so sorry I’m only just spotting this comment now. That’s really bad, is this still happening for you now? Really keen to fix asap!

  • nafees

    glad the security breach is taken care of.

    • LeoWid

      yup, tried to do that asap!

  • st

    Well done! All dealt with super professionally! You guys are awesome.

    One comment: I normally use the mobile app, and I only came onto the web because my Twitter posts were failing… So even though I knew you’d been hacked, I had assumed all was resolved until I realised my Twitter wasn’t being updated :(

  • Wim ten Brink

    Qoute: ” If access tokens were encrypted (which they are now) then this would have been avoided.” This is actually very good advice to all web developers! Always encrypt your tokens! Thanks for reminding us about this… :-)

    • LeoWid

      Hey Wim, yes, indeed, I think that’s a key lesson we’ve learned here!

  • Katrina Moody

    I’ve followed this from afar as I was away from home when everything went down – I’ve been so impressed with your handling of this breach! It shows your amazing dedication to service and transparency!

    • LeoWid

      thanks so much for the kind encouragement Katrina!

  • pll

    Does Buffer use any third-party code analysis tools to scan for security vulnerabilities such as Veracode, Inc. ?

    • LeoWid

      Hey there, that’s a great question, we don’t actually use anything like that right now, this would definitely be interesting to look at however!

  • http://robzie.wordpress.com/ Rob Zaleski

    You guys handled this right from the onset, and just keep doing so! Thanks for keeping everyone informed.

    • LeoWid

      Hey Rob, thanks so much for the heads up on this and glad we could keep you in the loop. We’re learning more right now, so hopefully we can publish soon!

  • http://www.markpack.org.uk/ Mark Pack

    Thanks for handling this so well. It’s very refreshing to have such regular and clearly written updates when a service runs into trouble.

    • LeoWid

      Hey Mark, thanks for stopping by and really glad that we could update you quickly on this. If you have any questions on the breach, please let me know, I’d love to answer!

  • http://about.me/sandmaxprime Lionel Faleiro

    I love the fact that you guys are so transparent about what happens behind buffer. Its kinda like an investment call that always brings smiles to many faces ;)

    My suggestion would be to get a Penetration Testing done. I know few days back MongoHQ also had a break in and so it could be related. If Adobe also uses Mongo as their DB then it could be a group that just hates Mongo as a whole and is targeting companies using it.

    No matter what happens, I’m always glad to see the entire team cool and optimistic.

    • LeoWid

      Hi Lionel,

      Thanks for stopping by and for the kind words here. Really glad that we could be so transparent here.

      And yes, that’s a great point, we’ve definitely been thinking about a lot of security improvements since the hack and have implemented a number of changes. Penetration testing is another great idea!

  • geekosupremo

    A hug to sooth the troubles, and :bro knuckles: for having our backs. Thank you for your swiftness and openness about this. It remind us that you’re made of humans, in all the best ways!

    • LeoWid

      thanks, both hugs and :bro knuckles are hugely appreciated. Yes, really keen to be up front about this!

  • http://brankicaunderwood.com/ Brankica

    My account wasn’t affected but I am really amazed at how you dealt with this. Happy you fixed it all and found what the problem was.

    • LeoWid

      Hi Brankica, thanks so much for the kind heads up on this, that’s really amazing to hear. So glad to have you on board with us!

  • usagi

    Amazing cover and clear explanations. Thanks 4 all

    • LeoWid

      thanks so much Usagi for the kind words!

  • Owen McGab Enaohwo

    @LeoWid:disqus and @joelgascoigne:disqus I love how y’all stepped up to the challenge, addressed your community, quickly made changes and let them know when it was solved.

    Cheers!

  • http://www.arseburgers.co.uk/blog/ theaardvark

    “I want to be clear that this is still our fault.”

    I find this line incredibly reassuring. Whilst you explain that the initial vulnerability was through a third party, the fact that you are not looking to pass the blame gives me confidence that you are taking the correct approach to this issue and are likely to do whenever any other issues arise.

    Once again folks, good work.

  • Peter Smith

    So the hackers got your code too? We will be seeing a competing free service soon. “Duffer” – personal social spambot!

  • http://www.woodstone.nu/salive Amma Rany

    Hi Joel. very pleased to see the explanation. waiting for other interesting posts at a time when that will come.
    From bvba Woodstone

  • http://www.woodstone.nu/salive Amma Rany

    Hi. Its really a nice post, the content of this blog is really awesome and extraordinary. waiting for other interesting posts at a time when that will come.
    From bvba Woodstone

  • Pingback: From Android Contractor to CTO: My Story As An Engineer - XTBlog!

  • rony009

    Hey I had the same issue with my computer. I wasted my time on it for

    many days but finally I got a solution from this link

    http://www.microsoftliveassist.com/were-sorry-but-word-has-run-into-an-error-that-is-preventing-it-from-working-correctly/

  • Pingback: The Best Customer Experience is Securing Customer Data