To get the lowdown, and keep in touch with all our news, head over to http://blog.twitterfeed.com – see you there!

This is a brief update on the current downtime:

Over night, a number of disks on our primary database server failed. The hosting provider has since replaced the disks, but was unable to restore the latest data. Now, we do of course have a fully up-to-date backup/slave database, but as we are in the middle of moving to a new hosting provider, we can’t (yet) just switch to the backup database server without delay. We are working as fast as we can on getting the service back up, but due to the enormous size of the database and the complexities of the hosting setup this may still take a few more hours.

We’d like to say how sorry we are for the problems this causes you, and at the same time want to reassure you that the move to the new hosting facility/provider will  - once completed – make these lengthy outages a thing of the past, as we’ll have full redundancy there, so will be able to hot-swap database  and other servers in the event of similar hardware failures. It’s incredibly unfortunate and frustrating to us that this current disk issue happened while we still haven’t fully completed the migration to the new hosting facility.

Dealing with this kind of explosive growth and pushing more than a million pieces of content to Twitter alone every day has its challenges, particularly around scaling and we’re working hard to make sure that Twitterfeed continues to increase the velocity of posting so that we can reach our goal of being able to publish your content in real time every time. If you like the idea of scaling big services, please come talk to us as we are hiring!

Look out for more new features coming down the line for Twitterfeed in the next few weeks!

Please do accept my apologies for this, it’s been an incredibly frustrating day for us, and it’s awful to think we let you down when you counted on us. Please do reach out to us @twfeed if you are still having problems tomorrow and we’ll make sure we fix them the cases that fell through the cracks!

Thanks again for being so supportive of us, it really made a difference today.

Luckily the backup we’re working from is pretty recent, but there could be issues with feeds created and items published in the last day or so. We’re working with bit.ly to see if we can come up with a clever way to identify these, so there’s a chance that this will affect relatively few publishers. Either way, I’d like to sincerely apologize for the problems and promise to keep you updated as we deal with this hardware issue.

When Twitterfeed was born back in May 2007, the sheer scale of traffic that it would have to deal with wasn’t even conceived of. Now we’re dealing with more than 700,000 feeds publishing millions of posts a day. Changing a legacy system to a new more scaleable system (as Twitter itself can attest) is a little like doing open-heart surgery with weighted boxing gloves on while the patient is still walking about. Everything becomes infinitely more difficult.

Nevertheless over the last few months we’ve been replacing key parts of the infrastructure and adding hardware capacity, as well as implementing standards like pubsubhubbub to increase the velocity of posting. We’ve also completely rewritten our feed processing back end and have been phasing that in as fast as the servers can handle it.

The last major part of the puzzle has been to migrate our massive database model over to a new more optimal model. We were hoping to do this without a major impact on the system, and we’ve been taking it slowly to try and minimize the effect, but even with those precautions, the system has at times been overloaded and some people’s feeds have suffered. Our best estimate is that the service will still be problematic for some over the next few days, but towards the end of the week should be running at full capacity again.

I can’t tell you how sorry we are that these growing pains have affected you and can only say that these changes are intended to build out a far more robust Twitterfeed that will be able to publish your feeds as swiftly as you post them, with some powerful new features coming down the line that we hope you’ll find invaluable.

• there was a Facebook bug which meant that some users couldn’t authorise posting to their Page(s) correctly, resulting in “200 Permissions” errors when we tried to post to these Pages. If you still see this error in your dashboard (click “view” to see the status line near the top), please edit the feed and re-authorise the page, it should then start working.
• we updated the format we post to Facebook to take advantage of Facebook’s features – so rather than just posting a text string with an appended link, we post title/link/description in a way that fits better with Facebook, see an example screenshot below. Note that the text “twitterfeed blog:” you can see in that screenshot is taken from the “Post Prefix” textbox under the “Advanced Settings” of your feed. So if you want to have some text appear at the top of every post, make sure you edit the feed and enter whatever info you want to appear as the “Post Prefix”.

Twitterfeed has grown pretty fast since its early side project days. There are now 350,000 publishers from the White House to CNN to John Cleese pumping out almost 600,000 feeds and generating around 5,500,000 clicks to your sites every day.

Faster, more reliable publishing

Dealing with such a flood of traffic on a service meant that Twitterfeed has had reliability issues; at times the system would simply get overwhelmed. Over the last few months we’ve been building a completely new architecture designed to handle the millions of posts we publish to Twitter and beyond each day.

We’re starting to roll out this new architecture today and this means that Twitterfeed should be able to publish feeds much faster than before. It will take a few days for us to move all the feeds over safely, but the result will be your posts reaching your audience when you want them to.

Pubsubhubbub real-time feed publishing

Twitterfeed now works with Pubsubhubbub. This means that sites enabled with Pubsubhubbub, like Typepad and Blogger sites, will be able to see their posts published in real time! We’ll shortly be announcing support for RSScloud too, so that wordpress sites can also be published in real time. You can find out more about pubsubhubbub here.

Publish to Facebook!

You can now automatically publish to Facebook from Twitterfeed. What’s more, you’ll see statistics for each post showing you just how many people clicked on posts from Facebook vs those from Twitter. You’ll be able to better understand just where your social media strategy is working.

Google Analytics Integration

Twitterfeed now adds special ‘UTM’ tags to each post that it publishes. Google Analytics can read these tags and integrate the information into its reporting. You’ll be able to understand how people clicking on your posts from Facebook differ from those clicking from Twitter and track the complete path of their visit to your site.

We hope you like the improvements and new features; you should log in and see how Twitterfeed can help your content spread across the social web!

Very best,

Mario, Tony and the Twitterfeed team

• if you follow @twfeed on twitter, the account should automatically follow you back within an hour or so
• once @twfeed is following you back, you can send it a direct message (not a public message starting with “@twfeed “, this won’t work, the bot only responds to direct messages)
• the content of this direct message should be “reminder” (without the quotes) – just that one word, nothing else, or it won’t be picked up by the automated service
• @twfeed will then search for any OpenID accounts that contain feeds posting to your twitter account, and send you back a direct message with the details

Note this only works for OpenID accounts, not for twitterfeed accounts that used an email address to sign up, but then we assume you can remember your email – the queries we got over the last few weeks were almost exclusively questions about forgotten OpenID accounts.

You can of course continue to send any other support questions to @twfeed, ideally as public messages addressed to @twfeed rather than direct messages, and they will be reviewed manually as before.

If you find any problems with this new automated functionality, please as usual report it at http://getsatisfaction.com/twitterfeed.

Greater stability: Bit.ly has taken great pains to have a stable service, building in multiple layers of redundancy so that bit.ly short urls can always resolve to their destination pages. Twitter recently also moved to bit.ly as its default and it made sense for Twitterfeed to also use the market-leader in this space.

Rich Analytics: Making bit.ly the default also means that Twitterfeed users can access bit.ly analytics and get far more information about what is happening to their posts once they are fed to Twitter. We’ll be working closely with the bit.ly team to develop richer analytics that give you the information you need to optimize how you interact with Twitter.

If for some reason, you would rather use another url-shortener you can still use the service of your choice, simply by changing your settings. However, we would strongly recommend that you use bit.ly to get the most out of Twitterfeed as we go forward.

Survey Winner

We’d also like to thank all of you who participated in our survey. We greatly appreciated your feedback and have chosen a random winner for the Amazon certificate: @cupcakeartist

Congrats! We’ll be getting in touch to deliver your prize.