Nursing RSS Graffiti

The troublesome 341 Facebook errors.

In the past days we have been experiencing serious issues in posting stories to Facebook. Many attempts to post a story to Facebook result in the following Facebook error (and fail): 

• Error Number: 341
• Error Identifier: API_EC_EDIT_FEED_TOO_MANY_USER_ACTION_CALLS
• Error Description: Feed action request limit reached

This roughly seems to mean that we done too many “user action calls” and thus reached the limit of calls Facebook imposes.

This sounds like bad news since we did not increase the number of calls we make before the error started appearing, which means that this limit (if it exists) is a new limit that was imposed by Facebook (or an old limit they just started enforcing).

The good news (unfortunately they don’t seem to get better than this) is that we don’t really know what is going on. Which means we don’t really know yet if this is a bug or a feature.

We, in RSS Graffiti, still regard this as a Facebook bug. To see this as a feature we need some proof that this limit exists and is purposefuly enforced  by Facebook. We need some documentation on this. And so far we found no documentation for this supposed limit what so ever.

There is a related bug (#7624) filed in Facebook’s bug tracker for this issue. You can find it here: http://bugs.developers.facebook.com/show_bug.cgi?id=7624

It is obvious that this issue is not bothering just us in RSS Graffiti. Facebook bug #7624 has at the time I’m writing this 621 votes and a status of “NEW”. Which means that basically it was filed and confirmed by popular vote, but Facebook has not assigned it yet to anyone for fixing it. So the issue still remains to attract Facebook’s attention to this issue. It’s been already over 3-4 days that this issue is affecting us and I’m not very optimistic on how this will turn out.

Now what can we do about this?

Here is our list of things to do about this:

1. Attract Facebook’s attention so that they either fix the bug or provide us with some documentation on this limits rules and how to live with it.
2. Prepare for the worse.

To attract Facebook’s attention we must bring the votes for this bug to over 1200. If you have a look at http://developers.facebook.com/live_status.php you will see in the bottom right section that the top bug in their list has over 4.000 votes and the last of the top bugs has over 1.000 votes. If we don’t get this bug in that list I personally doubt that it is going to get any attention anytime soon. If you are a developer and feel affected by this issue then please move all the votes you can spare to this bug.

How do we prepare for the worse? 

For the past weeks we have been hard at work to prepare our next release. In there there are quite a few new and interesting things for everyone, some of which will help us out in minimizing the effects of this bug. At the moment we are not ready to release everything yet but we are investigating the possibilities of releasing some of the stuff we have prepared and making some adjustments to allow us to live with this issue until it is cleared. We plan to do this next release on Tuesday (March 30th, 2010). After that, we will consider how we deal with this depending on any relevant information we can get from Facebook.

How does this bug affect RSS Graffiti at the moment?

The obvious result to everyone is that nothing is posted on your wall after the limit is hit for your account. The limit for RSS Graffiti seems to be currently at 30 posts per user per day. (At least that’s what Facebook says in RSS Graffiti’s dashboard about the request limit, but it is still unclear if this limit is the same as the one we are hitting).

Apart from the obvious though there are even more issues caused in RSS Graffiti as a result of this bug. Since there are many users that have a lot of feeds in their pages a long queue of unpublished stories is being built up for them by the minute. As a result the background processes that work on those feeds hit take a much longer time to finish due to mechanisms we have for retrying and logging errors. This in turn degrades the performance of the whole system and the 5 minute cycle we used to enjoy cannot be maintained all the time. We often see cycles of 20 or 30 minutes when lots of feeds with a huge backlog are being processed.

To cope with this we have a number of new features that we were planning for our next release in place.

Our next release is going to improve logging and decrease the time spent while logging all this huge number of errors. There is a lot more we have on the works about logging but we can release what we already have in place to ease this pain as soon as possible. 

Another thing that is coming up with our next release is a new communication system between RSS Graffiti and it’s users, which will allow us to simply disable all feeds that have permanent errors and send a message to their admins. This will pause a large number of feeds that fail due to permissions errors which are still being processed now despite the fact that they won’t work unless some action is take by their administrator. Don’t be alarmed because we are not going to disable all feeds that have any kind of error (like for instance this 341 error); we just going to pause the feeds that definitely require some action from the user in order to start working again. This will significantly unload the burden of processing cycles and help bring them back to out usual standards.

There is yet another thing that will help in this situation and is included in our next release: limits! RSS Graffiti will now give you the ability to specify the maximum number of items to be posted in every cycle. This was planned as a feature anyway, but it will help you control the effects of this issue too until it gets fixed.

At the same time we are planning for ways to handle the new Facebook request limit if it turns out to be real. We already had some things in mind (like aggregated stories) but we are trying to figure out in which ways we can give you control of the usage you are making on your API limits. Twitter has a nice system in place for their API limits. Facebook unfortunately still lacks this fine grained control but we do believe that if this request limit is real then they will probably provide developers with ways of checking the remaining number of requests per user and control the distribution of the remaining usage at any given moment.

We will keep monitoring this issue and will keep trying to find ways to minimize it’s impact (or even make the best of it). We hope to get Facebook involved with this serious issue very soon and get this out of our way fast because there is all sorts of exciting things we have been working on and want to put forward in RSS Graffiti. We hope and work for the best.