FeedBlendr.com

Due to the ongoing costs of operating FeedBlendr, I will be closing this site down at the end of this month (June). Your blends will no longer operate after that date, and all requests to this website will be directed to a single static page.

If you’d like a replacement, you might like one of these alternate providers:

• Hosted Services
  • Yahoo! Pipes
  • FeedRinse
  • xFruits
• Install on your own server
  • Big in Japan’s Frankenfeed
  • SimplePie
• Separate Server/Enterprise-oriented
  • RSSBus

Thanks to everyone who used FeedBlendr and who made it popular, and it’s a pity that I can’t keep it running.

I am currently looking for someone who is willing to acquire FeedBlendr and take over the operations of the service. It would be up to them whether they kept it publicly-accessible or took the technology and rolled it into part of something else.

Here’s some information if you’re interested in taking over (or just if you like numbers 🙂 )

• Runs from Amazon EC2, so if you’re willing to add additional instances to the pool, it can scale out.
• Currently almost 16,000 blends in the database.
• Those blends are compiled from over 65,000 source feeds.
• In February (short month), FeedBlendr pushed 150 Gig of data “in” and 200 Gig “out”.
• Approximately 50 new blends are created each day.
• According to Compete.com, FeedBlendr gets more than a couple thousand hits a day and growing.
• The site actually sees a couple hundred page impressions per day — the vast majority of requests are for feeds/blends from the system (that’s the point)
• The (MySQL) database is hosted externally (on DreamHost!) but you could move it somewhere else (and I’d recommend that you do)
• DNS is managed from DynDNS, but you could transfer it to anywhere you prefer.

Whoever acquires FeedBlendr will get the following:

1. The domain “feedblendr.com”,
2. All code (PHP) that powers the site including the app itself, WordPress templates, site UI,
3. The database behind the service (including this WordPress-powered blog)
4. Amazon AMI used to host the site (to be moved to your own S3 bucket and changed over to using your credentials),
5. My assistance transferring everything over to your servers/accounts/control,
6. Gratitude from the people using the service (and from me).

If you’re interested, please email beau@feedville.com directly.

In case you were wondering, FeedBlendr had some time off overnight. Starting yesterday there were some problems with the service which culminated in things being completely offline all night (PST).

Everything is back online now, and although under pretty heavy load, should be working “as normal”

And although it’s nothing to do with FeedBlendr, MyBabyOurBaby is the other project that I’ve been spending a lot of time on. Please check it out if you have kids and ever thought about creating a secure, online journal for them.

You can read more about it on Dented Reality if you like. Otherwise just go and set up a baby scrapbook or journal!

I probably should have mentioned it on here earlier, but last night I was the guest speaker at the new, monthly AWSome Meetup. It was a fun event, and a chance to tell people about my experiences with Amazon Web Services and some related tools and products. Around 20-25 people attended I think.
My talk seemed to be well-received, and people were very interested in how to get started, some of the teething problems I’d had, and the learning curve involved in using AWS. Hopefully some other people will be writing up some of the things I talked about, so I’ll link to them when I hear about them here (please ping this post if you’re writing about it!).
Thanks to Sebastian and Donnie for setting up the meetup and for inviting me to speak. I look forward to attending future events and meeting more people interested in and using AWS.

Slowly but surely, FeedBlendr is moving into one of the realms that it was originally intended to serve: being a true feed platform. Part of the goal of providing APIs and alternate ways of accessing blends is to help people use Blendr and the blended feeds that it provides in different contexts than just a normal feed reader. That’s exactly what Near-Time is doing – using FeedBlendr to provide feed services for their online publishing platform. In the words of Joel Bush, their Sales Director:

Near-Time is a hosted Web 2.0 collaboration and publishing platform that includes wikis, weblogs, RSS/Atom output, and a number of other tools. The platform is built around the concept of “spaces” – users can build an unlimited number of spaces and include an unlimited number of members. The sidebar of each space has a number of optional components – one of them is the ability to pull in an RSS feed from another Near-Time space, or anywhere on the web for that matter.

You can see the beginnings of this integration by checking out this growing Publishers 2.0 space, where Joel is combining the power of FeedBlendr to combine 6 different feeds together, then using Near-Time’s RSS module to pull in the contents of that feed and display them on the page.

It’s been a couple of months now that FeedBlendr has been hosted with Amazon EC2, and some people have shown interest in hearing more about that experience, so I thought I’d follow up with some observations about EC2 in general and my experiences/configuration in particular.

Here goes:

1. Dynamic DNS is a SLOW way of faking load balancing. It’s reasonably functional from a normal management perspective, but an emergency situation would not be pretty. Setting your TTLs down to 300 should mean that changes happen pretty quickly, but 48 hours after making changes and removing an instance from my DNS records, I still see requests coming direct to that instance. Amazon, please offer us internal load balancing between instances somehow!
2. Disposable Instances are something you need to get used to. I’ve already had one instance get itself into trouble because of a “degraded ephemeral data store” (according to an email I received from Amazon). If you’re in the habit of making instances completely disposable then you can just launch a new one and terminate the one with problems.
3. Instance Cycling is something I’m starting to believe in – periodically just starting new, clean instances, and moving your operations over to them, then shutting down the previous ones. This is also the case with a new code release for me. Rather than upgrading the code on my instances that are already running, I slide onto new instances via DNS changes.
4. Alternate AMIs are something I need to deal with, so that I can have a new AMI being bundled and tested, without throwing away my previous one. If there’s a problem with the new one, how do you roll back otherwise? This is something I need to figure out in my own deployment process.
5. Inconsistent Performance seems to be quite common on instances. I wasn’t expecting this one, but if you think about it, it probably makes sense. I can launch identical instances, and load them equally in DNS, but they will perform quite differently. I believe this is probably due to resource sharing/exhaustion on specific instances due to other users’ instances on the same physical machine within EC2. This is something Amazon probably needs to look into.
6. Slow External Connections are not something you want to be a required part of your core system. Currently my database is hosted outside the cloud (on DreamHost) and this seems to be a bit of a bottle-neck in the system. Also, I pay for the bandwidth generated by those communications which is a cost I could avoid by hosting the database within the cloud.
7. Pricing on EC2 is something interesting to consider. Right now, I run 2 instances, 24/7, which means I’m up for $144 per month plus bandwidth, so let’s just call it $200 pm for a self-managed service. Is that actually worth it compared to getting a dedicated machine somewhere else? More thoughts on this below.

EC2 vs. “Traditional Dedicated Host”

Potentially “FOR” EC2

• You can easily launch as many instances as you need, rather than just having one server to work with
• Setting up unique AMIs is relatively easy, so you can scale any part of your application
• Pay-as-you-drink means you’re not paying for infrastructure you’re not using
• No up-front/set-up costs
• Scalability through commodity computing
• More choice over operating system, because it’s entirely up to you

Potentially “AGAINST” EC2

• Because you are technically sharing hardware, you’re subject to resource-starvation/sharing with other users!
• No control over your network configuration (e.g. no availability of hardware load balancers, choice to put instances on the same switch etc, you get what you’re given and that’s it)
• No choices for custom hardware, so you can’t get better hard drives, more RAM etc
• You need to learn how to build an AMI and work with the Amazon tools for bundling and launching instances etc (not that it’s hard)
• You have to use Amazon S3 to store your AMIs in
• There is no bundled bandwidth with the service, so you only pay for what you use, but you pay for all of it
• No control panels, monitoring, reporting or statistics or anything like that provided (unless you install them yourself) other than very basic bandwidth numbers and “instance-hours” reporting
• No SLA at all, so you have no guarantees on uptime

Anyone else have any other ideas/additions to those lists?

I am in the process of moving over to a new set of internal code for FeedBlendr to help with future enhancements, so you might see some funkiness over the next few days (although hopefully not!). The new version introduces a few enhancements, including some slight changes to existing features:

• Internal APIs use Atom-based formats where possible now (OPML stuff is still there where it makes sense tho).
• Added a custom namespace (fv, for FeedVille) to the OPML output, and also some other outputs to provide additional information to users/developers.
• Added some new output format options (look for them on blend information pages soon).
• Added some customization options for output formats for restricting the number of entries displayed and limiting output to headlines only.
• LOTS of internal changes, but those shouldn’t affect you 🙂

Hopefully all servers will be switched over to the new code-base by the end of today, and some new documentation etc will be available to detail these changes.

UPDATE 2007-08-21: This code has been fully deployed on all servers as of 2007-08-17 and appears to be working successfully 🙂

I forgot to mention that I’m at excellent WordCamp 2007 in San Francisco today and tomorrow, so if you see me (wearing a WordCamp 2006 shirt), come and say hi! WordCamp is a collection of people who are passionate about WordPress getting together and discussing their experiences, looking at what’s coming up, and working together to improve WordPress.

Ouch – I’m an idiot. I only just realized (while testing a new code base for FeedBlendr/Feedville incidentally) that my feed URL was broken 🙁 I accidentally had things configured so that the feed generated out of WordPress (which is what powers this blog) was redirected to FeedBurner so that people would get my burned copy of the feed. Of course FeedBurner was configured to look at that same URL to get the content it was supposed to burn, so it was redirected to itself!

Ahhh endless redirection loops, how we love you. Naaahhhhtt.

It’s fixed now – sorry for depriving you of FeedBlendr News Love!