bcbiff(1) – Boxcar based IMAP biff

There’s one big common frustration among Gmail users on iPhone that you cannot receive instant notification for coming mails.  On the other hand, you know that Boxcar is one of the must-haves for iPhone bringing you a fully customizable push notification method that is flexible and easy to use.

It may seem enabling push notification for Gmail is as easy as setting up your Gmail account to forward important mails to your Boxcar address, but it actually isn’t.  Suppose you want to be only notified of mails hitting Inbox because you receive tons of mails on mailing lists or mail magazines that are tagged and archived by filters. You’ll find it not easy to write a filter rule that matches mails that will be dropping in Inbox, because in Gmail all filters are evaluated in parallel and you cannot define a rule based on tags added by other filters.

Another concern is your privacy.  Even if you managed to make up a filter rule and successfully set up forwarding to Boxcar, every mail forwarded would be relayed through external servers and end up having a copy on Boxcar.  I’m not saying I don’t trust Boxcar or relay servers, but there’s no point for you (nor for them!) to carry a private mail content including sensitive headers outside of Gmail just for notifying that a mail has received.

Here’s why I wrote bcbiff(1).  Bcbiff is a tool that checks the Inbox folder (or any specified folders) on an IMAP server for unread mails and sends a notification via Boxcar for each.  Mail bodies and headers other than From, Date and Subject are stripped off, and the Message-Id’s of the latest 100 unread mails are cached so that you get just one notification per mail.  Bcbiff supports multiple mailboxes and Boxcar accounts.

To use it, you need a server on which you can periodically run bcbiff (Ruby 1.8.7+ or 1.9.2+ required) and send email via the sendmail(1) command.  See README for details.

Have fun with it!

Making FriendFeed a perfect Twitter client

As you all know, FriendFeed is equipped with some features to cooperate with Twitter. FriendFeed gives you with automatic CC to Twitter for writing and realtime feed updates for reading, and helps you find your Twitter friends on FriendFeed. However, they are just not enough because you cannot expect everyone to join FriendFeed. For Twitter users who don’t or wouldn’t have their accounts on FriendFeed, you have to set up an “imaginary friend” for each one of them by hand to make their tweets come up on your FriendFeed timeline. Even after all these hard work, you are left with some friends who make their tweets private, which FriendFeed cannot import. What about @replies? You don’t even notice on FriendFeed that someone is talking @you when your friends on Twitter are enjoying spontaneous talk with others, including strangers.

Introducing tw2ff

Here’s my answer to all these inconveniences. I wrote a tool called tw2ff that makes your FriendFeed life much easier in terms of integration with Twitter. It consists of the following features:

  • Friend list synchronization: importing Twitter-only friends to FriendFeed as “imaginary friends”
  • @replies import: generating a custom RSS feed of @replies for importing to FriendFeed
  • Protected tweets import: generating a custom RSS feed of protected tweets for importing to FriendFeed
  • Favorites synchronization: synchronizing FriendFeed likes and Twitter favorites
  • .. and some more

Installation

tw2ff is bundled in my “friendfeed” ruby gem, so you can install it right away by just one command as follows:

The first thing you do after the installation is run tw2ff config and set up credentials for your Twtter and FriendFeed accounts. It invokes a text editor with the following content, so fill in with your login information and save the file.

The file is located at ~/.tw2ff/config.yml, which directory is created with mode 700. Make sure it is never read by anyone else.

Friend list synchronization

You are now ready to run tw2ff friends to import your Twitter-only friends to FriendFeed as “imaginary friends”. It’s automatic, it compares your friend list on Twitter with subscriptions on FriendFeed and creates one imaginary friend for each Twitter friend missing in FriendFeed, where the screen name is enclosed with parentheses so you can easily see it is an imaginary friend. The tw2ff friends command does not change anything with existing imaginary friends, so you can safely rerun it at any time to keep your FriendFeed timeline in sync with that of Twitter. You will likely want to run it every time you follow a new person on Twitter, but it is also an idea to register this as a daily cron job and just leave the task to it.
One note about replying to imaginary friends. If you post a comment to an imaginary friend’s entry with the “Also send this comment as an @reply twitter from ” box checked, it will be visible in Twitter but not on FriendFeed because imaginary friends are private to you.
After a while, you may happen to visit Twitter or run a Twitter client and notice that some of your friends have changed their icons. That’s when you run tw2ff icons, a command that updates all imaginary friends icons with those of their Twitter accounts. Maybe a candidate for a monthly cron job or so.

Import of @replies and protected tweets

The only way to import private feeds to FriendFeed is put the feed in a secret place no one else would know and feed it to your own imaginary friend. (FriendFeed does not support any kind of authentication in feed crawling)
First off, create an imaginary friend named, say, “@replies”. This is where your secret @replies RSS is fed to. Then set up a cron job that invokes tw2ff replies replies.rss and upload the file replies.rss to a secret location, preferably only fetchable via HTTPS. Now you can add a service to the above said imaginary friend and set the secret location. There are a couple of options to choose what kind of replies should be included to the feed: --include-friends and --include-protected. The default is both off. Specify them as needed. In addition, by the way, I highly recommend adding your favstar feed to the imaginary friend so you can see more as to how others are reacting to you. Currently there is no support in tw2ff for your tweets RT’d, but I’ll implement it some way in near future.
Okay, you’re done with @replies and now, you can pull protected tweets into your FriendFeed timeline the same way by importing a feed produced by tw2ff protected. Please be doubly sure that you won’t leak the feed out to public.

Favorites synchronization

I much prefer FriendFeed likes to Twitter favorites because they are visible to others, naturally including the one whose post is favorited. That makes it easier to share topics not just with your friends but across the much wider friends-of-friends community. Recently Twitter introduced the RT feature to help people share their attentions, but I don’t think it would have been needed if favorites had been implemented and featured properly.
Let’s get back to the subject. I use “likes” a lot in FriendFeed and whenever I “like” a tweet I want to “favorite” it on Twitter at the same time, and vice versa. That’s exactly what tw2ff favorites does. It looks into the recently liked entries for tweets, and the ones not yet favorited are favorited on Twitter. Likewise, it searches the recent entries on your FriendFeed timeline for your favorited tweets, and the ones not yet liked are liked on FriendFeed. All you need to do it make it a cron job. Problem solved!

To-Do’s

  • Support for RT’s
  • Switching Twitter client access to OAuth
  • Utilizing Twitter’s streaming API and eliminate the time lag of @replies/protected tweet feeds
  • Switching FriendFeed client access to v2 API
  • (You add one)

Conclusion

With the help of tw2ff as described above, I can live just with FriendFeed without the need for always keeping another window for Twitter. FriendFeed quite shines when it stands as the one-stop portal for sharing what you and your friends are up to. Empower your FriendFeed life with tw2ff!