So you've hacked your Tivo to the point where you can telnet to it, display weather maps on demand, and run a web server. Now what?

One of my frustrations with this otherwise wonderful product is its approach to discovery. Sure, Tivo makes suggestions, which you can help along by hitting Thumbs Up or Thumbs Down. But in my experience, the recommendations tend to be either wrong --- just didn't interest me --- or obvious because I'd given the program three up-thumbs.

How can we do better? Instead of relying on Tivo's suggestions, I tend to learn about new programs from people talking about them, usually friends. Having gotten my Tivo onto the network, I started thinking about how to eliminate the verbal middleman and give friends the ability push suggestions right to the Tivo.

The end product is Program My Tivo! (http://www.grumet.net/rsstv/programMyTivo), a collection of client and server scripts, plus a Web site, tied together into a neat bundle with some help from RSS.

How it works

Long before Program My Tivo! there was a hack called TivoWeb (http://tivo.lightn.org) through which the owner could add to their To-Do list from a Web browser. Add a reverse-proxying Web server to the mix, perhaps running on an old, beat-up x86 box sitting in the garage, and you have remote programmability.

The drawback of the TivoWeb architecture is its tight coupling. The end user either has full control or no control. So if you want to invite friends to suggest new programs, much less strangers, you have to be willing to also give them the ability to see what you're watching, delete programs, add season passes and so on. Additional concerns are the general security risk of putting one's Tivo directly on the public Internet, and potential performance problems if multiple friends try to access the system at the same time.

The Program My Tivo! architecture uses RSS syndication to achieve a loose coupling that mitigates security risks and performance risks. Program searching and suggesting are moved from the Tivo to a Web application that outputs an RSS feed with embedded program information (http://grumet.net/rsstv/feeds/pmt?f=Scheduled). The Tivo runs an RSS reader that detects this information and adds programs to the To-Do list if it can. I say "if it can" because the RSS reader is programmed to not override programs already on the schedule (another security benefit).

There are two parts to getting the hack working. First you'll need to install the RSS reader and its helpers onto your Tivo. Second, you'll need a compatible RSS feed to point it at. We assume below that you've already got telnet access to your Tivo and the ability to transfer files to it. If you haven't, you may find Raffi Krikorian's Tivo Hacks (http://www.oreilly.com/catalog/tivohks/) helpful or public resources on the Web (do a Google search on "tivo hacks").

RSS reader installation

If you haven't already, install Tivo Control Station (TCS) (http://www.zirakzigil.net/tivo/TCS.html) and TivoWebPlus (http://tivo.fp2000.org/twp), the successor to TivoWeb. Note that TCS is a shareware app and has a fee, $15 USD at the time of this writing.

Install TivoWebPlus first, according to the directions in the TivoWebPlus README file (http://tivo.fp2000.org/twp/README). The app is distributed with a fancy binary installer that they say can configure your Tivo to start the Web server when the Tivo boots. As I'll describe below, the Program My Tivo! hack doesn't actually require the Web server to be running, so I simply extracted the the server tarball using the extract-tgz option and copied it over and untarred by hand.

When configuring TCS, set it up to run without the network server. The network server is a performance optimization that allows TCS to delegate Web fetching to another machine, saving precious hardware resources on the Tivo. Because the RSS reader schedules programs on the Tivo as it parses the incoming data, it must run directly on the Tivo, so we can't take advantage of the network server optimization. In addition, when configuring TCS, be sure that the tivoweb and tivowebdir point to your TivoWebPlus installation.

Once you've got TivoWebPlus and TCS installed, you're ready to copy over the RSS reader. The code for the RSS reader is here: http://grumet.net/rsstv/RssReader.tcl.txt. Installation instructions are embedded in the code, near the top of the file. To summarize, quit TCS (9-9-Clear on your remote), rename RssReader.tcl.txt to RssReader.tcl, edit the CONSTANTS in the file, copy to the modules/modules subdirectory of your TCS installation, create the hardlink in the modules subdirectory and start TCS.

Because hardware resources on the Tivo are precious, the RSS reader is only programmed to handle a single feed. The feed is designated by the RssFeedHost and RssFeedPath constants in the CONSTANTS section of the file. When first installing the RSS reader, set these to any standard RSS feed of interest, perhaps your weblog feed. After restarting TCS, you should be able to view a summary of the feed overlaid on your TV screen by typing 6-7-Clear on your remote.

A final installation note for the RSS reader module. To keep resource utilization light, you may wish to deactivate a number of the standard TCS modules. This can be accomplished by deleting a number of .tcl files from the first modules subdirectory under the TCS home. I say the first modules subdirectory because TCS has a modules subdirectory and a modules/modules subdirectory. The latter directory is the repository for all available modules, whereas the former is a set of hard links to just the modules that you wish to run. To make things a little more concrete, here are the exact commands I used to deactivate a number of modules:

bash-2.02# cd /var/hack/tcs/modules
bash-2.02# rm MLB.tcl NBA.tcl NCAAB.tcl NCAAF.tcl NFL.tcl NHL.tcl Quotes.tcl SportsCommands.tcl

Create a compatible RSS feed

Driving the Tivo To-Do list requires embedding extra information not usually present in RSS feeds. Namely, the television channel, start and end times for the recording. The RSS reader is programmed to read RSSTV extensions (http://grumet.net/rsstv) bearing this information.

There are endless possibilities for generating these feeds, the simplest of which is to create one manually. This is the fastest way to get a taste for RSS driven To-Do lists. Feel free to use this sample template (http://grumet.net/writing/programmer/syndication/pmt-sample-feed.xml) to get started. Note that the format for the tvchannel element is "C" followed by the channel number on your cablesystem followed by the call sign for the channel. When you're up and running with an RSSTV-enabled feed, RSS item summaries should be followed by a status note indicating whether the reader was able to schedule that item for recording.

We don't at present have a source code distribution for the Program My Tivo! codebase, but we'll walk through its major components in the section below.

The RSS reader code

The RSS reader is implemented as a TCS module. TCS provides the provides the framework to poll and process Web pages at regular intervals, and the ability to display useful data on the TV screen in response to commands from the Tivo remote control. The code rougly follows the sample template found at modules/samples/WebTemplate.tcl.

We'll now outline the key parts of the code with reference to this listing of RssReader with the lines numbered: http://grumet.net/writing/programmer/syndication/RssReader-numbered.tcl.txt. First note that at startup time, TCS will execute the code in RssReader.tcl when it is found in the modules subdirectory. The majority of the code in this file creates the RSS reader library, whose operation we'll describe shortly. The rubber hits the road at line 450, where we call InstallRssReaderModule. This procedure, whose definition starts at line 392, sets a bunch of constants and then registers the module with TCS by calling InstallRemoteCommand. This call, which can be found at line 433, tells TCS to call the $periodicupdatecommand (defined on line 429) at $periodicupdatefrequency (line 428), and to look in the file specified by $RssReaderFile (line 422) to display results on the screen when $remotecommand (line 427) is issued from the remote control.

With the module installed, TCS will regularly call $periodicupdatecommand, or GetFreshRss, whose definition starts at line 358 of the code listing. This function manages a few variable initializations, then processes the RSS feed located from $RssFeedHost and $RssFeedPath (defined on lines 409-410) then writes the results file for later display on the TV screen.

RSS parsing is handled by the ProcessRssLine callback function whose definition starts on line 217. The parser is rudimentary by necessity, to keep resource utilization reasonably low. Hence there is no DOM parsing, no validation, no special handling of namespaces or character sets.

If RSSTV tags are found and RssRecordingEnabledP is set to 1, the RSS reader will attempt to create a To-Do item via RssTryToSchedule, whose definition starts on line 79. After performing a few data integrity checks, an attempt is made to resolve the TV channel at line 107.

107    set sfsid_lookup_info [$RssStationFsidCallback $TvChannel]

In early versions of the module, this part of the code simply extracted the channel number, e.g. "61" from "C61FOOD". A weakness of this approach is that it would be hard to share feeds with people on other cable systems, for whom channel 61 may be a different network. Later we added a call sign based lookup, since "FOOD" is standard for the Food Network, at least across North American cable systems. The code in line 107 calls a configurable $RssStationFsidCallback, which is assigned on line 414 and defaults to the more portable call sign based lookup. These lookups are performed on a local copy of the channel data, periodically refreshed into the RssActiveChannelsInfo global by RssBuildActiveChannelsInfo (line 33).

When all of the schedule data is verified and resolved, a check is made for conflicting To-Do items (lines 113-122), and if any of these are Tivo suggestions, these are deleted (123-128). This part of the code relies on the TivoWeb library, specifically modules/sched.itcl. The TivoWeb libraries are sourced just after installing the module, at lines 451-482.

Finally, the recording is actually scheduled at line 130.

The feed-generating app

They key to building feed-generating applications is good program guide data. Listings might be entered by hand, or pulled automatically through a listing application such as XMLTV (http://membled.com/work/apps/xmltv). Be aware that availability and terms of use for this data will vary depending on where you live.

The Program My Tivo! app stores program guide data in a relational database. A sample SQL table definition is provided in this listing: http://grumet.net/writing/programmer/syndication/pmt.sql.txt. The table, pmtivo_suggestions, stores basic information about the listing, who suggested it, when it was watched and what the viewer thought.

When it comes time to generate the RSS feed, you may find it handy to use an XML library to ensure well-formedness. This code example illustrates one way to do it: http://grumet.net/writing/programmer/syndication/pmt-make-feed-numbered.tcl.txt. The rsstv_make_feed function, appearing at the top of the listing, is a general-purpose builder that creates RSSTV feeds from systematically named SQL views. XML is built using tDOM (http://www.tdom.org) running inside the mighty AOLserver (http://www.aolserver.com/). The db_foreach macro is part of the OpenACS database API (http://openacs.org), and loops through query results executing the code block once for each row. DOM building tends to look the same from platform to platform, so the listing ought to be helpful no matter what platform you're familiar with. The same code example without line numbers may be found here: http://grumet.net/writing/programmer/syndication/pmt-make-feed.tcl.txt.

Hacking the hack

First of all, if the Tivo being hacked is your primary PVR, be sure not to hack it while any show you care about is being recording. Hacking in general, and this hack in particular, tend to tax the machine, potentially resulting in reboots or degraded recordings.

Second, install a text editor and tail on your Tivo, and get familiar with tivosh (see the Krikorian book). When hacking the client, I tend to keep a dedicated telnet window open with a tail -f running on the tcs log, which lives at /var/log/tcs. You'll also probably want to be sitting in view of the TV with the Tivo remote at hand, so that you can view the RSS reader output using 6-7-Clear, and shut down TCS with 9-9-Clear.

As far as new features go, the sky's the limit. But here's an idea. A major drawback of Program My Tivo! is that it deals in fairly specific program guide information. While I was able to remove dependencies on specific channel numbers in favor of call signs, the listing times are still specific to my part of the world and my cable system. As a result, I may not be able to share my Boston-specific feed with a friend in Los Angeles. Even if we added some code to shift the times by three hours, there's no guarantee that a given program will air simultaneously across the country --- often it won't. One solution would be to identify workable GUIDs for programs and episodes, build prototypes and propose additions to RSSTV for these data. This would open up the door to applications that enable sharable To-Do lists and opportunities to build online communities to support them.