Ditch artificial intelligence for human intelligence by letting your friends push TV programs directly onto your Tivo.
So you've hacked your Tivo to the point where you can telnet to it, display weather maps on demand, and run a web server. Now what?
One of my frustrations with this otherwise wonderful product is its approach to discovery. Sure, Tivo makes suggestions, which you can help along by hitting Thumbs Up or Thumbs Down. But in my experience, the recommendations tend to be either wrong --- just didn't interest me --- or obvious because I'd given the program three up-thumbs.
How can we do better? Instead of relying on Tivo's suggestions, I tend to learn about new programs from people talking about them, usually friends. Having gotten my Tivo onto the network, I started thinking about how to eliminate the verbal middleman and give friends the ability push suggestions right to the Tivo.
The end product is Program My Tivo! (http://www.grumet.net/rsstv/programMyTivo), a collection of client and server scripts, plus a Web site, tied together into a neat bundle with some help from RSS.
Long before Program My Tivo! there was a hack called TivoWeb (http://tivo.lightn.org) through which the owner could add to their To-Do list from a Web browser. Add a reverse-proxying Web server to the mix, perhaps running on an old, beat-up x86 box sitting in the garage, and you have remote programmability.
The drawback of the TivoWeb architecture is its tight coupling. The end user either has full control or no control. So if you want to invite friends to suggest new programs, much less strangers, you have to be willing to also give them the ability to see what you're watching, delete programs, add season passes and so on. Additional concerns are the general security risk of putting one's Tivo directly on the public Internet, and potential performance problems if multiple friends try to access the system at the same time.
The Program My Tivo! architecture uses RSS syndication to achieve a loose coupling that mitigates security risks and performance risks. Program searching and suggesting are moved from the Tivo to a Web application that outputs an RSS feed with embedded program information (http://grumet.net/rsstv/feeds/pmt?f=Scheduled). The Tivo runs an RSS reader that detects this information and adds programs to the To-Do list if it can. I say "if it can" because the RSS reader is programmed to not override programs already on the schedule (another security benefit).
There are two parts to getting the hack working. First you'll need to install the RSS reader and its helpers onto your Tivo. Second, you'll need a compatible RSS feed to point it at. We assume below that you've already got telnet access to your Tivo and the ability to transfer files to it. If you haven't, you may find Raffi Krikorian's Tivo Hacks (http://www.oreilly.com/catalog/tivohks/) helpful or public resources on the Web (do a Google search on "tivo hacks").
If you haven't already, install Tivo Control Station (TCS) (http://www.zirakzigil.net/tivo/TCS.html) and TivoWebPlus (http://tivo.fp2000.org/twp), the successor to TivoWeb. Note that TCS is a shareware app and has a fee, $15 USD at the time of this writing.
Install TivoWebPlus first, according to the directions in the
TivoWebPlus README file (http://tivo.fp2000.org/twp/README).
The app is distributed with a fancy binary installer that they say can
configure your Tivo to start the Web server when the Tivo boots. As
I'll describe below, the Program My Tivo! hack doesn't actually
require the Web server to be running, so I simply extracted the the
server tarball using the
extract-tgz option and copied it
over and untarred by hand.
When configuring TCS, set it up to run without the network
server. The network server is a performance optimization that allows
TCS to delegate Web fetching to another machine, saving precious
hardware resources on the Tivo. Because the RSS reader schedules
programs on the Tivo as it parses the incoming data, it must run
directly on the Tivo, so we can't take advantage of the network server
optimization. In addition, when configuring TCS, be sure that the
tivowebdir point to your
Once you've got TivoWebPlus and TCS installed, you're ready to copy
over the RSS reader. The code for the RSS reader is here: http://grumet.net/rsstv/RssReader.tcl.txt.
Installation instructions are embedded in the code, near the top of
the file. To summarize, quit TCS (9-9-Clear on your remote), rename
RssReader.tcl.txt to RssReader.tcl, edit the CONSTANTS in the file,
copy to the
modules/modules subdirectory of your TCS
installation, create the hardlink in the
subdirectory and start TCS.
Because hardware resources on the Tivo are precious, the RSS reader is
only programmed to handle a single feed. The feed is designated by
RssFeedPath constants in
the CONSTANTS section of the file. When first installing the RSS
reader, set these to any standard RSS feed of interest, perhaps your
weblog feed. After restarting TCS, you should be able to view a
summary of the feed overlaid on your TV screen by typing 6-7-Clear on
A final installation note for the RSS reader module. To keep resource
utilization light, you may wish to deactivate a number of the standard
TCS modules. This can be accomplished by deleting a number of
.tcl files from the first
subdirectory under the TCS home. I say the first modules
subdirectory because TCS has a
modules subdirectory and a
modules/modules subdirectory. The latter directory is
the repository for all available modules, whereas the former is a set
of hard links to just the modules that you wish to run. To make
things a little more concrete, here are the exact commands I used to
deactivate a number of modules:
bash-2.02# cd /var/hack/tcs/modules bash-2.02# rm MLB.tcl NBA.tcl NCAAB.tcl NCAAF.tcl NFL.tcl NHL.tcl Quotes.tcl SportsCommands.tcl
Driving the Tivo To-Do list requires embedding extra information not usually present in RSS feeds. Namely, the television channel, start and end times for the recording. The RSS reader is programmed to read RSSTV extensions (http://grumet.net/rsstv) bearing this information.
There are endless possibilities for generating these
feeds, the simplest of which is to create one manually. This is the
fastest way to get a taste for RSS driven To-Do lists. Feel free to
use this sample template (http://grumet.net/writing/programmer/syndication/pmt-sample-feed.xml)
to get started. Note that the format for the
element is "C" followed by the channel number on your cablesystem
followed by the call sign for the channel. When you're up and running
with an RSSTV-enabled feed, RSS item summaries should be followed by a
status note indicating whether the reader was able to schedule that
item for recording.
We don't at present have a source code distribution for the Program My Tivo! codebase, but we'll walk through its major components in the section below.
The RSS reader is implemented as a TCS module. TCS provides the
provides the framework to poll and process Web pages at regular
intervals, and the ability to display useful data on the TV screen in
response to commands from the Tivo remote control. The code rougly
follows the sample template found at
We'll now outline the key parts of the code with reference to this listing of RssReader with the lines
numbered: http://grumet.net/writing/programmer/syndication/RssReader-numbered.tcl.txt. First note that at startup time, TCS will execute the
RssReader.tcl when it is found in the
modules subdirectory. The majority of the code in this
file creates the RSS reader library, whose operation we'll describe
shortly. The rubber hits the road at line 450, where we call
InstallRssReaderModule. This procedure, whose
definition starts at line 392, sets a bunch of constants and then
registers the module with TCS by calling
InstallRemoteCommand. This call, which can be found
at line 433, tells TCS to call the
$periodicupdatecommand (defined on line 429) at
$periodicupdatefrequency (line 428), and to look in the
file specified by
$RssReaderFile (line 422) to display
results on the screen when
$remotecommand (line 427)
is issued from the remote control.
With the module installed, TCS will regularly call
whose definition starts at line 358 of the code listing. This
function manages a few variable initializations, then processes the
RSS feed located from
$RssFeedPath (defined on lines 409-410) then writes the
results file for later display on the TV screen.
RSS parsing is handled by the
function whose definition starts on line 217. The parser is
rudimentary by necessity, to keep resource utilization reasonably low.
Hence there is no DOM parsing, no validation, no special handling of
namespaces or character sets.
If RSSTV tags are found and
RssRecordingEnabledP is set
to 1, the RSS reader will attempt to create a To-Do item via
RssTryToSchedule, whose definition starts on line 79.
After performing a few data integrity checks, an attempt is made to
resolve the TV channel at line 107.
In early versions of the module, this part of the code simply extracted the channel number, e.g. "61" from "C61FOOD". A weakness of this approach is that it would be hard to share feeds with people on other cable systems, for whom channel 61 may be a different network. Later we added a call sign based lookup, since "FOOD" is standard for the Food Network, at least across North American cable systems. The code in line 107 calls a configurable107 set sfsid_lookup_info [$RssStationFsidCallback $TvChannel]
$RssStationFsidCallback, which is assigned on line 414 and defaults to the more portable call sign based lookup. These lookups are performed on a local copy of the channel data, periodically refreshed into the
When all of the schedule data is verified and resolved, a check is
made for conflicting To-Do items (lines 113-122), and if any of these
are Tivo suggestions, these are deleted (123-128). This part of the
code relies on the TivoWeb library, specifically
modules/sched.itcl. The TivoWeb libraries are sourced
just after installing the module, at lines 451-482.
Finally, the recording is actually scheduled at line 130.
The Program My Tivo! app stores program guide data in a relational
database. A sample SQL table definition is provided in this listing:
pmtivo_suggestions, stores basic information
about the listing, who suggested it, when it was watched and what the
When it comes time to generate the RSS feed, you may find it handy to
use an XML library to ensure well-formedness. This code example
illustrates one way to do it: http://grumet.net/writing/programmer/syndication/pmt-make-feed-numbered.tcl.txt. The
function, appearing at the top of the listing, is a general-purpose
builder that creates RSSTV feeds from systematically named SQL views.
XML is built using tDOM (http://www.tdom.org) running inside
the mighty AOLserver (http://www.aolserver.com/). The
db_foreach macro is part of the OpenACS database API (http://openacs.org), and loops through
query results executing the code block once for each row. DOM
building tends to look the same from platform to platform, so the
listing ought to be helpful no matter what platform you're familiar
with. The same code example without line numbers may be found here: http://grumet.net/writing/programmer/syndication/pmt-make-feed.tcl.txt.
First of all, if the Tivo being hacked is your primary PVR, be sure not to hack it while any show you care about is being recording. Hacking in general, and this hack in particular, tend to tax the machine, potentially resulting in reboots or degraded recordings.
Second, install a text editor and
tail on your Tivo, and
get familiar with
tivosh (see the Krikorian book). When
hacking the client, I tend to keep a dedicated
window open with a
tail -f running on the tcs log, which
/var/log/tcs. You'll also probably want to be
sitting in view of the TV with the Tivo remote at hand, so that you
can view the RSS reader output using 6-7-Clear, and shut down TCS with
As far as new features go, the sky's the limit. But here's an idea. A major drawback of Program My Tivo! is that it deals in fairly specific program guide information. While I was able to remove dependencies on specific channel numbers in favor of call signs, the listing times are still specific to my part of the world and my cable system. As a result, I may not be able to share my Boston-specific feed with a friend in Los Angeles. Even if we added some code to shift the times by three hours, there's no guarantee that a given program will air simultaneously across the country --- often it won't. One solution would be to identify workable GUIDs for programs and episodes, build prototypes and propose additions to RSSTV for these data. This would open up the door to applications that enable sharable To-Do lists and opportunities to build online communities to support them.