*sigh* I'm having massive TiVo woes this week. I guess it's a good thing I'm not going to the Little League World Series this year, 'cause it leaves me time to fix this. (Mostly... Hoos's wedding is in Schaumburg this weekend. It just doesn't last two weeks.)
Over the last couple of weeks, recordings have been marred by video encoding errors. Playback will freezeframe for a bit, audio will garble, and then it'll resume. Sometimes this will happen several times in succession, but not badly enough to make it impossible to understand a conversation, let alone a plot. I assumed it was because the box was getting full again, so I really needed to clean the last three Sopranos, the last six Shields, and three games from last August's LLWS off the box to clear it out. A week and a half ago, I finally got around to installing the CacheCard. Things got worse, and fast. Half of last week's Real Time was wrecked. Live TV was hit-or-miss whether it was viewable. When it wasn't, response to the remote control (or telnet sessions) lagged by minutes. The system began to hang once a day. The output would be a freezeframe, the box wouldn't even blink its light to acknowledge a remote control keypress. Rebooting it often didn't help--Live TV was corrupted immediately on restart.
Oh, and if you're wondering how I know it was encoding errors, it's cause I thought of pulling the S-Video input cable. Once the buffer caught up with "right now", everything worked fine again. (You know, for that odd variant of "fine" that excludes the primary purpose of the box--recording television.) So that ruled out decoding or disk-reading problems.
So fuck it, I said. I swapped the TurboNet back in. Things improved a lot, but nowhere near enough. Video encoding was still awfully corrupted. /proc/meminfo was reporting 180-240 K of the system's 16M of RAM available--at the worst of the CacheCard-present problems, that number was well under a hundred K.
What next? It's gotta be a disk problem. Those drives are three years old, and they have been spinning continuously the whole time. Maybe 24 hours' downtime due to power failures or system upgrades or whatnot. Otherwise they spin, and are accessed, 24/7. And the case cooling isn't great--I've been running it with the lid off for over a year due to system freezes I assumed were due to heat. (It quit freezing when I stopped buttoning it up again.) Maxtor's PowerMax diag utility showed no physical errors on either drive, even on the long scan.
So it had to be a logical problem on the media storage partitions. *sigh* Great... that stuff is not officially, publically documented. It is dark, dark magic only moderately understood by the TiVo hacker community authors. People talked on the forums about solving similar problems by running 'fsfix'. No dice. I held my breath and ran fsfix with the arguments given in /etc/rc.d/rc.sysinit:
/tvbin/fsfix(I most definitely want fsfix killing the TiVo recording and display processes--no sense in trying to write to a filesystem we're trying to fix!)
-nokill-splash -uncollide -rehash -salvage -reboot
There is considerable improvement (MemFree shows 2200-2400K now,) but it's still not perfect. I left the living room TV on for the last few hours, tuned to ESPN. Little hiccups were very common. Every 30-40 minutes or so, I'd hear a brief audio chirp as things got really wonky. About midnight, things really seemed to be grinding. About then, I finished burning the last DVD of programming backlogged on the G4's drives. Firing up TyStudio to start pulling more off the TiVo, I saw that all the tyserver indexes were gone. One of the reboots in the network card tweaking had killed /var/index/. (And /var/devbin/, where I put common utilities that TiVo leaves out--like
ls.) Fortunately, kernel modules and rc.sysinit aren't affected, so the G4 is still NFS mounted, where I keep copies of all that jazz.
Anyways, tyindex automatically rebuilds its indexes if they disappear, and that process tends to take a while. Neither tyindex nor tyserver are very friendly processes. They're both known to be CPU hogs, affecting encoding, so I've made a point of not extracting programs during scheduled recordings. It's still indexing, so I'll have to see how things are in the morning. I'll go to bed with my fingers crossed...