Broadcatch

Broadcatching is the automation of queuing of files, usually BitTorrents files. For idea, complaints, or feature requests discuss it on the talk page or Email me.

Contents 1 Background 2 Installation 2.1 Requirements 2.2 Files 2.3 Crontab Installation 3 To do list 4 See also

Background

About a half year ago I was look for a way to broadcatch my anime. At the time Azureus was slow, memory hungry, and the web GUI just sucked. I being working on a bash script to automatically check and queue torrent files into mldonkey. This is the third version the of script (in continual beta), it can handles various formats of RSS files.

Installation

It probably is a good idea to place these files into its own directory. The files debug.log, rssdata, and cast.log will be created on run.

Requirements

• bash?
• wget
• sed
• xsltproc
• mldonkey (with the web interface enabled)

Files

broadcatch.sh

The script, still is development phase so there's still debugging function active. The mlnethost var will need configure to provide communicate to the mldonkey server.

#!/bin/sh
# Script based on http://linc.homeunix.org:8080/scripts/bashpodder
# Last revision 10/15/2006 
# Released Dec 31, 2006
# Release Notes: as you can see I've been testing it for quite a
#  while, so it should be fairly stable; config name has changed and 
#  debugging is much less verbose only reporting matches.  As always
#  you can look to see what exactly was changed between releases. 
#  And
#                      A Happy New Year to Everyone!

# User Vars
# Address to contact the web GUI of mldonkey with username and password
MLHOST=username:password@localhost:4080
CONFG="bc.conf"

# Debug Log (set to /dev/null to turn off)
DEBUG="debug.log"

# Make script crontab friendly:
cd $(dirname $0)
echo -e "\nExecuting $0 on $(date)" >> $DEBUG

# feed dump reset
rm -f rssdata

# Read the bp.conf file and wget any url not already in the catch.log file:
while read subscription
     do
     xmldata=$(wget $(echo "$subscription" | sed 's/[^@]*@\(.*\)/\1/') -q -O -)
     expression=$(echo "$subscription" | sed 's/\([^@]*\)@.*/\1/')
     
     # If $expression is blank or is the same as the source then use a wildcard
     if [ "$expression" = "$subscription" ] || [ -z "$expression" ];
         then
         expression="."
     fi
     
     # Parsing xml depending on where the torrent url is located inside <link> tags or as the value for the attribute enclosure
     if  fgrep -iq enclosure <<< "$xmldata"
         then
         file=$(echo "$xmldata" | xsltproc parse_enclosure.xsl - 2> /dev/null)
     else
         file=$(echo "$xmldata" | xsltproc parse_link.xsl - 2> /dev/null)
     fi
     
     # Protect against the white space gotchas
     file=$(tr ' \\\t\r' '_/__' <<< "$file")
     
     for url in $file
         do
         if  echo "$url" | egrep --color -i $expression &> /dev/null
             then
             torrent=$(sed 's/\([^#]*\)#.*/\1/' <<< "$url")
             if ! fgrep -i "$torrent" catch.log > /dev/null
                 then
                 # URL and Mininova fixer
                 torrent=$(echo "$torrent" | sed -e "s/:/%3A/g" | sed -e "s/mininova.org\/tor/mininova.org\/get/g")
                 
                 echo -e "Submitting URL $torrent to $MLHOST on $(date '+(%r %D)')\n$url" >> $DEBUG
                 
                 # Submit torrent URL to mldonkey for downloading and log if sucessful 
                 wget -q -O /dev/null "http://$MLHOST/submit?q=dllink+$torrent" && echo $url >> catch.log
             fi
         fi
         # rssdata is for test matching
         echo "$url" >> rssdata
     done
done < $CONFG

bp.conf

Sample configuration file, this one will downloads from using mininova.org and revision3.com (diggnation feed). The case-insensitive regular expression string is placed before the delimiter @, if empty (or removed) it will proform a wildcard match (.*) and queue all files in the feed. It should also be possible (FileTP needs to be enable) to download regular podcasts too.

daily.show.*(ds|TV)rip@http://www.mininova.org/rss.xml?cat=8&num=50
tw.*death.*note|eclipse.*kanon@http://www.animesuki.com/rss.php
http://revision3.com/diggnation/feed/large.xvid.avi.torrent.xml

General RSS feeds

• mininova
  • tvtorrent.info
• The Pirate Bay
• baka-updates

See the sites respective FAQ and policies for information and use on their RSS feeds. See also List of BitTorrent RSS feeds (Wikipedia)

parse_enclosure.xsl

The XSL parse for RSS feeds which use the <enclosure url="..."> tag.

<?xml version="1.0"?>
<stylesheet version="1.0"
	xmlns="http://www.w3.org/1999/XSL/Transform">
	<output method="text"/>
	<template match="/">
		<apply-templates select="/rss/channel/item"/>
	</template>
	<template match="item">
		<apply-templates select="enclosure"/>
		<text>#</text>
		<value-of select="title"/>
		<text>&#10;</text>
	</template>
	<template match="enclosure">
		<value-of select="@url"/>
	</template>
</stylesheet>

parse_link.xsl

The XSL parse for RSS feeds which use the <link>...</link>

<?xml version="1.0"?>
<stylesheet version="1.0"
	xmlns="http://www.w3.org/1999/XSL/Transform">
	<output method="text"/>
	<template match="/">
		<apply-templates select="/rss/channel/item"/>
	</template>
	<template match="item">
		<value-of select="link"/>
		<text>#</text>
		<value-of select="title"/>
		<text>&#10;</text>
	</template>
</stylesheet>

Crontab Installation

You will need to install into your crontab file for automation. Bellow is a sample corntab that runs at different time on weekday for morning viewing (bias).

# m hour           dom mon dow  command
00  4,7,10,14,18    *   *  1-5  /home/user/bc/broadcatch.sh
30  */3             *   *  0,6 /home/user/bc/broadcatch.sh

To do list

• Rewrite the configuration system
  • Commenting
  • A NOT regex option
  • Episode Counter (so it doesn't re-download from different groups)
• Options to test configuration
• Allow regex-less config
• Use # as a separator instead of "?" (Nicer log files)

See also

• TipsAndTricks
• Broadcatching (Wikipedia)
• Podcasting (Wikipedia)

---

Languages:  English