Weblog

Well here we are once again. This time to conquer the pile of paper laying besides, on and under your desk. My girlfriend went nuts by the sheer load of paper laying around everywhere. I’m kind of messy but when I start to organize things I’m kind of a perfectionist. So here I went and put all the papers in one big pile.

After going through the pile I realized that there were a lot of papers worth trowing away but at the same time we’re worth keeping. A dilemma. So I had a good cold beer and started thinking about the situation. The beer worked like oil on the brains and I came up with a good solution. ‘Let’s scan this pile of toiletpaper!’ and so I provided my HP 7310 with some juice and put all the paperwork on the automatic paper input of the device. I inserted an empty SD-Card and started gaming while the HP started to do what it was made for.

After a few hours ( well actually the scanner was long done before that… plz don’t tell my girlfriend ) the scans were complete and I put the files on my debian server. So far so good. Now at least I had a backup of all the paper versions. After that I was rather satisfied and threw away a large pile of paper I no longer needed as I had a digital backup now.

But a real geek doesn’t stop here. What I wanted next was to be able to search through my digital paperwork fast and get the papers that I need. So I had a look and found that there is a great open source ocr package called tesseract. I downloaded some packages and started trying out some things. I found out that it was only capable of handling tiff images at this point and that it was best to avoid color in the images. To get the required images I installed ImageMagick to do the converting from jpg to grayscale uncompressed tiffs.

So far so good. This will result in a txt file containing the text in the file. Pretty neat. Now I can use grep to look for any string matches and then open the matching jpg. Easy as 1,2,3.

After this I created the following script that will parse any new images automatically with the ocr software.

#!/bin/bash
 
basedir="/share/downloads/Scans"
 
# Parses a scan if not already processed
# $1 = JPG file
parse_scan(){
        jpg_file=$1
        base_name=${jpg_file:0:(${#jpg_file}-4)}
        tif_file=$base_name.TIF
        txt_file=$base_name.txt
 
        # If tiff file does not exist, use imagemagick to convert
        if [ ! -e $txt_file ]; then
                echo "Converting: $jpg_file into $tif_file"
 
#               echo $base_name
#               echo $tif_file
#               echo $txt_file
 
                # Convert jpg into tiff file
                convert $jpg_file -format tiff -colorspace gray -depth 8 -compress none $tif_file &> /dev/null
 
                # Use tesseract for ocr on tiff file
                tesseract $tif_file $base_name -l nld &> /dev/null
 
                # Remove tiff file
                rm $tif_file &> /dev/null
        fi
}
 
for file in $basedir/*; do
        filename=$file
        length=${#filename}
 
        if echo $filename | grep -q '.jpg$'; then
                # Create TIF filename
                parse_scan $filename
        elif echo $filename | grep -q '.JPG$'; then
                # Create TIF filename
                parse_scan $filename
        fi
done

You can of course make the basedir an argument which can be passed into the shell script. But that is a personal choice. Now you can get rid of your pile of paper too. Let’s start recycling that pile of paper.

To be able to start and stop my subversion server nicely I created the following script to startup and shutdown under debian.

#! /bin/sh
# /etc/init.d/svnserve: start and stop svnserve
 
# Exit immediately if a command exits with a nonzero exit status.
set -e
 
# svnserve exists and is executable
test -x /usr/bin/svnserve || exit 0
 
# Directory Where the Repository is located, created with svnadmin create
REPOS_DIR="/opt/svn"
 
# The pid-file
PIDFILE="/var/run/svnserve.pid"
 
case "$1" in
  start)
        echo -n "Starting Subversion (SVN) Server"
        start-stop-daemon --start --quiet \
                --exec /usr/bin/svnserve -- -dr $REPOS_DIR
        # Also tried the following line but it didn't work ok
        #       --make-pidfile --pidfile $PIDFILE
        PID=`pidof svnserve` || true
        echo $PID > $PIDFILE
        echo "."
        ;;
  stop)
        echo -n "Stopping Subversion (SVN) Server"
        start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
        if [ -f $PIDFILE ]
        then
                rm $PIDFILE
        fi
        echo "."
        ;;
 
  *)
        echo "Usage: /etc/init.d/svnserve {start|stop}"
        exit 1
esac
 
exit 0

Place the script in /etc/init.d/svnserve and run the following command:

update-rc.d svnserve defaults

If you have apache installed as your main http server you might want all requests to be handled by apache. But what if you want to have a standalone tomcat install? Well this is still possible by using apache’s mod_proxy. You might need to install it if you are running debian. Do some ‘apt-cache search apache proxy’ abracadbra and install mod_proxy if it is not already installed. You might even need to enable it with ‘a2enmod proxy’.

Edit your configuration (I added it to the sites-available/default) and add something like this:

    # Tomcat Proxy
    RedirectMatch ^/tomcat$ /tomcat/
    ProxyRequests Off
    ProxyVia Off
    ProxyPass /tomcat/ http://localhost:8082/
    ProxyPassReverse /tomcat/ http://localhost:8082/

You also might want add some access rules to make sure noone will be able to abuse your proxy (especially when running a forward proxy).

<proxy *:80>
Order deny,allow
Deny from all
</proxy>
<proxy *:8080>
Order deny,allow
Deny from all
Allow from 127.0.0.1
</proxy>

This should do the trick (after reloading apache of course, /etc/init.d/apache2 reload). The redirectmatch will make sure that users that reach http://serveraddress/tomcat will be redirected to http://serveraddress/tomcat/ because otherwise they would not reach the page.
The proxypass points to /tomcat/ note the trailing slash. If you omit the trailing slash your images will not load correctly. port 8082 is tomcat’s proxy port which will allow proxied connect calls.

You can use the same trick for any other webserver you have running on another port. (webmin, azureus web html, etc)

note: I found out that you can also use the ssh -D option to have ssh function as a SOCKS server to get similar behaviour without the hassle of configuring apache.

Are you at work behind a big bad evil proxy? Afraid of your privacy? Set up a proxy on your local home server so you can browse safely.

What are we going to use:

• Putty
• Apache 2

Is that all we need? Yes that’s all.

Ok let’s set up our apache 2 proxy first. It is a good idea to add some security to your proxy server so not everyone can reach it. You might want to restrict it to the localhost only. This tutorial is based on debian install of apache 2. So hang on and let’s go.

First change the ports apache2 is listening in to. Edit /etc/apache2/ports.conf for this purpose and add the line:

Listen 8080

This will make the apache2 server listen to port 8080. If there is a line that makes apache2 listen on port 443 (https) you might want to disable it. We are going to use putty to connect to this port.

You might need to download mod_proxy for apache2 to be able to use proxying. If it is not already enabled use:

a2enmod proxy
a2enmod proxy_connect
a2enmod proxy_html
a2enmod proxy_ftp

This will enable it. If mod proxy is not yet installed at all use an apt-cache search mod proxy to locate and install it through apt.
the mod proxy_connect is required to be able to handle SSL calls through your proxy.

Create a new entry in /etc/apache/sites-available and name it ‘proxy’ for example. Insert something like the following:

<virtualhost *:8080>
        ServerAdmin webmaster@localhost
 
        ProxyRequests On
        ProxyVia On
 
        #Add ports you want to be able to connect to through your proxy here
        AllowCONNECT 443 563 1863 10000
        #443   = SSL
        #563   = TLS
        #1863  = MSN Messenger
        #10000 = Webmin
 
        DocumentRoot /var/www/
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
 
        ErrorLog /var/log/apache2/proxy-error.log
        TransferLog /var/log/apache2/proxy-transfer.log
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel notice
</virtualhost>

Enable this new site by typing:

a2ensite proxy

It would be really wise to limit the access to your forward proxy.
edit the proxy.conf file in /mods-available/proxy.conf.
Add something like this to allow only connections from localhost:

<proxy *:80>
    Order Deny,Allow
    Deny from all
</proxy>
<proxy *:8080>
    Order Deny,Allow
    Deny from all
    Allow from 127.0.0.1
</proxy>

reload the webserver after this by running:

/etc/init.d/apache2 reload

Add 443 to your ssh listen ports by opening /etc/ssh/sshd_config.
Edit it so it reads:

# What ports, IPs and protocols we listen for
Port 22
Port 443

Restart the ssh daemon by calling:

/etc/init.d/ssh restart

Now you are ready to go. You can try your proxy now by using putty to connect to your server and tunnel port 8080 to another port on your local machine. It might also be a good idea to enable zip compression on your connection (Putty:Connection->SSH->Enable Compression) to speed things up a bit.
Now you can use firefox or another app and connect on localhost:[bound putty port] to connect to your proxy.

If you want firefox to do the dns lookups on the remote end you should open your about:config page by typing this in the address bar. Lookup the value:

network.proxy.socks_remote_dns

Set the value to true to do remote dns lookups.

I was getting fed up with having to synchronize several folders containing photos. So I was looking for a good way to synchronize these images across the network. I first tried to put the images into Subversion but this turned out to be overkill. So that’s why I put my focus on Rsync.

Rsync has support for incremental file exchange. Not that I am going to use it but it still is cool. I will only use Rsync to keep my photo directories synched with the server. So lets install it.

apt-get install rsync

Edit the rsyncd.conf file in the /etc directory and create some entries here. Example is my photos entry here.

#/etc/rsyncd.conf
#file containing username+passwords in the form <name>:
<password> the file should be readable only BY user OR GROUP.
#so chmod 660 or 600 would be necessary.
secrets file = /etc/rsyncd.secrets
motd file = /etc/rsyncd.motd 
 
#Below are actually defaults, but to be on the safe side...
READ only = yes
list = yes
# with what user permissions should rsync handle directories?
uid = nobody
#gid = nobody
#On debian group nobody is mapped to nogroup
gid = nogroup
#if stricts mode is true secrets file should not be readable by all. To disable this turn stricts mode to off. (built in for cygwin users)
strict modes = true
 
[photos]
comment = Family Photos
path = /share/photos
auth users = wytze,wytske
READ only = no
hosts allow = 192.168.*
hosts deny = *
list = false
</password></name>

Start the rsync daemon:

rsync --daemon

Now you can start synchronizing. Synchronizing works like copying files with scp. So it’s fairly straightforward.

To retrieve a list of entries if listing is enabled:

rsync -avz wytze@debian::

To synchronize files from server:

rsync -avz wytze@debian::photos /opt/my-local-photos

To synchronize files to the server:

rsync -avz /opt/my-local-photos/ wytze@debian::photos

Please mind the trailing slashes. Try out the difference with and without the trailing slash. Pretty straightforward there.

For running Azureus headless on my linux machine I use screen so it runs in a separate screen.

The command to do this is the following:

screen -d -m -S azureus su -p azureus -c "/opt/azureus/azureus.sh"

This will create a detached screen with the name “azureus” which will run the command “su -p azureus -c “/opt/azureus/azureus.sh”

To send a command to this detached screen we can use the following command structure: (Which I use to tidily close the process)

screen -S azureus -p 0 -X eval 'stuff quit\015'

-p 0 tells screen to use window 0 on the specified screen. Using the eval function makes sure the \015 char will be evaluated back into an enter character. The stuff command is used to fill the input buffer of the screen program.

After this the command “quit” is sent to the azureus client which will cause it to quit and close the attached screen.

After installing moblock (a p2p guardian program) I found that my hamachi was no longer working.

To fix this I had to modify the moblock.conf file in the /etc/moblock directory.
I did so by specifying a network address with a netmask.

Hamachi’s ip-range starts with a 5. So the address would be 5.0.0.0
The netmask is 255.0.0.0 to allow all hamachi ip’s to connect. Because 255.0.0.0 means we have 8 bits on 1. The line becomes: 5.0.0.0/8

Add this line to the IP_TCP_IN line for example and restart moblock with moblock-control restart. After I did this I was able to reconnect with my linux server from hamachi. Hurray!

[update]Ok, maybe you also want to add LOGMEIN to your ignore block list. :S
Do so by changing the IP_REMOVE line into IP_REMOVE=”LOGMEIN”.
Do a moblock-control reload after this to reload the new configuration[/update]

…. :S Schaam me diep…

M’n eigen site (samen met Wytze) en ik heb er nog helemaal niks opgezet…

Dus.. bij deze dan;

Hallo Allemaal en Welkom op onze site!

P.S. Er wordt nog wel wat aan gewerkt