Clayton Shepard

Compiling vnStat on iPhone

2010-02-16T05:05:00.001-08:00

vnStat is a neat little bandwidth monitoring cli utility for Linux and FreeBSD.

Actually the hardest part about this is just getting the toolchain running, after that it is a couple modifications to the Makefiles.

So there is quite a bit of documentation out there on how to get the toolchain running. The easiest way I found was on the device itself, using these instructions.

After you do that then you need to just change the top of the Makefile to:


CC=arm-apple-darwin9-g++
Sysroot=/var/toolchain/sys30

LDFLAGS=        -multiply_defined suppress \
                -L$(Sysroot)/usr/lib \
                -Wall \
                -Werror \
                -march=armv6 \
                -mcpu=arm1176jzf-s \
                
CFLAGS= -I$(Sysroot)/usr/include

# bin and man dirs for Linux
BIN = $(DESTDIR)/usr/bin
SBIN = $(DESTDIR)/usr/sbin
MAN = $(DESTDIR)/usr/share/man

# bin and man dirs for *BSD
BIN_BSD = $(DESTDIR)/usr/local/bin
SBIN_BSD = $(DESTDIR)/usr/local/sbin
MAN_BSD = $(DESTDIR)/usr/local/man

default: vnstat

...

This worked for vnStat 1.10 on an iPhone 3GS 3.0.1.

You also need to sign the files using ldid:


ldid -S src/vnstat
ldid -S src/vnstatd

Or you can just put them in the src/Makefile so it does it automatically for you:


...
all: vnstat vnstatd vnstati

vnstat: $(OBJS)
 $(CC) $(LDFLAGS) $(OBJS) $(LDLIBS) -o vnstat
 ldid -S vnstat
vnstatd: $(DOBJS)
 $(CC) $(LDFLAGS) $(DOBJS) $(LDLIBS) -o vnstatd
 ldid -S vnstatd
...

The make install didn't work well for me so I just run it from my /var/root/vnstat directory. You also want to make your db directory and copy the rc file over:

mkdir ~/.vnstat
cp cfg/vnstat.config ~/.vnstatrc

Then modify the ~/.vnstatrc to set the DatabaseDirectory correctly:


# location of the database directory
DatabaseDir "/var/root/.vnstat"

Apple Late Model iPhone 3GS Clarifications

2010-02-14T01:47:00.000-08:00

A couple things I've learned over the past couple days:

iRecovery reports the iBoot version wrong on all the 3GSs I have tried. It always reports 636.66, when in fact the versions were 359.3.2 and 359.3.

The 4th and 5th digits of the serial number are the manufacture week. According to theiphonewiki iPhones made after week 40 are not vulnerable to the 24kpwn (untethered) exploit. This isn't quite true, as I have at least one week 41 that was vulnerable to 24kpwn.

I have yet to find a good way to put 3GSs in DFU mode, or more importantly, get them out of DFU mode, which is the only way I have seen to reliably tell which iBoot version they have. I have tried holding the power and home button for 30 seconds, even a minute, or just the power, or just the home button for that long. iRecovery won't even detect it.

To build iRecovery you need a couple packages, namely readline and libusb. On the Mac this apparently requires darwin ports then libusb, but I never actually got it to build on OSX. For ubuntu you just need to apt-get libusb-dev and libreadline5-dev (although I also installed libusb-1.0-0-dev before I saw libusb-dev, just in case you need both). Also, while building on Ubuntu, I had to add "include <signal.h>" to irecovery.c, or else it couldn't find SIGINT.

Also, from what I can tell, even if you have your SHSH saved, you can never downgrade your baseband, so if you accidentally upgrade to 3.1.3 you lose your ability to unlock.

Finally, switching back and forth from blackra1n and redsn0w has worked fine so far.

Update: Switching from redsn0w to blackra1n caused iTunes not to be able to sync, giving error message "iTunes cannot read the contents of the iPhone xxxx. Go to the Summary tab in iPhone preferences and click Restore to restore this iPhone to factory settings." The fix is to delete /private/var/mobile/Media/iTunes_Control/iTunes/iTunesDB and any files under /private/var/mobile/Media/iTunes_Control/Music.

A Couple Issues Jailbreaking iPhone OS 3.1.2

2009-11-09T20:34:00.000-08:00

I ran in to three issues jailbreaking iPhone 3GS OS 3.1.2 with PwnageTool 3.1.4:

1) Error 1064 when iTunes tried to deploy a custom image
2) No cellular signal after jailbreak
3) Cydia immediately crashing when you try to open it

--------------------------1--------------------------

The 1064 error was solved simply by pwning the phone again. The hardest part was getting the phone out of DFU mode. A couple of the published "hold home and power for 8 seconds, then hold home and power for 8 seconds, then again, but keep holding the home button for 20 seconds" methods didn't work. iRecovery worked perfectly first try. After downloading and extracting it run:


./iRecovery -s
#after the prompt appears enter:

setenv auto-boot true
saveenv
/exit

Note: In Windows XP this needs to be run in compatibility mode and as administrator...

Then just run the PwnageTool again and tell it the phone hasn't been pwned. This only happened to one of the phones I was messing with, so it could have been a user error.

--------------------------2--------------------------

This is just because I told PwnageTool to activate the phone even though I had a legite cell plan... Disabling the activation feature made everything work fine again.

--------------------------3--------------------------

I traced this issue back to using expert mode of PwnageTool and telling it to install all of the outdated pre-downloaded packages that come with PwnageTool. The fix is to either just not preinstall any packages, or to update the packages before they are installed.

The reason that simply hitting the refresh button under Cydia Settings->Download Packages doesn't work is because it is using a different repository than Cydia ends up using on the iPhone. So you have to go to "Manage sources" and remove the existing "http://apt.saurik.com/dists/tangelo/main/binary-iphoneos-arm/Packages" repository (click it and hit delete), then add "http://apt.saurik.com/dists/tangelo-3.7/main/binary-iphoneos-arm/Packages". Now go back to "Download packages", select the 3.7 repository, sort by Status, highlight all of the packages that have a new version available (shift+click), and click add to queue and wait for them to download. Now you can add all the packages you want via "Select packages" and create an image that won't cause Cydia to immediately crash.

-----------------------------------------------------

Another tidbit: iTunes asked me if I wanted to update my carrier settings. I was a bit skeptical after having just jailbroken the phone, but all it does is update some APN information and such, not the baseband or firmware or anything. So it is fine to say yes to AFAIK.

Also, I did notice the lowered WiFi signal after installing blackra1n, but a network settings reset seemed to fix it.

Drawable Image View in Android

2009-08-21T09:39:00.000-07:00

I recently needed an image view that scaled a bitmap to its size and could be drawn on using a touchscreen.

Its not very polished or extensible (the line size and color is hardcoded), but it works well for a prototype: Update: I went ahead and made the color and line width configurable:


import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Matrix;
import android.graphics.Paint;
import android.view.MotionEvent;
import android.view.View;

public class DrawableImageView extends View {
    private Bitmap mBitmap;
    private Bitmap pic;
    private Canvas mCanvas;
    private final Paint mPaint;
    private int a = 255;
    private int r = 255;
    private int g = 255;
    private int b = 255;
    private float width = 4;
    
    public DrawableImageView(Context c, Bitmap img) {
        super(c);
        pic = img;
        mPaint = new Paint();
        mPaint.setAntiAlias(true);
        mPaint.setARGB(a,r,g,b);

        
        Bitmap newBitmap = Bitmap.createBitmap(img.getWidth(), img.getHeight(), Bitmap.Config.RGB_565);
  Canvas newCanvas = new Canvas();
  newCanvas.setBitmap(newBitmap);
  if (img != null) {
   newCanvas.drawBitmap(img, 0, 0, null);
  }
  mBitmap = newBitmap;
  mCanvas = newCanvas;
        
        mCanvas.setBitmap(mBitmap);
    }

    public DrawableImageView(Context c, Bitmap img, int alpha, int red, int green, int blue) {
     this(c, img);
     setColor(alpha, red, green, blue);
    }    
    public DrawableImageView(Context c, Bitmap img, int alpha, int red, int green, int blue, float w) {
     this(c, img, alpha, red, green, blue);
     width = w;
    }
    
    public Bitmap getBitmap() {return mBitmap;}
    public void setWidth(float w) {width = w;}
    public void setColor(int alpha, int red, int green, int blue) {
     a = alpha;
     r = red;
     g = green;
     b = blue;
        mPaint.setARGB(a,r,g,b);
    }
    public void Undo() {
  mCanvas.drawBitmap(pic, 0, 0, null);
  invalidate();
 }

    float scaleX;
    float scaleY;
    float scale;
    @Override protected void onSizeChanged(int w, int h, int oldw, int oldh) {        
     scaleX = (float) w/mBitmap.getWidth();
     scaleY = (float) h/mBitmap.getHeight();
     scale = scaleX > scaleY ? scaleY : scaleX;
    }

    @Override protected void onDraw(Canvas canvas) {
        if (mBitmap != null) {
            Matrix matrix = new Matrix();
         matrix.postScale(scale, scale);
         canvas.drawBitmap(mBitmap, matrix, null);
            //canvas.drawBitmap(mBitmap, 0,0, null);
        }
    }

    float lastX;
    float lastY;
    @Override public boolean onTouchEvent(MotionEvent event) {
        mPaint.setStrokeWidth(width/scale);
        
        float curX =  event.getX()/scale;
        float curY =  event.getY()/scale;
  switch (event.getAction()){ 
   case MotionEvent.ACTION_DOWN:{
             mCanvas.drawCircle(curX, curY,width/2/scale, mPaint);
             break;
   }
   case MotionEvent.ACTION_MOVE:{
    mCanvas.drawLine(lastX, lastY, curX, curY, mPaint);
             mCanvas.drawCircle(curX, curY,width/2/scale, mPaint);  //fix for weird jaggies occur between line start and line stop
             break;
   } 
   case MotionEvent.ACTION_CANCEL:
   case MotionEvent.ACTION_UP:{
    mCanvas.drawLine(lastX, lastY, curX, curY, mPaint);
             mCanvas.drawCircle(curX, curY,width/2/scale, mPaint);
             break;
   }
   
  }
  lastX = curX;
  lastY = curY;
        invalidate();  //invalidate only modified rect...
     
     return true;
    }
}

TouchPaint was used extensively to figure out how to do this, along with the standard 2d drawing doc.

For me the most non-obvious part of the whole 2d drawing thing was how canvases work. I originally didn't quite understand how a given canvas was linked to what was actually drawn on the screen. In short you have your own canvas, in this case mCanvas, that operates on your own bitmap, mBitmap, then when the onDraw(Canvas) method is called, you copy your bitmap over to the canvas that method provides. The canvas provided by onDraw is backed by the bitmap rendered to the screen, which is how your modifications actually make it to the screen.

Simple Hacking on Cisco WCS for Seamless Anonymous Browsing

2009-07-15T20:22:00.000-07:00

My university has a Cisco network, which has all sorts of nice management features such as a wireless control system (WCS), vlans, CleanAccess, and I'm sure many more goodies. The wireless network broadcasts two SSIDs: one is for people associated with the university, requires credentials, and puts you in the appropriate vlan, the other is for visitors and provides completely anonymous access. The visitor network requires you to click a button to accept some University policy before you gain access, and this annoying redirect is reset every 6 hours or so. This is particularly cumbersome because if I open Firefox before accepting the policy then it redirects every saved tab to the policy page, and then to the university homepage.

I use the visitor network for various reasons, one is that because of some ridiculous network policies from the student vlan I can't get to some ports (svn, rdp, etc) on the DMZ vlan where I have a couple servers, despite the fact that 99% of the internet can access those ports. I think this is being fixed, but until then I need to use the visitor network. Also, I believe anonymous access to the internet is critical, but I am going to refrain from philosophical/political rants on a technical blog.

Just to procrastinate real work I decided to attempt to automate the policy accept process, and thus save my Firefox tabs.

I started by taking look at the policy page html and javascript, but it was a bit obtruse, so I went ahead and installed Firebug and took a look at its "Net" tab. Unfortunately since the policy page uses 20 gazillion different redirects, it kept changing pages before I could see what it was sending. (If anyone knows a workaround for this please let me know; I tried enabling redirect warnings in the Firefox preferences, but it didn't help.)

So I changed my hosts file to point wirelessauth.university.edu to 127.0.0.1, and fired up netcat. Turns out the policy page was accessing wirelessauth over ssl, so netcat was kind of useless. This also meant packet sniffers would be useless... So I just installed apache w/ ssl. This worked like a charm and I was able to use Firebug to see that it was POSTing some stuff to https://wirelesssauth.university.edu/login.html:

POST /login.html HTTP/1.1
Host: wirelessauth.university.edu
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/20090331
00 Ubuntu/9.04 (jaunty) Firefox/3.0.8
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.university.edu/it/network/visitor_wireless/?switch_url=https://wirelessauth.university.edu/login.html&ap_mac=00:0b:85:7f:26:40&wlan=Visitor&redirect=woot.com/
Cookie: __utma=64401068.180182089960766140.1247531434.1247531434.1247531434.1; __utmz=64401068.1247531434.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=64401068

buttonClicked=4&err_flag=0&err_msg=&info_flag=0&info_msg=&redirect_url=

Notably it is sending all sorts of crap: cookies, host, referrer, url encoded values, etc. I replicated everything except for the cookies and used wget to send a login request. This worked just fine:

wget --post-data="buttonClicked=4&err_flag=0&err_msg=&info_flag=0&info_msg=&redirect_url=" --user-agent="Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.8) Gecko/2009033100 Ubuntu/9.04 (jaunty) Firefox/3.0.8" --referer="http://www.university.edu/it/network/visitor_wireless/?switch_url=https://wirelessauth.university.edu/login.html&ap_mac=00:0b:85:7f:26:40&wlan=Visitor&redirect=woot.com/" http://wirelessauth.university.edu/login.html

Turns out all you really need is:

wget --post-data="buttonClicked=4" https://wirelessauth.university.edu/login.html

Which is easy enough to remember :)

I did leave out one little detail... that is how I was able to play with the policy/login page without having to wait 6 hours for it to timeout. This is obvious for network guys, just disassociate from the wifi network and change your MAC address:

sudo ifconfig wlan0 down
sudo ifconfig wlan0 hw ether ##:##:##:##:##:##
sudo ifconfig wlan0 up

While disconnecting and forgetting a network in NetworkManager is relatively easy:

gconftool-2 --recursive-unset /system/networking/connections/1

I haven't figured out an easy way to add a connection from the cli. It is pretty simple to associate to a wifi network with other command line tools, but I would like to go through gconf/NetworkManager. This code snippet just removes the first remembered connection, there is probably a relatively easy way to do it by name too.

Since I'm on an anonymous wireless network (and booting from a livecd for that matter) it would be nice to have a random MAC address too. Putting it all together:

#!/bin/bash

gconftool-2 --recursive-unset /system/networking/connections/1

sudo ifconfig wlan0 down

#randomize...  Almost all legite MACs start with 00
eth=00
for i in `seq 1 5`; do
 eth=$eth:`head -c1 /dev/random |hexdump -e '1 "%x"'`
done
sudo ifconfig wlan0 hw ether $eth
sudo ifconfig wlan0 up

#need code to join network... for now just click, then do the wget manually
#wget --post-data="buttonClicked=4" https://wirelessauth.university.edu/login.html

After concocting my random MAC address generator I realized somebody else probably already did it. Sure enough here is one of the first hits. I like my method better because it actually uses the appropriate random number generator, not /proc/interrupts. That example leaves off the first byte, which should be 0. Both because most legitimate MAC addresses start with 00, and the 7th and 8th bits signify multi/unicast and global/local uniqueness. Granted you could probably just do:

MAC=00:`head /dev/urandom|md5sum| sed -r 's/^(.{10}).*$/\1/; s/([0-9a-f]{2})/\1:/g; s/:$//;'`

or:

MAC=00:`head -c5 /dev/urandom|hexdump -e '"%x"'| sed -r 's/(..)/\1:/g; s/:$//;'`

I'm using urandom rather than random because random can block, especially without mouse or keyboard movements. I don't know how random md5sum is (probably more than enough for this application), but the second example doesn't use it. Apparently bash also has a $RANDOM variable, but it is in decimal, not hex.

QualNet (Scalable Networks Technologies) Website and Issues

2009-06-20T22:19:00.000-07:00

I had a number of issues with the QualNet website, which ended up resulting in this email dated June 3rd to license@scalable-networks.com and
evals@scalable-networks.com:

Dear Scalable Networks,

I recently signed up for a 30 day evaluation of QualNet. When I log in I get a welcome message that states the following:

"Thank you for evaluating QualNet. Your 30-day Evaluation License File has been generated and posted to your download page."

However, when I go to the download page and click the link to the license file (http://www.scalable-networks.com/distributions/download/license/evals/qualnet-4.5.1-eval-2009.07.03.lic) I am immediately redirected to the sitemap (http://www.scalable-networks.com/sitemap.php). Since I am on Linux/Firefox and was having other issues with your site (such as broken CSS suckerfish dropdown menus), I decided to try logging in on a Windows PC.

This attempt was even more broken/disturbing, [since] as soon as I logged in I received a non-html page that stated:

INSERT INTO ErrorLog (UID, address, SID, error, login, password) VALUES ('', '#.#.#.#', 'nodnhkck0rlkh0qc2442skijh5', 'Password is not correct', 'my@email.com', 'MyPassword'')

Subsequent attempts to log in on the Windows machine do not even take me to the login page, they simply show this SQL query. I initial[ly] thought my password was correct, however if you look closely there is an extra single quote at the end of it, which I must have hit while submitting the form with the enter key. I had to clear my cookies in order to be able to log in correctly. Unfortunately the license file link was still broken (as were the dropdown menus).

In summary:

1. Your CSS suckerfish dropdown menus have extra pixels [between] them and the popup which causes them to break in Linux and Windows Firefox.
2. You are likely not sanitizing your user input before inserting it in to your SQL database, which is a huge security hole.
3. You are storing your passwords in plain-text which is a very, very, bad practice. A simple SQL injection could potentially reveal every stored password.
4. You are sending passwords in plain-text over email which is not necessarily encrypted; this is also a very bad security practice.
5. And most importantly, I still cannot download my evaluation license file. Please let me know what I need to do to get this.

Thank You,

Clayton Shepard

P.S. http://xkcd.com/327/

(I apologize for the typos, noted in brackets. Clearly I removed my plaintext password, IP, and email.)

I guess I just find it ironic, and somewhat amusing, that a technology oriented company has such glaring flaws with their website and security. In their defense they very promptly replied with the license file. Also, the SQL table was just the "ErrorLog", which means they could still be using a hash for the actual passwords somewhere else; arguably this is still a security threat though, since mistyped passwords are likely very close to the original. For obvious reasons I did not try a SQL injection attack, so I can not verify whether or not that is actually a security hole (although my guess is that it is).

On entirely different note, I was unable to get QualNet 4.5 running on Ubuntu 9.04 AMD64 because of library issues. In short they require gcc 4.0 and glibc2.3 or earlier to compile/install. Getting glibc2.3 to run on 9.04 is a huge pain. When asked, their official tech support reply is to use Ubuntu 6.06 :(. Unfortunate.

This reminds of the MSDNAA (MSDN Academic Alliance) website which also sends plaintext passwords in emails. Sigh.

Showing scp's Progress Using zenity (pseudo terminal example)

2009-06-17T23:21:00.000-07:00

I recently discovered zenity for making quick, straightforward, graphical interfaces for shell scripts. One such script I wrote uses scp for backups. Since scp prints its percentage progress to stdout I figured it would be pretty easy to simply use a regex to parse the output and pipe it to zenity. I guessed this venture would take about ten minutes (I am not very experienced with regex), since zenity --progress just expects a number printed to its stdin in order to update the progress bar. I couldn't have been more mistaken; after four hours of hacking and wtfs I gave up. Finally, after a discussion with my old operating systems professor and a little bit of coding I finally have a solution.

After piping scp's output to various c and perl programs I determined that the problem was probably a terminal mode problem. A standard (canonical) terminal sends input to the program as lines, thus if there are never any new lines (such as in scp's output) the program would never receive the data. After figuring out how to put c (and perl) in to raw mode, as well as playing with stty, I managed to get a couple programs working that could read input before a newline was sent. I tested these with stdin and pipes and they seemed to work fine, but they still wouldn't work with scp. After a bit of debugging I figured out that when the data was piped to my c program I couldn't put the terminal in to raw mode. Go figure, when you pipe data to a program it doesn't use a terminal... it uses a pipe (duh!). This is the point where I gave up, as I didn't really feel like digging through the scp source code, and I certainly didn't want to run a modified copy of scp (that would kind of defeat the whole purpose).

After discussing the problem with an old professor he hypothesized that scp was checking if it was being run on a terminal, and if it wasn't then it was disabling output. Sure enough, on line 396 of scp.c:


 if (!isatty(STDOUT_FILENO))
  showprogress = 0;

As a possible solution he mentioned pseudo terminals. Apparently back in the old days psuedo terminals were quite a pain since you had to find a free one manually before you could use it. Luckily linux provides a convenient function, forkpty(), that finds a free pseudo terminal, forks a new process, and attaches the new process to the terminal. Sweet. Now with some simple fileio it works fine:


#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>

main(int argc, char **argv) {
 int fd;
 pid_t pid;
 char c,c1,c2;

 if (argc != 3) {
  printf("usage: [[user@]host1:]file1 [[user@]host2:]file2\n\nThis is a program that wraps scp and prints out the numeric progress on separate lines.\n");
  fflush(stdout);
  _exit(1);
 }

 pid = forkpty (&fd, NULL, NULL, NULL);
 if (pid == 0) {
  execlp ("scp", "scp", argv[1], argv[2], (char *) NULL);
  _exit (1);
 } else if (pid == -1) {
  return 1;
 } else {
  FILE *F;

  F = fdopen (fd, "r");

  //reading character by character isn't horribly efficient...
  while((c = fgetc(F)) != -1) {
   if (c == '%') {
    if (c1 == ' ')
     printf("%c\n", c2);  //one digit progess
    else if (c1 == '0' && c2 == '0')
     printf("100\n"); //done
    else
     printf("%c%c\n", c1,c2); //two digit progress
   }
   fflush(stdout);
   c1 = c2;
   c2 = c;                   
  }

 fflush (F);
 wait (0);
 }
 return 0;
}

Notably, I went ahead and parsed the scp output in c. It seemed to be a cleaner solution than piping this output to a perl script and then to zenity. I initially missed the fflush() in the while loop, which took me a while to figure out :/.

To compile it you need to use the -lutil switch. So first copy and paste this c program in to scpwrap.c. Then run "gcc -lutil scpwrap.c -oscpwrap". Before you can use it you have to get rid of scp's authentication prompt by setting up some authentication keys:


ssh-keygen -t rsa
cat .ssh/id_rsa.pub | ssh user@example.com 'cat >> .ssh/authorized_keys'

Now to use the wrapper you can do something like:


./scpwrap /home/ubuntu/somefile user@example.com:~ | zenity --progress

Of course you can add whatever zenity switches you want. You cannot, however, add any scp switches without modifying the c program... Sorry.

---------------------Notes and Sources---------------------------

Terminal raw mode "stty raw".

Perl expression to parse the scp output:
perl -015 -l -ne 'print /(\d+)%/'
Notice the "-015" that tells it to use carriage returns as the line delimiter. Pretty cool, it probably would have worked if scp wasn't checking if it was running on a terminal.

Perl code to put terminal in to raw mode:


#!/usr/bin/perl

use Term::ReadKey;

ReadMode 5; # Turn off controls keys
while (($key=getc) ) {
  print "$key\n";
  last if $key eq "q";
}; 
print "Get key $key\n";
ReadMode 0; # Reset tty mode before exiting

C code to put terminal in to raw mode.

Zenity Progress Example 1:


(for a in `seq 1 100` ;


do
echo $a;
sleep 0.03;


done) | zenity --auto-close --progress \


--text="Slow counting from 1 to 100" \
--title="Example Progress"

Zenity Progress Example 2:


#!/bin/sh
(
echo "10" ; sleep 1
echo "# Updating mail logs" ; sleep 1
echo "20" ; sleep 1
echo "# Resetting cron jobs" ; sleep 1
echo "50" ; sleep 1
echo "This line will just be ignored" ; sleep 1
echo "75" ; sleep 1
echo "# Rebooting system" ; sleep 1
echo "100" ; sleep 1
) |
zenity --progress \
--title="Update System Logs" \
--text="Scanning mail logs..." \
--percentage=0

if [ "$?" = -1 ] ; then
zenity --error \
--text="Update canceled."
fi

Update: I modified the scp source (openssh5.2p1) to include a '-n' switch to print out the progress meter on new lines, regardless of whether stdout is a tty. I proposed this new feature to the openssh-unix-dev listserv, but I doubt they will adopt it. The email thread and patch can be found in their archives.

Update 2: Hrm, the pseudo terminal hack doesn't work if you run it from a launcher (kind of the whole point).

Printing from DOS in VirtualBox

2009-04-29T16:17:00.000-07:00

VirtualBox doesn't support LPT ports. DOS has limited ability to print to USB printers, and the program I am using definitely doesn't support it. Thus I need to find an automatic, or at least semi-automatic, user friendly way to print from DOS.

----------------------------------- The Mount Method -------------------------------------

My first approach was to print to a file in DOS, then simply mount the .vdi and read/print the file.

This has three drawbacks:

1. You need sudo to mount the .vdi.
2. You can't use it with snapshots or non fixed sized disks.
3. I don't like mounting a disk that is already in use.

The first problem can be solved by adding an entry in the fstab or using FUSE. The second and third problems can be solved by printing to a floppy image and then mounting that instead.

So the basic process is go to Devices and mount a floppy image, print to the floppy using filename "print", unmount the floppy, then run this script in Linux:


#!/bin/bash
#we need recode (sudo apt-get install recode)
#add the floppy image to fstab to avoid sudo.
#mount writable and erase the print file so DOS doesn't prompt to overwrite?

mkdir -p ~/dosprint/printtemp
sudo mount -o loop,ro ~/dosprint/flopppy.img ~/dosprint/printtemp/
cp ~/printtemp/print ~/dosprint/printing
recode IBM437/CR-LF ~/dosprint/printing
lpr ~/dosprint/printing
umount ~/dosprint/printtemp
rmdir ~/dosprint/printtemp
rm ~/dosprint/printing

Notice that I am calling recode to convert the old DOS formatting. Obviously you will have to change floppy.img to match the floppy image you printed to.

This isn't a terrible option, but it does make the user have to remember the exact filename to print to, or else the script won't work. The whole mount/unmount process is a bit tedious too.

----------------------------------- The Serial Port Method -------------------------------------

To fix this I started looking at VirtualBox's COM emulation. It can connect the VM's COM port to a socket on the filesystem (it calls it a pipe, but it is really a socket... I probably would have preferred a real pipe). Anyway, after fiddling with it a bit I came up with a, hopefully, better solution.

After enabling COM1 on the VM to output to a "pipe" start up the VM. Then run socat -u UNIX-CONNECT:$HOME/dosprint/printpipe GOPEN:$HOME/dosprint/printfile in a terminal. You may have to do a 'sudo apt-get install socat' if it isn't already installed. The DOS VM now can print to COM1. After printing in DOS simply run this script:


#!/bin/bash
printfile="$HOME/dosprint/printfile"

#check to make sure the file exists and is not empty.
if [ ! -f $printfile ] || [ "`stat -c %s $printfile`" -le "1" ]; then
 zenity --error --text "The Print File is Empty or Does Not Exist\!"
 echo "" > $printfile
 exit 1
fi

lpr $printfile

#check to make sure it printed
if [ "$?" -ne "0" ]; then
 zenity --error --text "Error Printing!"
 echo "" > $printfile
 exit 1
fi

echo "" > $printfile
zenity --info --text "Printing\!"

There are probably some better ways to use socat here, but this worked the best so far. I would prefer to use a pipe, but it didn't work as expected. Also notice I didn't have to use recode, which was nice; this however is dependent on the printer driver DOS is using. In one incarnation I had to use dos2unix; unix2dos; then recode to get it to print properly. (In Ubuntu dos2unix is in the package 'tofrodos'.)

The biggest drawback here is that socat has to be run every time the VM is started, and you can't even run it as a startup script because it returns immediately unless the VM is running. At least it gets rid of the mounting and naming issues.

Here is a simple script to start the VM and run socat automatically:


#!/bin/bash
VirtualBox --startvm DOS&
echo "" > ~/dosprint/printfile
sleep 5
socat -u UNIX-CONNECT:$HOME/dosprint/printpipe GOPEN:$HOME/dosprint/printfile

You can create a launcher (right click desktop or nautilus window) if you want to be able to open this script without being prompted to run in terminal or display.

P.S. Zenity is very nice :).

Thanks to Maciej Bliziński for some nice info on VirtualBox's serial port pipes.

Update: I ended up using iconv rather than recode, since recode had some problems with some special characters. Careful with iconv, it changed between the version on Ubuntu 8.04 and 8.10 (mainly the overwrite in place functionality).

Mount Xen and VirtualBox Images in Linux

2009-04-27T13:12:00.000-07:00

If you use Xen disk images then you can mount them fairly easily to directly modify them on dom0 or another system:

Use "fdisk -luC 530 WinXP.disk" to print the partition table of the disk (where WinXP.disk is your disk image...).
Then multiply sector size by the sector start of the partition you want, then simply mount using that offset:
"sudo mount -o loop,offset=32256 WinXP.disk /media/mountpoint"

For example:

fdisk -luC 530 WinXP.disk 

Disk WinXP.disk: 0 MB, 0 bytes
255 heads, 63 sectors/track, 530 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x6c896c89

     Device Boot      Start         End      Blocks   Id  System
WinXP.disk1   *          63    41913584    20956761    7  HPFS/NTFS
Partition 1 has different physical/logical endings:
     phys=(1023, 254, 63) logical=(2608, 254, 63)

Then 63 sectors * 512 bytes per sector =32256, so:
sudo mount -o loop,offset=32256 WinXP.disk /media/mountpoint

This info was found here.

In VirtualBox the operation is similar, but since the .vdi format is different you have to jump through another hoop. Note this only works with fixed size images. First you have to find the start of the disk, then use that to find the start of the partition:

od -j344 -N4 -td4 image.vdi | awk 'NR==1{print $2;}'

Now use the number it returns as #offset# in:

dd if=image.vdi of=vdstart bs=1 skip=#offset# count=1b
/sbin/sfdisk -luS vdstart

This takes the partition table of the virtual disk, dumps it to a file, then reads the partition table to tell you where the partition starts. Now the process is similar to Xen, multiply the sector size by the sector offset of the partition to find the start of the partition, but you also have to add the vdi offset you found before. You can then use that offset to mount the partition:
mount -o loop,offset=39424,ro image.vdi /media/image/

For Example:


$ od -j344 -N4 -td4 DOS.vdi | awk 'NR==1{print $2;}'
7168
$ dd if=DOS.vdi of=vdstart bs=1 skip=7168 count=1b
512+0 records in
512+0 records out
512 bytes (512 B) copied, 0.00523881 s, 97.7 kB/s
$sfdisk -luS vdstart 
Disk vdstart: cannot get geometry

Disk vdstart: 0 cylinders, 255 heads, 63 sectors/track
Warning: The partition table looks like it was made
  for C/H/S=*/64/63 (instead of 0/255/63).
For this listing I'll assume that geometry.
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
 vdstart1   *        63   3253823    3253761   6  FAT16
 vdstart2             0         -          0   0  Empty
 vdstart3             0         -          0   0  Empty
 vdstart4             0         -          0   0  Empty
$sudo mount -o loop,offset=39424,ro DOS.vdi /media/dos/

The mount offset was calculated by: 7168 (vdi header) + 63 (sectors) * 512 (bytes per sector) = 39424

Works like a charm :) Thanks Przemoc. More info can be found here.

I am still looking for a way to do it with Snapshots though :/. On a different note, this looks like it could be useful.

WPA Enterprise on Android HTC G1 Dream

2009-04-12T15:02:00.000-07:00

Unfortunately Android still does not have a GUI interface for connecting to corporate/enterprise networks; however, it does support it. While this isn't a solution for most users, it is fairly easy for those who have already rooted their phone or aren't scared of the command line.

First, you need to root your phone or else you can't get to the files you need to configure. Newer firmwares aren't as easy to root as older ones; either way this tutorial should walk you through what you need to do. It is very user friendly, and if you have any problems then it is probably addressed later in the forum comments. I had two problems when I did it:

1) I skipped the reformatting of the SD card since mine was already FAT. For some reason the phone needs FAT32 in order to recognize boot files... So don't skip this step.

2) The first link, and corresponding MD5 sum, to the modified OS images are to the BuildEnvironments rather than the actual OS image. Just go to the second link and download the actual image (it will be a zip file and 40.5MB rather than a .tar.gz that is 7.5MB). Hopefully Koush will update the post and fix this.

Anyway, after your device is rooted pull up a command prompt on the phone somehow. If you already know how to do this then skip ahead. You have a couple options:

1) All on the phone: I am pretty sure the terminal emulator comes pre-installed on that image. If not then you can just download it from the Market. You can just use this, but I wouldn't recommend it unless you really like typing on your phone.

2) Through telnetd: Use the terminal emulator then type "su" then enter. This will pull up the super user whitelist prompt. Tell it "Yes" that is okay for the Terminal to run as super user. Meanwhile the terminal may timeout, in which case you need to type it again. You can that you are root by running "whoami" which will tell you that you are uid 0 if you are root. Make sure the phone is connected to wifi. You can get the IP from the Wireless gui or running "ip a". Now run "telnetd". From another computer on the network run "telnet #android_IP#" where #android_IP# is the IP of the phone.

3) Using the debugger: This involves doing this then looking at this. If your device is all set up and plugged in to a computer this entails basically running "adb shell".

Now all you have to do is edit the /data/misc/wifi/wpa_supplicant.conf file. You can do this by copying it to your sdcard, editing it on a computer, then copying it back. I prefer just to edit in place. Update: Sorry I forgot to mention you need to run "su" first, or else you will get permission denied errors.

Regardless, you probably first want to backup the existing wpa_supplicant.conf file:


cd /data/misc/wifi/
cp wpa_supplicant.conf wpa_supplicant.conf.orig

To copy to sdcard (make sure you have one in there):


cd /data/mis/wifi/
cp wpa_supplicant.conf /sdcard

To edit in place:


cd /data/mis/wifi/
vi wpa_supplicant.conf

The wpa_supplicant file will contain all of your remembered wifi networks. Something like:


ctrl_interface=tiwlan0
update_config=1

network={
    ssid="linksys"
    key_mgmt=NONE
   priority=11
}

Now add the parameters for your network. Many schools/business will provide this config for you, so you can just copy it over. Mine looks like:


ctrl_interface=tiwlan0
update_config=1

eapol_version=1
fast_reauth=1
network={
    ssid="Some SSID"
    scan_ssid=1
    key_mgmt=WPA-EAP
    eap=TTLS
    identity="USERID"
    password="PASSWORD"
    phase2="auth=PAP"
    priority=1
}

network={
    ssid="linksys"
    key_mgmt=NONE
   priority=11
}

This will now make "Some SSID" show up as remembered on the WiFi config GUI screen, and when in range, if everything is configured right, it will connect automatically. Note that in this case my password is stored in plain text on a file in my phone. This is bad, but I'm not sure there is a way around it at this point.

More information on how to configure this can be found here.

Notably Android automatically deleted the eapol_version=1 and fast_reauth=1 lines, however it still works fine.

There is a relate Google code issue here.

Update: I am having problems with the phone rapidly associating and disassociating from every AP in range. I filed a bug report here.

Update: This is fixed in Haykuro's latest build.

Update: Allegedly there is a program in the market that can do all of this from a GUI (although you still need a rooted phone). It is called Wifi Helper by Fan Zhang; I haven't tested it personally yet. I kind of doubt it fixes the AP hopping issue on older versions.

WTF Nokia? N810 and E71 Won't Charge from USB?

2009-02-09T05:36:00.000-08:00

This is completely retarded. It almost makes them unusable. Even worse, the frakin charger is is 5v... I'm half tempted to tear the devices open and put a trace wire from the usb connector to the charger connector so it will automatically charge. I like being able to use my laptop as a backup battery for my other devices. I like being able to use one cord to charge any device. I like being able to carry a small generic cord that I can plug in to the front of any computer rather than a large charger that I have to dig under the desk to plug in...

It doesn't matter how awesome your device is, but you can really piss people off if you do something stupid like this. On that note, why the hell doesn't the iPhone support A2DP (or HID, or USB Mass Storage, or SD...)?

Ugh.

Forms: FDF, PDF, and PHP

2009-02-09T04:49:00.000-08:00

I have always been somewhat opposed to PDF documents. While they are great for cross-platform viewing and printing, the inability to edit them without buying special software was philosophically unacceptable. Also, not being able to save forms you fill out is really stupid. While there are some free alternatives now, they are really a kind of pain in the ass compared to Adobe Professional. So, unfortunately, that is what I used to get this to work quickly and easily. Thanks to Justin Koivisto, who's tutorial really helped.

For those of you that don't know (as I didn't before starting this project), FDF files are used to save filled out PDF forms, however they don't store the original PDF document. Instead the FDF file just has a pointer to the PDF document which can either be on the local computer or on the web.

So here is the scenario:

1) User fills out an online form.
2) PHP parses the form, and creates a VCard as well as an FDF.
3) PHP emails the files as attachments, and displays the filled out PDF for the user to print.

This is very useful for populating PDF forms that already exist, and may be in use in paper form. This makes it easy to transition to an electronic system.

The first step is a standard HTML form that will submit the data to a PHP script (application.html):


              <html>
 <body>
  <form method="post" action="./submit.php">
    <table>
     <tr><td>First Name</td><td><input type="text" name="firstName" /></td></tr>
     <tr><td>Last Name</td><td><input type="text" name="lastName" /></td></tr>
     <tr><td>Mobile Phone Number</td><td><input type="text" name="mobilePhone" /></td></tr>
     <tr><td>Home Phone Number</td><td><input type="text" name="homePhone" /></td></tr>
     <tr><td>E-Mail</td><td><input type="text" name="email" /></td></tr>
     <tr><td>Date of Birth (MM-DD-YYYY)</td><td><input type="text" name="dobMonth" maxlength="2" size="2"/>-<input type="text" name="dobDay" maxlength="2" size="2"/>-<input type="text" name="dobYear" maxlength="4" size="4"/></td></tr>
     <tr><td>Address</td><td><input type="text" name="address" /></td></tr>
     <tr><td>City</td><td><input type="text" name="city" /></td></tr>
     <tr><td>State</td><td><input type="text" name="state" /></td></tr>
     <tr><td>Zip Code</td><td><input type="text" name="zip" /></td></tr>
     <!--<tr><td></td><td><input type="text" name="" /></td></tr>-->
     <tr><td>Age</td><td><input type="text" name="age" /></td></tr>
    </table>
    <input type="submit" value="Submit" />
  </form>
 </body>
</html>

Next is the PHP script to create the VCard and FDF, emails them, and displays the FDF (submit.php):


<?php

 include("./vcard.php");
 include("./fdf.php");
 include ("./mail_attachment.php");
 
 $emailto = "someone@example.com";
 
 //The pdf file that the fdf file points to:
 $pdf_doc='http://example.com/application.pdf';
 
 
 //We need to check POST variables to make sure the form was submitted correctly, we should also make sure they don't enter more than one email address, and that they enter an email address
/*
 if(isset($_POST["firstName"])) {
 echo $_POST["firstName"];
 }
 else
  echo "BAD!!!!";
*/
 $v = new vCard();

 $v->setPhoneNumber($_POST["mobilePhone"], "PREF;CELL;VOICE");
 $v->setPhoneNumber($_POST["homePhone"], "PREF;HOME;VOICE");
 $v->setName($_POST["lastName"], $_POST["firstName"], "", "");
 $v->setBirthday($_POST["dobYear"]."-".$_POST["dobMonth"]."-".$_POST["dobDat"]);
 $v->setAddress("", "", $_POST["address"], $_POST["city"], $_POST["state"], $_POST["zip"], "US");
 $v->setEmail($_POST["email"]);
 //$v->setNote("You can take some notes here.\r\nMultiple lines are supported via \\r\\n.");
 //$v->setURL("http://www.thomas-mustermann.de", "WORK");
/*
 $output = $v->getVCard();
 $filename = $v->getFileName();

 Header("Content-Disposition: attachment; filename=$filename");
 Header("Content-Length: ".strlen($output));
 Header("Connection: close");
 Header("Content-Type: text/x-vCard; name=$filename");

 echo $output;*/
 
 /*$outfdf = fdf_create();
 fdf_set_value($outfdf, "volume", $volume, 0);

 fdf_set_file($outfdf, "http:/testfdf/resultlabel.pdf");
 fdf_save($outfdf, "outtest.fdf");
 fdf_close($outfdf);
 Header("Content-type: application/vnd.fdf");
 $fp = fopen("outtest.fdf", "r");
 fpassthru($fp);
 unlink("outtest.fdf");*/
 
 foreach($_POST as $name => $val) {
  $data[$name] = $val;
  //echo $name."<br/>";
 }

 
 // generate the file content
 $fdf_data=createFDF($pdf_doc,$data);
 $attachments[0]['data']=$v->getVCard();
 $attachments[0]['filename']=$_POST['firstName'].$_POST['lastName'].".vcf";
 $attachments[0]['mime']="text/x-vcard";
 
 $attachments[1]['data']=$fdf_data;
 $attachments[1]['filename']=$_POST['firstName'].$_POST['lastName'].".fdf";
 $attachments[1]['mime']="application/vnd.fdf";
 
 $from=$_POST['firstName']." ".$_POST['lastName']." <".$_POST['email'].">";
 
 mail_attachment($from,$emailto,"Rental Application", "Lease: from ".$_POST['leaseStart']." to ".$_POST['leaseEnd'], $attachments);
 Header("Content-type: application/vnd.fdf");
 echo $fdf_data;

?>

This shows an example of how to use the multi-attachment email script shown below. We should be checking the POST inputs to make sure someone isn't trying to do something nasty, there could be a security problem there.

Notably, there included files that you need. The first is the VCard creator (vcard.php):


<?

/***************************************************************************



PHP vCard class v2.0

(c) Kai Blankenhorn

www.bitfolge.de/en

kaib@bitfolge.de





This program is free software; you can redistribute it and/or

modify it under the terms of the GNU General Public License

as published by the Free Software Foundation; either version 2

of the License, or (at your option) any later version.



This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

GNU General Public License for more details.



You should have received a copy of the GNU General Public License

along with this program; if not, write to the Free Software

Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.



***************************************************************************/





function encode($string) {

 return escape(quoted_printable_encode($string));

}



function escape($string) {

 return str_replace(";","\;",$string);

}



// taken from PHP documentation comments

function quoted_printable_encode($input, $line_max = 76) {

 $hex = array('0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F');

 $lines = preg_split("/(?:\r\n|\r|\n)/", $input);

 $eol = "\r\n";

 $linebreak = "=0D=0A";

 $escape = "=";

 $output = "";



 for ($j=0;$j<count($lines);$j++) {

  $line = $lines[$j];

  $linlen = strlen($line);

  $newline = "";

  for($i = 0; $i < $linlen; $i++) {

   $c = substr($line, $i, 1);

   $dec = ord($c);

   if ( ($dec == 32) && ($i == ($linlen - 1)) ) { // convert space at eol only

    $c = "=20"; 

   } elseif ( ($dec == 61) || ($dec < 32 ) || ($dec > 126) ) { // always encode "\t", which is *not* required

    $h2 = floor($dec/16); $h1 = floor($dec%16); 

    $c = $escape.$hex["$h2"].$hex["$h1"]; 

   }

   if ( (strlen($newline) + strlen($c)) >= $line_max ) { // CRLF is not counted

    $output .= $newline.$escape.$eol; // soft line break; " =\r\n" is okay

    $newline = "    ";

   }

   $newline .= $c;

  } // end of for

  $output .= $newline;

  if ($j<count($lines)-1) $output .= $linebreak;

 }

 return trim($output);

}



class vCard {

 var $properties;

 var $filename;

 

 function setPhoneNumber($number, $type="") {

 // type may be PREF | WORK | HOME | VOICE | FAX | MSG | CELL | PAGER | BBS | CAR | MODEM | ISDN | VIDEO or any senseful combination, e.g. "PREF;WORK;VOICE"

  $key = "TEL";

  if ($type!="") $key .= ";".$type;

  $key.= ";ENCODING=QUOTED-PRINTABLE";

  $this->properties[$key] = quoted_printable_encode($number);

 }

 

 // UNTESTED !!!

 function setPhoto($type, $photo) { // $type = "GIF" | "JPEG"

  $this->properties["PHOTO;TYPE=$type;ENCODING=BASE64"] = base64_encode($photo);

 }

 

 function setFormattedName($name) {

  $this->properties["FN"] = quoted_printable_encode($name);

 }

 

 function setName($family="", $first="", $additional="", $prefix="", $suffix="") {

  $this->properties["N"] = "$family;$first;$additional;$prefix;$suffix";

  $this->filename = "$first%20$family.vcf";

  if ($this->properties["FN"]=="") $this->setFormattedName(trim("$prefix $first $additional $family $suffix"));

 }

 

 function setBirthday($date) { // $date format is YYYY-MM-DD

  $this->properties["BDAY"] = $date;

 }

 

 function setAddress($postoffice="", $extended="", $street="", $city="", $region="", $zip="", $country="", $type="HOME;POSTAL") {

 // $type may be DOM | INTL | POSTAL | PARCEL | HOME | WORK or any combination of these: e.g. "WORK;PARCEL;POSTAL"

  $key = "ADR";

  if ($type!="") $key.= ";$type";

  $key.= ";ENCODING=QUOTED-PRINTABLE";

  $this->properties[$key] = encode($name).";".encode($extended).";".encode($street).";".encode($city).";".encode($region).";".encode($zip).";".encode($country);

  

  if ($this->properties["LABEL;$type;ENCODING=QUOTED-PRINTABLE"] == "") {

   //$this->setLabel($postoffice, $extended, $street, $city, $region, $zip, $country, $type);

  }

 }

 

 function setLabel($postoffice="", $extended="", $street="", $city="", $region="", $zip="", $country="", $type="HOME;POSTAL") {

  $label = "";

  if ($postoffice!="") $label.= "$postoffice\r\n";

  if ($extended!="") $label.= "$extended\r\n";

  if ($street!="") $label.= "$street\r\n";

  if ($zip!="") $label.= "$zip ";

  if ($city!="") $label.= "$city\r\n";

  if ($region!="") $label.= "$region\r\n";

  if ($country!="") $country.= "$country\r\n";

  

  $this->properties["LABEL;$type;ENCODING=QUOTED-PRINTABLE"] = quoted_printable_encode($label);

 }

 

 function setEmail($address) {

  $this->properties["EMAIL;INTERNET"] = $address;

 }

 

 function setNote($note) {

  $this->properties["NOTE;ENCODING=QUOTED-PRINTABLE"] = quoted_printable_encode($note);

 }

 

 function setURL($url, $type="") {

 // $type may be WORK | HOME

  $key = "URL";

  if ($type!="") $key.= ";$type";

  $this->properties[$key] = $url;

 }

 

 function getVCard() {

  $text = "BEGIN:VCARD\r\n";

  $text.= "VERSION:2.1\r\n";

  foreach($this->properties as $key => $value) {

   $text.= "$key:$value\r\n";

  }

  $text.= "REV:".date("Y-m-d")."T".date("H:i:s")."Z\r\n";

  //$text.= "MAILER:PHP vCard class\r\n";

  $text.= "END:VCARD\r\n";

  return $text;

 }

 

 function getFileName() {

  return $this->filename;

 }

}





//  USAGE EXAMPLE

/*

$v = new vCard();



$v->setPhoneNumber("+49 23 456789", "PREF;HOME;VOICE");

$v->setName("Mustermann", "Thomas", "", "Herr");

$v->setBirthday("1960-07-31");

$v->setAddress("", "", "Musterstrasse 20", "Musterstadt", "", "98765", "Deutschland");

$v->setEmail("thomas.mustermann@thomas-mustermann.de");

$v->setNote("You can take some notes here.\r\nMultiple lines are supported via \\r\\n.");

$v->setURL("http://www.thomas-mustermann.de", "WORK");



$output = $v->getVCard();

$filename = $v->getFileName();



Header("Content-Disposition: attachment; filename=$filename");

Header("Content-Length: ".strlen($output));

Header("Connection: close");

Header("Content-Type: text/x-vCard; name=$filename");



echo $output;*/

?>

Next is the FDF creator (fdf.php):


<?php

/*

KOIVI HTML Form to FDF Parser for PHP (C) 2004 Justin Koivisto

Version 2.1.2

Last Modified: 9/12/2005



    This library is free software; you can redistribute it and/or modify it

    under the terms of the GNU Lesser General Public License as published by

    the Free Software Foundation; either version 2.1 of the License, or (at

    your option) any later version.



    This library is distributed in the hope that it will be useful, but

    WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY

    or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public

    License for more details.



    You should have received a copy of the GNU Lesser General Public License

    along with this library; if not, write to the Free Software Foundation,

    Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 



    Full license agreement notice can be found in the LICENSE file contained

    within this distribution package.



    Justin Koivisto

    justin.koivisto@gmail.com

    http://koivi.com

*/



/*

*   createFDF

*

*   Takes values submitted via an HTML form and fills in the corresponding

*   fields into an FDF file for use with a PDF file with form fields.

*

*   @param  $file   The pdf file that this form is meant for. Can be either

*                   a url or a file path.

*   @param  $info   The submitted values in key/value pairs. (eg. $_POST)

*   @result Returns the FDF file contents for further processing.

*/

function createFDF($file,$info){

    $data="%FDF-1.2\n%????\n1 0 obj\n<< \n/FDF << /Fields [ ";

    foreach($info as $field => $val){

        if(is_array($val)){

            $data.='<</T('.$field.')/V[';

            foreach($val as $opt)

                $data.='('.trim($opt).')';

            $data.=']>>';

        }else{

            $data.='<</T('.$field.')/V('.trim($val).')>>';

        }

    }

    $data.="] \n/F (".$file.") /ID [ <".md5(time()).">\n] >>".

        " \n>> \nendobj\ntrailer\n".

        "<<\n/Root 1 0 R \n\n>>\n%%EOF\n";

    return $data;

}

?>

There is an official PHP FDF creator, but it uses some ridiculous dll file that you need to install. PHP is great at string manipulation, why the heck do we need a dll? Anyway, thanks a lot Justin Koivisto.

Finally a custom email script I wrote to email multiple attachments (mail_attachment.php):


<?php

function mail_attachment ($from , $to, $subject, $message, $attachments){

 $email_from = $from; // Who the email is from 

 $email_subject =  $subject; // The Subject of the email 

 $email_txt = $message; // Message that the email has in it 

 

 $email_to = $to; // Who the email is to

 $headers = "From: ".$email_from;
 

 $semi_rand = md5(time()); 

 $mime_boundary = "==Multipart_Boundary_x{$semi_rand}x"; 

    

 $headers .= "\nMIME-Version: 1.0\n" . 

            "Content-Type: multipart/mixed;\n" . 

            " boundary=\"{$mime_boundary}\""; 

 

 $email_message .= "This is a multi-part message in MIME format.\n\n" . 

                "--{$mime_boundary}\n" . 

                "Content-Type:text/html; charset=\"iso-8859-1\"\n" . 

               "Content-Transfer-Encoding: 7bit\n\n" . $email_txt . "\n\n"; 

      

 foreach ($attachments as $a) {

  $data = chunk_split(base64_encode($a['data'])); 



  $email_message .= "--{$mime_boundary}\n" . 

                   "Content-Type: {$a['mime']};\n" . 

                   " name=\"{$a['filename']}\"\n" . 

                   "Content-Transfer-Encoding: base64\n\n" . 

                  $data . "\n\n";

 }

 

 $email_message .= "--{$mime_boundary}--\n";



 $ok = @mail($email_to, $email_subject, $email_message, $headers); 



 if($ok) { 

 } else { 

  die("There was a problem sending the email. Please check your inputs."); 

 } 

}



?>

Thank you about.com for the HTML Converter.

Howto: Xen Bridging with Static and DHCP NICs

2009-01-28T20:59:00.000-08:00

I have been meaning to figure out how to put Xen virtual machines on different networks for over a year. Every once in a while I would start poking at it, but never could get it to work. Now a project I'm doing requires VMs on different networks, and to easily be able to switch them, so I finally took the time to learn. Notably it is more linux networking 101 than anything.

This would have been relatively easy if both NICs used static IPs, this tutorial would have worked more or less out of the box. It turns out that throwing dhcp in the mix really screws with routing and dns. Even worse the dhcp daemon running in the background keeps screwing it up unless you disable it.

-------------------------------------------------

Howto:

Note: This is on Ubuntu 8.04 and Xen 3.2. I want dom0 to access the internet via a static ip on eth0.

First create /etc/xen/scripts/multi-network-bridge then put the following in it:


#!/bin/sh
/etc/xen/scripts/network-bridge $@ vifnum=1 netdev=eth1 bridge=xenbr1
/etc/xen/scripts/network-bridge $@ vifnum=0 netdev=eth0 bridge=xenbr0

ifconfig xenbr1 up
ifconfig xenbr0 up
ip route flush table main
ip route add X.X.X.X/X dev xenbr0
ip route add default via X.X.X.X
cp /etc/resolv.conf.bak /etc/resolv.conf

Where X.X.X.X/X is your static network such as 192.168.1.0/24 and X.X.X.X is your static gateway (192.168.1.1). /etc/resolv.conf.bak is a copy of your static dns servers.

Now make the script executable and disable the dhcp daemon:


sudo chmod +x /etc/xen/scripts/multi-network-bridge
sudo update-rc.d -f dhcdbd remove

Finally, find the line in /etc/xen/xend-config.spx that has "(network-script network-bridge)" and change it to "(network-script multi-network-bridge)".

Now check to make sure that /etc/network/interfaces is configured properly, mine looks like:


auto lo
iface lo inet loopback

auto eth1
iface eth1 inet dhcp

auto eth0
iface eth0 inet static
 address x.x.x.x
 gateway x.x.x.x
 netmask x.x.x.x

For DomUs to use the bridge you need to change the cfg file to look like:


vif         = [ 'mac=xx:xx:xx:xx:xx:xx,ip=x.x.x.x,bridge=xenbr0' ]

For the static network, or:


dhcp        = 'dhcp'
vif         = [ 'mac=xx:xx:xx:xx:xx:xx,bridge=xenbr1' ]

For the dhcp network.

Reboot Dom0 to make sure that networking comes back right.

------------------------------------------------

If you don't specifically name the bridges, then they will automatically be named eth0 and eth1 (and you probably won't have to do the ifconfig xxxx up either). This makes the output of ifconfig look slightly more normal.

A couple other things I found out along the way:

1. The interfaces should already be up and configured properly (with ips) before the bridge script is run. This should happen automatically if /etc/network/interfaces is set up properly.

2. Dom0 connects to the bridge directly, which is why it has an ip assigned.

3. "ip route" and "brctl show" are your friends.

4. The tapX devices are from hardware virtual machines.

Note: Doing this can confuse firefox in to thinking it is offline. If you have issues with firefox make sure File->Work Offline is unchecked (sometimes checking it then unchecking it helps).

Update:

Here is my new multi-network-bridge script:


#!/bin/sh
/etc/xen/scripts/network-bridge $@ vifnum=1 netdev=eth1
/etc/xen/scripts/network-bridge $@ vifnum=0 netdev=eth0

ip route flush table main
ip route add X/X dev eth0
ip route add default via X
cp /etc/resolv.conf.bak /etc/resolv.conf

If you don't specify a bridge then it defaults to eth0.

GWT-WishList

2008-12-20T11:00:00.000-08:00

As a sort of test project for a bunch of other things I was learning/doing such as AuthSub, iBatis, SQL joins, gwt authentication, etc. I created a little pet project to manage wishlists:

http://code.google.com/p/gwt-wishlist/

It is a good, relatively small, project for looking at how to integrate iBatis and GWT or AuthSub and GWT. The code is in a pretty ugly/uncommented state right now. If I get a chance I will make a blog entry about how it actually works... and comment my code.

Howto Ubuntu Intrepid Ibex 8.10 Xen Dom0

2008-12-17T07:17:00.000-08:00

Intro

So... much to my dismay Ubuntu quit supporting Xen as a Dom0. The default kernel intrepid ships with is supposed to support being a domU, but that is just because of the addition of paravirt_ops into the standard linux kernel.

Anyway, they aren't supporting xen anymore because they are going with KVM as their server virtualization technology, and virtualbox as their destkop virt tech. Not that I am opposed to KVM, it just seems like this is a bit more complicated than xen-create-image... especially since I am not familiar with it.

As it so happens there are a couple branches of 2.6.27 linux xen kernels, and while they are not official, they seem to be working well. I think that the plan is to integrate dom0 functionality into the paravirt_ops, which basically means creating the backend drivers, but since that is not completed yet, we will need to use one of these unstable branches to get a functioning dom0 with a recent kernel (although they seem very stable to me -- the one I'm using is from Novell Suse).

Howto:

Install a fresh Ubuntu Server 8.10 AMD64. I opted for lvm, but that is certainly not necessary. Luckily Xen 3.3.0 is in the intrepid repos, unfortunately it doesn't have a compatible linux kernel, so we have to download, compile, and install one.

Note: MAKE SURE ITS AMD64, apparently there are build problems with 32bit. Besides there really is no point to have a 32bit dom0 -- it can't see more than like 3gb of ram, and you can always make 32bit domUs.


 sudo apt-get install ubuntu-xen-server build-essential libncurses5-dev gawk mercurial
 mkdir -p /home/#user#/build/linux-2.6.27-xen
 cd /usr/src/
 sudo hg clone http://xenbits.xensource.com/ext/linux-2.6.27-xen.hg
 cd linux-2.6.27-xen.hg
 make O=/home/#user#/build/linux-2.6.27-xen/ menuconfig
 make O=/home/#user#/build/linux-2.6.27-xen/ -j12 #put number of cores X1.5 here#
 sudo make O=/home/#user#/build/linux-2.6.27-xen/ modules_install install
 sudo depmod 2.6.27.5
 sudo mkinitramfs -o /boot/initrd-2.6.27.5.img 2.6.27.5

The most important part here is the menuconfig, by default none of the xen stuff is enabled. I had to disable 10000 Mbps ethernet (ixgbe), the slub allocator, and perfmon (or maybe I selected that and shouldn't have) because they wouldn't compile. You can either build two kernels, one with backend drivers (dom0), and the other with frontend drivers (domUs), or you can put both of them in one build and use the kernel for both dom0 and domUs. See this and this for various xen options. Pushing h in the menuconfig helps a lot

Now you have to edit your /boot/grub/menu.lst to boot to the new kernel under xen. It should look something like:


title         Xen 3.3 / Ubuntu 8.10, kernel 2.6.27-xen
uuid          538bd858-c116-45e8-bbb5-93c073464889
kernel        /xen-3.3.gz
module        /vmlinuz-2.6.27.5 root=/dev/mapper/LVMVG-LV ro console=tty0
module        /initrd-2.6.27.5.img

You should now be able to boot in to xen by simply rebooting. If your dom0 doesn't come up then simply reboot and change the grub selection to the non-xen and fix whatever it is you broke ;p.

Oddly the ubuntu-xen-server doesn't include hardy or intrepid configs for it's xen tools configs. This will fix that though:


sudo ln -s /usr/lib/xen-tools/edgy.d /usr/lib/xen-tools/hardy.d
sudo ln -s /usr/lib/xen-tools/edgy.d /usr/lib/xen-tools/intrepid.d

Now you can create an intrepid image using:


sudo xen-create-image --hostname=somehost.example.com --size=10Gb --swap=2Gb --ide --mac=00:00:00:00:00:00 --memory=2Gb --arch=amd64 --lvm=VolumeGroup --kernel=/boot/vmlinuz-2.6.27.5 --initrd=/boot/initrd-2.6.27.5.img --install-method=debootstrap --dist=intrepid --mirror=http://archive.ubuntu.com/ubuntu/ --passwd --accounts --dhcp

---------------------Cut---------------------------
You might be able to fix this somewhere in the default config, but you need to change the console both in the generated .cfg file and in the actual disk image it created:


sudo echo "console='xvc0'" >> /etc/xen/somehost.example.com.cfg
sudo mkdir /media/domudisk
sudo mount /dev/mapper/LVMVGroup--somehost.example.com--disk /media/domudisk
cd /media/domudisk/etc/event.d/
sudo nano tty1
cd ~
sudo umount /media/domudisk

On the last line change 'tty1' to 'xvc0'.
---------------------------------------------------
Update: You can fix this with a config:


sudo su 
echo "console='xvc0'" >> /etc/xen-tools/xm.tmpl

(sudo echo doesn't work for me...)
Then add the line "sed -i "s/tty1/${serial_device}/" ${prefix}/etc/event.d/tty1" to /usr/lib/xen-tools/edgy.d/30-disable-gettys, so that last if section reads:


#
#  Are we using an alternative serial device?
#
if [ ! -z "${serial_device}" ]; then

    serial_device=`basename ${serial_device}`
    echo "Setting DomU Serial Device to $serial_device"
    # Let the user know.
    logMessage "Replacing default serial device (tty1) with ${serial_device}"

    # replace existing device.
    sed -i "s/tty1/${serial_device}/" ${prefix}/etc/event.d/tty1
    mv ${prefix}/etc/event.d/tty1  ${prefix}/etc/event.d/${serial_device}
    sed -i -e s/tty1/${serial_device}/ ${prefix}/etc/inittab

    # make sure that it is allowed to login.
    echo $serial_device >> ${prefix}/etc/securetty
    echo "Serial device set"
    echo $serial_device
fi

and, finally, uncomment "serial_device = xvc0" and "disk_device = xvda" in /etc/xen-tools/xen-tools.conf

There are other goodies in /etc/xen-tools/xen-tools.conf (such as defaults for image creation) too.

The vanilla kernel that ships with Ubuntu as well as building a new kernel from kernel.org (if you enable the paravirt_ops and frontend drivers), will both run as domUs, you simply have to change the hda/sda references in the cfg file to xvda manually, or you can uncomment the "disk_device = xvda" in /etc/xen-tools/xen-tools.conf. MAKE SURE YOU COPY YOUR /lib/modules/2.6.27.whatever over to the domU drive!!! xen-tools will not do this for you (without more script mods at least -- a simple hack would be to replace what /usr/lib/xen-tools/edgy.d/80-install-modules does with cp -r /usr/lib/modules/* ${prefix}/usr/lib/modules).

To build the latest linux kernel:


mkdir -p /home/user/build/linux-2.6.27/
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.27.y.git
cd linux-2.6.27.y
make O=/home/#user#/build/linux-2.6.27-xen/ menuconfig
make O=/home/#user#/build/linux-2.6.27-xen/ -j12 #put number of cores X1.5 here#
sudo make O=/home/xepra/build/linux-2.6.27.y/ modules_install install
depmod 2.6.27.9
mkinitramfs -o /boot/initrd.img-2.6.27-9 2.6.27.9

Notes:

I will update with some more details... hopefully

On another note -- I actually did this with a gentoo install cd initially and booted directly to a xen kernel. It worked surprisingly well, especially since I am a gentoo neophyte, but there weren't any image creation tools. After manually creating a domU and getting it running I had all sorts of networking problems that I never had with Ubuntu... so I went back to Ubuntu.

I also went through this process on Debian lenny, but also had some problems; I think xen 3.3 wasn't in the repos so I had to compile it from scratch... I also was using grub2, which was a bit different, but not hard to figure out and get working.

Update: It looks like Boris Derzhavets used this entry and did a follow up with a lot more detail, screen shots, and output.

Passwords

2008-11-20T20:28:00.000-08:00

In this post I mentioned that you should not use the same password for everything due to a number of security reasons. I also mentioned that you shouldn't store your passwords unencrypted anywhere.

Most people aren't going to remember a different password for every website/computer/email address they use. So what should you do?

Well the first option is to store them in an encrypted file (which requires a password to access...). Also, if you ever lose that file (stolen, deleted, hard drive crash, etc) then you are SOL.

The way I solve this problem is to use a predefined pattern to modify a base password. For a simple example say your base password is "VerySecret7" then you could make your password for Gmail "GerySecret7" and your password for Facebook "FerySecret7". Granted this is not a very secure example (yet still much more secure than the same password!), but you can extend this to make more complicated patterns. For example:

1. Use the next letter of the alphabet so the connection isn't as apparent. (thus "HerySecret7" for Gmail and "EerySecret7" for Digg)

2. Convert letters to numbers. ("7erySecret7" Gmail and "25erySecret7" for Yahoo).

3. Take multiple letters from the service name. (IE take the first and third letter and put them at the end -- "VerySecret7Ga" for Gmail or "VerySecret7Fc" for Facebook)

4. Use classes of passwords. (For banks use "SuperSecret39" for your base password, then apply your normal pattern to it.) This is probably overkill if you have a half decent pattern/base password.

5. If the first letter of the service starts with a letter after M spell your base password backwards. (Also probably overkill)

6. You get the idea, basically whatever you can think of. Combinations of patterns will be more secure, and symbols, caps, and numbers always improve password security.

Get creative, but not so creative that you won't remember it... Just be consistent. A relatively simple pattern such as take the letter after the first letter of the service name and add it to the front, then convert the last letter to a number gives you a pretty secure password and fairly unrecognizable pattern ("Hery11ecret7" for gmail).

It would be very hard for someone who has one of your passwords to both know you are using a pattern and discover your pattern (if it is half decent, not VerySecret7Gmail...). On the other hand it *should* remain easy for you to remember all of your passwords without writing them down.

The big drawback of this is that you can't change just one password without breaking the pattern, so to change one password you need to change all of them...

This technique is essentially a simple hash function. If you have a better solution (IE it solves the above problem), please let me know :).

iPhone Goodness

2008-11-17T04:45:00.000-08:00

Some useful iPhone tidbits:

1. All of the below is contigent on jailbreaking. Really this is the only way I could ever justify buying an iPhone... Thanks a lot to the iphone-dev team. You guys rock.

2. If you want to develop an iPhone application, and don't want to pay apple, wait for their certificate, or go through their application review process then you should try: http://www.saurik.com/id/8 and Cydia for development, deployment, and distribution. In general everything by saurik is badass and worth reading.

3. If you want to be able run official applications in the back ground (for example listen to pandora while looking up something on maps) checkout "backgrounder" on cydia.

4. If you write your own app, and don't care about releasing it in the app store, you can run it in the background using:



@interface Thing : UIApplication {
}
@implementation Thing
- (void)applicationSuspend:(UIApplication *)application {
    [super applicationSuspend: application];
}

Thanks AlJaMa in iphone-dev irc for that. (I haven't tried it yet)

5. If you want to be able to use Fring (or other wifi limited applications) over the cellular network, then there is a patch on cydia for that too: Voipover3g. You can apparently specify which applications to spoof wifi access for by editing /Library/MobileSubstrate/DynamicLibraries/VoIPover3G.plist

The default config looks something like: "Filter = {Bundles = ("com.Fringland.Fring", "com.apple.AppStore", "com.audiofile.Interstate", "com.apple.MobileStore");};"

6. I haven't tried Erling's accelerometer hack on 2.x firmware yet, but hopefully it still works (allows you to adjust sample rate): http://blog.medallia.com/2007/08/iphone_accelerometer_source_co.html

UPDATE: It appears that there is a way to use the SDK to set the sample rate through UIAccelerometer.

7. http://www.cocoamachine.com/blog/ has also been a good resource. And of course there is always #iphone-dev on irc.osx86.hu

8. There are some headers missing from the official framework. You should be able to do a class dump:

class-dump -H /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator2.0.sdk/System/Library/Frameworks/UIKit.framework/UIKit -o output-directory

The file it spits out is mach-o.

9. PDANet also from cydia, allows you to share the iphones 3g connection via wifi.

10. Netatalk appletalk / general storage.

11. DiskAid

Other: terminal, hp calc, stumbler plus, and on the app store Rooms (irc) and motionX dice.

GWT RPC Exceptions

2008-11-17T03:49:00.000-08:00

So... Most of GWT's documentation is pretty good, but the part about RPC exceptions is kind of lacking.

I found some posts such as:

http://groups.google.com/group/Google-Web-Toolkit/browse_thread/thread/c048700dcaee7b2a
http://astithas.blogspot.com/2007/08/case-of-disappeared-exception-message.html
http://blog.platinumsolutions.com/node/198

But all of them mentioned the now deprecated SerializableException. According to the API this was deprecated because Exception implements Serializable, and thus a normal exception should work fine... (http://google-web-toolkit.googlecode.com/svn/javadoc/1.5/com/google/gwt/user/client/rpc/SerializableException.html)

Unfortunately I was still getting the "the call failed on the server see server log for details" message from exceptions that reached the client.

Anyway, the solution I reached was very, very, simple. My ServiceImpl class throws a standard RuntimeException ("throw new RuntimeException("Epic Fail");"). But this is what I was already doing that didn't work. I had to add "throws Exception" to the method, and thus the client side service interface as well. Worked like a charm after that.


public interface SomeService extends RemoteService {
    void doSomething(Object someObject) throws exception;
}

public interface SomeServiceAsync {
    void doSomething(Object someObject, AsyncCallback callback);
}

public class SomeServiceImpl extends RemoteServiceServlet implements SomService {
    public void doSomething(Object someObject) throws Exception {
        if(epic.fail())
            throw new RuntimeException("Epic Fail");
        else 
            doSomethingCool();
    }
}

Too many choices for RIAs...

2008-10-08T01:18:00.000-07:00

Notably this, like most of my ramblings, is mainly for thought aggregation and self-reference later, in this case mainly keywords to look up, but if it helps you out then good :).

The number of options for web development is overwhelming! The more complex the application is, the more options you have. For a simple web page its easy -- HTML, and if you want to get fancy you use javascript and CSS. More than that is when it starts to get complicated, you have Javascript libraries like jQuery and Dojo or embedded stuff like Flash, Java Applets, and, now, Silverlight.

Now what if you want to do something on the back-end? There's SSI, PHP, ColdFusion, Python, Java Servlets, JSP, Java Restlets, Ruby... and the list goes on. Hell, if you wanted to you could write the whole thing, web server and all, in C or C++ -- at least it would be fast as hell :).

Now what about databases? MySQL is kind of the open source standard that seems to scale well, but enterprises tend to lean towards PostgreSQL or commercial options like MSSQL and Oracle. I also took a look at FirebirdDB, which looks very interesting. In all honesty I am surprised that more people don't use FirebirdDB and PostgresSQL over MySQL. Then there is also Hadoop... but thats not even SQL.

Then there are database abstraction layers (DALs), and the selection depends on your language. With PHP the most promising ones (for me) are PEAR DB and adoDB (which actually can run some native C code to speed things up, very cool). adoDB works with Python too. In Java there are Spring, Hibernate, and iBatis. I don't even know all the options for the other languages. While there are arguments against database abstraction, some of the solutions make coding much simpler and faster. ORMs in particular look very useful.

Once you start getting more complex interfaces you have to worry about cross browser compatibility which is just nasty. Coding for IE6 as well as standards compliant browsers was awful, just awful. Not to mention debugging and writing maintainable code is non-trivial (thank god for firebug). In short it is a layer that a system architect for a complex RIA just does not want to deal with. There are various options for circumventing this, even some CMSs such as Drupal and Wordpress do this indirectly, but there are a couple packages that do this very elegantly and quickly such as OpenLaszlo, Adobe Flex, and Google Web Toolkit, and one other one I can't think of right off. Here is an interesting article comparing those three.

Next you need to consider reliability, performance, scalability, and code maintenance. In enterprises performance is irrelevant if it is scalable -- you can always just throw more servers at it. This is one reason Java has been so well adopted on an enterprise level; even though it is very slow when compared with C or C++, it is scalable, more flexible, and produces more maintainable code. "Programmer performance is worth far more than system performance ;-)." [1].

For scalability you can look at products like Squid which will do reverse caching and load balancing for web pages as well as sqlrelay which will do connection pooling and load balancing for sql servers. http://danga.com has some cool stuff too.

Since most of these technologies are interchangeable, you can mix and match almost any combination of them; this creates a huge number of deployment options. Hmm, maybe I will do GWT -> PHP -> adoDB -> SQLRelay -> MySQL. Or you could even do PHP -> Python -> PostgresSQL, if you decided you like Python for some of your server interactions and PHP just for templating, and, in this case, you opted not to have a DAL. These are important decisions to make before a project begins, as backtracking can be prohibitively expensive and choosing the wrong architecture can cause problems with stability and development; even worse is its hard to predict how it will scale until the application is completely written. Luckily for small, homebrew, applications almost all of these technologies will be able to handle any load you throw at it, so the common recommendation is do it however you are comfortable with, but when you get in to large enterprise applications then things like persistent database connections, clustering, stability, and collaborative code development need to be carefully considered.

This brings me to a minor point - in my experience it is much easier to write bad PHP code than good PHP code, where as it is much easier to write good Java code than bad Java code. Granted there are exceptions, but the loose typing and script like nature of PHP can very easily lead to unreadable code. Java, on the other hand, is strictly object oriented and tends to lead to better design patterns. Sure a bad coder can screw up either royally, but it is something to consider while choosing a collaborative language.

So far, after looking at all of these packages, GWT -> JBoss -> iBatis -> WhateverSQL [2] looks very, very, promising for quickly producing scalable and maintainable RIAs. GWT makes communication between a browser and a Java Servlet seamless and lets you stay away from the HTML/Javascript pit of death. JBoss has all sorts of neato scalability. iBatis looks like a great ORM that doesn't abstract the SQL too much; it also does connection pooling and balancing. Lets assume WhateverSQL has seamless clustering and replication ;). This looks much easier (and cleaner!) than HTML -> JavaScript -> PHP -> adoDB -> SQLRelay -> WhateverSQL. We'll see how it goes. Here is a nice little tutorial for GWT + iBatis.

[1] rycamor
[2] I actually just don't want to put that I am using Oracle for this particular project... Who the hell pays $40,000+ for one license when there are so many free options? It may have some cool features, which I have yet to see, but unless it writes the damn webapp for you then just use Postgres.

Alphabet Soup:

PHP -> PHP: HypeText Preprocessor
RIA -> Rich Internet Application
SQL -> Structured Query Language
HTML -> HyperText Markup Language
GWT -> Google Web Toolkit
MSSQL-> Microsoft SQL
CSS -> Cascading Style Sheet
adoDB-> Active Data Objects DataBase
ORM -> Object Relational Mapping
SSI -> Server Side Includes
JSP -> Java Server Pages

Multiple Desktops in Linux (even Remote ones...)

2008-10-04T21:09:00.000-07:00

Although I love the ability to control Linux PCs and servers remotely with ssh and X tunneling, I was never satisfied with VNC as a complete Remote Desktop solution. For starters it is insecure and to secure it you have to jump through a couple hoops. The biggest problem, however, is that you already have to have a session running on the remote computer in order to use VNC.

I wanted an easy to use, built-in, secure way to get a complete remote desktop on a Linux machine at any time, from anywhere.

Luckily I found one:

1) go to a new terminal via ctrl-alt-f2 and login.
2) type "xinit -- :1". This should pop up a new instance of x with a terminal running. I had to mouse over the terminal to type.
3) type "ssh -CX user@host" and login to the remote server (C is compression...)
4) now type "gnome-session" or "startkde" or whatever desktop you want to start.

Thats it. No special programs and it is already encrypted through an ssh tunnel =).

To close this desktop type "ctrl-alt-backspace".
To get back to your primary desktop type "ctrl-alt-f7".
To switch back to the remote desktop type "ctrl-alt-f9".

You can open another desktop if you would like even with these two open, just use "xinit -- :2" which will be on the f10 terminal... You can do as many as 5 or 6.

If you want another local, rather than remote, desktop you can just ignore the ssh step (3) and just start the new desktop locally. This will allow you to run kde, gnome, and xcfe side by side and switch between them with a single key combo.

Also, you may find it useful to know that many programs have a --desktop option that allows you to open programs on any desktop from any terminal. ie "firefox --desktop :1". If you have multiple monitors you can even do "firefox --desktop :1.1" to specify the second monitor.

If you want to do this from windows just use a program like Xming :). Although the beta xrdp project is kind of interesting, it still seems a bit rough around the edges. The only thing that this doesn't really allow you to do is resume already running sessions :/

Update: What I was looking for was Xephyr. Works like a charm: http://ubuntuforums.org/showthread.php?p=3816948#post3816948 (The whole terminal thing is still cool tho =p)

Compiling VLC in 64 Bit Ubuntu 8.04.1

2008-08-29T16:06:00.000-07:00

If you haven't compiled VLC recently then you may be caught off guard by a couple changes made. The --with-ffmpeg-tree is now deprecated because ffmpeg started using the pkg-config system. Also the version of live555 in the Ubuntu repositories is currently too old - you will have to compile that yourself as well.

Hopefully, since you are trying to compile something, you are tech savvy enough to understand a shell script just about as well as my ramblings, so I'll just paste that:

#!/bin/bash

#get dependencies
sudo apt-get install cvs build-essential subversion git git-core automake1.9 libtool libgcrypt-dev libfaad-dev libtwolame-dev libqt4-dev libjack-dev libfaac-dev liblame-dev libxpm-dev libcddb2-dev liblua5.1-0-dev libzvbi-dev libshout-dev wget

#get the latest live555
wget http://www.live555.com/liveMedia/public/live555-latest.tar.gz
tar -xzvf live555-latest.tar.gz live
cd live
#modify config.linux to add -fPIC to the first line (for 64bit only)
mv config.linux config.linux.orig
echo `head -1 config.linux.orig` -fPIC > config.linux
tail -16 config.linux.orig >> config.linux
./genMakefiles linux
make -j 8

#get vlc
cd ~
git clone git://git.videolan.org/vlc.git
cd vlc/
./bootstrap

#git x264
cd ~/vlc/extras/
git clone git://git.videolan.org/x264.git
cd x264
./configure --enable-pic --disable-asm
make -j 8
sudo make install

#ffmpeg (need amr?)
cd ~/vlc/extras/
svn co svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-postproc --enable-pthreads --enable-libmp3lame --enable-libfaac --enable-swscale --disable-static --enable-shared
make -j 8
sudo make install

# this is for that pkg-config system ffmpeg now uses
export PKG_CONFIG_PATH="/usr/local/lib/pkgconfig/"

#build
cd ~/vlc
mkdir build
cd build

#originally I had this configured for no gui - you may need to play with it.
#watch for warnings - it will still compile with missing functionality if you are missing needed libraries
../configure --prefix=/usr \
--enable-snapshot --enable-debug \
--enable-dbus --enable-dbus-control --disable-musicbrainz \
--enable-shared-libvlc --disable-mozilla \
--enable-lirc \
--enable-x264 --with-x264-tree=../extras/x264 \
--enable-shout --disable-taglib \
--enable-v4l \
--enable-dvb \
--enable-realrtsp --enable-real --disable-xvmc \
--enable-svg --enable-dvdread \
--enable-dc1394 --enable-dv \
--enable-theora --enable-faad \
--enable-twolame --enable-real \
--enable-flac --enable-tremor \
--enable-skins2 --enable-qt4 --enable-x11 --enable-xvideo\
--enable-ncurses \
--enable-aa --enable-caca \
--enable-esd --disable-portaudio \
--enable-jack --disable-xosd \
--enable-galaktos --enable-goom \
--enable-ggi \
--disable-cddax --disable-vcdx \
--disable-qte --disable-quicktime --disable-lua \
--with-live555-tree=~/live \
--enable-libmpeg2 --enable-mkv \
--enable-ogg --enable-vorbis --enable-release --disable-hd1000v > config.out

make -j 8

Note that I use make -j 8 rather than just make because I am compiling with multiple cores (in this case it was 6 - it is usually good to start a couple more threads than the number of cores you have).

I had to add "--disable-static --enable-shared" to the ffmpeg or else it wouldn't compile - I think it had to do with some 64 bit PIC problems.

Notice the --disable-asm in the x264 config. This is BAD. The latest version of yasm in the Ubuntu repositories is not new enough for x264. This will be fixed in the next release of Ubuntu, but if I get around to it I will post an update before then.

Security

2008-08-29T15:29:00.000-07:00

Most people, tech and non-tech alike, have some very basic, easily correctable, user created security holes. Here are the most common I run in to:

1. If you use Firefox to store your passwords and don't set a master password then anyone who uses your computer can view your password in clear text with about 5 clicks - or about three seconds. Either don't use Firefox to store your passwords or set a master password. Be aware that other browsers have similar security issues when storing passwords.

2. Do not use the same password for everything. If someone manages to hack, crack, steal, or find out your password any way then they will have access to all of your accounts that use that password. Anyone who runs a website has access to the password you use at their website; do you trust them? Also many sites do not use encryption while authenticating you, thus your password is sent in clear text over the internet for anyone to see.

3. Check to see if a website is using encryption, especially for authentication. (You can check by looking at the url, which should start with https://, and there is usually a padlock in the bottom right corner of your browser.) Gmail by default only uses encryption to authenticate you; all of your email is sent unencrypted - this includes any passwords that are emailed to you by other websites. Last month they (finally) added a setting to always use encryption, but it isn't enabled by default... If a website does not use encryption for authentication then be very, very, sure to use a different password for that website.

4. Your email is a security hole for all the accounts tied to it. Be careful to not leave your email logged on anywhere other people have access to it. If you use email on a mobile phone/device then be sure to that it is protected with a pin or password. (*cough* iPhone *cough*)

5. Anyone with physical access to your computer can access all of your files in the time it takes to reboot the PC, regardless of whether you have it password protected. It is ill-advised to store passwords in a text file on your computer. There are plenty of programs that allow you to store passwords in an encrypted format (including Firefox, if you set a master password). If you have a lot of sensitive data then you may want to look in to full disk encryption, such as LUKS (Linux only...).

Darwin Streaming Server 6.0.3 on Linux

2008-08-29T12:00:00.000-07:00

Quick tutorial for installing DSS on Ubuntu 8.04.1 64bit (x86_64).

Most of these details can be found here: http://dss.macosforge.org/trac/ticket/6

Hopefully they will integrate these patches in to the next release, but until then here are the build instructions for Ubuntu:

Download DSS 6.0.3: http://static.macosforge.org/dss/downloads/DarwinStreamingSrvr6.0.3-Source.tar

Download this patch: http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-6.0.3.patch (Thank you Sverker Abrahamsson!)

And this patch: http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-hh-20080728-1.patch

Finally get the modified Install script: http://dss.macosforge.org/trac/raw-attachment/ticket/6/Install

The basic process is:

1. Get the build dependencies from the repositories.
2. Set up the qtss user (this is because the Install script does it in a way that is incompatible with Ubuntu)
3. Extract the source code.
4. Apply both patches.
5. Replace the Install script and chmod +x it.
6. Compile.
7. Install.

Notably for ssl admin you need the openssl and libnet-ssleay-perl packages, unfortunately even with these packages I was unable to get ssl admin to work.

I created the modified Install script; you can do a diff with the original if you want to see what I changed. Mostly just the paths were broken, but I am guessing that some other modules and such will probably not work.

Here is the bash script I used to do everything (including automatically download everything):


#!/bin/bash

sudo apt-get install build-essential wget
sudo addgroup --system qtss
sudo adduser --system --no-create-home --ingroup qtss qtss

wget http://static.macosforge.org/dss/downloads/DarwinStreamingSrvr6.0.3-Source.tar
tar -xvf DarwinStreamingSrvr6.0.3-Source.tar
mv DarwinStreamingSrvr6.0.3-Source DarwinStreamingSrvr6.0.3-Source.orig
wget http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-6.0.3.patch
patch -p0 < dss-6.0.3.patch
mv DarwinStreamingSrvr6.0.3-Source.orig DarwinStreamingSrvr6.0.3-Source
wget http://dss.macosforge.org/trac/raw-attachment/ticket/6/dss-hh-20080728-1.patch
patch -p0 < dss-hh-20080728-1.patch
#need to answer n then y
cd DarwinStreamingSrvr6.0.3-Source
mv Install Install.orig
wget http://dss.macosforge.org/trac/raw-attachment/ticket/6/Install
chmod +x Install
./Buildit
sudo ./Install

Update 2/2/2009: I changed the links to all point to dss.macosforge.org, which is a much more reliable host. Everyone should also take a look at that thread, as there are a lot of updates, including a newer patch from Horace Hsieh. I haven't played with this in a while, but if I get a chance I will try to update this with the newer patch. Please let me know if there are any problems (I just turned comment notifications on :/).