Sep 27 2011

Securing IPv6 with IPSec, protecting yourself in an end-to-end world!

Category: Linux,NetworkingMike Lovell @ 8:52 pm

With IPv6 there is no more NAT.  All your machines have end-to-end connections to their destination.  It’s taken for granted the implied security that NAT provides, by preventing incoming connections.  There’s also no longer a need to use a VPN, if I want to connect to my work PC I merely use it’s publicly available IPv6 address.

So what we need is a good (default deny) firewall policy to start with, but then we have the problem of encryption without using a VPN.  This is where IPSec comes in, providing end-to-end encryption at the packet level.  That way we can then happily use normally unencrypted  protocols securely.  We can also harden already encrypted protocols (SSH, RDP).

 

First thing we need to do is install “racoon” and “ipsec-tools” on both machines we wish to securely communicate between, in my case my Lubuntu 11.10 laptop and a Ubuntu 11.04 Server VPS.  These two machines have the following IPv6 addresses (anonymized):

Laptop    2604:xxxx:xxx::500
VPS       2607:xxxx:xxx::f49

To install the relevant packages on Debian based systems:

#sudo apt-get install ipsec-tools racoon

Then I need to edit “/etc/ipsec-tools.conf” to tell each machine I expect IPSec to be used to/from a certain IPv6 address, on my laptop it looks like this:

#!/usr/sbin/setkey -f
flush;
spdflush;

spdadd ::0/0               2607:xxxx:xxx::f49 any -P out ipsec esp/transport//require ah/transport//require;
spdadd 2607:xxxx:xxx::f49  ::0/0              any -P in  ipsec esp/transport//require ah/transport//require;

So any IP on my laptop (::0/0) to my VPS outgoing, then my VPS to any IP on my laptop (::0/0) incoming. Then same on the VPS but switching the IP:

#!/usr/sbin/setkey -f
flush;
spdflush;

spdadd ::0/0               2604:xxxx:xxx::500 any -P out ipsec esp/transport//require ah/transport//require;
spdadd 2604:xxxx:xxx::500  ::0/0              any -P in  ipsec esp/transport//require ah/transport//require;

Then we need to configure the level of encryption and key exchange we use on both machines.  These settings must match exactly or key exchange and encryption will fail.  These settings are defined in “/etc/racoon/racoon.conf”.  The man pages have information on the algorithms you can use, in my case I’ve gone for “uber paranoid” settings like this:

# Author: 	Mike Lovell
# Modified:	2011-08-16

path pre_shared_key "/home/mikelovell/.ipsec/config/keys.txt";

remote anonymous
{
	exchange_mode main;
	lifetime time 12 hour;

	proposal
	{
		encryption_algorithm   aes;
		hash_algorithm         sha512;
		authentication_method  pre_shared_key;
		dh_group               modp4096;
	}
}

sainfo anonymous
{
	pfs_group                      modp4096;
	lifetime time                  12 hour;
	encryption_algorithm           rijndael, aes;
	authentication_algorithm       hmac_sha512;
	compression_algorithm          deflate;
}

I store my IPSec keys inside “/home”, which is encrypted (mounted on boot) on my machines – You’re probably best keeping your keys in “/etc/racoon/keys.txt”.  In here you need to put the destination IP along with a password to use to communicate – To generate the password I always use “apg -a 1 -m 128″, here’s what the file might look like on the laptop:

2607:xxxx:xxx::f49   not_a_good_password!

And on the VPS:

2604:xxxx:xxx::500   not_a_good_password!

All that’s left to do now is restart the IPSec service and racoon, make sure you have another way into your machine (not via IPv6) before you do this – If something doesn’t work you don’t want to be locked out!

#sudo /etc/init.d/setkey restart
#sudo /etc/init.d/racoon restart

Once you’ve done that on both machines as soon as you establish a new connection (anything, try a ping6) you should see IPSec successfully negotiate in your syslog:

initiate new phase 1 negotiation: 2604:xxxx:xxx::500[500]<=>2607:xxxx:xxx::f49[500]
begin Identity Protection mode.
received Vendor ID: DPD
ISAKMP-SA established 2604:xxxx:xxx::500[500]-2607:xxxx:xxx::f49[500] spi:d____________f
initiate new phase 2 negotiation: 2604:xxxx:xxx::500[500]<=>2607:xxxx:xxx::f49[500]
IPsec-SA established: AH/Transport 2604:xxxx:xxx::500[500]->2607:xxxx:xxx::f49[500] spi=11___2(0x6____)
IPsec-SA established: ESP/Transport 2604:xxxx:xxx::500[500]->2607:xxxx:xxx::f49[500] spi=1___1(0xc____)
IPsec-SA established: AH/Transport 2604:xxxx:xxx::500[500]->2607:xxxx:xxx::f49[500] spi=1___5(0xb____)
IPsec-SA established: ESP/Transport 2604:xxxx:xxx::500[500]->2607:xxxx:xxx::f49[500] spi=96___3(0x5____)

Viola!  You care confirm if you like by sniffing the network traffic via “wireshark”.


Sep 26 2011

IPv6 without native support from your provider, via SiXS

Category: Linux,NetworkingMike Lovell @ 9:33 pm

You’ll need a Linux machine or device supporting AICCU and ip6tables to act as IPv6 router.  It doesn’t have to be the same machine/device as IPv4 router.  If you’re using DD-WRT on your IPv4 router then there’s a tutorial covering this here.

First you need to pop over to www.sixxs.net and sign up for a account (it’s free).  Once that’s approved you’ll need to request a tunnel.

For safeties sake, as some connectivity providers block other protocols, select “Dynamic NAT-traversing IPv4 Endpoint using AYIYA” as your endpoint.

Then you’ll need to select a PoP that’s close to you (to keep latency low) and give a reason why you want an IPv6 tunnel (use your imagination!)

After that’s approved you’ll need to request a subnet

Now we just need to setup our IPv6 router (in my case a Debian VM).  First lets setup AICCU which will configure a tunnel device for IPv6 with SixXS.  During the ncurses installer, you will be asked for your SixXS username and password:

# apt-get install aiccu

An “ifconfig” should now show the tunnel to be present:

sixxs     Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet6 addr: fe80::xxxx:xxx:xx:2/64 Scope:Link
          inet6 addr: 2604:xxxx:xxx:xx::2/64 Scope:Global
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1280  Metric:1
          RX packets:1989470 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1187540 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:1401336999 (1.3 GiB)  TX bytes:306453776 (292.2 MiB)

Then we need to enable IPv6 forwarding (as root):

#echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

To make this persistent you’ll need to add/ammend this rule in “/etc/sysctl.conf”:

net.ipv6.conf.default.forwarding=1

Lets now setup an IPv6 address from subnet we’ve been given to act as our IPv6 gateway, in my case my subnet is (anonymized):

2604:xxxx:xxx::/48

So in “/etc/network/interfaces” I put the following:

iface eth0 inet6 static
	address	2604:xxxx:xxx::1
	netmask	48

Now whenever the network comes up I want to add some routes in to make sure my IPv6 traffic is directed correctly, to do this I created a script called “/etc/network/if-up.d/ipv6routes” and marked it as executable:

#!/bin/sh

ip -6 route add 2604:8800:112::1/64 dev eth0
ip -6 route add 2604:8800:112::/48 dev lo

Then we just need to setup the ip6tables rules to forward the traffic correctly, in my case:

ip6tables -P FORWARD DROP

ip6tables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
ip6tables -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

ip6tables -A INPUT -i lo -j ACCEPT
ip6tables -A OUTPUT -o lo -j ACCEPT

ip6tables -A FORWARD -i eth0 -s 2604:xxxx:xxx::/48 -j ACCEPT
ip6tables -A FORWARD -i sixxs -o eth0 -d 2604:xxxx:xxx::/48 -j ACCEPT

To make these persistent you’ll need to set them to be loaded when the network comes up, what I usually do is this:

ip6tables-save > /etc/iptables-ipv6.conf

Then create a script called “/etc/network/if-up.d/iptables” and mark it as executable with the following inside it:

#!/bin/sh

ip6tables-restore < /etc/iptables-ipv6.conf

Now we assign an IPv6 address to a different machine in the network and set the gateway as the address we setup previously (2604:xxxx:xxx::1), here’s what the “ifconfig” looks like:

wlan0     Link encap:Ethernet  HWaddr 00:23:14:53:56:3c
          inet addr:192.168.x.xxx  Bcast:192.168.x.xxx  Mask:255.255.x.x
          inet6 addr: 2604:xxxx:xxx::500/64 Scope:Global
          inet6 addr: fe80::xxx:xxxx:xxxx:xxxx/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1280  Metric:1
          RX packets:403511 errors:0 dropped:0 overruns:0 frame:0
          TX packets:366141 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:195242106 (195.2 MB)  TX bytes:60409846 (60.4 MB)

And hopefully, it all works!

# traceroute6 -n www.sixxs.net
traceroute to nginx.sixxs.net (2001:1af8:4050::2) from 2604:xxxx:xxx::500, 30 hops max, 24 byte
 1  2604:xxxx:xxx::1  2.871 ms  2.892 ms  9.606 ms
 2  2604:xxxx:xxx:31::1  137.17 ms  150.576 ms  168.446 ms
 3  2620:0:6b0:a::1  204.796 ms  245.039 ms  329.039 ms
 4  2001:1900:2100::171  326.815 ms  256.593 ms  385.289 ms
   ...                         ...


Sep 25 2011

My base “iptables” / “ip6tables” script

Category: Linux,NetworkingMike Lovell @ 7:16 pm

I thought I’d post my base iptables/ip6tables script.  This is what I dump on every machine I install, then tweak from there.

It also creates  a script to load the rules up (on Debian based systems) when the network starts (if-up.d)

Download:
firewall.tar.gz


Jan 26 2011

Extracting music from flash videos using mencoder and mplayer

Category: Linux,Sound,Video ProcessingMike Lovell @ 2:30 pm

If you’ve ever downloaded music videos from a website (for example via “youtube-dl”) you may have been in the position of wanting to extract the music in “MP3″ format from the “FLV” (flash video) file, here’s how you can do this:

#mencoder source.flv -ovc frameno -oac mp3lame -lameopts br=128 -noskip -o resampled.avi
#mplayer -dumpaudio -dumpfile final.mp3 resampled.avi

Both “mencoder” and  “mplayer” can be used on both Windows and Linux (in the latest version of Ubuntu both are available via the package manager).


Jan 26 2011

X11 Forwarding of Pidgin with Sound using PulseAudio

Category: Linux,Pidgin,PulseAudio,SSHMike Lovell @ 1:41 pm

With the aid of PulseAudio it’s possible to remotely host applications such as Pidgin and still have sound enabled (something that’s very handy with an IM client for notification purposes).

First, on the client and server, you’ll need to make sure PulseAudio is installed (it should be there by default in Ubuntu 10.10 if that’s your client), for the client you can install all you need like this:

#apt-get install pulseaudio pulseaudio-utils paprefs

And for the server:

#apt-get install pulseaudio pulseaudio-utils

Then on the client allow network access to the local sound devices (you can open the preferences by typing “papref” at console or going to “System | Preferences | PulseAudio Preferences”):

"paprefs" Dialog

(Security Note: The port being used (TCP/4713) is blocked by the firewall on this machine to anything but LocalHost, you should do the same)

Then, when we connect to the server, we want to enable X11 forwarding and setup a reverse tunnel to allow us to send sounds to the client, we do this like so:

#ssh -X R 4713:127.0.0.1:4713

Start up Pidgin:

#pidgin &

Then change the sound preferences to use “paplay” to play the sounds down the reverse tunnel:

Pidgin Sound Dialog

And there we go, job done.

I like to have Pidgin running on a virtual machine at home to prevent any monitoring of my IM communications when at work, on-site or using an untrusted network.


Mar 21 2010

Hybrid Cryptosystem

Category: CryptographyMike Lovell @ 12:11 pm

When using ‘asymmetric’ (public-private key) encryption, there are a few limitations:

  • Slow Speed - Symmetric (key) encryption is far quicker
  • Single Recipient – You can only encrypt to a single public key

However we can combine ‘asymmetric’ and ‘symmetric’ cryposystems to create a hybrid system which will allow us to harness the speed of ‘symmetric’encryption and allow multiple recipients by simply encrypting the shared ‘symmetric key’ to each public ‘asymmetric key’ we want to use.

So lets create a simple format for a ‘hybrid cryptosystem‘.  We’re going to be using ‘RSA‘ for our ‘asymmetric encryption‘, and AES (Rijndael) for our ‘symmetric encryption‘.


1 Byte    Recepient count: 1-255 recepients allowed
4 Bytes   Recepient 1 data: Size (Int32)
20 Bytes  Recepient 1 data: SHA1 hash of public key
n Bytes   Recepient 1 data: Shared key/password (encrypted to public key)
...
4 Bytes   Recepient n data: Size (Int32)
20 Bytes  Recepient n data: SHA1 hash of public key
n Bytes   Recepient n data: Shared key/password (encrypted to public key)

n Bytes   Encrypted data: Encrypted using AES (Rijndael) and shared key/password

 

Now we have our format defined, we can implement a simple C# application that follows this.  First we’re going to implement a ‘CryptoProvider‘ class.  This is going to handle both ‘asymmetric‘ and ‘symmetric‘ encryption and decryption, key generation, hash generation and random byte generation.  Our key/password is going to be randomly generated and 48 bytes long.  The first 16 will be the ‘IV‘ vector, the last 32 will be the ‘key‘.

   1:  // Author: Mike Lovell (mike.lovell@gotinker.com)
   2:   
   3:  class CryptoProvider
   4:  {
   5:      private    RSACryptoServiceProvider    rsaProvider;
   6:   
   7:   
   8:      public CryptoProvider(int keySize)
   9:      {
  10:          rsaProvider = new RSACryptoServiceProvider(keySize);    // Generate new key
  11:      }
  12:   
  13:   
  14:      public CryptoProvider(string xmlKey)
  15:      {
  16:          rsaProvider = new RSACryptoServiceProvider();
  17:   
  18:          rsaProvider.FromXmlString(xmlKey);    // Import XML Key
  19:      }
  20:   
  21:   
  22:      public string ToXmlString(bool includePrivateKey)
  23:      {
  24:          return rsaProvider.ToXmlString(includePrivateKey);    // Export XML Key
  25:      }
  26:   
  27:   
  28:      public byte[] Hash()
  29:      {
  30:          var sha1 = new SHA1Managed();
  31:   
  32:          // Hash from the modulus
  33:          return sha1.ComputeHash(rsaProvider.ExportParameters(false).Modulus);
  34:      }
  35:   
  36:   
  37:      public byte[] Encrypt(byte[] data)
  38:      {
  39:          return rsaProvider.Encrypt(data, false);
  40:      }
  41:   
  42:   
  43:      public byte[] Decrypt(byte[] data)
  44:      {
  45:          return rsaProvider.Decrypt(data, false);
  46:      }
  47:   
  48:   
  49:      public static byte[] RandomSequence(int size)
  50:      {
  51:          var random        = new byte[size];
  52:          var rngProvider    = new RNGCryptoServiceProvider();
  53:  
  54:          rngProvider.GetBytes(random);
  55:   
  56:          return random;
  57:      }
  58:   
  59:   
  60:      public static byte[] Encrypt(byte[] data, byte[] password)
  61:      {
  62:          var rijndael    = Rijndael.Create();
  63:   
  64:          rijndael.KeySize    = 256;    // Maximum key size
  65:          rijndael.IV            = SubBytes(password, 0, 16);    // First 16 bytes is vector
  66:          rijndael.Key        = TrimBytes(SubBytes(password, 16, password.Length - 16), 32);    // Last 32 is key
  67:          rijndael.Mode        = CipherMode.CBC;
  68:   
  69:          var stream            = new MemoryStream();
  70:          var cryptoStream    = new CryptoStream(stream, rijndael.CreateEncryptor(), CryptoStreamMode.Write);
  71:   
  72:          cryptoStream.Write(data, 0, data.Length);
  73:          cryptoStream.FlushFinalBlock();
  74:   
  75:          cryptoStream.Close();
  76:   
  77:          var encryptedData = stream.ToArray();
  78:   
  79:          stream.Close();
  80:   
  81:          return encryptedData;
  82:      }
  83:   
  84:   
  85:      public static byte[] Decrypt(byte[] data, byte[] password)
  86:      {
  87:          var rijndael    = Rijndael.Create();
  88:   
  89:          rijndael.KeySize    = 256;    // Maximum key size
  90:          rijndael.IV            = SubBytes(password, 0, 16);    // First 16 bytes is vector
  91:          rijndael.Key        = TrimBytes(SubBytes(password, 16, password.Length - 16), 32);    // Last 32 is key
  92:          rijndael.Mode        = CipherMode.CBC;
  93:   
  94:          var stream            = new MemoryStream(data);
  95:          var cryptoStream    = new CryptoStream(stream, rijndael.CreateDecryptor(), CryptoStreamMode.Read);
  96:   
  97:          var    length            = cryptoStream.Read(data, 0, data.Length);
  98:          var    decryptedData    = new byte[length];
  99:   
 100:          stream.Read(decryptedData, 0, length);
 101:   
 102:          cryptoStream.Close();
 103:          stream.Close();
 104:   
 105:          Buffer.BlockCopy(data, 0, decryptedData, 0, length);
 106:   
 107:          return decryptedData;
 108:      }
 109:   
 110:   
 111:      public byte[] DecryptPasswordAndData(byte[] data)
 112:      {
 113:          var    buffer            = new MemoryStream(data);
 114:          var    keyCount        = buffer.ReadByte();    // First byte is key count
 115:          var    thisHash        = Convert.ToBase64String(this.Hash());
 116:          var    password        = (byte[])null;
 117:   
 118:          if (keyCount < 1) throw new Exception("No keys found");
 119:  
 120:          var    keyLengthBuffer    = new byte[4];
 121:  
 122:          for (int i=0; i < keyCount; i++)
 123:          {
 124:              // First 4 bytes are Int32 of encrypted hash/password size
 125:              buffer.Read(keyLengthBuffer, 0, 4);
 126:   
 127:              var    keyLength    = BitConverter.ToInt32(keyLengthBuffer, 0);
 128:              var    keyHash        = new byte[20];    // First 20 bytes is SHA1 hash
 129:              var    keyPassword    = new byte[keyLength - 20];    // Remaining byes are encrypted password
 130:   
 131:              buffer.Read(keyHash, 0, 20);    // Read first 20 bytes (SHA1 Hash)
 132:              buffer.Read(keyPassword, 0, keyPassword.Length);    // Read remaining bytes (password)
 133:   
 134:              if (Convert.ToBase64String(keyHash) == thisHash)
 135:              {
 136:                  password = Decrypt(keyPassword);    // Decrypt password
 137:              }
 138:          }
 139:   
 140:          if (password == null) throw new Exception("Data not encrypted to your key");
 141:   
 142:          // remaining bytes are encrypted data
 143:          var encryptedData     = new byte[buffer.Length - buffer.Position];
 144:   
 145:          // Read remaining bytes
 146:          buffer.Read(encryptedData, 0, encryptedData.Length);
 147:   
 148:          buffer.Close();
 149:   
 150:          return CryptoProvider.Decrypt(encryptedData, password);    // Decrypt data
 151:      }
 152:   
 153:   
 154:      private static byte[] SubBytes(byte[] source, int offset, int length)
 155:      {
 156:          var    subData    = new byte[length - offset];
 157:   
 158:          Buffer.BlockCopy(source, offset, subData, 0, length - offset);
 159:   
 160:          return subData;
 161:      }
 162:   
 163:   
 164:      private static byte[] TrimBytes(byte[] source, int maxLength)
 165:      {
 166:          if (source.Length > maxLength)
 167:          {
 168:              var trimmedData = new byte[maxLength];
 169:   
 170:              Buffer.BlockCopy(source, 0, trimmedData, 0, maxLength);
 171:   
 172:              return trimmedData;
 173:          }
 174:   
 175:          return source;
 176:      }
 177:  }
 178:   

Then lets create a collection of ‘CryptoProvider‘ which will allow us to encrypt our data to multiple public keys.

 179:   
 180:  class CryptoProviderCollection : List<CryptoProvider>
 181:  {
 182:      public byte[] EncryptPasswordAndData(byte[] data)
 183:      {
 184:          if (this.Count < 1 || this.Count > 255) throw new Exception("Too few or too many recepients");
 185:   
 186:          var password    = CryptoProvider.RandomSequence(16 + 32);    // Random password
 187:                                                                      // First 16 = Vector
 188:                                                                      // Last 32 = Key
 189:          var stream        = new MemoryStream();
 190:  
 191:          stream.WriteByte((byte)this.Count); // First byte is recepient count
 192:   
 193:          var encryptedPasswords    =
 194:              (
 195:                  from    item
 196:                  in        this
 197:                  select    CombineBytes
 198:                      (
 199:                          item.Hash(),            // Public key hash
 200:                          item.Encrypt(password)    // Password encrypted for recipient
 201:                      )
 202:              );
 203:   
 204:   
 205:          foreach (var encryptedPassword in encryptedPasswords)
 206:          {    // For each recepient
 207:   
 208:              // Store encrypted password length as Int32 (first 4 bytes)
 209:              stream.Write(BitConverter.GetBytes(encryptedPassword.Length), 0, 4);
 210:              // Store encrypted password
 211:              stream.Write(encryptedPassword, 0,  encryptedPassword.Length);
 212:          }
 213:   
 214:          var encryptedData = CryptoProvider.Encrypt(data, password);    // Encrypt data
 215:   
 216:          stream.Write(encryptedData, 0, encryptedData.Length);        // Store
 217:   
 218:          var encryptedBytes = stream.ToArray();
 219:   
 220:          stream.Close();
 221:   
 222:          return encryptedBytes;
 223:      }
 224:   
 225:   
 226:      private byte[] CombineBytes(byte[] dataA, byte[] dataB)
 227:      {
 228:          var combinedData = new byte[dataA.Length + dataB.Length];
 229:   
 230:          Buffer.BlockCopy(dataA, 0, combinedData, 0, dataA.Length);
 231:          Buffer.BlockCopy(dataB, 0, combinedData, dataA.Length, dataB.Length);
 232:   
 233:          return combinedData;
 234:      }
 235:  }
 236:   

Now lets see if it works.  We’ll create 3 ‘RSA‘ keys then encrypt the data for the first two recipients ONLY, just to see if the 3rd cannot decrypt the data (which should be the case).

 237:   
 238:  class Program
 239:  {
 240:      static void Main(string[] args)
 241:      {
 242:          var    key1    = new CryptoProvider(1024);    // Make a new key
 243:          var    key2    = new CryptoProvider(2048);    // ..
 244:          var    key3    = new CryptoProvider(1024);    // ..
 245:   
 246:          var col    = new CryptoProviderCollection();
 247:  
 248:          col.Add(key1);    // Add only key1 and key2
 249:          col.Add(key2);    // to the collection
 250:   
 251:          // Encrypt data and random password for key1 and key2
 252:          var encryptedData    = col.EncryptPasswordAndData(Encoding.UTF8.GetBytes("Test Encrypted Data"));
 253:  
 254:          // Decrypt using key1
 255:          Console.WriteLine(Encoding.UTF8.GetString(key1.DecryptPasswordAndData(encryptedData)));
 256:          // Decrypt using key2
 257:          Console.WriteLine(Encoding.UTF8.GetString(key2.DecryptPasswordAndData(encryptedData)));
 258:  
 259:          try
 260:          {
 261:              // Decrypt using key3, that we DIDN'T use for encryption, this will fail
 262:              Console.WriteLine(Encoding.UTF8.GetString(key3.DecryptPasswordAndData(encryptedData)));
 263:          }
 264:          catch (Exception e)
 265:          {
 266:              Console.WriteLine("ERROR: {0}",  e.Message.ToString());
 267:          }
 268:   
 269:          Console.ReadLine();
 270:      }
 271:  }

And the result you should see is the following:

Test Encrypted Data
Test Encrypted Data
ERROR: Data not encrypted to your key

You can expand the classes to carry out ‘exception handling‘, and build a robust ‘hybrid cryptosystem‘ from this basic implementation.

Download Visual Studio 2010 Project (8.33k)

Tags: , , , , , , , , , ,


Mar 18 2010

Audible Ping

Category: Networking,SoundMike Lovell @ 8:32 pm

This is either the greatest or most annoying tool ever!  I have to reboot a lot of machines and I always kick of a continuous ping against them to see them go down and come up again.  I thought it would be great if I could make a hospital heart-monitor style beeping ping utility, so I did.  Here it is.

First lets write a ‘ToneHandler‘ class to control our sound.  We’ll give this the capability to either make a single ‘beep‘ or a continous tone (indicating the machine is down).  We’re going to use ‘Console.Beep‘ for this.  We’ll run our continuous tone function in its own thread so we can terminate it when the machine starts pinging again.

   1:  // Author Mike Lovell (mike.lovell@gotinker.com)
   2:   
   3:  class ToneHandler
   4:  {
   5:      private    Thread    constantToneThread;
   6:   
   7:   
   8:      public void SingleTone(int frequency, int duration)
   9:      {
  10:          Console.Beep(frequency, duration);
  11:      }
  12:   
  13:   
  14:      public void ConstantToneBegin()
  15:      {
  16:          if (constantToneThread != null) return;    // If already making tone, exit
  17:   
  18:          constantToneThread    = new Thread(new ThreadStart(delegate()
  19:              {
  20:                  while (true)
  21:                  {    // Infinite loop, we'll kill it later
  22:                      Console.Beep(500, Int32.MaxValue);
  23:                  }
  24:              }));
  25:   
  26:          constantToneThread.IsBackground = true;
  27:          constantToneThread.Start();    // Kick it off
  28:      }
  29:   
  30:   
  31:      public void ConstantToneEnd()
  32:      {
  33:          if (constantToneThread == null) return;    // If already not making tone, exit
  34:   
  35:          Console.Beep(500, 1);    // Send another Beep, this will terminte the other one
  36:   
  37:          constantToneThread.Join(500);    // Terminate thread
  38:   
  39:          constantToneThread = null;
  40:   
  41:          return;
  42:      }
  43:  }
  44:   

Now lets make a little wrapper for the .NET ‘Ping‘ class.  Because we’re going to link the tone and duration of the tone to the round-trip time of the ping, all we want our ping class to return is the time the ping took, or ‘-1‘ if the ping failed.

  45:   
  46:  class PingWrapper
  47:  {
  48:      private    IPAddress    ipAddress;
  49:   
  50:      private    Ping        ping        = new Ping();
  51:      private    PingOptions    pingOptions    = new PingOptions();
  52:   
  53:      private    byte[]        buffer        = new byte[32]
  54:          {    // 32 byte buffer
  55:              0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
  56:              0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
  57:              0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
  58:              0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF
  59:          };
  60:   
  61:      public PingWrapper(string host)
  62:      {
  63:          // Resolve the host once, not at every ping
  64:          ipAddress = Dns.GetHostAddresses(host)[0];
  65:   
  66:          pingOptions.DontFragment = true;
  67:      }
  68:   
  69:   
  70:      public int Send()
  71:      {
  72:          var reply = ping.Send(ipAddress, 1000, buffer, pingOptions);
  73:   
  74:          // Only send a ping every 1 second
  75:          Thread.Sleep(1000 - (int)reply.RoundtripTime);
  76:   
  77:          if (reply.Status == IPStatus.Success)
  78:          {
  79:              return (int)reply.RoundtripTime;
  80:          }
  81:          else
  82:          {
  83:              return -1;
  84:          }
  85:      }
  86:  }
  87:   

Now we’ll take the first argument of the command line and use it as the host we’re going to ping.  We’ll setup a continuous loop in a different thread (so we can terminate it from the main thread) and get it to call our tone class based on the output of ping.  We’ll also display a 1 character status of what happened (success ‘_‘ or failed ‘!‘)

  88:   
  89:  class Program
  90:  {
  91:      static void Main(string[] args)
  92:      {
  93:          if (args.Length != 1)
  94:          {
  95:              Console.WriteLine("Usage: gotinker.audibleping <hostname or IP>");
  96:              return;
  97:          }
  98:   
  99:          var    terminate    = false;
 100:          var    toneHandler = new ToneHandler();
 101:          var    ping        = new PingWrapper(args[0]);
 102:   
 103:          var loopThread    = new Thread(new ThreadStart(delegate()
 104:              {
 105:                  while (true)
 106:                  {
 107:                      if (terminate) break;
 108:   
 109:                      var result = ping.Send();
 110:   
 111:                      if (result == -1)
 112:                      {
 113:                          Console.Write("!");
 114:                          toneHandler.ConstantToneBegin();
 115:                      }
 116:                      else
 117:                      {
 118:                          Console.Write("_");
 119:                          toneHandler.ConstantToneEnd();
 120:                          toneHandler.SingleTone(500 - (result / 4), ((result * 4) + 100));
 121:                      }
 122:                  }
 123:              }));
 124:   
 125:          Console.WriteLine("(Press any enter to exit)");
 126:   
 127:          loopThread.Start();
 128:   
 129:          Console.ReadLine();
 130:   
 131:          terminate = true;    // Our loops checks for this to exit
 132:   
 133:          while (loopThread.ThreadState == ThreadState.Running)
 134:          {
 135:              Thread.Sleep(50);    // Wait fot the thread to terminate
 136:          }
 137:      }
 138:  }

And the result, here’s me pinging something while I reboot it (Download Audio of Demo):

(Press any enter to exit)
________!!!!!!_________!_____________

Download Visual Studio 2010 Project (7.16k)

Tags: , , , , , , , , , , , ,


Mar 16 2010

Handling Unhandled Exceptions

Category: Events,Exception HandlingMike Lovell @ 7:34 pm

If an exception occurs outside a ‘try-catch block‘, it’s nice to have a way of either logging this, or having yourself notified.  The ‘EventLog‘ might be a way to go, but in the case of software running on your end-users PC, you might want to make it easy for them to send ‘unhandled exceptions‘ to you directly.  After all, if it’s complicated they may not have the technical expertise to get the information to you.  Something you might find easy might be beyond the reach of your end-users.

Here’s a quick example of both how to implement a ‘UnhandledException‘ EventHandler, and how to allow your end-user to easily email the information to you.

First lets make a little function that’s designed to fail!  This following function is going to always throw an Exception (divide by zero).

   1:  class Program
   2:  {
   3:      // Author Mike Lovell (mike.lovell@gotinker.com)
   4:   
   5:      private static int OtherFunction(int i)
   6:      {
   7:          var j = 0;
   8:   
   9:          return i / j;    // Divide by zero exception
  10:                          // will happen here
  11:      }
  12:   

Now the handler.  Inside ‘AppDomain.CurrentDomain‘ lives the ‘UnhandledException‘ event handler, this is fired when an exception is thrown (and of course, not handled!).  What we’re going to do is open up the end-users default mail client, with the ‘to‘, ‘subject‘ and ‘body‘ prefilled with the information we need.  In this case, we’re going to take the date, the full path and filename of the executable and the full exception (including stack trace).  Because we want the exception exactly as it would have appeared, and including any non-latin characters (their mail client may not support) we’ve Base64 encoded the UTF8 bytes of our exception string.

We can very easily reverse this process at the other end, to get to the exception.

  13:   
  14:      static void Main(string[] cargs)
  15:      {
  16:          AppDomain.CurrentDomain.UnhandledException += delegate(object sender, UnhandledExceptionEventArgs args)
  17:              {
  18:                  var exception = ((Exception)args.ExceptionObject);
  19:   
  20:                  var mailProcess = Process.Start
  21:                      (
  22:                          String.Format
  23:                              (
  24:                                  "mailto:null@gotinker.com?subject=Unhandled Exception ({0})&body={1}%0A{2}%0A%0A{3}",
  25:                                  HttpUtility.HtmlEncode(exception.Message),
  26:                                  DateTime.UtcNow.ToString("yyyy-MM-dd hh:mm:ss"),
  27:                                  Process.GetCurrentProcess().MainModule.FileName,
  28:                                  Convert.ToBase64String(Encoding.UTF8.GetBytes(exception.ToString())
  29:                              )
  30:                          )
  31:                      );
  32:              };

Now we want to call the function so we actually cause the exception to be thrown

  33:   
  34:          OtherFunction(123);
  35:      }
  36:  }

And the result?  Should look something like this:

GoTinker Exception Handler Email

Exception Handler Email

Download Visual Studio 2010 Project (6.78k)

Tags: , , , , , , ,


Mar 14 2010

Inheriting EventArgs for better events

Category: EventsMike Lovell @ 10:41 pm

In a lot of cases, you need to return some custom data from your events.  Inheriting from ‘EventArgs’ gives you the opportunity to do this.  Here’s an example of how you do it.  Fortunately this time, it’s all self explanatory.

   1:  // Author Mike Lovell (mike.lovell@gotinker.com)
   2:   
   3:  class Firey
   4:  {
   5:      public class MyEventArgs : EventArgs
   6:      {
   7:          public    bool    IsSomethingNice;
   8:          public    int        SomeNumber;
   9:          public    string    SomeText;
  10:      }
  11:   
  12:   
  13:      public    event    EventHandler<MyEventArgs>    MyEvent;
  14:   
  15:   
  16:      public void FireEvent()
  17:      {
  18:          if (MyEvent != null) MyEvent(this, new MyEventArgs()
  19:              {
  20:                  IsSomethingNice    = true,
  21:                  SomeNumber        = 666,
  22:                  SomeText        = "Hello World"
  23:              });
  24:      }
  25:  }
  26:   
  27:   
  28:   
  29:  class Program
  30:  {
  31:      static void Main(string[] pargs)
  32:      {
  33:          var firey    = new Firey();
  34:   
  35:          firey.MyEvent += delegate(object sender, Firey.MyEventArgs args)
  36:              {
  37:                  Console.WriteLine
  38:                      (
  39:                          "Event Fired:nIsSomethingNice: {0}nSomeNumber: {1}nSomeText: {2}",
  40:                          args.IsSomethingNice,
  41:                          args.SomeNumber,
  42:                          args.SomeText
  43:                      );
  44:              };
  45:   
  46:          firey.FireEvent();
  47:   
  48:          Console.ReadLine();
  49:      }
  50:  }

Download Visual Studio 2010 Project (6.35k)


Mar 13 2010

A simple custom thread queue

Category: ThreadingMike Lovell @ 8:05 pm

If you use the built-in ‘ThreadPool’, you lose a lot of control over the thread after you queue up your work.  Here’s an example of a custom thread queue.  You can add more work to it and adjust the maximum threads whilst it’s running, like the built-in ‘ThreadPool’But you can issue a ‘Stop’ and terminate all the currently running threads.  You can expand the classes to add even more functionality but I’ve just covered the basics.

As always, I’m going to handle all my ‘work’ using the ‘Action’ class.  I’ll be using the ‘Queue’ class to hold all my ‘work’ inside, and a ‘List’ to hold my threads, for which I’ll be making a simple wrapper for the ‘Thread’ class called ‘ManagedThread’.

First lets define the ‘ManagedThread’ class.  We’ll tack onto the end of the ‘work’ a new delegate to call a ‘Completed’ event.  So this will code will run after the ‘work’ is completed and fire an event that can be monitored later in our code.

   1:  // Author: Mike Lovell (mike.lovell@gotinker.com)
   2:   
   3:  class ManagedThread
   4:  {
   5:      private    Thread    thread;
   6:      private    Action    work;
   7:      private bool    isRunning = false;
   8:   
   9:   
  10:      public bool IsRunning
  11:      {
  12:          get { return isRunning; }
  13:      }
  14:   
  15:   
  16:      public    event    EventHandler<EventArgs>    Completed;
  17:   
  18:   
  19:      public void AssignWork(Action work)
  20:      {
  21:          this.work = null;    // Clear old work
  22:   
  23:          this.work += work;
  24:      }
  25:   
  26:   
  27:      public void Start()
  28:      {
  29:          if (isRunning) throw new InvalidOperationException("The thread is already started");
  30:   
  31:          work += delegate()    // Add our completed handler to the end of the work
  32:              {
  33:                  if (Completed != null) Completed(this, EventArgs.Empty);    // If there's a completed
  34:                                                                              // handled, call it!
  35:   
  36:                  isRunning = false;
  37:   
  38:                  thread = null;
  39:              };
  40:   
  41:          thread = new Thread(new ThreadStart(work));
  42:   
  43:          isRunning = true;
  44:   
  45:          thread.Start();
  46:      }
  47:   
  48:   
  49:      public void Stop()
  50:      {
  51:          if (!isRunning) return;    // Not running anyway
  52:   
  53:          if (thread.ThreadState != ThreadState.Stopped) thread.Abort();
  54:      }
  55:  }
  56:   

Now lets create a class to hold the threads and handle the work queue.  We’re going to add a ‘Completed’ event to this class also, but this is going to only fire when we’re ‘Completed’ ALL the work.  It can potential fire more than once (it will fire when the queue is empty, and there are no threads running – i.e. all the work assigned has been completed).

  57:   
  58:  class ThreadQueue
  59:  {
  60:      public enum QueueStatus
  61:      {
  62:          Stopped,
  63:          Stopping,
  64:          Started
  65:      }
  66:   
  67:   
  68:      public    event    EventHandler<EventArgs>    Completed;
  69:   
  70:      private            Queue<Action>            workQueue    = new Queue<Action>();
  71:      private            List<ManagedThread>        threads        = new List<ManagedThread>();
  72:   
  73:      public    int         MaxThreads            = 10;
  74:      private    int         runningThreads        =  0;
  75:      private    bool        threadingMoreWork    = false;
  76:      private QueueStatus status                = QueueStatus.Stopped;
  77:   
  78:   
  79:      public QueueStatus Status
  80:      {
  81:          get { return status; }
  82:          set { status = value; }
  83:      }
  84:   
  85:   
  86:      public int RunningThreads
  87:      {
  88:          get { return runningThreads; }
  89:          set { runningThreads = value; }
  90:      }
  91:   
  92:   
  93:      public void AssignWork(Action work)
  94:      {
  95:          lock (workQueue)
  96:          {
  97:              workQueue.Enqueue(work);
  98:          }
  99:      }
 100:   
 101:   
 102:      public int WorkQueueSize()
 103:      {
 104:          return workQueue.Count;
 105:      }
 106:   
 107:   
 108:      public void Start()
 109:      {
 110:          if (status != QueueStatus.Stopped) throw new InvalidOperationException("Cannot start when Status != Stopped");
 111:   
 112:          status = QueueStatus.Started;
 113:   
 114:          ThreadMoreWork();
 115:      }
 116:   
 117:   
 118:      private void ThreadMoreWork()
 119:      {
 120:          threadingMoreWork = true;
 121:   
 122:          while (status == QueueStatus.Started && runningThreads < MaxThreads)
 123:          {    // If there's more to do, thread more work
 124:              lock (workQueue)
 125:              {
 126:                  if (workQueue.Count > 0)
 127:                  {
 128:                      lock (threads)
 129:                      {
 130:                          var thread = (ManagedThread)null;
 131:   
 132:                          if ((thread = AvailableThread()) == null)
 133:                          {    // If there's no ManagedThreads not 'Running', create a new one
 134:                              thread = new ManagedThread();
 135:   
 136:                              thread.Completed += delegate(object sender, EventArgs args)
 137:                                  {    // When the thread is completed, reduce the runningthread counter
 138:                                      runningThreads--;
 139:  
 140:                                      lock (workQueue)
 141:                                      {
 142:                                          if (workQueue.Count == 0 && runningThreads == 0)
 143:                                          {    // All work is done, queue is empty
 144:                                              lock (threads)
 145:                                              {
 146:                                                  threads.Clear();
 147:                                              }
 148:   
 149:                                              status                = QueueStatus.Stopped;
 150:                                              threadingMoreWork    = false;
 151:   
 152:                                              if (Completed != null)Completed(this, EventArgs.Empty);
 153:                                          }
 154:                                      }
 155:   
 156:                                      if (!threadingMoreWork) ThreadMoreWork();    // Thread more work again
 157:                                  };
 158:   
 159:                              threads.Add(thread);
 160:                          }
 161:   
 162:                          thread.AssignWork(workQueue.Dequeue());    // Assign work from the queue to the new thread
 163:   
 164:                          runningThreads++;    // Increase the runningthread counter
 165:   
 166:                          thread.Start();        // Start the thread
 167:                      }
 168:                  }
 169:              }
 170:          }
 171:   
 172:          threadingMoreWork = false;
 173:      }
 174:   
 175:   
 176:      public void Stop()
 177:      {
 178:          status = QueueStatus.Stopping;    // Indicate stopping is in progess
 179:   
 180:          lock (threads)
 181:          {
 182:   
 183:              var active    =
 184:                  (    // Get a list of the active threads
 185:                      from    item
 186:                      in        threads
 187:                      where    item.IsRunning
 188:                      select    item
 189:                  ).ToList();
 190:   
 191:              active.ForEach(delegate(ManagedThread thread)
 192:                  {    // Stop each thread
 193:                      thread.Stop();
 194:                  });
 195:   
 196:              RunningThreads = 0;    // There are no more threads running, so set to 0
 197:   
 198:              threads.Clear();    // Clear out the ManagedThread List
 199:   
 200:              if (Completed != null) Completed(this, EventArgs.Empty);    // If there's a completed
 201:                                                                          // handler call it
 202:          }
 203:      }
 204:   
 205:   
 206:      private ManagedThread AvailableThread()
 207:      {
 208:          var matches =
 209:              (    // Use LINQ to get the threads not running
 210:                  from    item
 211:                  in        threads
 212:                  where    !item.IsRunning
 213:                  select    item
 214:              );
 215:   
 216:          if (matches.Count() == 0) return null;    // if there is none, return a null
 217:   
 218:          return matches.First();    // Otherwise, return the first one in the enumeration
 219:      }
 220:  }
 221:   

As you can see, when new work is assigned, we enter it into our ‘Queue<Action>’ class called ‘workQueue’ .  The function ‘ThreadMoreWork’ is used to start up new threads (if needed).  When we create a new thread and add it into ‘threads’ we hook up the ‘Completed’ event of that thread to call ‘ThreadMoreWork’ (if it’s not already running).  This means as soon as a thread is completed, we’re going to check if we need to start another one – If we do, well, we do!  In our ‘stop’ function, we’re using LINQ to find all the running threads, and using the very handy ‘ForEach’ function to Stop all the individual threads.  Then we fire the ‘Completed’ event.

Now we have a simple Console application to demonstrate the classes.  It’s going to queue up 100 pieces of ‘work’ (all will just print a ‘.’ to Console).  We’ll leave the maximum threads at the default we defined earlier (10).  We also have a timer setup to run every 5 seconds and report the progress.  We’ve hooked up the ‘Completed’ event to print ‘Completed’ to console when we’re all done.

 222:   
 223:  class Program
 224:  {
 225:      static void Main(string[] cargs)
 226:      {
 227:          var threadQueue = new ThreadQueue();
 228:          var checkTimer    = new Timer(delegate
 229:              {    // Poll for updated progress every 5 seconds
 230:                  Console.WriteLine("nThread Information Update! Running={0}, Remaining={1}", threadQueue.RunningThreads, threadQueue.WorkQueueSize());
 231:   
 232:              }, null, TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5));
 233:   
 234:          threadQueue.Completed += delegate(object sender, EventArgs args)
 235:              {
 236:                  // We've finished, stop the checkTimer and notify Console
 237:                  checkTimer.Dispose();
 238:                  Console.WriteLine("nCompleted!");
 239:              };
 240:   
 241:          for (int i=0; i<100; i++)
 242:          {    // Queue up our test work
 243:              threadQueue.AssignWork(delegate()
 244:                  {
 245:                      Console.Write(".");
 246:   
 247:                      Thread.Sleep(new Random().Next(5000));
 248:                  });
 249:          }
 250:   
 251:          Console.WriteLine("(Press enter to stop early)");
 252:   
 253:          threadQueue.Start();    // Start processing
 254:   
 255:          Console.ReadLine();
 256:   
 257:          if (threadQueue.Status != ThreadQueue.QueueStatus.Stopped) threadQueue.Stop();
 258:   
 259:          Console.ReadLine();    // Pause so we can see the output
 260:      }
 261:  }

Run it, and the output should look a little like this…

(Press enter to stop early)
......................
Thread Information Update! Running=10, Remaining=78
................
Thread Information Update! Running=10, Remaining=62
....................
Thread Information Update! Running=10, Remaining=42
....................
Thread Information Update! Running=10, Remaining=22
......................
Thread Information Update! Running=10, Remaining=0
Completed!

Job done.

Download Visual Studio 2010 Project (8.03k)

Tags: , , , , ,


Next Page »