Category Archives: Project

Projects I work on

An interesting experience: Writing to 400 candidates

During the last week, I worked on an interesting project. But instead of programming, this time it involved politics: I wrote a message to 403 german candidates to the European Parliament in the upcoming european elections, on the topic of “digital rights”.

It all started when I heard of WePromise.eu. In a nutshell: Candidates can promise to follow a charter of basic digital rights, supporting laws that strengthen these rights and opposing those that seek to reduce them. The charter contains a lot of very obvious, sensible points, and some less obvious but also very sensible points like export controls for surveillance / censorship equipment. Voters, in return, can promise to vote in the elections, and vote for a party whose candidates support these rights.

Now, my original plan was to write a physical letter to some candidates from my area, but when I asked the people behind WePromise for some material, they also supplied me with a list of all german candidates, including their eMail addresses. And since tools like Mail Merge make sending a lot of personalized eMails very easy, I decided to just write to each and every single candidate from that list. I quickly removed all candidates that had already pledged their support to the project, and all candidates without a known mail address, leaving me with 403 candidates, ranging from people almost guaranteed a spot in the european parliament to people on the 88th spot of a tiny party that may or may not get one or two candidates into the parliament. I quickly wrote up a message detailing the project, the aims, why I support it, and asking them to support it as well (or, alternatively, write me a quick mail detailing why they do not want to support it). I fed the message and the spreadsheet to Mail Merge, waited two minutes, and the mails were sent.

I received a bunch of autoreplies and some error messages concerning incorrect eMail addresses (which I tried to correct and update in my spreadsheet, sending the corrected addresses back to WePromise). Then I waited. That was one week ago.

Until today, I have received replies from over 25 candidates, ranging from the aforementioned 88th spot on the ÖDPs list to current members of the european parliament. The number of german candidates has jumped from 22 to 37, with a few more candidates having promised their signature and not yet appeared on the website. I had some very interesting discussions concerning the advantages and disadvantages of online anonymity, during which I convinced at least one candidate to change his views (two more discussions are ongoing). I also received three replies from parties that are generally not considered to be very pro-internet (all of them stating that they would not sign the pledge, but would “continue to fight for data protection”, and all of them from supporters of data retention laws. I’ll leave you to figure out how the hell that is supposed to work, because I have no idea).

So, some statistics from one week into the project:

  • 15 new signatures (6x Die Grünen, 4x ÖDP, 3x SPD, 2x FDP)
  • 4 signatures pledged that have not yet appeared on the website (2x Linke, 2x SPD, 1x ÖDP)
  • 6 refusals for different, mostly acceptable reasons (2x CSU, 2x AfD, 1x CDU, 1x SPD)

As you can see, a large majority of candidates has never replied. That was to be expected. Still, it has been quite an interesting experience, interacting directly with people that may, in the near future, be called upon to represent my interests in the european parliament.

I can only recommend contacting (some of) your candidates. Ask them their opinion on a cause close to your heart, maybe even have a (civil!) discussion on the matter if they have a different opinion. My experience has shown me that, at least in the smaller parties, you can actually change someones opinion on some matters. And who knows, maybe the candidate will actually be elected to the european parliament. And maybe, just maybe, your discussion will change their vote on a crucial issue… And wouldn’t that be worth the 10 minutes it takes you to write up a mail?

Introducing the SMTP GPG Proxy

I frequently encounter software that allows me to send mails, but has no GPG support out of the box (sometimes not even using plugins). This annoys me greatly, especially if it is software like FusionInvoice, which may transport sensitive information in its mail messages. Since FusionInvoice (and many other programs) support SMTP for sending their mail, and since I had a few spare hours, I decided to see if I could hack something together to add GPG support to those programs. And the result was…

…the SMTP GPG Proxy

The SMTP GPG Proxy, besides having an awful name (name proposals welcome), is a Python program. It provides an SMTP Server and will accept incoming mail messages, encrypt / sign them according to its settings and magic strings in the mail subject, and then forward them to the upstream SMTP server.

Since the basic python smtpd-Module does not support encrypted connections, I used the modified “secure-smtpd”-Module by bcoe. It extends the basic smtpd with support for SSL-encrypted connections while providing an almost identical interface. For the encryption itself, I used the standard “python-gnupg”-wrapper, which isn’t ideal but gets the job done most of the time.

Setup

Setting up the SMTP GPG Proxy is quite easy. Just grab the latest version from the GitHub-Repository, install the dependencies, rename the config.py.example to config.py and fill in the settings (everything should be documented in there), and then launch the main program. Next, point your SMTP-speaking program at the IP and port you just configured (it is highly recommended to do this via localhost only, as incoming connections into the Proxy are, as of right now, not encrypted), and mail away.

Usage

To get the SMTP Proxy to encrypt a message, just send the mail and add the KeyIDs (including the “0x”) to the subject line, seperated by whitespaces. They will be automatically parsed and removed from the subject, so if you want to send a message with the subject “Invoice #23”, encrypted with 0x12345678 and 0x13374242, you would choose the subject “Invoice #23 0x12345678 0x13374242”. KeyIDs can be in short (8 characters) or long (16 characters) form, as well as full fingerprints (without whitespaces and prefixed by “0x”).

Depending on the settings, missing public keys will either lead to the message being rejected, sent unencrypted, or keyservers may be polled before rejecting or sending unencrypted if no public keys are found. You can also configure the program to GPG-sign all messages, or only encrypted messages, or no messages at all.

Development status

The program is currently in alpha, but it works very well for me. Still, as of right now there are some open issues with it, which I may or may not be working on. If you set up everything correctly, you should not encounter any problems. It is the border cases like incorrect SMTP passwords that are currently not dealt with very well.

Roadmap

If I find the time, I will keep developing the program, removing bugs, making it more stable, and adding more features like opportunistic encryption. However, I may not have the time to fully fix everything, and bugs that are annoying me will obviously be fixed faster than those I will never encounter in my usage.

However, as the program is open source and on GitHub, feel free to fork and submit pull requests. The code is, as of right now, shamefully undocumented, but as it has only about 200 lines, it should still be fairly easy to understand.

License

Like almost all my projects, I am releasing this program under the BSD 2-Clause License.

The “Humble Store” Method: Results

So, the Steam Christmas Sale is basically over, let’s see what the results of the Humble Store Method are.

First, some notes. I decided to also apply this method to games purchased on other sites (GoG, the Humble Store (ironically), Humble Bundle and Humble Weekly Sale, as well as any other store where I may buy games). I also decided to follow the method throughout the year, as opposed to only during sales. It shouldn’t make a big difference, as I get almost all of my games during sales, but this makes it easier to follow.

After looking at my purchase history, I am also not sure if 10% are actually enough. I’ll have to think about increasing it to 20% or even 50%, as 10% appearently do not scare me enough.

My purchases were:

  • Bioshock Infinite (7.49 €)
  • 2x Chivalry: Medieval Warfare (5.74 € each)
  • 2x Shogun 2: Total War (7.49 € each)
  • CS:GO (3.49 €)
  • Risk of Rain (5.03 €)
  • Humble Weekly Sale: Puppy Games (6.66 $ => 4.49 €)
  • Stronghold Crusader HD (4.99 $ => 3.77 €)

This leaves us with a grand total of 50.73 €. Damn.

Looking back, I probably should not have purchased Shogun 2 and the Puppy Games Weekly sale. I haven’t played Bioshock infinite yet, Chivalry is great (already sunk 10+ hours into that one), CS:GO is okay (I need more people to play this with), Risk of Rain is RSI-inducingly fun, and, judging from my experience with the original version, Stronghold Crusader HD is probably fun, although I have not played the HD version yet.

So, following the 10% rule, that leaves us with 5.07 €. That’s not very much. So, I’ll just raise the percentage to 25% for this sale, leaving me with 12.68 €. I’ll have to think about the percentage for the next time, but I think 25% is a good first step. I’d set it higher, but I just donated a bunch of money to different organizations at the 30c3 conference and so am a bit short of money at the moment (which is totally not the fault of a lack of restraint on the steam sale, as you can clearly see above. Totally. *ahem*).

So, the only thing left to decide is who will get the money. At the moment, I see a few options:

  1. The CCC, as they did a great congress this year and did not get any donation from me so far (also because they did not accept it when I tried to give it on day 4 of the congress)
  2. Netzpolitik.org, for generally being awesome and informative.
  3. The Tor Project, for generally being awesome and always in need of money
  4. …?

So, if you have any proposals on who should get the money, write it into the comments. I’ll decide over the weekend and update this post with the results.

Update: I decided that the money will go to netzpolitik.org this time around. I’ve been planning to donate to them for a long time (probably more than a year by now) and somehow never got around to it. And since 12.68 € is a terrible number, I’ll just round up a bit. The only thing left to decide is if it will be 13.37 € or just a plain-old 15 €.

Next time, I’ll pick a different organization, but I’ll decide on that once it becomes relevant.

Surviving the Steam Holiday Sale and… doing good? Aka: The “humble store” method

From past experience, I know that I always spend way too much money on the big sales of the Steam Game Distribution platform. I always buy a lot of games that I never play, or only play for half an hour before never starting them again, and I always end up repeating the same mistake on the next sale, because DAMN CS:GO FOR ONLY 3.49 €!

So, since I am obviously unable to learn my lesson, let’s try to at least do some good while buying games I will never play.

The “humble store” method

For those of you who do not know what the humble store is: It is a store for (mostly indie-)games by the guys behind the humble indie bundle. And, the most important part for the purpose of this article, 10% of the price of any game is donated directly to charities like the American Red Cross, the EFF, Child’s Play Charity, World Land Trust, and Charity: Water. They have already raised over $280k using that store, and combined with all the humble bundles they have sold to date, have raised over $29 million for charity.

So, starting with this sale, I pledge to keep track of how much I spend on those sales, and, after the sale has finished, tally it all up and donate (at least) 10% of the money to a charitable organization of my choice. This can be the EFF, but it could also be the wikipedia, Worldbuilders (the charity of my favourite author), Netzpolitik.org (a german blog I read a lot which could use a bit more money) or any other charitable organization.

Now, 10 % isn’t very much. BUT:

  1. I think any charity, having to choose between a small donation and no donation at all, would choose the small donation.
  2. Nothing prohibits me from donating more than I strictly have to.
  3. If 100 people would do the same, it could still easily make a difference of a few hundred euros.

My challenge to you:

Think about where you are spending way too much money. Maybe you are playing a free-to-play game and are constantly buying more virtual currency for it? Maybe you are drinking too much starbucks coffee? Maybe you are reading too many books No, that’s impossible. Whereever you find yourself spending way too much money, think about this: Could the “humble store” method work for this? Would you be willing to turn your vice into something that helps the world a little bit each time you give in to it?

I’m interested in your feedback on this idea. I will try it out for this sale and report back afterwards.

Introducing IssueBot, a Jabber MUC notification bot for GitHub

As part of a development team working on improving Enigmail, I recently found myself in need of a bot sending notifications to a Jabber multi-user chat (MUC) when the Issues of a GitHub-project receive an update (we already had a similar program for new commits in place, called commitbot). I searched for a while and did not find anything that fulfilled our requirements.

So, being a CS student and all that, I decided to write one.

Introducing IssueBot

IssueBot is a Python / Twisted bot, using the GitHub API to fetch information about the issues of a GitHub project. Those are then sent to a Jabber MUC. It can monitor multiple repositories at the same time, authenticate itself using an OAuth token (if you generate one manually) to increase the rate limit on the GitHub API, and will generate Notifications in the following conditions:

  • New ussue
  • Issue closed
  • One of the following has changed about the issue:
    • Title
    • State (open -> closed or vice versa)
    • Assignee
    • New comments

Development is still actively going on, with new features being planned and bugs being fixed, so keep an eye on the GitHub-Repository.

Configuration is done using a .tac-file (an example file is provided with the program). Just update the variables in it and you should be good to go. Instructions on how to use it can be found in the README.

As parts of the code are derived from the aforementioned commitbot, the Code is licensed under the GNU GPLv3. Feel free to fork, improve and send pull requests on the project page on GitHub.

The making of IssueBot

As it turns out, it is actually really easy to query the GitHub API. You send a request to a specific URL (for example, https://api.github.com/repos/octocat/hello-world/issues) and get a response with some JSON and some headers telling you how many requests you have remaining for the current hour (the API limit for unauthenticated users is 60 requests per hour).

Since it is pretty easy to do this in Python, and it has some nice support for JSON built-in using the standard json module, it was pretty easy to query the API, parse the result into a Python dictionary, and parse that into the local database. Then, the changes could be determined and notification messages generated. The only thing missing was the interface to Jabber.

For that, I decided to reuse some code from the commitbot project. This blew up the list of dependencies, but made it possible to somewhat painlessly work with Jabber MUCs. As an added bonus, one of the dependencies, Twisted, deamonizes the process automatically, saving me the trouble.

In the end, it took me about three hours to hack together the current version of the program. Most of the time was spent trying to figure out how to get Twisted to work with my main loop, which was actually non-trivial until I stumbled upon the LoopingCall-Instruction provided by Twisted.internet.task. As Twisted has some rather… interesting views on how it should be used, and my use case did not quite fit into that pattern, I found documentation on how to use it hard to find.

The program is now happily running on my server, spitting out the occasional notification into our chat, and works like a charm.

Drinking games in Vatican: Implementing the SSH RandomArt-Algorithm in Javascript

I recently read an article about the algorithm used to generate the RandomArt-Images used for SSH keys (the “drunken bishop” algorithm). I looked around for a bit and did not find an implementation in Javascript, so I quickly hacked one together.

The algorithm is explained really well in the article I linked above, so I’ll not go over it again. The algorithm is pretty easy to implement, the main parts of this algorithm are:

  • Representing the field the bishop moves on (easy enough with a 2D array of integers)
  • Converting Hex to binary (surprisingly, there is no builtin function for that, but there is a good answer on StackOverflow to that)
  • Isolating the bit-pairs (by iterating through two-character substrings of the words)
  • Checking if moves are valid (by checking if the target coordinates are within the limits of the array)
  • Showing the results (by writing them into a div with a monospaced font set)

So, all in all, it was mostly an exercise in javascript and nothing really fancy, but it was fun to implement it, debug it (too much python, forgetting semicolons all over the place), get the whitespaces to work properly (there are about one million kinds of white spaces, and appearently only one does what I want), and finally testing it.

No effort was made to keep this code simple, efficient, or beautiful to look at. It was a proof of concept and some exercise for my rusty JavaScript skills, mixed with a general interest on how the RandomArt algorithm works.

You can take a look at the code in action at jsfiddle, or just look at the horrible code in the gist on github and make fun of me in the comments.

Results from the inofficial Enigmail “Night of Code”

Yesterday, we had a small “Night of Code” in Hamburg. Basically, five hackers met up in the rooms of the CCC Hamburg and tried to improve Enigmail, the Thunderbird extension for PGP-encrypting Mails. It was a hell of a lot of fun, and we actually made quite a bit of progress on several improvements.

It all started with a discussion on the mailing list of the computer science department of the University of Hamburg. We had a lengthy discussion on what is wrong with Enigmail and PGP, and some of us decided to do something about it. Someone organized a room, called for a “Night of Code”, and a few people responded, me among them.

We started with a short introduction on the architecture of Enigmail and what the important files are. Afterwards, we discussed what needed improvements (the consensus being “basically everything about the UI”) and everyone chose one of the proposed improvements and started working.

I don’t want to spoil the surprise on what the others have been working (although all of it will come in pretty handy, once it is finished and hopefully merged into the main project), but I can say a bit about what I worked on.

So, one of the important things when using PGP is to manage your Web of Trust. This includes the signing of the keys of other people (after you validated that they are, in fact, the person they are saying they are). For that purpose, there are Key-signing parties. And one of the major annoyances about those parties is the distribution of freshly signed keys.

On Linux, there is a neat command line tool called caff. It takes any number of Key-IDs, downloads the public keys, signs each ID seperately and mails it (encrypted) to the provided eMail address. The problem is that caff is pretty annoying to set up, and only works on Linux.

The Feature I am working on is something along those lines. I added a new checkbox on sigining keys…

The second checkbox is new, in case you were wondering.
The second checkbox is new, in case you were wondering. And the keys are totally legit, I checked. 😉

If you select the checkbox and sign a key (and no error occurs during the signing process), a new Message composition window will open:

The new Message

It will contain some sort of preset text and have the signed public key attached.

Now, this is all working great already, but there are still some things to do:

  • Save the last decision on whether to mail the key or not (currently, due to some weird behaviour of the Enigmail preferences function that I still need to figure out, it is not saved)
  • Automatically set the mail to be encrypted and signed, regardless of the settings.
  • Perhaps encrypt the public key before attaching it, to make sure the recipient needs his private key to get the new signatures?
  • Perhaps choose the sending account based on the private key that was used to sign the public key?

Now, the experience to work on Enigmail has been interesting and somewhat cool, but not without its problems.

  • To say the documentation of Enigmail is bad would be misleading, as it implies that there actually is a documentation, which is not the case. Everything you want to use, you need to figure out yourself, possibly by using the addon with debug output active and seeing which functions are used in what order.
  • Thunderbird isn’t much better. Many of the important functions (adding an attachment!) had to be reverse engineered from other addons or from the very helpful thunderbird-stdlib-Project on GitHub, as the documentation has some pretty big holes in significant places.

If you are an Enigmail dev and reading this: Please provide at least some documentation on what is done where in the code, and what APIs can be used for new features. I know you probably understand the code, but it makes the entry barrier for new devs very high.

If you are a Thunderbird dev: See above. The current docs are not enough, and the function names are in parts weird enough to make it almost impossible to find out how to actually use them without checking the source files, which takes time and is extremely annoying.

All in all, I enjoyed my time hacking on Enigmail. But it could have been a lot more productive if there was some form of documentation one could use. As for the new feature: I will try to get it to work properly and then submit a patch to the devs, but I do not know how long that will take, as my time is currently pretty limited because of other things I need to take care of (my bachelors thesis among them).

As for the others: I don’t know when their features will be finished, but we already have a bunch of ideas on what to do next, and if we find the time, we’ll create some more new features. Some of our ideas have the potential to vastly increase usability, so I am very curious as to the reactions of the devs. Let’s hope for the best.

Coding Project of the day: seinfeld.sh

Today, I was looking for a command line program that implements the seinfeld calendar / seinfeld chain. As it turns out, there are a lot of programs on GitHub that do something like that, but none that did exactly what I was looking for (actually, that’s wrong. One Project did pretty much exactly what I wanted, but it refused to install).

Well, building a simple seinfeld chain tracker is not hard, so I quickly hacked together a small shell script that did exactly that. Introducing seinfeld.sh.

Now, the script is pretty simple, as I said, and probably has a bunch of bugs as well (and, frankly, some horrible code). At the moment, it does not even automatically detect new days, so I helped myself by copying the script on my server, putting the text file the script used into my dropbox, and triggering the “newday” routine via a cronjob on the server every day at midnight.

Current Features of the script:

  • seinfeld.sh do [project]: Mark a project as “done” for the day (represented by a “#”)
  • seinfeld.sh undo [project]: Mark a project as “not done” for the day (represented by a “-“)
  • seinfeld.sh newday: Add a new day to every project (represented by appending a “-“)
  • seinfeld.sh ls: Show the current chains
  • The script will ignore lines beginning with a “#”, so use that for comments
  • Update May 15th: Chain links are now color coded (green for “done”, yellow for “not done”)
  • Please read the instruction on the GitHub-Page

Feel free to clone the code, make improvements and send a pull request.

Find this Project on GitHub

Coding Project of the Day: attmail

(I have no idea if there are any projects out there that are named attmail. No copyright violations intended, it’s just my woking title 😉 )

Today, I had to get a backup script running on the server of my company. One part of the script should send an eMail with the backup attached every week.

Easy thing, right? I’ve got SSH access, so I can use just about anything a Linux Server has to offer.

Yeah, as it turns out, sending mails with attachments is actually not that easy if you can’t install your own stuff on the machine. The only (working) eMail solution available was Sendmail, and for that, you need to basically provide the whole eMail, encoded and everything. And, to make matters even worse, I had no access to uuencode. At least, I had Base64, so I knew that, somehow, it had to be possible.

Luckily, I found this thread, explaining how to send Base64-encoded attachments via sendmail. Bingo.

Being the computer science student that I am, I was not satisfied with the script (I am never satisfied with scripts that have everything fixed. I like my parameters). So, I set out to rework the script to make it more dynamic.

Now, about 3 hours later, I finally have a working solution (and have learned a lot about bash scripting). I know that any experienced Linux guy will probably claw his eyes out on seeing this code, but I’ll post it nontheless.

The Code on Pastebin (including usage instructions)

This Project on GitHub, in the unlikely case you want to fork and modify it.

It should be obvious that the code requires a working installation of sendmail to work correctly.

As always, feel free to comment with suggestions on how to improve the code, report any bugs (Untested usage scenarios include full path to attachment file (should work, I think), Filenames with Spaces or special characters except for .-_, and more). The code works for the scenario I intended it for, and that’s all I wanted. If it helps you, even better, but I will not offer too much in the way of support, due to time constraints and other problems.

Thanks, and have a nice day 😉