Ill Logic Tech

Haskell - Can your language's type system do this?

2011-07-16T20:42:00.036-05:00

Haskell is known for its very strict, and yet very versatile type system. There's so much to learn about it, and I'm barely scratching the surface myself. But I've thought of a simple example of something impressive that I think people who know nothing about the language could still appreciate, and hopefully it will convince you to check it out.

Note: There is something in Haskell called "Type level programming". I'll leave you to look it up and decide whether what I'm doing here is an example of this or not. I never figured that out. but what I'm about to show you is at very least something you probably can't do in your language, unless your language is even cooler than Haskell.

Let's start with this simple program:

data Purse a b c = Purse a b c

data Keys = Keys

data Phone = BlackBerry | IPhone | Android

data Wallet = LeatherWallet Float | MoneyClip Float

addKeys (Purse ( ) b c) newkeys = Purse newkeys b c

addPhone (Purse a ( ) c ) newphone = Purse a newphone c

addWallet (Purse a b ( )) newwallet = Purse a b newwallet

emptyPurse = Purse ( ) ( ) ( )

leaveHouse :: (Purse Keys Phone Wallet) -> ( )

leaveHouse purse = ( )

In short, this program makes sure that you add your keys, a phone, and a wallet to your purse before you leave the house. However, unlike most languages, your compiler will make you do this, rather than the runtime.

So first, let me decipher this thing for you.

data Purse a b c = Purse a b c

This creates a data type called Purse. It is a parametric data type, meaning that it takes other types as parameters, in this case 3 different ones. This is very similar to Java or C++, with something like Vector< int >. a, b, and c can be anything, at least for now. On the right side of the equal sign, we describe what it requires to construct an object of type Purse. This also takes three parameters, however these parameters are values, not types, but the values must be of types a, b and c respectively. (Note that in Haskell, it's fairly common for the type, and the value of the type, to have the same name. It might seem confusing, but if you understand the context they're each used in, it's unambiguous which is meant. So here, on the left, Purse is the type, but on the right, Purse a b c represents the value, or more properly one would say that on the right Purse is the type constructor)

This is basically like a struct in C. So in all, we have something like a templated struct in C++, with one data member of each parametric type. For example, based on this declaration, you could create a variable of type Purse Int Float Int that had the value Purse 1 2.3 4 or Purse 2 5.7 8. You could also (in the same program) create a variable of type Purse Int Char Char, and it could hold the value Purse 5 'a' '%' .

data Keys = Keys

data Phone = BlackBerry | IPhone | Android

data Wallet = LeatherWallet Float | MoneyClip Float

Phone and Wallet are also datatypes. They are not parametric, however they have a different feature. The variables of these types can each take on multiple forms. A Phone variable can either have the value BlackBerry, IPhone, or Android. Very similar to an enum in C, they basically are constants, and variables of type Phone are restricted to them. Wallet however, is a bit different. In addition to having multiple forms, it also has a Float parameter. If you remember Purse variables had parameters of unspecified types, however in this case we specify that it must be a Float. Keys is simplest of all, it can only have one value, Keys, with no parameters.

addKeys (Purse ( ) b c) newkeys = Purse newkeys b c

This is a function definition. It takes two parameters. Similar to def addKeys(purse, newkeys): in Python. However, Haskell uses something called pattern matching. You can find this in Erlang, and I'm sure a good handful of other languages as well. What this generally does is allow you to define a function multiple times, for different input, as long as every definition has the same type signature. However in this case we're only doing one, because we just want to use it to (partially) define the type signature. The first parameter of the Purse object has to be ( ), which is of type ( ). This is in some ways like None or NULL in other languages, however it's its own type, with only one value. Variables a and b can be any value of any type. What it returns is a new Purse, with newkeys as its first parameter, rather than ( ). The other parameters depend on what's passed in.This also means that the new Purse has a different type than the first one: Purse Keys b c, rather than Purse ( ) b c

Similar in Python would be:

def addKeys(purse, newkeys):

if purse.a == None:

newpurse = copy(purse)

newpurse.a = newkeys

return newpurse

The difference being that the Haskell version restricts the type of purse, the consequences of which we'll see in a bit.

addPhone (Purse a ( ) c ) newphone = Purse a newphone c

addWallet (Purse a b ( )) newwallet = Purse a b newwallet

Exactly the same as addKeys, except that ( ) are in different slots, appropriate to where this new phone or wallet should go. Again, note that the type of purse returned is different than then one entered, and that the type entered is partially parametric. For instance, for addPhone, c can either be of type Wallet or of type ( ) . Actually, again it could really be of any type, but if the purse was created with the functions defined in this program, it would be one of those.

emptyPurse = Purse ( ) ( ) ( )

This is defining a value called emptyPurse. It is a Purse of type Purse ( ) ( ) ( ) and of value Purse ( ) ( ) ( ) .

leaveHouse :: (Purse Keys Phone Wallet) -> ( )

leaveHouse purse = ( )

Finally, we have a function with an explicit type signature. Until now we've been taking advantage of type inference. However, here we want to force leaveHouse to have a specific type signature. The one argument to this function is of type Purse Keys Phone Wallet. Basically, any wallet with everything in it. No ( ). The return value of this function happens to be ( ) , but this is only because this is for demonstration. For our purposes, we don't care what it returns.

Ok cool, so what's the point? Well let's fire up the interpreter, ghci:

Prelude> :l purse.hs

[1 of 1] Compiling Main ( purse.hs, interpreted )

Ok, modules loaded: Main.

*Main> let p1 = emptyPurse

*Main> let p2 = addKeys p1 Keys

*Main> let p3 = addWallet p2 (LeatherWallet 50.24)

*Main> let p4 = addPhone p3 BlackBerry

*Main> leaveHouse p4

( )

Cool, we've left the house successfully. Let's inspect the types of each purse:

*Main> :t p1

p1 :: Purse ( ) ( ) ( )

*Main> :t p2

p2 :: Purse Keys ( ) ( )

*Main> :t p3

p3 :: Purse Keys ( ) Wallet

*Main> :t p4

p4 :: Purse Keys Phone Wallet

Remember, these aren't the values, these are the types.

Now let's restart out interpreter, and try to skip a step:

Prelude> :l purse.hs

[1 of 1] Compiling Main ( purse.hs, interpreted )

Ok, modules loaded: Main.

*Main> let p1 = emptyPurse

*Main> let p2 = addWallet p1 (MoneyClip 5.20)

*Main> let p3 = addKeys p2 Keys

*Main> leaveHouse p3

<interactive>:1:11:

Couldn't match expected type `Phone' against inferred type `()'

Expected type: Purse Keys Phone Wallet

Inferred type: Purse Keys () Wallet

In the first argument of `leaveHouse', namely `p3'

In the expression: leaveHouse p3

*Main> p4 = addKeys p3 Keys

We now have an error, because it was expecting a purse without ( ) anywhere in its type parameters. Remember, it's not complaining about a bad value, this pass doesn't know about the value, because this is a compiler error. It's complaining about the type. This doesn't look too different from something you might set up in Python, but remember, with Haskell, if you put these lines into a program, you would get these errors without having to run it (you wouldn't even be able to).

Now for fun, with the same interpreter session, let's try making a new purse, adding the keys to p3 a second time:

*Main> let p4 = addKeys p3 Keys

<interactive>:1:17:

Couldn't match expected type `()' against inferred type `Keys'

Expected type: Purse () t t1

Inferred type: Purse Keys () Wallet

In the first argument of `addKeys', namely `p3'

In the expression: addKeys p3 Keys

It's expecting a Purse with the first parameter of type ( ). Any Purse that has addKeys in its "history" will not have this in its type, it will be of type Keys, so the compiler will not let you call addKeys again.

Now, remember one particularly interesting thing here: when desiging the addKeys, addWallet, and addPhone functions, we did not know exactly what type of Purse would be going into them. We just set one restriction, that one particular slot had to be empty. The actual types of purse going into the functions were realized by what called the functions.

Why would you want to do something like this? Supposing you were creating a library function to call on some sort of API. You want to allow for multiple function calls to initialize some sort of object before kicking it off to the library. In a language like C or Python, failure to set some sort of initialization variable would lead to a runtime error you have to decipher. Here, the compiler won't let you run the program until you to set all of the necessary parameters, and you can set them in any order*. There's a benefit to having things fail before you even have a chance to run them. It's a pain to get working, but once it does, your program will have far fewer errors.

This trick I've described to you, however confusing I made it sound, is based on the very basics of the Haskell type system. There's a whole crazy world out there. It's complicated, but once you get to know it, you can force yourself into some pretty stable programs.

* Actually, the way I have it set up, while you can run these functions in any order, it (for better or for worse) forces you to call the functions in the same order every time you use them, if you ever use it more than once in the same program. However, using more advanced features (typeclasses) I think it would be possible to overcome this, though I've not tried it.

Lower the Barrier for Scratching Open Source Itches!

2011-04-29T11:53:00.008-05:00

Ok, this is an idea I've had for some time now, I think it's time to put it out there and see what a large audience thinks. I'm a bit ignorant about how distributions are set up, maybe it's just impractical for some reason I'm unaware of.

I think there should be an easy way for users of a distro to jump right into development. This stuff is done for free, we need all the help we can get. It would help to eliminate any barriers to entry we can. My proposal is that it should be integrated into the package manager.

Here is the process I envision: I, the user with coding skills, have an itch with Program X. I issue one command, let's say "sudo apt-get --collab programx". Here's what it does:

It automatically creates a fork of the project on my account on Github (or equivalent)
It pulls, via Git (or equivalent), the very version of Program X that I am running currently. Now, this is important because I don't want to worry about different behavior in the program, I don't want to deal with a newer version of the program requiring different versions of libraries. I want the same thing I just ran, with the same bug, and I want the source code that generates it.

The build environment is all set up. I don't want to hunt for build dependencies, compiler options, etc. Enough said. "apt-get -b" Seems to do most of what I described thus far, minus the crutial Git part.

I fix my problem. I make my changes, commit, and push. It shows up on upstream's Github fork queue (or equivalent). They decide whether to accept it.

Github has already done a great part of this, compared to a few years ago, by lowering the barrier to entry with the fork queue. Install via source already exists in apt. Would it be a huge task to coordinate the two?

I think that I would probably have scratched a few itches by now if it were this straightforward. Instead, I have to look up the specific build setup for the project (on the project's site, not Ubuntu's site), figure out build dependencies, etc. Or, I can do apt-get -b, but then it's not ready to commit my changes back (afaik).

The limit of my patience, and free time, is reached much earlier in the process as things currently stand. Remember, this is for people who are perhaps a few levels less involved than the sort of user who would run the bleeding edge Ubuntu Beta. This is a regular user with some coding skills, who might be able to fix a problem or two if the setup were handed to them. They have a different mentality. This is about getting a new class of developers involved.

Again, I'm ignorant about the details of package management and open source project management, so I'm probably leaving holes in this idea that I don't know about. I'm just a developer with an idea. The question is, can these holes be ironed out, or does this have a fundamental problem because of package management, as it stands today?

Or does this already exist and I just never heard of it? (in which case, it should just be promoted more!)

I found the perfect project to dive into Haskell

2011-01-29T23:16:00.003-06:00

I've had a recurring project in my life, a modular software synth. See here to get an idea of what I'm doing. The difference being that with software synths, you're not limited to how many components you have and how they're configured. I somehow thought I came to this revelation on my own years ago, but there's plenty of this sort of thing out there, such as SuperCollider, which sounds like it's fairly popular.

I've been trying to get into Haskell, but have been struggling to get myself out of my Python comfort zone. A friend of mine actually told me about SuperCollider recently, and I realized that my own synth would be the perfect project to get me started on Haskell, and one day last week I got inspired to get started. I found a simple example to start with of a sine wave being played through Pulse Audio and I went from there. Here's my repo

Unfortunately, you need PulseAudio to run this. I'm working on either getting Alsa output or file generation working soon.

When I was making this a few times before, I made it in C++, the latest instance being several years ago. Amazingly enough, despite still being a novice in the language, I found that doing this in Haskell is easier. The infinite lazy lists work perfectly as signals. Before I considered each component to be an object that had a value, and input signals. And I had to have a global "tick" that conveyed info between items. (And I thought that was really neat at the time.) Now I just have components be functions that "output" (return) infinite lists, and take infinite lists as inputs. It all sortof just sorts itself out.

Speed is sacrificed to be sure, at least so far, but real-time synths have been made for Haskell, so I bet I can profile it and optimize it significantly.

Also you will notice that everything is hard coded! I sortof like it that way, it's amusing, particularly when it starts making beat sequences (which is a point I got to in my old version), but I'll probably make an interface at some point. Or maybe not, Haskell is a nice interface.

Here's why I'm writing now of all times though. On top of being functional with the lazy infinite lists and such, Haskell also has a type system from Nazi Germany. This is actually an advantage, though. I have some trouble remembering all the unit conversions involved in the oscillators, when I'm dealing with cycles, seconds, and samples. So today, I made a type framework that provided functions that did the conversions properly. When I was writing out an improved version of my oscillator function, I used these types.

It took me a long time to figure out exactly how I wanted it all to work, and how to make it work. I would start on an expression, and then realize that I was adding different units, and Haskell wouldn't let me do it. Or sometimes the compiler told me so. But eventually I got through it. This is the monstrosity that resulted.

And the kicker: I used this to make a new version of the square wave oscillator, and it sounded exactly the same as the old one, the first time I ran it.

Testing Multiple Login Sessions Simultaneously

2011-01-02T17:45:00.006-06:00

One annoyance in developing websites is that you sometimes have to log in and out all the time to test interaction between multiple users.

Have you ever visited or administered a website (say, www.example.com) which lets you visit "www.example.com" or "www2.example.com", etc, and doesn't forward to "example.com"? Did you ever try logging in at one subdomain, and then switch to another? You'll be logged out, it's a different login session. If you needed to test something remotely with multiple users logging in at once, that's a nice trick to use.

Now let's do the same thing locally (*nix systems only afaik, sorry):

In /etc/hosts you should see:

127.0.0.1 localhost

Add the following:

127.0.0.1 localhost2
127.0.0.1 localhost3
127.0.0.1 localhost4

And so on for however many you need. Now each one will access your site with a different session, so you can log in as a different user for each.

Cryptonomicon: A Lesson for my Hyper-Logical Friends

2010-12-18T23:15:00.028-06:00

I'm currently reading Cryptonomicon by Neil Stephenson. I'm not very acquainted with literature at large, so forgive me if I'm being ignorant here, but it seems that this book is unique or among very few that are in wide release and yet somewhat esoteric. That is to say, anybody can appreciate it, but I think it speaks specifically to computer programmers and mathematicians, and may not be 100% understood by those who are unfamiliar with certain mathematical and engineering concepts, and who don't share that mentality. Then again, the purpose could be to provide some insight to outsiders who may want to understand the hyper-logical nerd mentality. Tom Wolfe seems to do a similar thing, for instance, with the investment bankers in Bonfire of the Vanities.

Though I think Neil Stephenson must have a closer personal connection with this mentality. It's a great book for a nerd because it's literature we can really relate to. It's told from the perspective of those of us who try to make logical sense of everything, see patterns all around us, and are confused by strange things like social niceties.

All in all I think it teaches an important lesson to nerds and non-nerds alike. I only just now crossed the 1/3 way mark (it's like 1100 pages), but I just came across some particular dialog which I think is particularly insightful. In this scene, Randy Waterhouse pulls Eberhard Föhr aside during a business meeting, and explains to him why, for their own legal protection, information has been withheld from them by one of their business partners, Avi. Ebehard, being of this nerd mindset, is frustrated that his business partners are not behaving logically. Randy, being of the same mindset but somewhat more enlightened, explains to Ebehard the realities of dealing with illogical people, but he does so in logical terms that Ebehard can relate to. This conversation is amusing like a lot of things in this book, because it demonstrates how us analytical types like to deconstruct everything.

Rather than risk inviting Neil Stephenson's lawyers (I have no idea how likely a scenario this is be but I don't care to do the research right now) I'll just invite you to read this page via Google Books.

I appreciate a couple things about this passage. Firstly, I appreciate that Randy's character is sort of an enlightened techie, who we should aspire to, who respects the qualities of other sorts of people, even if he doesn't understand their mentality. Business people clueless about technology, idealistic designers with a vision, techies who can't design a usable interface to save their life, we should all accept our own limitations of understanding, respect the others, and occasionally yield our own ideals for the sake of other ones. (ex: if "doing it right" means taking twice as long, and failing in the market, what use is your ideally laid out code if nobody's going to use it?)

The other thing I like about this passage is, as I mentioned above, the logical way that it approaches illogical people. Some nerds have a tendency to refuse to approach the world in anything other than a logical manner. Normal People may try to explain to them that the world, particularly other individuals, aren't rational at all, and we should stop seeing things so logically. I include myself in this group of nerds, so honestly, this line of argument is ridiculous to me. The universe is logical. But, I think that sometimes we as nerds are just Doing It Wrong, and we can take a cue from Randy here.

What we need to do is to appreciate that the fact that people act irrationally, out of emotion, is just a condition of the world. Just as we accept that animals are irrational, or that the sun is hot. It's a datum. Further, accept that you yourself, the nerd, are also emotional, particularly when people don't act logically. This frustration with others' illogical behavior is based on an expectation for people to act contrary to their nature. You're ignoring a data point. You're mad at the sun for being hot. You're a non-techie who's mad at your computer for doing something other than exactly what you told it to. Now look who is being irrational? I'm going to agitate a little and propose that we are in fact being hypocritical here.

The main problem I think we sometimes have is the distinction between Logic and Logical Faculties. The expectation of perfection in Logic is not the same as expecting a human to have perfect Logical Faculties. The universe works by rational laws. People are part of the universe, so their workings are rationally explainable. But this is entirely distinct from their Logical Faculties being able to perfectly model the world around them. Furthermore, people's Logical Faculties being able to model the world around them is distinct from their ability to defend it from any of their Emotional Faculties getting in the way. We humans are but animals who happen to possess a limited amount of logical faculties.

Expecting people to act in a rational straightforward manner is like expecting a computer to compute beyond its capacity. A problem may be Logically solvable. There is a perfect Logical progression toward the answer. If we treated computers the same way we sometimes treat other humans, we would demand that we should be able to stick the problem into a computer and get an instant output. But again, Logical Faculties are in limited supply. Somehow we don't seem to have a problem accepting this in computers. In fact, we have entire sub-fields of computer science, taking RAM, HD, and time limitations as data, and creating a whole new set of Logical problems. Why not accept the same limitations and challenges in humans?

Perhaps it's that there is one fundamental difference between computers and humans, which is that our departure from being perfect logic solvers is not just in our processing capabilities, but also, as Randy pointed out in the passage linked above, in our interfaces. Human interfaces are more like neural networks than serial connections. To gain access to the Logical Faculties, one must enter a pattern that is accepted by the neural network. The patterns include such things as social niceties and innuendo. Some of us have simpler interfaces than others. (And as Randy described, some may even require other humans to act as intermediate interfaces. When I worked at Oracle, there was a guy who was fluent in both Engineer and Customer, and intermediated all conversation. I understand this is a common thing to have in a company.)

And you, the nerd, are a neural network, at your core, not a Turing machine. You operate in that domain. That means you have the natural ability, however impaired by years sitting in front of the computer, to interface with other neural networks, if you would just accept your nature. This is in fact the only way you can communicate with other humans, so you might as well accept it for what it is. You may try to approximate a Turing machine, but your neural network nature will still show on occasion. For instance, as I pointed out above, when you are frustrated about others not behaving like Turing machines.

allCombinations: leaveOut

2010-11-15T00:48:00.004-06:00

leaveOut

Ok, on my previous post I brought up a small Python module I was inspired to throw together while writing tests. It's still sitting in a gist, though I'll probably move it to a real repo before too long:

https://gist.github.com/674715

So I admit, as I was posting it, it occurred to me that to a large extent this stuff could be replaced with a nested for loop. For instance, this:

for lst in allCombinations([1, 2, oneOf(3,4), oneOf(5,6)]):

can be pulled off with:

for x in (3, 4):
for y in (5, 6):
   lst = [1, 2, x, y]

Not a huge gain necessarily on my part. So as I was using it in my testing I realized I once again had engineered something for a tiny use that, neat as it is, could have been done much faster by brute force. But then, I realized another thing I could add that would make my code much more concise. I've added another keyword called "leaveOut". It lets you opt to not have the element show up at all. Here's an example:

allCombinations([1,2, oneOf(3, leaveOut), oneOf(4, leaveOut)])

This will return:

[ [1, 2, 3, 4], [1, 2, 3], [1, 2, 4], [1, 2] ]

And of course, the "leaveOut" case will omit dictionary entries and object data members as well.

BTW

I should also mention another use case I thought of, "leaveOut" aside, that might be a real pain to do without an aide such as allCombinations, which is dynamically created structures, with an arbitrary amount of variables:

allCombinations( [ oneOf(1, 2) ] * x )

I've just generated all possible lists of either 1 or 2, of an arbitrary length, which can be set at runtime. Or how about something a bit more fun:

allCombinations( [ oneOf( *range(y) + [leaveOut] ) for y in range(x) ] )

Taking all combinations of lists of length x, where each element can equal any integer from zero to its index, and then adding combinations where items are omitted. Not horribly useful, but complicated.

To do these in a standard way you'd need x for loops, which you can't do directly. (I bet you could do it with recursion).

Fixes

I'll also mentione that I fixed a couple general errors. oneOf on Data members had a big bug. And now if you don't have oneOf in your structure, allCombinations just returns a list containing only the original structure, instead of looping to death.

allcombinations - generating combinations of python structures

2010-11-12T15:34:00.005-06:00

Alright, on a whim I decided to make another tricky thing in Python. This one is less of a hack, and is more likely to be useful.

So let's say you want to do something with all combinations of... something.

[5, 6, oneOf(7,8,9), oneOf(10, 11, "shazaam")]

So you want to turn this structure into all the possibilities represented within:

[
[5, 6, 7, 10],
[5, 6, 7, 11],
[5, 6, 7, "shazaam"],
[5, 6, 8, 10],
[5, 6, 8, 11],
[5, 6, 8, "shazaam"],
[5, 6, 9, 10],
[5, 6, 9, 11],
[5, 6, 9, "shazaam"],
]

Well with allcombinations, you can do just that:

from allcombinations import allCombinations

allCombinations( [5, 6, oneOf(7,8,9), oneOf(10, 11, 12)] )

Here it is. The gist includes more complicated example.

Features
The oneOf should be able to reside almost anywhere in your expression. It can be in a list (as seen here), in a dict, or even in the attribute of an object. It can also reside in a list within a dict within an object's attribute, etc, as long as it's nowhere within an unsupported container type.

Limitations:
This will only work if oneOf resides in a list, dict, or an object's attributes. It shouldn't work anywhere within a set, or any other structure I can't think of. If you try it in something unsupported, the oneOf object should just stick around in all your combinations.

I'm probably actually going to use this, particularly (again) for testing. Anyone else think they'd find it useful? Should I package it?

Testing (Django) views with pyquery

2010-11-12T09:39:00.008-06:00

PyQuery is basically what it sounds like. Using jQuery syntax, you can query and even manipulate XML files. Obviously we don't (yet!) have Python in the browser, so it's not useful in the same domain, but it can help out in dealing with XML in general, in the same way as, say, lxml, but without having to learn about things like ElementTree for simple cases. It's particularly good for XHTML because jQuery (and thus PyQuery) uses CSS syntax for class= and id=. Which brings me to how I'm using it:


from django.test.client import Client
from pyquery import PyQuery
from django.test.testcases import TestCase

...

class TestSomeViews(TestCase):

   def testAView(self):

       client = Client()

       ...

       response = client.get("/someurl/")

       self.assertTrue("expected text" in PyQuery(response.content)("#someid").html() )

(If you're unfamiliar with testing Django views, see this.)

For some basic tests, you can just search the entire response html, and not have to worry about where it shows up. But suppose you're searching for a username in a particular part of your response. You're pretty likely to find that username elsewhere on the page, so you have to select out the part of the file you expect it. I think this is much easier than using a regex.

So what this bit of PyQuery does is find the tag with the id of "someid" (presumably there's only only one, being an id), and returns the html within that tag. (If you search for a class that returns multiple tags, it seems that a simple call to .html() will only return the contents of the first one. This very well may match jQuery's behavior, I'm admittedly not that familiar, but just a head's up.) For more details look at the PyQuery API.

Django Model Validation

2010-10-27T00:52:00.004-05:00

I'm really excited about model validation, in Django circa 1.2. It's going to save me from the disastrous hack of ModelForms I've used so far. One wants something to validate data before saving it, since validating it by saving it is apparently a Bad Idea, since you may have already made changes to the database by the time the error is thrown.

I wanted this done automatically, and Django only had form validation until 1.2, so I hacked forms into some sort of all-purpose wrapper for models. I've since learned to program Django like an adult, and model validation came around just in time anyway. But there's a problem I have with it.

First I should point out that there's a few different things that get validated, but the two I'll point out are A) checking that required fields are set and B) whatever I put into the clean() function. Judging by some errors I've gotten while debugging/testing, Django seems to be checking B before A.

Now, when I use a ModelForm, sometimes I want to use the commit=False option when I save. This returns a model that hasn't been committed to the database. Sometimes there's extra data I want to add to the model that the form didn't supply. Sometimes that data is in fact necessary for the model to be valid. So clearly Django shouldn't check A, and it doesn't. Here's the funny thing though: it does check B. Why would it do that? I can understand checking when I call is_valid(), I can control when that's called, making sure I added everything first.

So far the consequences of checking B early have been trying to access members that haven't yet been set, in my clean() function. So in clean() I just check for those items, and just let it pass if they don't exist. I figure, yeah, it'll pass certain tests when it shouldn't. But if it's really running through validation (not just B as in the save (commit=False) case) that means it'll catch the missing members on the A pass anyway.

Maybe I got something wrong, but I thought it was a weird design decision.

Significant Whitespace in Python Data Structures

2010-10-05T10:08:00.006-05:00

I recently wrote a program in Python for parsing files. I'm pretty naive still when it comes to functional programming, but I'm still excited about it, so I wanted it to be more functional in style. It had a complicated data structure representing the file structure, instead of a loop with bunch of if-thens. By Python standards I may have gone a bit overboard. Guido probably would not have approved of my code (not to mention what follows in this blog post).

So as a result, most of the program became whitespace irrelevant. Huge dicts of lists of tuples, etc. It made me think that relevant whitespace might become handy for data too. And while talking on IRC about it this morning I realized I could sortof hack it using decorators and generators. So here's what it looks like:

http://gist.github.com/611646

So I have two examples:

example.py - This shows how you can define a more complicated structure with whitespace instead of a bunch of ){(}[].

inlinefunc.py - This demonstrates a sort of side-effect benefit. You can have multi-line functions inline in a list (or tuple or dict). Usually you're stuck with lambdas, and of course that starts to look confusing too.

I'd like to clean up the syntax. Obviously having to have a decorator before and a yield after isn't great. I'll have to think about how I could do that. Maybe make an even dirtier hack by doing introspection.

Any thoughts?

A useful script: copytoclipboard

2010-03-13T17:30:00.005-06:00

Another quick tool I figure I'd share. Made this one a while ago. When you're doing stuff on the command line and you need to copy something for a desktop app, you end up having to select text on your terminal screen. Well if it's a big text file, you have to copy, paste, scroll to the next part, copy (make sure you didn't copy any part twice), paste, etc.

Well, copytoclipboard takes standard input and puts it into the gtk clipboard (haven't tried this on KDE).

#!/usr/bin/env python

import sys

import pygtk
pygtk.require('2.0')
import gtk

c = gtk.Clipboard()
c.set_text(''.join(sys.stdin.read()))
c.store()

For instance, to paste this code, I went into my terminal and typed:

cat copytoclipboard | copytoclipboard

and pasted the results here. For a less confusing example:

uptime | copytoclipboard

lets me paste this:

17:42:14 up 3:23, 2 users, load average: 0.35, 0.39, 0.41

EDIT: Or I could just do a google search and find that xclip already exists. Oh well.

EDIT2: No, xclip seems to be its own thing. I just tried it, doesn't seem to work with gtk. I guess I still win!

"New Music" script

2010-02-25T10:03:00.002-06:00

I just thought of a script (can't test it at work so I'll actually write it later, but it'd be only a few lines long). Would work thusly:

Desktop>newmusic britney_spears_leak_trance_remix.ogg ~/music/various/

It would act like mv, just move the song over to the "various" directory, but it would also add a symlink to

~/music/new/

That way, I'll remember to give it a listen. And when it's no longer novel I can just delete the symlink. Otherwise this crap gets accumulated on my Desktop. Maybe happens to you too.

Maybe it could be made into a Nautilus script or something, but that seems a bit more tricky if you're dealing with drag n drops.

Making new twitter followers marginally more convenient

2010-02-14T17:06:00.003-06:00

Well, one tiny annoyance that I just randomly decided I felt like itching - I get a handful of emails from twitter saying so-and-so is following me. Usually it's somebody I'm not interested in, but on occasion it's a friend. What annoys me is that I have to open their profile to see. I'd prefer to see the bio and their last few tweets in the email itself. Would help me decide much faster. Well, I've started on a real hack of a solution. It should make it a little less annoying:

http://gist.github.com/304335

Look it over. If you trust it, run it on a command prompt (works on Linux, all I can say). It asks for your gmail password. It opens browser tabs for all the twitter accounts that have followed you that are still in your gmail inbox.

Now, I'm making some wild assumptions about the emails that Twitter is sending us (namely that the first Twitter url is the profile in question), so this may not open up everything correctly. Especially if Twitter changes the email. So, I also open up a tab with a gmail search for all those emails. You can look it over and confirm that it opened up the right tabs, plus this helps you archive them right away.

If it doesn't work, do this first:

https://www.google.com/accounts/DisplayUnlockCaptcha

I guess it tells Google that your computer isn't engaging in any anti-human activities. I had to do it.

I may eventually have it generate an html page with all the useful info right there, so you can look it over quicker. I'll also look for a way to have a GUI password entry thing, so you won't need a cmd prompt. Or, you know, if any of you Open Sourcerers want to do it yourself and send back the update, that would be great too.

Ping!

2009-10-28T22:41:00.004-05:00

Pinging is the act of sending ICMP packets to another device, and waiting for a response. It's a good way of seeing if we're online, or if the hopeful recipient is online. It is prevalent enough that it's become slang for contacting someone, to see if they're around and listening. And it's gone beyond that; I just realized that Google Wave's use of "Pinging" someone makes it official in a way.

But even in the ICMP packet sense, it was in a sense a slang usage. From Wikipedia:

"Mike Muuss wrote the program in December, 1983, as a tool to troubleshoot odd behavior on an IP network. He named it after the pulses of sound made by a sonar, since its operation is analogous to active sonar in submarines, in which an operator issues a pulse of sound at the target, which then bounces from the target and is received by the operator. (The pulse of sound in sonar is analogous to a network packet in ping)."

Here's the other half of the story. I always feel inclined to theme my computer. Really put some life into it. I thought of a really silly idea, and a great way how to do this today while sitting at the Skylark.

So here's where it all comes together: Noisy Ping (for lack of better name)

When you set this up, ping will emit a sonar sound. And if you get a response, you will hear a subdued version of the same sound.

Code licensed under WTFPL, sound was from a creative commons site, so it's under Sampling Plus 1.0. Don't sue me if a whale tries to mate with your computer. Or if my program does something bad (though I promise I didn't mean to do anything bad, and that I'm running this on my own computer).

This was sortof hacked together, because frankly I have better things to do than to do this "properly", but I did my best to make sure that the python script relayed ping's inputs and outputs and kill signals faithfully (though I have a failsafe SIGTERM, followed by SIGKILL, at the end). if you have any suggestions on how to make this more safe, less crash prone, more portable, etc, please feel free to send me a better version.

And hell, it's sortof useful too. If you're pinging something and you don't want to watch the terminal to see if you get anything back.

So there you go.

What about a "rabbithole" paradigm for all GUIs?

2009-03-07T15:32:00.005-06:00

I'm watching a presentation on Zimbra, and I'm noticing that the interface is really busy. I can begin to see why it must be a pain in the ass to set up and maintain (as I've heard). I know I'd hate to have to use it. Where I currently work, as I would imagine is the case at most large businesses, you have a million different services that you can use, a couple of which you have to use, and a couple which you may want to use (the latter group will vary by person or group). A whole lot of things just don't get used by your group, but you keep hearing about it from the company above, it keeps showing up in your web interfaces, and it's just annoying.

A couple posts ago I mentioned an idea of a "rabbithole paradigm" for configurations. What if you did the same thing for your whole GUI?

You start with a big button that says Zimbra. You click on it and get three buttons called Communication, Collaboration, and Settings. You click Communication and you get your email, chat. Click Collaboration you get file sharing, Wiki, etc. Settings gives you GUI settings and Account Settings, each of which gives you more things. Everything is completely straightforward, if cumbersome, as to where you should find them. Certainly no settings or shortcuts that you don't need are in your main interface.

Well, that would make things a lot less confusing, and a pain to get to the services you need, because you have to go all the way down the rabbit hole to get to them. But, as you decide to use them, they would get added to your front page. This includes configuration too, I suppose, as I mentioned in my other post. That way, you always have exactly the services and settings you need and know how to use on your front page and in your menus.

Backing up stuff from websites.

2009-02-20T16:04:00.002-06:00

Ok so if I'm going to give in and start being a social media whore, can I at least back up my info? I know my friend Kevin was telling me that "open websites" let you can grab all your info from their DB. I wonder how possible that is for other sites. Could I, say, back up my Twitter contacts? My google reader feeds? My delicious links? (yet to join the latter two). I can concede that these outside sources know what I'm doing and what I'm interested in and who my friends are and where I live, but I don't like depending on them to keep my data.

I should start by setting up IMAP for my Gmail account, sheesh...

What about a "rabbithole" paradigm for configuration GUIs?

2009-02-20T15:55:00.004-06:00

You get very basic settings on the main screen, but there's a button for things a bit more advanced. The screen that follows that button has categories, and each category again has buttons for things slightly more advanced. You go as far down the rabbithole as you feel comfortable. You could have everything be configurable, but it's not a choice between a Fischer Price toy and an Airplane Cockpit.

And, and I think this is a kicker here, you save which buttons you clicked, which screens you edited. So you sortof have your own customized config GUI. It's tailored to each user for whatever they feel comfortable with.

xmsg

2009-02-14T19:51:00.002-06:00

Made a handy little script I wanted to share.

I call it xmsg:


#!/bin/bash
"$@" | xmessage -file -

Now, you can put xmsg before any program whose console output you want, and it'll show up in a (albeit archaic looking) dialog box. This is useful when you want to know the console output of something just with the Alt-F2 run dialog, and you don't want to bother opening an actual console.

Put it on the PATH so you don't have to put the whole path to the script, you can just type "xmsg" followed by your command into the Alt-F2 menu (or a console if you want). Quick example:


xmsg df -h

Will give you a nice popup telling you how much space you have on each mount. Now wasn't that more convenient than having to open a console?

How it works (in case a little education can come of this).

"$@" represents all arguments to the script after the name of the script itself. So, this just runs the command you put after xmsg. This pipes to xmessage. xmessage generally makes a dialog with whatever you give it as an argument, but if you say "-file" it will read from a file. If you give "-" as the file, it will read from standard in, which in this case is the output of "$@".

The quotes, btw, on "$@", are because of Bash's annoying quoting rules. Not using them will, I am told, cause arguments with spaces to retokenize.

Twitter

2009-02-10T10:21:00.002-06:00

I'm only announcing my Twitter usage here for now because I got a great quote from my friend Tim, and it won't fit in Twitter's max characters:

"dude, twitter is perfect for you. you send out a message to no one in particular. If anyone wants to listen, then you're talking to them. You do that without the internet."

I guess if you want to follow me:
http://twitter.com/orblivion

I suppose I'll be extra cool and get identi.ca soon.

Where is Google Apps free edition?

2009-01-28T06:58:00.001-06:00

This answered my question:

http://googlesystem.blogspot.com/2008/11/free-google-apps-more-difficult-to-find.html

This gave me the same feeling as discovering a secret area in a video game.

Joe sent me

2008-11-24T09:59:00.000-06:00

I moved into a new place recently. I need Internet access, and I wanted to avoid giving any more money to AT&T, and I've heard that Comcast isn't much better. I looked into Copowi first. We couldn't get them to work at our last apartment (though it's at LEAST 50% AT&T's fault, I'm sure more). I'd say they're not too bad, I had a support ticket and a support person that was willing to try different options to get my stuff to work. In the end, they refunded my money and even let me keep my modem. I was thinking it over again; do I want to deal with AT&T for a phone line again, and then wait for Copowi, potentially dealing with the same fiasco? Well this made my decision very simple:

Copowi: Preventing The Depression

Thanks, but I'll get my information "they don't want you to know" from Alex Jones, and I'll get my Internet access from Speakeasy.

I'd heard about Speakeasy from a couple people. They're a smaller ISP like Copowi. One person told me he used it and they had great service. Turns out, I can get the nets from them without even getting a phone line from AT&T. (Copowi doesn't offer that in Chicago yet, though I think it does in other cities). Speakeasy is pretty expensive (I'm too embarrassed to give the actual amount), you can look up the rates yourself. But it's worth it to not have to give money to or otherwise deal with The New AT&T (you can ask me about my experience with them if you'd like, I won't get into it here, this post is too long as it is). Plus, I'm paying for very impressive service, as you will soon see.

The following might seem like trivial reasons to fawn over a company, but think of all the little annoyances that come up when you deal with companies. Particularly telecom. As usual, it's the small things that make a company great.

I went to Speakeasy's site to see if they were available in my area. For some reason the form got confused about my zip code. There was another form below, however, to request someone to contact me to help. I chose email over phone (I'm an engineer, I don't like to talk to people). Plus email is much more out of the way.

So I believe the next day I get an email from the guy. I asked him about my address, and he responded saying that I should be good, though I can't get more than 1.5 Mbps. I fire off a ton of questions at him all day, (it's a tough decision, a year contract, a high price). He responds to all my questions, probably within 15 minutes every time. I ask at one point if I can use a modem I had on hand (from copowi) instead of the modem that comes with the plan.

"There is no real way to tell as it is all based on the chipset used to sync with the dslam at the central office. I can definitely set up the order for you using your own modem, and if worse comes to worst and it doesn't work you can just get our Modem if you like."

See, that strikes me well, because he's a sales guy, but he uses terms like chipset. Good to know that the people selling me stuff know what they're talking about.

Eventually I say that I'm interested. He sends me a form to fill out and fax (I preferred not to send my credit card number over email or cell phone, and I didn't feel like asking about GPG). Today I get my form email giving the details of the account. I was impressed by the fact that this form email said the following:

"You have instructed your sales person to NOT order hardware for this installation. You have reported owning the following bridge/router:

Broadxent 8012-V

If an onsite technician attempts to deliver unordered hardware, simply inform them that you have your own, approved and supported CPE."

So they got my modem model wrong, but think about the fact that they had this option in a form email. I would have expected just a lack of charge for the modem, or they would charge me, but then the support guy would tell me "yeah, ignore that. I put in a note". I'm impressed that "customer has his own modem" is a standardized option for them.

So I log into my account, and find that I can easily change my mailing and shipping address. The order info page has more technical and contact info (for the folks setting up my connection) than I know what to do with. It has a list of 8 "Stages" to getting a working connection (Order Received, Order Placed, Install Confirmation, IP Addresses & Circuit Configured, etc). It shows details for each, including how long I should expect to wait, and of course which one I'm currently on. And, there's a dropdown box where I can choose how often I want email status updates (24, 48, 72, hours, or never) Underneath, there's some sort of "up to the minute report":

"The work log below represents the available documented efforts of Speakeasy and vendor (as indicated by their respective logos) to complete installation of your service.

Speakeasy's service activation team proactively works to install your circuit at the highest level of priority. This real-time installation progress report provides the most current, complete and accurate update on your installation status...."

Wow. Overkill, honestly. Though, if this were for a business I would probably appreciate this more.

To top it all off, while I wait for my DSL connection, they offer a complimentary dialup connection that I can use right now. I don't have a phone line to use it on, but still, that's some really slick Business Fu if you ask me.

One thing I don't really like is that the prices he gave me over email don't seem to be the same scheme that they have posted on their site. I may have just gotten confused on that though. And yes, for sending my credit card info, they should have had an encrypted web form for ordering service. Heck maybe they did, I didn't check, but I sortof wanted to go through this guy, since he was already helping me out and could sort out any issues I had, plus maybe he's getting commission. Finally, for some reason they didn't send my password or account number in my form email, but I emailed the support guy and got those right away.

There's more to come with this, I'm sure. If it's anything spectacular I may post again.

Some Intrepid notes

2008-11-16T10:00:00.000-06:00

I figured I'd mention this in case other people have these problems (and solutions) and want to know if they're alone.

So on my laptop, Dell Inspiron 6400, I went from Gutsy to Intrepid within a couple days. I tried Hardy very briefly. My Gutsy->Hardy upgrade was rocky, but Hardy->Intrepid was smooth as a baby's bottom.

So, starting with Hardy, Sendmail takes about a minute to start. Though I'm told that it's because I never configured it. I never use it. So I uninstalled it. I don't even remember why I had it. BUT, it never did this under Gutsy.

Starting with Intrepid, something related to dhclient takes about a minute, dare I say maybe two minutes, to start during my boot sequence. I'm working on resolving it at the moment. Pretty sad.

Gnome doesn't have as many compiz bugs when starting up as I've had in Gutsy, but overall I feel like it takes longer before things are usable. Once it's up though, it's just fine. I think Firefox is faster, yay.

Now, some of my run-on-startup program configs got wiped, I think going from Hardy->Intrepid. I hear this has something to do with Gnome sessions stuff. I never was in love with Gnome anyway, maybe it's time to switch. I'm intrigued by wmii, and all the cool kids in my LUG are using it, so I might have to try it if I want to still be cool. I tried running it once, just to see if it had a usable default. It didn't really, but it started up pretty much instantly. Sortof scary. Or maybe it's sortof scary how much useless garbage Gnome starts up. Thank God it's Linux and I have a choice.

Also, I use Tilda. I don't have the package installed since I was doing some development on it. For some reason, twice now, my config got wiped just randomly; it went back to the default config. I'm going to guess it has something to do with the Hardy or Intrepid updates. This has never happened before, I wonder what's up.

Now, wireless, that's delicious. I have:

0b:00.0 Network controller: Broadcom Corporation BCM4311 802.11b/g WLAN (rev 01)

It's been getting progressively better over the semi-year releases. But up to Gutsy, it was still flaky. When I started up, I would have to reload the driver before getting it to work (unless I waited a while before logging in. Hmm.) Anyway, none of that in Intrepid (probably Hardy, I never really gave it a spin, but I think that's when my new driver came in). It also connects to anything reasonable that I've thrown at it, things that I could only connect to after a struggle, at best, before. I can steal the nets from my apartment where I couldn't in Gutsy. The whole issue of "wireless sucks on Linux" seems to be history for my particular configuration. I know your mileage will probably vary.

How did a quick rant become so long?

Reversing lists and strings in python; third field in [] is for step

2008-09-08T10:27:00.000-05:00

This is the simplest way to reverse a string or list in Python:


>>> "aoeu"[::-1]
'ueoa'

I was about to say this was pretty shitty because it doesn't follow any sort of pattern, it seems to be a special thing just out of nowhere. But then I tried -2, and -3. Turns out the third field is for step.


>>> "aoeu"[::-1]
'ueoa'
>>> "aoeu"[::-2]
'uo'
>>> "aoeu"[::-3]
'ua'
>>> "0123456789"[::]
'0123456789'
>>> "0123456789"[::1]
'0123456789'
>>> "0123456789"[::2]
'02468'
>>> "0123456789"[::3]
'0369'
>>> "0123456789"[::-1]
'9876543210'
>>> "0123456789"[::-2]
'97531'
>>> "0123456789"[::-3]
'9630'
>>> "0123456789"[2:4:-1]
''
>>> "0123456789"[4:2:-1]
'43'

So, with [4:10] it takes the 4th through 10th item in the list. With [4:10:2], it takes every other item from 4 to 10. With [4:10:3] it takes every third item, and so forth. With [4:10:-1], well it seems to do nothing, but with [10:4:-1] it takes every item from 4 through 10, but in reverse order. With [10:4:-2] it takes every other item in reverse order.

The more you know. No rainbows please.

Feed Getter

2008-02-11T14:52:00.000-06:00

I've mentioned in a previous post an RSS aggregator that I'm working on. It's tentatively called Feed Getter. I'm putting it up now because it's usable, and it would be nice to get feedback from anyone that wants this kind of thing. Here's the pitch:

I can't find any RSS aggregators that just save the files I ask for. I don't want a program to download and play my podcasts for me, I just want the files on my machine, where I ask them to be, so I can view and play them with my usual programs. I want it to be automated, I don't want an interface. This is especially useful for automated things, like a desktop background changer. If you agree, Feed Getter may be for you.

Now, it's in early stages, but it gets your files for the most part. The biggest issues: a) It gets files again, even if you have them already. I do this because I'm not sure if files on your machine are incomplete or not, so I assume they're incomplete and get the whole thing over again. The solution is to write to a temp file, and do an atomic copy to the main file only when it's done. That way I'm sure that if it's in the main file, it's complete. This will probably be in version 0.2. b) It doesn't delete old files. Probably version 0.3.

Now, at this point I don't run this as my main user, and I especially won't once I start deleting files, and you may consider doing the same. I want to make sure that my code regarding restricting file creation/deletion to certain paths is safe before I do. If you're interested in using this program, and you're experienced in path safety in python, maybe you could review my code.

So Here it is. Check it out, let me know what you think.

Optional sections of your document using LaTex

2008-02-09T19:11:00.000-06:00

If you're like me, you have more that you could put on your resume than you can fit. You have to cut it down to what's relevant, but sometimes the best things to put in differ based on who you're sending it to. So you have to make different versions of your resume. And then it turns out you need to reformat something in your basic info, and you have to change every version. If you used a word processor for this, you know how this could be a pain in the ass.

This is why I liked the idea of LaTeX; it's technically a programming language. TeX is anyway. I could make one giant resume, and put a bunch of conditionals in there. Now, I figured out how to do this, and I think it's a hack, I'm not really sure how this language works and I don't really care to find out. Point is, this works.

Here's an example. Let's say there's an internship that I did a while ago that I might not want to include in all versions of my resume. First, I define this variable at the top:

\global\let\includeinternship\relax

Then I test if it's undefined:


        \ifx\includeinternship\undefined
                \textbf{Internship}
                \begin{itemize}
                        \item Brought coffee to employees
                        \item Sat around
                        \item Oh one time my boss had me make a spreadsheet
                \end{itemize}
        \fi

You defined it, so it won't show up. Sortof backwards, but it works. Again, it's a hack, and I'm too lazy to figure out how to do it right. So, to make it show up, comment out the definition:

%\global\let\includeinternship\relax

So you could put all the variables on top, and that's a lot more convenient than what you had before. But it could still be better. Lets say you have different versions of your resume.tex handy, with different variable settings. If you have to change some formatting or details in your resume, you're back where you started; you have to make the change in all versions of the file. So, you should have only one version of resume.tex, and include the file variables.sty:

In resume.tex:

\usepackage{variables}

In variables.sty, in the same directory:


%\global\let\includeinternship\relax
\global\let\includecrappygpa\relax
%\global\let\includemacaroniartproject\relax

So now, you keep around multiple versions of variables.sty, with different combinations of variable definitions commented out. Any changes to your resume, you only have to do once. Now, what if you're updating your resume and you decide to add variables. Well, that's the downside. You would have to add it to all the variable files. But that's a lot better and more straightforward, in my opinion, than making a more complicated change to multiple versions of a whole resume file.

At this point, you can make multiple versions of variables.sty, or you can make a quick change to variables.sty and make a new resume.pdf. With multiple versions, you still have to rename them to variables.sty to use them. So, you just put it in my resume generating script:


cp $1 variables.sty
pdflatex resume.tex

Of course, you have to be careful not to have anything you want to keep in variables.sty, because the cp will overwrite it. Keep your actual work in different files that you pass to this script. For instance:

./makeResume.sh version3.sty

Submitted For Your Approval

2008-02-06T21:20:00.000-06:00

I talked about Differential Equation Munchers a bit in my previous post. I decided that I'm not going to wait until the game is "presentable" before releasing it. So Here You Go. It's in a "fully functional" state, but it's not pretty. I did get around to at least stealing all the Muncher artwork from the original game, but the Troggles still look MS Paint quality. Let me know if you decide to try it out.

To My Naysayers

2008-01-07T11:19:00.000-06:00

You know you you are. I have successfully created a way to generate examples of first and second order differential equations. A little backstory is in order I guess.

I mentioned my game engine in an earlier post. At this point, it's far from done, but it's functional. I put that on hold to make my game demo, which uses what I have so far of the engine. Independent of all this, I had the idea to make a game called Differential Equation Munchers, a parody of the classic Number Munchers, which the lucky among us had a chance to play in our elementary school Apple labs. I've decided to use this opportunity and make this as my demo.

In the original game, you're given a category of things, such as "Multiples of 4". You walk around the board and eat the numbers that fit into that category. But watch out for the Troggle! Troggles can come into the board and eat you, so avoid them. You also die if you eat a number not in the category. You win the level when you eat all the numbers in the category.

For my version, the categories are related to differential equations. Which means that I have to somehow render differential equations. My thought was that I would generate them on the fly. Someone else suggested that I should have a table instead. Supposedly, writing a generator would take too long. I didn't want to make a table, I said. Either I would have too small a pool of possible answers, or it would be a real pain in the ass making a huge table by hand in Gimp. Plus, come on, that's lame. Right?

Well, this weekend, I made a generator for First and Second order differential equations. I decided that, yes, if my categories were things like solutions to the differential equation y' = 5xy + 7, I may be facing way more work than I intended, because I would have to solve Dif Eq's on the fly. However, if it's just categories like First Order Differential Equations, then it's a simple matter of generating an equation that happens to have a y' in it (and some other rules). My program doesn't have to have any notion of solving or understanding a differential equation, it's just a dumb image generator. Read 'em and weep:

(click to embiggen)

Granted, this is a simplified version of what I want ultimately. I can take more time to make generators for that stuff later. This is good enough for my demo. Also, for the more complicated stuff down the line, there is a Python library called sympy that can help me manipulate algebraic expressions.

Anyway, I'm happy about that. The game is in fact playable now. I have to add things like number of lives, score, splash screens, difficulty, etc. Then it'll be fully functional. Then I'd have to go clean up pictures (making the dif eq's more readable would be a good start). Also putting the original artwork in. Then I'll have to add more categories of equations. Then it'll be suitable for a demo.

RSS gripes

2007-12-27T13:32:00.000-06:00

I should have a gripes tag.

I am working on an RSS aggregator. I want it to get files for me. That's all. Put them where I want them. Is that so much to ask? I think it is, so I had to do it myself. All the better, I can customize it a lot this way. (It's written in Python using feedparser, I will post it to this blog when I'm done, probably public domain.) I want it to download my music for immediate initiation to my library, videos too perhaps, images definitely (comics; probably for a desktop changer later), and also maybe text; not sure what for yet.

So at this point I have it pretty much working. Some of the nice things I don't quite have yet; filtering for certain entries, deleting files, putting the timestamp in the filename to further prevent collisions, etc. But for now I can use it, and it made me happy. Until, xkcd gave me this:

<img src="http://imgs.xkcd.com/comics/blade_runner.png" title="Blade Runner: Classic, but incredibly slow." alt="Blade Runner: Classic, but incredibly slow." />

This, and the html page (linked from the feed), are the only reference to the actual png. No direct link, so I have to now parse this bit of html, or the link that the rss feed points to. No way am I going to make a custom bit of code just for some emo nerd comic. (the fanboys already made me irritated with this comic, so it's convenient) But, it turns out that Perry Bible Fellowship does the same thing; the supposed image link is actually an html link to a page that contains the image. All the podcasts have direct links to the mp3 files; after all, podcast players like to have direct links. I guess there aren't enough, if any, comic readers yet.

And this is worse than parsing the feed, because there's no standard place to look for the link within the HTML; it's different for every comic. And the HTML page is subject to change at any moment. At least there's a standard in RSS. Some RSS feeds may have stuff in non-standard places, but I made a very easy way to define a non-standard place to look for stuff*, and I don't think they'll move it. And XML parsing is less error prone that HTML.

Well, fortunately there's a couple html parsers for Python, I'll probably use pullparser. My hope now is that it's as easy as feedparser, and I can make my custom settings a similar way*. Though, there are bound to be multiple images in the html page, so I have to somehow identify the one I want. Perhaps by tag id, or maybe the directory that holds the image. God help me if I have to write regexes; I want this to be simple, it's weird enough that my configs are a list of dictionaries in Python.

*Here's how I have it get files in non-standard fields. feedparser puts the whole feed xml file into a tree of dictionaries and lists. Within each entry, your podcast (for instance) item is usually at entry["enclosures"][0]["href"]. This is what I have as the default. But what if I wanted something at entry["link"][1]["href"] (semi-madeup example). My config file is written in Python too. I have a list of dictionaries that define each feed. To my dictionary I add "url-basis":["link", 1, "href"], and it knows to look at entry["link"][1]["href"]. It's very convenient that in Python you can say x[a], and it will try to treat x as a dictionary or list, depending on what type a is (and of course, throw an exception if you guessed wrong).

Idea for an anti-phishing plugin

2007-12-19T15:10:00.000-06:00

I just thought of a decent anti-phishing scheme, which would probably make a good plugin for Firefox. Particularly useful for those sites that use unicode for evil.

The plugin would give you a button you can push that tells you if you're on one of a whitelist of sites. If you're not, and you thought you were, you're being phished. It would be tedious to whitelist every single site out there, of course. Mainly your important accounts (bank, paypal, email). Listing as many as 20 things sounds like it's worth my time for safety.

Another variation: a small popup (not a dialog you have to click on), or maybe a change of color of a widget on your browser, that comes when you are on a whitelisted site. Sounds a bit backwards at first, but really, you can't get warned when you're on a phishing site unless you have a perfect blacklist (and if there were one, there would be no use for my plugin). My thought is, eventually you'll get used to seeing the popup every time you're at on of your important sites. Then, one day you go to a phishing site. Because of habit, you'll hopefully think that something is a little off when the popup doesn't show, and take notice. Unlike the first variation, I think this variation could work for general carelessness, not just the unicode trick. You would probably combine the two, really.

And while I'm at it

2007-09-28T12:39:00.000-05:00

I made a chat program in Visual Basic in college. It sucked as a program, but it had a neat feature I always wanted out of a chat client, which is multiple input windows. Here's an example:

____________________________________
CoolGal27: Hey how's it going?
SmooveDood69: Not bad.
____________________________________
Enter Message:
Actually, I was just getting done with my dissertation on bacteriophages. It's really quite fascinating, these virus-like agents actually attack bacteria, in a similar way to how viruses

...

____________________________________
CoolGal27: Hey how's it going?
SmooveDood69: Not bad.
CoolGal27: Want to meet up for dinner?
____________________________________
Enter Message:
Actually, I was just getting done with my dissertation on bacteriophages. It's really quite fascinating, these virus-like agents actually attack bacteria, in a similar way to how viruses dangerous to you and me would attack our cells. My personal thesis focused on

Goddammit! See, now I have to copy all this text, type in, "yeah sure, but I can't until 7", and paste my long discussion back into the input window.

I'd make a plugin that would give you two or more input windows, so you could have that long thing you were typing along side the quick thing you send over.

Another idea I had was the ability to send a message to multiple people at once. Not in a chat room, just a quick message or two privately. As it stands, if there's a kool link ^_^ that I need all my friendz 2 see, I need to paste it into all the different windows, etc. I'd like to make a plugin that gives the ability to multicast messages.

And dammit why did 2.2 get rid of the font sweep-away icon?!? The greatest thing Pidgin ever did.

Pidgin Gripes

2007-09-28T11:03:00.000-05:00

I like Pidgin, it's straightforward, faster than AIM, etc. Last I checked (like 5 years ago or so) Trillian was slow as hell, though I don't use Windows at home anymore anyway. However there are a couple things that really annoy me about Pidgin, and sometimes IM clients in general, so I'm thinking of starting to make plugins to fix it. Kindof like I was going to make Tomboy plugins and still have yet to do. Here are my gripes:

1. Handling of Incorrect Passwords

First and FOREMOST, in any other interface I can think of, if I put in an incorrect password, the application informs me of the situation, and offers me another try. Not so with Pidgin. Maybe you never noticed because you save your password on your machine; that's not for me. (though a Keyring plugin would be super kosher).

No, what it does (if I remember correctly) is inform me that my password is wrong, and offers me to "change my account settings", "connect" (with the same wrong password, as if it'll work the second time), or just give up. Now, when I "modify my account", the bad password I just entered is in the password field. That password field is there for those that want to save the password on the disk. This worried me (since I don't want it on the disk) so I always deleted it from the field. Why was it trying to save my password when I told it not to, I wondered. Turns out that it wasn't actually going to save it, which is a bad interface choice.

Anyway, you go ahead and modify it. Then you disable and enable the account and for whatever reason it still prompts you for a password, so that's how you try logging in again. Lame.

Admittedly, with Pidgin 2.2 (which I just installed and which is strongly diffusing my argument here as I type it) it just says "Pidgin will not attempt to reconnect the account until you correct the error and re-enable the account.", and gives you a cancel button, which isn't much better but is less ironic at least. I still have to go to the menu and re-enable the account. That's much less annoying than it was before. But still, I shouldn't have to do even that, it's easy to make an immediate prompt.

So, a plugin I would make would fix this, and give me a prompt to put in my password again, if I want.

2. Sending messages while away

Remember when this was an optional feature, rather than the standard? I bet AIM still remembers this. I personally end up forgetting to take down my away message when I come back. People remind me. It makes the whole away message thing pointless, it should be up when I want people to think I'm away. Maybe some people don't forget, that's why it's a good option for them.

Though, I still like it in theory. What I would do, and this is something that would solve a gripe with all chat clients, not just Pidgin, is make a plugin that would remind me every X minutes that I was away, if I was typing while away. For convenience, the dialogue would give me the option of coming back from away, too, and probably "don't bug me about it again this time" as well. That way, I could go away, send those few messages I forgot to send, go to lunch, come back, message Cezar, and he wouldn't have to remind me that my message still says I'm at lunch.

3. Away Messages Don't Expire

Whereas the first gripe was with Pidgin and the second with IM clients in general, the third one is with myself. I forget or don't get around to changing my away message. I was apparently showering all day yesterday. What good does this sort of an away message paradigm do anybody? What I would make is the option to set an expiration time for an away message. I'd set an hour for the shower message. Then I would set a subsequent message. Like, work. Maybe I want another subsequent message (because when I get from work, I don't go and take myself off from away first I get home), this message would be a catch-all "At this point I'm not where I was when I last set my message, I can't tell you where I am, but I'm not at the computer, sorry" sort of message.

Game Demo Progress

2007-09-01T07:59:00.000-05:00

I think I should post my progress as long as I have this Web Log and people are forced to listen by virtue of a feed associated with a certain User Group of a certain class of operating systems. I probably never mentioned this in the first place, but I've been commissioned (probably no money involved; I had a discussion with a friend, I'm pretty sure I can still technically say "commissioned") to make a 2D game engine.

I initially chose C, with Python for scripting. Then I decided that for a prototype I should just do it in Python. Then I figured, why the hell not just make most of the actual product in Python? I would just use C for anything time intensive, such as looking for collisions ( O(N^2), unless there's some industry trick there that I have to learn ) and image effects (generated fire, as opposed to pre-animated fire).

So far I've drafted up my requirements and cleared it with the guys planning the game itself. It was a decent sized doc too, I'm reasonably happy about getting that done. Next is code design, but it was pretty hard to think of how to design it without trying to code it first (a catch 22 of sorts), so I decided to make a prototype(/demo) to help give me the ideas. Turned out downright vital that I did this because things came up that I didn't anticipate at all, as I'm sure is always the case.

For starters, I'm working on the Actor framework. Everything in this game is an actor. They send messages to each other, they have actions every frame, and then they draw. I'm not even worried about them showing themselves at this point, I'm mainly worried about laying out the code and making everything communicate properly. Input isn't really pressing either. I'll probably make a curses output module for starters, just for the heck of it, and just to show off (to myself, I guess) the modular nature (I hope) the engine will have. Actually, I'd mainly do it because it would help demonstrate that everything's working properly, and it's faster to get working than SDL, I would imagine.

So I just got message passing going. I think next I'll put in a quick framework for input (but not bother implementing it yet), but soon go onto other important things like searching for collisions, etc.

After this thing is done, or at least showing enough interesting aspects of my engine, I hope to have a presentation for this user group about game programming and Python. This would probably be my first presentation; I'm not used to doing them and it might be a while before I get around to that.

In this post I'd also like to chime in and say I'm impressed that Gnash works on Youtube, at least on a rudimentary level. Not Google Video though, sadly. So now I won't have to bust my head, having a 64 bit system, until Gutsy comes out and I can easily use the nspluginwrapper (I went from a naive Windows user that would install programs in a semi-irresponsible manner to a naive Linux user that refuses to install anything not from an official repository). Heck I may just stick with gnash, I've heard that it uses Open GL, so it should be faster than Flash. I think it is, really, the load time for the YouTube app is damn near instantaneous, though my computer is fast. Maybe I'll run a little comparison.

Capitalism and Linux

2007-07-01T09:22:00.000-05:00

I read Freddy Martinez's blog post about not bashing Windows and started writing a comment, and realized that it was much longer than his initial post, so I figured I'd just post it to my own blog, since I don't post here enough anyway.

Like Freddy, I have capitalist ideals. When people were telling me about Linux a few years ago, I had the hardest time comprehending how a free operating system could even get off the ground. There are a couple things to consider.

Firstly, I realized that free human interaction naturally and rightly includes both selfish competition as well as selfless collaboration. I've realized that the bottom line is freedom, not so much capitalism, if we are to define capitalism as selfish competition within a free market. (If you define capitalism just as a free market, that can also include the selfless collaboration if you think about it). So, people often get a lot of things done in the name of selfishness, as they're rewarded for their work with money. But people also can also accomplish a lot in a collaborative environment, something I was too naive to recognize back then. As much as human nature can be selfish and may often require capitalism to accomplish anything, there is also a nicer side of it. Remember that we have friends and families. As it is, capitalism and communism coexist peacefully, in a sense. (Yes I do consider free software somewhat of a a form of communism, and since there no regulations and nobody's forced to do anything it doesn't bother me). I heard a funny thing one day. A family should work like communism; from each according to his/her ability, to each according to his/her needs. Honestly, the only reason to hate communism is if it a) forces people into it (government imposed, particularly) or b) doesn't work. Free Software is free market communism, and so far it seems to work, and I think it's fantastic. Though, to be honest, I'm still pretty surprised it works as well as it does, but I think I'll grow to understand it eventually.

Secondly, and hopefully not completely nullifying the first point, Linux does make money, and as long as it's Free to the users and developers, I'm pretty sure it even has Stallman's blessings. Think about this, there's a utility that company A and company B both want. They could both invest the effort to make it, or they could both spend half the resources to make it together. Now, as the tool improves over time, the companies will patch it. Would it make sense to keep the patch for themselves? Not so much, because they would then have to keep patching each new version as it came out. Might as well submit their patch so it gets included in new versions. They make the software freely available (rather than secret between their two companies) so that Companies C and D come into the picture and start using and improving it further. Still, in a way this somewhat fits the communal view. I think it's similar to committees formed by multiple companies for their mutual benefit. (Yes the RIAA is a communist drone and must be stopped)

taking Simple Notetaking to ridiculous ends

2007-06-30T19:09:00.000-05:00

Anybody that knows me well enough knows that I'm very slow to change my habits. When I feel comfortable with a certain process, I feel rather uncomfortable changing it. This is particularly true when it comes to using a convenient tool, God forbid that ti has a GUI, that does a bunch of stuff for me. Instead, I do things in a very mundane and unsophisticated manner with the future (realistically far future if ever) intention to establish a more sophisticated system myself. I'd rather know everything my tools are doing, I want all interfaces to be transparent, so I can integrate everything as I want whenever I want. It's through those barriers that I broke to finally try Tomboy.

Tomboy, a Gnome app, is a pretty great tool on it's own if you're not crazy like me. It's a simple wiki-esque note taking system, automatically saves, nice keybindings and quick workflow (albeit with a few problems with key bindings, at least on my Dvorak keyboard). The big selling point is that you can make links to other notes by their name. Any text in any note (other than in the note's title) that is the name of another note automatically becomes a link to that note. It's a simplified Wiki with no "edit" screen, for a sped up work flow.

It's good for my psychotic note taking practices, however I never tried it because I wanted a more integrated, database-based thing, where I could easily move note items around, generate lists as I want them, etc. I was using vim and text files until now (and have like 100 files left to manually import, thankfully I found a python script for copying a file from standard input to the clipboard), figuring I could more easily import to whatever it is that I wrote more easily with text, but Tomboy uses XML so it's not too bad, even if I still ended up going that route.

Fortunately my issues aren't too important anyway, because Tomboy has plugins. Herein lies the heart of this post. Now I can still make any sort of integration I want. It's nice because instead of starting from scratch and making some crappy program that'll end up losing all of my notes, I can start from something relatively stable and very convenient.

Tomboy is written in Mono. It's pretty funny because the plugin files are .dll files. But it's still GTK. It was easy enough to build a sample plugin. I envision myself making a ton of these before I'm done. I also hope to describe the API on this wiki page. At this point the only documentation is the code for the existing plugins.

Why does a note taking program need plugins? Well, there are plugins available for integration with Evolution, Trac, and Gaim. Simple stuff like that which allow you type links that will activate things in those programs.

I plan to do a couple like that, but hopefully I can eventually realize the whole of my note-taking psychosis. It would be nice to integrate it with some sort of calendar support. Some sort of automation of listing things in a certain class would be nice too. This is more database-esque behavior, so I may think about integrating it with some sort of database. Though the newest version apparently supports tagging, perhaps that's a good enough platform.

I guess that's the end of it for now. I'll let you know when I make some plugins.

This is my tech blog

2007-05-07T10:04:00.000-05:00

This is where I will start posting about technical things that are on my mind. It will eventually be aggregated to www.chicagolug.org/planet/ with other members of the LUG.