MUD Nostalgia

For programmers of a certain age - those who saw their first internet connection in the 90s - MUDs [Multi User Dungeons - basically D&D online] were de rigueur. Open Source was in its infancy and this new thing called linux came on a stack of 35 3.5" floppy disks (configuration was equally hostile). So if you wanted to contribute to a collaborative project MUDs were where it was at. A number of python people have outed themselves as [ex?]mudders, including Glyph (Twisted founder, Twisted started as a mudlib), Doug Napoleone (PSF board), Jesse Noller (PSF board), Richard Tew (Stackless Python), Will Kahn-Greene (Miro, and mud client Lyntin), and yours truly (I maintain telnetlib, need I say more?).

Learning is in the doing, so what I got of college was mostly lots of free time to do stuff. I spent maybe 400 hours in four years doing programming for classwork and over 4000 hours writing LPC code for a now defunt MUD* (using /bin/ed - the only supported editor). Writing a mudlib was my "turtles all the way down" moment, when it clicked that user level programs were just programs that were composited from other programs; The trick was that the lower level stuff had an agreed upon naming convention but otherwise wasn't special at all.

I'll admit to being somewhat jealous of the college students of today - it is easy to contribute to projects that actually matter. Being able to rely on ping times being under 500ms is nice too (Europe used to just _go away_, sometimes for days).

I haven't left MUDing completely behind. I still use it to try out new methodologies in coding and just play around in general. Leanlyn (http://bit.ly/leanlyn) is my Lyntin fork that is about 10 years old and will never be finished (or be less than user hostile).


* The dead mud was Ether Realms. I can still be found sometimes at OverDrive (a Lehigh staple) and Three Kingdoms.
** College came with lots and lots of free time. I spent way more than 4000 hours just chasing tail. Imagine me but with shoulder length hair, an eyebrow ring, and a flannel shirt (it was the 90s)

PyCon Speaking

Do read Moshe's How to give an OK talk, esp the Narrative part. It feels quite silly to repeat yourself on every slide but you are preforming live and not writing on paper - so tell the audience why slide N+1 is important and for the same reason as slide N at every opportunity. During my talk I felt as though I was being overly repetitive, but on watching it after the fact I feel like I didn't hammer home the point enough (you should do more shit like this!).

Speaking of speaking: I asked a few popular conference warriors (Martelli & Hettinger) how many talks they had given and when they first felt comfy giving talks. They didn't know. I should have asked Beazely and C.T. Brown but they're professors and get more practice in a year than most people get in a lifetime.

But seriously, those popular conference regulars didn't know. Hettinger estimated he had spoken 30 times. His wife then pointed out he did 15 talks last year and so he upped his estimate to "more than 15 twice." Martelli estimated 50-100 talks but was equally uncertain. Myself, I've only done 7-8 and still get nervous if the room is less than 30 or more than 300 people. Hettinger said something like "I still get nervy when it the audience is 10x what I'm used to." I couldn't coax an admission out of Alex (I had only vague thoughts on the topic when I asked, and no narrative).

Take it all as paraphrased hearsay, but anytime I have to step up to a podium I just say to myself "suck it up, princess." It mostly works.

PyCon Detritus

PyCon proper is over and sprints have begun.

Language Summit

The Summit was boring again this year, and this is good. The language moratorium is over so PEP 380 ("yield from") is moving forward. The stdlib will get its own repository so it can be shared between CPython/Jython/IronPython/PyPy. Likewise all the commiters from the not-CPython projects will have commit privs to CPython (most already do).

My Talk

Useful Namespaces: Decorators and Context Managers went over well: video available here and here's the live Convore thread from the talk.

Every year I say I'm not going to do another talk, but it is years like this that make it worthwhile. Dozens of people I don't know have offered me thanks, jobs, free beer, and ZJs. There is no worse feeling than spending 40 hours on something and having it be less than good. I'm a happy camper.

The first question was asked by Larry Hastings ; it's become an inside joke because Larry has asked the first question at ALL my past PyCon talks (having a confederate to break the ice on questions is handy). I zinged him a bit in my response ("thank you for pointing that out, and may I point out it's just an example slide"); apparently most people thought I was razzing a random questioner even though I was addressing him by name.

Another questioner objected to my use of a module named "UnicodeNazi" because it isn't nice to talk about Nazis even (or especially) jokingly. I restated his objection and said he should take it up with the module's maintainer. That maintainer is Austrian, natch.

The room was pretty full. Not going out the night before definitely helped my delivery. There were some A/V problems at the start (lots of those this year) so I rushed the last few slides. Better that than running short though.

Other People's Talks

So far, so good. Yesterday my talk was in the afternoon so I spent all day preparing and didn't make any talks. Today I went to three: Alex Gaynor's "Python's Data Structures," Larry Hasting's "The Python That Wasn't," and Richard Saunder's "Everything You Wanted to Know About Pickling..." Most of my time was consumed by the Hallway track.

Hotel

The hotel has banned alcohol (or at least implied a ban) in public spaces that is not purchased from the hotel. I don't know if this is because of money or just because they want fewer drunks walking around. Annoying either way.

Diverstiry

A woman came up to me and said that something I told her last year had stuck with her. Last year she was sitting at a table with myself and another core dev, and holding her own when discussing eldrich corners of Python. The topic turned to contributing to open source and she had a list of reasons she felt she couldn't contribute, none of them very good. The good news is that she sprinted remotely last year, is sprinting in person this year, and has contributed code to projects over the past year. Oh, and the thing I said that stuck with her? "Suck it up, princess." So I have struck a blow both for and against diversity. [FYI, I don't remember saying that but it is the kind of thing I might say after a few beers]

Happy PyCon to Everyone

I'm in the happy majority of Python commiters that mostly idle except for the scheduled holidays like bug weekends and release alphas, and only awake for the high holidays like PyCon. I'll be speaking again this year and I'll be there for the language summit before and two days of sprinting afterward.

This weekend I stopped researching my talk "Useful Namespaces" and started actually writing slides; the theme is a riff on "Namespaces are one honking great idea -- let's do more of those!" and the first slide is "I monkey patch the shit out of everything." Coincidentally I won't be sprinting 100% on core stdlib this year but will instead be trying to fix up my monkey patches and tests on other projects that haven't yet made it upstream.

On Deployment Policy

Q: Don't we have a policy about drinking during deployments?
A: Yes, we always drink during deployments.
Q: I thought the policy was "Friends don't let friends deploy drunk?"
A: If you can still type how drunk can you be?

Encouraging Arbitrage in Virtual Economies

I'm always surprised by how different companies conduct interviews - I shouldn't be after being surprised so many times; so my surprise at being surprised is surprising (to be all meta). Today I had to give a talk to a group of devs on the topic of Anything. I started out with snippets of my canned talks and then it diverged as people asked questions. Anyway, it is a virtual goods company so here is the talk I should have given, but the idea didn't hit me until I was in the car on the way home.

Encouraging Arbitrage in Virtual Economies

Wikipedia defines arbitrage as

In economics and finance, arbitrage is the practice of taking advantage of a price difference between two or more markets: striking a combination of matching deals that capitalize upon the imbalance, the profit being the difference between the market prices.

So arbitrage is being the classic and much maligned middle-man: buy from one guy and then resell to another guy at a slightly higher price. People hate middle-men because they seem to make money from doing nothing (they don't make anything dammit!), but they serve the very useful function of smoothing out prices. In return for their cut they add information to the system and more information is a good thing.

When there are a lot of people arbitraging a good you get more stables prices. When there are several pawnshops close by you can bet that they will charge very similar prices. You can walk into one and sell your iPod without doing research and be pretty sure you got a fair price.

When there is competition the gap between what you get paid for that iPod and the price of buying one also shrinks. This has huge ramifications because it basically allows you to rent items by buying them and then selling them back. Need an ebook reader for the weekend? buy it today and sell it back on Monday for $10 less.

What this means for virtual economies

In a heavily arbitraged market users can try out new things cheaply: instead of renting that ebook reader they can try out a FarmVille tractor or that fancy new Vorpal blade, safe in the knowledge that they can sell it back at a small loss. Users like to do new things, so this is good. Since arbitrage adds information to the system it also tells users what they should be doing. They can see through prices what other users value and then do more of that.

Note that I added some big qualifiers in there: heavily arbitraged, competition, a lot of people. To get those things you need to make arbitrage as easy as possible. You need a market that supports both "bid" and "ask" prices so the gap is obvious even to casual users. You also want as many people doing hard-core arbitrage as possible so give them as much information on past prices as possible in a very easy to use format. I really can't emphasize that enough: I've ruined many in-game economies by actively removing information from the system by say, buying up all the copies of an item. Other people didn't have any current information on the prices, and the games didn't offer historical information so it was only me who had it. This is possible even on sites that offer some historical information (Duels only listed the last N sales so I just sold to myself a bunch of times to clear the buffer).

If you have a bunch of people doing this and the transactions only happen in-game you also have a ready made money sink. Tax the transactions at X% and put the proceeds in /dev/null. An interesting question is what that tax rate should be. If it is high you get less arbitrage because there is no money to be made, and as a knock on the other users get less benefit because the cost of "renting" an item is higher.

PS, on a python-related note: there are lots of python folks that care about this stuff in real life markets too. Corner Hettinger or Glyph sometime when you have a few hours to spare.

Comparing the Ruby/PHP/Python C Interpreters

The other day I went poking around the Ruby and PHP interpreters (the current stable versions). I hadn't looked inside PHP since the 4.x series and Ruby I had never checked out. Like CPython the internals of both PHP and Ruby look something like their resulting language, but in C. For each interpreter I just compiled it and looked at how core types and extension types were implemented.

Ruby 1.9.1

Ruby-the-language has lots of syntax and its core types are just as extensible at run time as classes written in ruby (you can monkey patch core types). The compile was clean and runs with -Wall, generating just a couple warnings. All the unit tests passed. The grammar is implemented with lex/yacc and the resulting parse.c file took 10 minutes to compile on my 1.5GHz machine. Did I mention the grammar is big?

There is no difference between ruby core types and extension types written in C. That is mostly true in python but ruby goes all the way. The C-struct that holds information about the ruby type has a hash map that contains all the type's methods - and I mean all of them. Here is the interface for adding a __add__ method (cBignum is the core integer type)

rb_define_method(rb_cBignum, "+", rb_big_plus, 1);

The "+" is the not-so-magic name for the addition operator. The type's hash uses "+" as a key that points to the value of the addition function. That is a beautiful interface compared to CPython, where you have to put the __add__ method in the right place in a struct[1]. As an optimization the "hash" is actually a list if the number of methods is small; method strings are interned and assigned a number - I'm not sure why this is faster than just keeping the hashkey on the string and always using a dict, but I assume someone benchmarked it.

PHP 5.2.13

[NB, I should have looked at the 5.3.x release but the 5.2.13 release was at the top of the homepage when I went looking]

I hadn't looked at PHP since the 4.x series (see my why I started using Python post). PHP has added some nice features since then, like namespaces, but the interpreter looks much the same. The compile uses a custom wrapper around gcc and is very spammy: a dozen -I include directories on each line for hundreds of C files. It does not use -Wall by default so if you want really really spammy turn that on. After compiling PHP I ran the unit tests and 7 failed[2]. All 7 had to do with bad conversions between signed and unsigned numbers (a negative signed int is a positive unsigned int). This is a production release so those failures are not confidence inspiring.

Like PHP-the-language the C interpreter makes a big distinction between core types and extension types. The core types are int, string, and list/hash (a hybrid). The C-struct is a union that has is either an integer, string, list/hash, or "resource" (everything else). Extension types can't do operator overloading so the interpreter has if/else clauses for handling the core types. Methods are added by registering them by resource number in a global registry.

Objects get passed around in the core as pointers to pointers, and sometimes as pointers to pointers to pointers. I'm not sure why, but this can't be good for speed.

Python 2.5+ 3.x

I'll lump all releases of Python after 2.5 together because the internals are very similar. The AST (abstract syntax tree) that the byte compiler uses was rewritten and simplified for the 2.5 release and there haven't been any big changes to the internals since then. The 3.x releases made some big simplifications, but they still use the same framework.

Like Ruby, Python compiles cleanly and uses -Wall, generating few warnings. The test suite passes. Python doesn't make a distinction between core types and extension types: if you copied Objects/dictobject.c and renamed it "mydict" [insert dict joke here] you could ship it as a module and "import mydict". The only difference is that the byte compiler knows that when you type "d = {}" you mean "d = dict()".

The C-struct for python types is a bit more complicated than the ruby one. It has specific slots for all the magic methods like __add__ instead of keeping them in a hash map like it does for pure-python classes. Like PHP the execution loop does have some if/elses for core types like integer, but unlike PHP this is just a speed hack and not a requirement (I assume Ruby does something similar).

Conclusion

So there you have it. All three interpreters look much like their parent language once you get under the hood. I'd mention the perl interpreter too but it's been years since I dove into that one; but guess what? It looks like perl.

[1] python-dev has several threads about adding a similar simple interface. Someone just has to do the work (at PyCon Hastings said he's exploring it).
[2] I downloaded PHP 5.3.2 and the 7 test failures I saw are fixed, but I get 9 new and different failures.

PS, blogger hates H4 tags. Why the extra newline?