PyCon on The Charles: Night 2

We had the 2nd night of PyCon rehearsals tonight at the Boston Python Meetup and it was my turn to practice. My PyCon talk is Stop Writing Classes. The talk presents some examples of code that were overly complicated and how I refactored or avoided them completely. Five years ago I couldn't have given this talk because it would have been just a spleen cleaning list of things I found icky but couldn't describe. After many years of forcing myself to write constructive things in code reviews I know better why those things are icky.

I was supposed to go first but there were, of course, A/V problems[1]. I've used that same laptop in the same room before but this time - no dice. Anytime I changed slides the sound would blurp static and the picture would drop; I got a curious tingling in my hand when I power cycled the machine. Then I was supposed to go second and this time I plugged the laptop into the same ground as the projector - still no dice. Last and finally I was supposed to go third: this time with my slides (they're HTML!) on Ned's laptop which had faithfully delivered his own talk (in HTML!) Pragmatic Unicode, or, How do I stop the pain?. No. Fucking. Dice. But then he restarted his laptop a couple times and it was gravy from then on.

The official format for the event is that speakers will deliver their talk as-prepared for PyCon. For speakers that feel most comfortable delivering their talks as-prepared this is very useful rule (Ned is a beautiful fscking machine at delivering his talks as planned). For speakers that deliver their talks more ad-hockily the official format is weakened or ignored; last week Glyph gave his first best guess as a talk, and this week..

To say I spent little prep time on my talk would be unfair. To say I spent just 4 hours reformatting my research notes into a talk would be completely fair. As it usually happens I presented my notes as a talk while speaking extemporaneously [first drafts happen].

I panicked a bit when I had gone through 16/20 slides in *only ten minutes*. I had four slides left [two of which I knew I didn't really like] and fifteen minutes to fill. Two minutes a slide is a pretty good rule-of-thumb and I'd completely blown it. At this point I told all the stories about the first 75% of the slides (many of those stories were about redemption) and then reiterated a dozen times in a dozen ways why simpler things were both simpler and better. The laughs and nods were contagious and I hit my 25 minutes.

So, as in past years of dress rehearsals I learned something. And now I will get on with the usual thing of compulsively rehearsing and rewriting the talk until it is good enough that I think it won't be wasting other people's time. 40+ hours of rehersals and rewrites. Mostly because I love you guys and partly because I don't want to suck.

[1] There was a guy who complained that the A/V problems were easily avoided and as such were cutting into his personal time (his personal time attending an event with free talks, free pizza, and free beer). Multiple people reminded him that volunteers shall not be questioned[2], and many more people privately offered that they would drop the guy if he complained twice[3].
[2] Ned & Jessica.
[3] Oh, everybody but him.

PyCooooooonnnnn!

The Talk schedule has been published and I'm on it. For which I'm very greatful - I saw the full list of ~400 submissions and there were many I'd like to see which didn't make the cut. [To all those people I'll let you in on a secret: submit the same thing at a smaller conference. They have 80%+ acceptance rates - basically they just toss out the crazies and let everyone else speak. PyCon was like that not too long ago. So go and speak wherever they will have you and get some practice: my first PyCon talk never would have been accepted under today's standards.]

Name Dropping

Normally I'm a Hallway Track kind of guy but there are some talks from friends I kinda want to see as it happens as an event: Larry Hastings, Doug Hellman, Jeff Rush, Benjamin Peterson (is he old enough to drink yet?), David Mertz, Moshe Zadka (long time/first time?), Ned Batchelder, Alex Martelli, Barry Warsaw, Dino Viehland, Maciej/Gaynor/Rigo (pypy), Armin Ronacher, Glyph (I'll tell you his real name in trade for a beer), Raymond Hettinger, Gary Bernhardt, and Jim Baker.

And that's just the list of speakers I know will be good, there are many more that look good on paper that I want to see.

You'd better nail it kid

There are some perennial talk topics because it is always nice to have a fresh take on important things. That said, being the second (or third, or fourth) person to retread the same ground is difficult.

Larry Hastings is treading old ground with Stepping through CPython, a tour of what really happens when your python code runs. I did it in 2006 as "Writing your own python types in C" and Ned Batchelder did it much better in 2009 as "A Whirlwind Excursion Through Python in C". I'm looking forward to Larry's take.

Getting the most out of Python imports by Eric Snow. Note that he speaking about imports and his name isn't Brett Cannon.

The Magic of Metaprogramming by Jeff Rush and Metaclasses by Luke Sneeringer. Not just well tread ground, but ground always loaded with fresh landmines.

Decorators and Context Managers by Dave Brondsema. I'm very much looking forward to this one, as those things have been my topics in the last few years. I haven't met Dave but I imagine he has giant brass balls.

And the other thing

Raymond Hettinger will be presenting The Art of Subclassing and I will be presenting Stop Writing Classes. Raymond and I have agreed on pretty much everything in the decade I've been involved with python (he was already a commiter when I showed up*). I doubt we are in disagreement about classes or subclassing, rather we are giving talks about two different topics. His is "here is how you do hard things well" and mine is "don't do hard things, they are hard and you'll probably fuck it up". Two weeks ago I wrote a caching class that used __new__ and this past week I refactored it to not do that; the code was half the length and achieved the same result. My only question to the team was "why did no one yell at me two weeks ago?"

* My favorite PyCon story is my first one: 2003 in DC and I got to meet everyone that was previously just names on a mailing list. Hettinger was working the registration desk and I was like "thats neat" and when I stepped out into the smoker's lounge I introduced myself to the three hangers about. "Hi, I'm Jack Diederich" was followed by "Hi, I'm (Alex Martelli|Tim Peters|Christian Tismer)". That was awesome.

Analog Analogies

At work there has been a recent interest in explaining tech things in terms non-tech people can understand. Not just because it helps explain tech things to non-tech people but because it also clarifies to tech people what the purpose of the thing is.

We hit on the term "analog analogies" because one of my coworkers kept saying "analog" instead of "analogy" and I didn't think of the two terms as synonymous. I thought of "analog" as opposite "digital", but not for a second did I think "analogue" was synonymous with "analog" (it is). Hence the redundancy. So "analog analogies" became the phrase meaning "explaining a digital thing by analogy."

By way of for example[1] I'll be leading off my five minutes at the Q3 kickoff with one of my favorite analog analogies: the high and low laundry equilibriums. There are only two stable equalibriums in laundry: you can keep all your clothes clean and do laundry every time you have a basket of dirty, or you can have a full clothes hamper and only do a load when you need something to wear.

It is a not terrible analogy for technical debt. But the analogy falls down because even in a medium sized project you might have several different laundry equilibriums across different aspects: are we keeping a lid on our code bottlenecks? backend queries? can we rent/buy faster machines? And the worst part is you might not know which is lacking. I think the Germans have a word for how you know: messerschitz. You gotta do that too.

[1] I like to pretend to be all Dutchy on occasion.

MUD Nostalgia

For programmers of a certain age - those who saw their first internet connection in the 90s - MUDs [Multi User Dungeons - basically D&D online] were de rigueur. Open Source was in its infancy and this new thing called linux came on a stack of 35 3.5" floppy disks (configuration was equally hostile). So if you wanted to contribute to a collaborative project MUDs were where it was at. A number of python people have outed themselves as [ex?]mudders, including Glyph (Twisted founder, Twisted started as a mudlib), Doug Napoleone (PSF board), Jesse Noller (PSF board), Richard Tew (Stackless Python), Will Kahn-Greene (Miro, and mud client Lyntin), and yours truly (I maintain telnetlib, need I say more?).

Learning is in the doing, so what I got of college was mostly lots of free time to do stuff. I spent maybe 400 hours in four years doing programming for classwork and over 4000 hours writing LPC code for a now defunt MUD* (using /bin/ed - the only supported editor). Writing a mudlib was my "turtles all the way down" moment, when it clicked that user level programs were just programs that were composited from other programs; The trick was that the lower level stuff had an agreed upon naming convention but otherwise wasn't special at all.

I'll admit to being somewhat jealous of the college students of today - it is easy to contribute to projects that actually matter. Being able to rely on ping times being under 500ms is nice too (Europe used to just _go away_, sometimes for days).

I haven't left MUDing completely behind. I still use it to try out new methodologies in coding and just play around in general. Leanlyn (http://bit.ly/leanlyn) is my Lyntin fork that is about 10 years old and will never be finished (or be less than user hostile).


* The dead mud was Ether Realms. I can still be found sometimes at OverDrive (a Lehigh staple) and Three Kingdoms.
** College came with lots and lots of free time. I spent way more than 4000 hours just chasing tail. Imagine me but with shoulder length hair, an eyebrow ring, and a flannel shirt (it was the 90s)

PyCon Speaking

Do read Moshe's How to give an OK talk, esp the Narrative part. It feels quite silly to repeat yourself on every slide but you are preforming live and not writing on paper - so tell the audience why slide N+1 is important and for the same reason as slide N at every opportunity. During my talk I felt as though I was being overly repetitive, but on watching it after the fact I feel like I didn't hammer home the point enough (you should do more shit like this!).

Speaking of speaking: I asked a few popular conference warriors (Martelli & Hettinger) how many talks they had given and when they first felt comfy giving talks. They didn't know. I should have asked Beazely and C.T. Brown but they're professors and get more practice in a year than most people get in a lifetime.

But seriously, those popular conference regulars didn't know. Hettinger estimated he had spoken 30 times. His wife then pointed out he did 15 talks last year and so he upped his estimate to "more than 15 twice." Martelli estimated 50-100 talks but was equally uncertain. Myself, I've only done 7-8 and still get nervous if the room is less than 30 or more than 300 people. Hettinger said something like "I still get nervy when it the audience is 10x what I'm used to." I couldn't coax an admission out of Alex (I had only vague thoughts on the topic when I asked, and no narrative).

Take it all as paraphrased hearsay, but anytime I have to step up to a podium I just say to myself "suck it up, princess." It mostly works.

PyCon Detritus

PyCon proper is over and sprints have begun.

Language Summit

The Summit was boring again this year, and this is good. The language moratorium is over so PEP 380 ("yield from") is moving forward. The stdlib will get its own repository so it can be shared between CPython/Jython/IronPython/PyPy. Likewise all the commiters from the not-CPython projects will have commit privs to CPython (most already do).

My Talk

Useful Namespaces: Decorators and Context Managers went over well: video available here and here's the live Convore thread from the talk.

Every year I say I'm not going to do another talk, but it is years like this that make it worthwhile. Dozens of people I don't know have offered me thanks, jobs, free beer, and ZJs. There is no worse feeling than spending 40 hours on something and having it be less than good. I'm a happy camper.

The first question was asked by Larry Hastings ; it's become an inside joke because Larry has asked the first question at ALL my past PyCon talks (having a confederate to break the ice on questions is handy). I zinged him a bit in my response ("thank you for pointing that out, and may I point out it's just an example slide"); apparently most people thought I was razzing a random questioner even though I was addressing him by name.

Another questioner objected to my use of a module named "UnicodeNazi" because it isn't nice to talk about Nazis even (or especially) jokingly. I restated his objection and said he should take it up with the module's maintainer. That maintainer is Austrian, natch.

The room was pretty full. Not going out the night before definitely helped my delivery. There were some A/V problems at the start (lots of those this year) so I rushed the last few slides. Better that than running short though.

Other People's Talks

So far, so good. Yesterday my talk was in the afternoon so I spent all day preparing and didn't make any talks. Today I went to three: Alex Gaynor's "Python's Data Structures," Larry Hasting's "The Python That Wasn't," and Richard Saunder's "Everything You Wanted to Know About Pickling..." Most of my time was consumed by the Hallway track.

Hotel

The hotel has banned alcohol (or at least implied a ban) in public spaces that is not purchased from the hotel. I don't know if this is because of money or just because they want fewer drunks walking around. Annoying either way.

Diverstiry

A woman came up to me and said that something I told her last year had stuck with her. Last year she was sitting at a table with myself and another core dev, and holding her own when discussing eldrich corners of Python. The topic turned to contributing to open source and she had a list of reasons she felt she couldn't contribute, none of them very good. The good news is that she sprinted remotely last year, is sprinting in person this year, and has contributed code to projects over the past year. Oh, and the thing I said that stuck with her? "Suck it up, princess." So I have struck a blow both for and against diversity. [FYI, I don't remember saying that but it is the kind of thing I might say after a few beers]

Happy PyCon to Everyone

I'm in the happy majority of Python commiters that mostly idle except for the scheduled holidays like bug weekends and release alphas, and only awake for the high holidays like PyCon. I'll be speaking again this year and I'll be there for the language summit before and two days of sprinting afterward.

This weekend I stopped researching my talk "Useful Namespaces" and started actually writing slides; the theme is a riff on "Namespaces are one honking great idea -- let's do more of those!" and the first slide is "I monkey patch the shit out of everything." Coincidentally I won't be sprinting 100% on core stdlib this year but will instead be trying to fix up my monkey patches and tests on other projects that haven't yet made it upstream.

On Deployment Policy

Q: Don't we have a policy about drinking during deployments?
A: Yes, we always drink during deployments.
Q: I thought the policy was "Friends don't let friends deploy drunk?"
A: If you can still type how drunk can you be?

Encouraging Arbitrage in Virtual Economies

I'm always surprised by how different companies conduct interviews - I shouldn't be after being surprised so many times; so my surprise at being surprised is surprising (to be all meta). Today I had to give a talk to a group of devs on the topic of Anything. I started out with snippets of my canned talks and then it diverged as people asked questions. Anyway, it is a virtual goods company so here is the talk I should have given, but the idea didn't hit me until I was in the car on the way home.

Encouraging Arbitrage in Virtual Economies

Wikipedia defines arbitrage as

In economics and finance, arbitrage is the practice of taking advantage of a price difference between two or more markets: striking a combination of matching deals that capitalize upon the imbalance, the profit being the difference between the market prices.

So arbitrage is being the classic and much maligned middle-man: buy from one guy and then resell to another guy at a slightly higher price. People hate middle-men because they seem to make money from doing nothing (they don't make anything dammit!), but they serve the very useful function of smoothing out prices. In return for their cut they add information to the system and more information is a good thing.

When there are a lot of people arbitraging a good you get more stables prices. When there are several pawnshops close by you can bet that they will charge very similar prices. You can walk into one and sell your iPod without doing research and be pretty sure you got a fair price.

When there is competition the gap between what you get paid for that iPod and the price of buying one also shrinks. This has huge ramifications because it basically allows you to rent items by buying them and then selling them back. Need an ebook reader for the weekend? buy it today and sell it back on Monday for $10 less.

What this means for virtual economies

In a heavily arbitraged market users can try out new things cheaply: instead of renting that ebook reader they can try out a FarmVille tractor or that fancy new Vorpal blade, safe in the knowledge that they can sell it back at a small loss. Users like to do new things, so this is good. Since arbitrage adds information to the system it also tells users what they should be doing. They can see through prices what other users value and then do more of that.

Note that I added some big qualifiers in there: heavily arbitraged, competition, a lot of people. To get those things you need to make arbitrage as easy as possible. You need a market that supports both "bid" and "ask" prices so the gap is obvious even to casual users. You also want as many people doing hard-core arbitrage as possible so give them as much information on past prices as possible in a very easy to use format. I really can't emphasize that enough: I've ruined many in-game economies by actively removing information from the system by say, buying up all the copies of an item. Other people didn't have any current information on the prices, and the games didn't offer historical information so it was only me who had it. This is possible even on sites that offer some historical information (Duels only listed the last N sales so I just sold to myself a bunch of times to clear the buffer).

If you have a bunch of people doing this and the transactions only happen in-game you also have a ready made money sink. Tax the transactions at X% and put the proceeds in /dev/null. An interesting question is what that tax rate should be. If it is high you get less arbitrage because there is no money to be made, and as a knock on the other users get less benefit because the cost of "renting" an item is higher.

PS, on a python-related note: there are lots of python folks that care about this stuff in real life markets too. Corner Hettinger or Glyph sometime when you have a few hours to spare.

Comparing the Ruby/PHP/Python C Interpreters

The other day I went poking around the Ruby and PHP interpreters (the current stable versions). I hadn't looked inside PHP since the 4.x series and Ruby I had never checked out. Like CPython the internals of both PHP and Ruby look something like their resulting language, but in C. For each interpreter I just compiled it and looked at how core types and extension types were implemented.

Ruby 1.9.1

Ruby-the-language has lots of syntax and its core types are just as extensible at run time as classes written in ruby (you can monkey patch core types). The compile was clean and runs with -Wall, generating just a couple warnings. All the unit tests passed. The grammar is implemented with lex/yacc and the resulting parse.c file took 10 minutes to compile on my 1.5GHz machine. Did I mention the grammar is big?

There is no difference between ruby core types and extension types written in C. That is mostly true in python but ruby goes all the way. The C-struct that holds information about the ruby type has a hash map that contains all the type's methods - and I mean all of them. Here is the interface for adding a __add__ method (cBignum is the core integer type)

rb_define_method(rb_cBignum, "+", rb_big_plus, 1);

The "+" is the not-so-magic name for the addition operator. The type's hash uses "+" as a key that points to the value of the addition function. That is a beautiful interface compared to CPython, where you have to put the __add__ method in the right place in a struct[1]. As an optimization the "hash" is actually a list if the number of methods is small; method strings are interned and assigned a number - I'm not sure why this is faster than just keeping the hashkey on the string and always using a dict, but I assume someone benchmarked it.

PHP 5.2.13

[NB, I should have looked at the 5.3.x release but the 5.2.13 release was at the top of the homepage when I went looking]

I hadn't looked at PHP since the 4.x series (see my why I started using Python post). PHP has added some nice features since then, like namespaces, but the interpreter looks much the same. The compile uses a custom wrapper around gcc and is very spammy: a dozen -I include directories on each line for hundreds of C files. It does not use -Wall by default so if you want really really spammy turn that on. After compiling PHP I ran the unit tests and 7 failed[2]. All 7 had to do with bad conversions between signed and unsigned numbers (a negative signed int is a positive unsigned int). This is a production release so those failures are not confidence inspiring.

Like PHP-the-language the C interpreter makes a big distinction between core types and extension types. The core types are int, string, and list/hash (a hybrid). The C-struct is a union that has is either an integer, string, list/hash, or "resource" (everything else). Extension types can't do operator overloading so the interpreter has if/else clauses for handling the core types. Methods are added by registering them by resource number in a global registry.

Objects get passed around in the core as pointers to pointers, and sometimes as pointers to pointers to pointers. I'm not sure why, but this can't be good for speed.

Python 2.5+ 3.x

I'll lump all releases of Python after 2.5 together because the internals are very similar. The AST (abstract syntax tree) that the byte compiler uses was rewritten and simplified for the 2.5 release and there haven't been any big changes to the internals since then. The 3.x releases made some big simplifications, but they still use the same framework.

Like Ruby, Python compiles cleanly and uses -Wall, generating few warnings. The test suite passes. Python doesn't make a distinction between core types and extension types: if you copied Objects/dictobject.c and renamed it "mydict" [insert dict joke here] you could ship it as a module and "import mydict". The only difference is that the byte compiler knows that when you type "d = {}" you mean "d = dict()".

The C-struct for python types is a bit more complicated than the ruby one. It has specific slots for all the magic methods like __add__ instead of keeping them in a hash map like it does for pure-python classes. Like PHP the execution loop does have some if/elses for core types like integer, but unlike PHP this is just a speed hack and not a requirement (I assume Ruby does something similar).

Conclusion

So there you have it. All three interpreters look much like their parent language once you get under the hood. I'd mention the perl interpreter too but it's been years since I dove into that one; but guess what? It looks like perl.

[1] python-dev has several threads about adding a similar simple interface. Someone just has to do the work (at PyCon Hastings said he's exploring it).
[2] I downloaded PHP 5.3.2 and the 7 test failures I saw are fixed, but I get 9 new and different failures.

PS, blogger hates H4 tags. Why the extra newline?

Some Odd Observations as a PyCon Speaker

1) Your answer to the first question after your talk will be simple, neat, and wrong.
2) That question will have been asked by Larry Hastings.

Extrapolation from my experience might fail in your particular circumstances because Larry isn't omnipresent. It might fail for my talks too: the pycon video archives only go back to last year.

NB "wrong" in the sense of "less than optimally correct." I included a fuller answer on my published slides both years. Which no one will see.

PyCon Wrapup II: Python Stuff

[People stuff is trees-and-forest, so here is a post on what was done about Python at PyCon]

The talks were good and the 5x (as opposed to 4x) tracks didn't seem to hurt. Worst case: any talk you missed you can watch on pycon.blip.tv. Speakers were aware that their talks would be recorded so using laser pointers (instead of highlights or spoken words) is going away. This is a sideways move - laser pointers were useful right up until they weren't.

The Language Summit was far more boring than last year. Python 3.x issues are mostly settled from the core-dev standpoint so the big issues were disutils (how, and at what level should python packages care about packaging) and alternate implementations. Unladen Swallow was the talk of the town not because they are the first alternate implementation of Python but because they are the first implementation that plans to ship with benefits and no tradeoffs. Did I say no tradeoffs? It was unanimus that both disutils2 and Unladen Swallow would be integrated once the tradeoffs were wholly positive. Who can't get behind that?

PyCon sprints were smaller for python-dev this year but I couldn't tell if that was true for other major groups like Twisted and Django (the rooms were more broken up this year). I can say for certain the python-dev sprint had both fewer regulars and fewer pure-newbies; One local EE postgrad wanted to help out and simultaneously tried to make me care that his badge and his name didn't match; He went by "Cedric" but his badge said something else (I believe he was Caribbean). I have a long standing amusement with first names: my birth certificate doesn't say "Jack," Titus Brown's doesn't start with "Titus," and Alex Martelli's doesn't say "Alex." For that matter Guido's may say "Guido" but he doesn't care how you pronounce it. In fact all the groups I'm a participant and care about most don't care who you are legally, and don't ask for ID at the door (to riff on my last post, caring about legal ID is a "negative trust" cue).

The PSF (Python Software Foundation) exists to serve two different classes of users: end-users and People-who-hire-end-users. To put it differently the PSF is a single purpose organization that wants more users both from the bottom-up and top-down. The bottom-up stuff has been easier to organize in the form of "your-locality-here" Cons. While the PSF wants to help people to do more of that they also want to aid the corporate users who have an interest in Python. Getting companies to spend money and organize sprints has happened quite successfully before, but very irregularly (see the Need for Speed sprint). Quite happily I can say that if half the events that were spit-balled at PyCon come to be then sprints will be even more prolific in the near future (both bottom-up and top-down) and they will be just as free but even more topic specific (2to3 porting, hardcore dev stuff). At least four groups have intentions to do an event in Boston, for instance.

The CPython bug tracker has 2000 outstanding issues (a mix of bug reports/feature requests/doc requests). A new status field named "languishing" was added because there are a lot of bugs that have +1/-1 comments by core-devs but no resolution; it is a classic "middle school dance"* deadlock where no-one feels they have authority and is just waiting for someone else to pronounce. AMK and I closed about 20 of these during sprints (some applied, some rejected) but there are still a ton of these bugs outstanding. They just need a champion (for or against) to get resolution. Alex Gaynor recently did a post about who-gets-what-commit-rights on various python projects (not including python-dev). Python-dev can be disfunctional because there are 120 committers and everyone assumes there is someone else who knows better for any particular bug (the "middle school dance").

Steve Holden and Michael Foord both have the audio bug: imagine what camera crazy people spend on digital SLR gear but apply that to audio eq. Between the two they captured tens of hours of audio at PyCon and some of that should start showing up soon. Editing is the hard part in making raw into general interest so maybe four or five hours of that will appear for general publication. The blackmail snippets are easy to produce; if any exists you've already received it and the adjoining demands (Foord sensibly priced his at slightly less than a trans-Atlantic plane ticket; Holden has yet to publish a price list).

* "middle school dance" is comp-sci jargon for a deadlock where party A is waiting for party B to do something and the reverse. The allusion is to boys standing on one side of the gym waiting for the girls to ask them to dance and the girls standing on the other side, etc.

High Trust, High Responsibility Communities

[This is a blog version of a lightning talk I didn't give at PyCon]

People Hacking is as old as time, and every long lasting organization engages in it. One example of hacking people (and the title of this post) is to tell people you trust them. This "trust cue" invites reciprocity -- people behave well because you've signaled that you expect them to. Google's "don't be evil" motto works this way; the company has promised the outside world that it should be trusted so internal employees feel pressure to live up to the promise.

You can do the opposite and tell people that you don't trust them ["negative trust cues"] and people being people their behavior is predictably cautious and non-cooperative. I once worked for an ex-DEA prosecutor and to put it mildly the employer-employee relations weren't stellar.

The Python community has many positive trust cues going for it. The coding end is open source so everyone is a volunteer by definition, and volunteering for anything is a positive trust cue. Likewise PyCon introduces people face-to-face and having met someone as a real-live-person (even once!) invites an obligation to be more forgiving in the sterile world of email and bug trackers (was he being a dick or just having a bad day? I've met him, must have been a bad day). I got my commit bit after never having one of my many patches to CPython accepted*, but curiously not long after I met the guy who had rejected them (Raymond Hettinger, who was working the registration desk when I checked in).

I am/was a member of a large number of long standing volunteer orgs (most are fraternal**) and have seen many variations on trust cues: the big ones are the Boy Scouts, Habitat for Humanity, the Kappa Alpha Society, and the Free Masons. None are as old or put as high an emphasis on trust cues as the Masons - everything is setup to do charity and avoid conflict. It is prohibited to drink before/during meetings***, discuss politics, or discuss religion. The taboos are so strong that there aren't legal punishments for violating them - you are trusted not to violate them so (by reciprocity) no one does. Very few of the members are half as smart as the average lunch table at PyCon, but it is impossible not to like the members of my lodge because every time I see them they are doing charity: giving blood, contributing to food banks, etc. The trust cues are through the roof (one side effect of that is a very high attrition rate -- people quit because the activities are quite stolid and boring).

So the Python community in general and PyCon in particular inculcate positive trust cues (when was the last time your saw a post titled "PyCon sucked?" never). That said, I ran away from the python-d7y list because it was so loaded with negative trust cues. See Anna Ravenscroft's PyCon2010 talk for very healthy ways to promote positive trust cues and eschew negative ones. The topic mandated the opposite of the "no politics, no religion" of my other, more boring, org and the results were predictable. The d7y list was mostly harmless but 5% of the talk was by people that had no skin in the game (and hence no expectation of kindness or reciprocity). To be vague and ablative the demands of the 5% was a laundry list of negative trust cues: censorship, name-and-shame, and other minor atrocities. As a result most discussion of the list actually took place off the list because plain discussion on the list was impossible (par exemplar was one of the moderators emailing me off-book "I wish I could +1 but I can't. We need a new list.").

In closing the Python community exploits human nature in the form of trust == responsibility pretty well. I am happy to be exploited thusly because I'm human and it tickles my humanity. Assign bugs to me and - if I've met you or have met someone who has met you - I'll be quick to close it with a more delicate comment than otherwise.

* My patches weren't bad or wrong, they just weren't enough of a speed boon considering the maintenance overhead to be added to the stdlib.
** I have three brothers and no sisters so that's where I'm at ease. Though until recently (last century) "fraternal" meant "brotherhood of man" instead of the narrower "male only" so there still exist some mixed sex and female-only orgs that have "fraternity" in the name.
*** The "no booze" is very unusual for voluntary associations in the US. Most clubs have a members only bar that is the major fundraiser for the org. For instance, when I lived in Pennsylvania I was a social member of the King of Prussia Volunteer Fire Company - I was one of the thousand members who didn't fight fires but paid $30/year for my membership card (you needed to be nominated by an actual fire-fighter to join but like most orgs this was a technicality -- they were mainly interested in keeping out people who couldn't find one related person to say something nice about them)

PyCon Wrapup

Here are my assorted thoughts on PyCon this year. Long story short: Marvelous.

The Atlanta location was actually in downtown Atlanta and not like the Chicago or Dallas PyCons where it was hosted nearer the airport than the city itself. There were dozens of restaurants and bars at every price-point within walking distance, and even more than that if you took the train a few stops. My favorites were the cheap local places like the Vietnamese soup joint (via Titus Brown) and the chili cheese dog joint (via Larry Hastings). I liked the high price steak joints too, but you can get good food anywhere so it's less of a kick. Now that we have some institutional knowledge about eating in Atlanta I expect to eat even better next year.

The conference organizers, program committee, and network guys all did a perfect job this year. You won't hear about it much because when things "just work" there is nothing to talk about. As an anecdote about how well things were setup: during sprints one developer was loading up his computer with video because the 'net download speed was better than he could get at home over fiber. Another anecdote: during sprints a different conference was coming in and had signs that listed our sprint rooms as part of their conference venue; I know nothing about the particulars but there was a flurry of panicked activity and the situation was handled.

Labor must be cheap in Atlanta because every business had more employees per square foot than I've seen in any other American city - including many Southern ones.

Flying sucks. While PyCon-qua-PyCon wouldn't be possible without cheap flights (trans-Atlantic esp) I would really have liked a heads-up that I would be stuck in Philadelphia for five hours. I could have called up some kith & kin and hit the downtown instead of sitting in the terminal for a dog's age.

I mostly got to talk to the people I wanted to see but haven't seen since last year (including the other Boston locals). I also met some new and interesting folks. Unfortunately one week isn't enough to do enough of both no matter how little sleep you get; but I'm pretty happy with the results.

A big shout-out to CCP games. They have an Atlanta office so an even bigger chunk of the Iceland office was at PyCon than usual. I met their Atlanta marketing director who is the guy who behind the viral marketing cum bombscare for ATHF a few years back. Good to see he landed on his feet. The CCP guys are a big reason why I slept very little, and happily so. Allow me a semi-related shout-out to Mr & Mrs Wayne & Wendy Witzel; I met them at a 'con or two ago, and Wayne volunteered as my session chair this year. They are both big EVE players and know Reykjavik better than I do (I don't know much). [Also, her name is not "Wendy" which I have been told repeatedly but refuse to acknowledge. It should be Wendy, dammit.]

Every secret project tried to do a release in time for PyCon, or so it seems. Some of the company ANNs include Sauce Labs and Nasuni. Additionally the conference was a big bag of leaky secrets so expect more announcements quite soon (I'm not contractually obligated to not-say but I will so as to not blunt their PR).

Sprints were smaller this year than last, especially python-dev. The sprint rooms at late night were also less, umm, vigorous than in past years. Sprints are still ongoing but for the first few days, when I was there, I can say there were zero card games played. Not holdem, not asshole, not set, not nothing.

I normally don't ask about late night sprint activities out of a sense of propreity, but I'll make an exception in this case. Was anyone around circa 3am on Wednesday? I have a lump on my head and I'm somewhat curious about its origins.

PyCon Prep

If you are giving a talk take the time to watch AMK's How to Give a Python Talk. I posted some thoughts on speaking last year too. For anyone attending: go listen to the Pre-PyCon podcast where several regular PyConers give a conference HOWTO. It features some solid advice such as "accept the first dinner invitation you get" to which I'd add my favorite "eat lunch at a table with no one you know." It is easy to get insular especially for repeat speakers* and/or python-devs so the lunch rule helps me break out of that.

• Finalize my talk and cut to length. "Finalize" is a bit of a joke because I tend to rewrite it every time I do a practice run. My talk last year got rewritten in the wee hours the night before I gave it too. Cutting to length isn't much of a problem; If you've done many rewrites and know how many slides you have you can adjust how much you talk about each slide on-the-fly. Relatedly, I'm impressed by the people that give just the talk they rehearsed - it's something I can't do.


• Load up my Kindle. I'm not much of a gadget guy - I wear a watch and carry a pocket knife (I'm down to my last one, @#*@$^ TSA). The Kindle is a gift that I haven't had a chance to use because I don't travel much.


• Load up my laptop. I rarely turn it on because my home office has three nice LCDs so I need to 'svn up' all the python trees and pull some bzr repositories for other projects. (I guessed wrong and started using bzr before Python announced the switch to mercurial).


• Get my home network in order. I need to open up an ssh port so I can tunnel over a secure connection at the con.


• Work on my lightning talks. I'm not 100% sure these will happen but be on the lookout for "High Trust, High Responsibility Communities" and "The Physics of Bowling." The community one will not include farkers, goons, or /b/tards but in my research I did come across this interesting article about EVE Online (EVE uses python heavily and always sends a contingent to PyCon) which includes a player .sig "You may be playing EVE Online, but be warned: we are playing Something Awful." My talk is about hacking people via trust cues and what does and doesn't work in some all-volunteer groups I'm in (Python-dev and Masons feature heavily).


• No beta-blockers, again. I keep meaning to use performance enhancing drugs when I talk (beta-blockers are used by concert violinists to quell the symptoms of stress during performances) but never bother to procure them. Instead I just forgo coffee the day of my talk, which kinda sucks.


• No mustache, again. I enjoyed it but it was a one time thing. Anecdotally: people who know you without facial hair have a hard time recognizing you with it. The opposite is not true: people who know you with facial hair have no problem recognizing you shaved.

* Jack's Rule of Badges: If two people in a group have speaker's badges then everyone in the group will have speaker's badges.

PyCon on the Charles, Night 2

50+ members of the Boston Python Local turned out for PyCon on the Charles, Night 2; a dress rehearsal for PyCon speakers from the Boston area. Ned did an informal survey of the audience and 1/3rd will be at PyCon proper. I did an even more informal eyeball survey of the audience and 20% had beards and 10% were women (the groups were mutually exclusive). Only 2% of the attendees brought cookies and I would have berated her for reinforcing gender stereotypes but I was too busy thanking her for the oatmeal cookies with chocolate chips.

Peter Portante led off with his talk "The words 'Non-Blocking' and 'Asynchronous' Are Not Synonyms and Here's Why" (I paraphrase) which started with examples of documentation that use the terms synonymously and then detailed how they the two concepts are, in fact, different. The talk is a broad tour of what one thing means versus the other as well as who-owns-what as far as kernel-vs-userpace goes. Peter also footnoted the hell out of it so if you want to learn the difference between select(), epoll(), and kqueue() you can.

Glyph gave a beta* of his talk "Turtles All The Way Down" (not a paraphrase). The title references the fact that python behaves in unsurprising ways by design. His explicit goal wasn't to explain how all the "Python's Dusty Corners" work but to hammer home that what even a python newbie knows mostly holds up in strange, dark places because python's authors and maintainers are learned refugees from other camps that publish tomes like the "C++ FAQs (3rd ed)."

Gylph's talk was of particular interest to me because he's talking about the same damn topic as I am and in the same time slot. Thankfully the world is a big place so his talk is also completely different. This is my 7th PyCon so I've seen many topics repeated and I'm always pleasantly surprised by how differently the same material is presented by different people; part of it is personality, part of it is style, and part of it is voice. I, for one, would pay to listen to Alex Martelli read a phonebook.

Antonio Rodriguez gave an alpha of his keynote address. Keynotes are fun because they get to give BIG BOLD AVDICE. Hyperbole is fun because like advice from your father things need to be said twice as loud because you'll only follow half of it. The alpha talk was 3x as loud as necessary but I think he'll trim it down to the keynote normal of 2x based on feedback (and some topics will get moved to the Language Summit instead).

Edward Abrams debuted his "DJ-ing in Python: Audio processing fundamentals" talk. It is an industy talk/python success story. Not my favorite milieu but industry talks do a good job of highlighting tools and best practices. Abrams gave a brief overview of why audio manipulation is hard followed by a list of which python tools he used to solve his problems. NumPy and threads (for I/O) were featured heavily.


* version 6beta2 if his intro slide is to be believed

PyCon on the Charles

The first three Boston PyCon speakers spoke last night at PyCon on the Charles, Night I. NB: If you are near Boston do show up on Feb 3rd for PyCon on the Charles, Night II with Glyph Lefkowitz, Peter Portante, and Edward Abrams.

Francesco Pierfederici did a "python success story" industry talk on using a mix of Python and Fortran-77 (via the "ctypes" module) to eat terabytes of telescope image data in real-ish time - the requirements include picking out all new asteroids from the thousands of known moving objects during the 60 second window before the next 3giga-pixel image arrives. Approximating thousands of N-Body problems in limited time is a challenge, and that's Francesco's job.

Ned Batchelder gave his talk "Go Easier on Yourself: You don't test enough and the reason you don't test enough is because you don't write easily testable code." With good examples. No, great examples. Ned is an invited speaker this year because he is concise. His talk from last year covered the same topic I have talked on before and I kept thinking "He didn't mention X" followed by "I should have omitted that too."

My talk on "Python's Dusty Corners" earned it's more accurate original twitter title "Strange Python Shit." I scrapped my originally planned "overview of everything" and 1000-slide opus that covered comparisons, descriptors, namespace lookups, and context managers. I had to because that morning I tried to put all my well researched and thoroughly unit-tested examples into power point and it came out at .. I stopped kidding myself at 40 slides and gave up. Instead I threw up the bullet point slides on what I had and just tried to talk extemporaneously; The result was about 10 minutes of speaking and 30 minutes of Q & A. This suits me just fine - as I said about Vilnius (host of EuroPython 2008) "I love this town, it is unkempt but not ruinously so."

Most questions boiled down to asking for what purpose this "strange python shit" existed and why the stuff didn't seem more useful in the generic:

Guy: what good is [protocol X] other than that it makes it easy to implement [core python feature Y]?

The answer was usually "none" which is a compelling story by itself.

Based on feedback my final talk will include more graphics (arrow pointing at the dot in "foo.bar") and include a description of why magic can happen (and why it rarely does!) at the dot. My original code examples that implement the various features like instance lookup in pure python will end up on a blog or wiki once I get them formatted. They were way too long to fit on a slide but about right for a web tutorial.

PyCon Plans

My flight is booked, I'm registered, and I have otherwise been pushed, filed, stamped, indexed, briefed, debriefed and numbered*. I'll be there for the Language Summit through two and a half days of sprints.

My "Python's Dusty Corners" talk is coming along. I'll be giving a version of it next week at the Microsoft NERD Center (the too-clever acronym for their "New England Research & Development" house) at the first night of the PyCon on the Charles PyCon dress rehearsal. The talk still lacks a strong narrative - "things you might not know about python" is too loose for my tastes. I'll be polling the audience after to pick their favorite bits which will hopefully allow me to narrow the talk to one of two narratives:

1. Here is a feature that you don't know about and here is what it looks like when a lib/framework does use it and throws an error.


2. Here are all the ways that attribute lookup can be magic.

.. but if the audience feedback is that a hodge-podge of decorators, protocols, and namespace lookups actually is the interesting story I'll stick with that.

The conference organizers owe me some free drink tickets: The schedule is up and I have a good time slot (afternoon on the middle day) but I'm up against Glyph's "Turtles All the Way Down" talk. Hell, I want to see that talk. Thankfully I can because Glyph is giving a version at PyCon on the Charles night II. Feast and famine: I lucked out last year and had a packed talk because Bruce Perens was scheduled to give a keynote on the exact same topic as my talk (thus priming the pump) but then had to pull out at the last moment due to injury. Does Glyph ski? I'll have to ask him.

Be seeing you,

-Jack

* "The Prisoner" remake was terrible. I disliked it so much I felt the need to write about it on my non-python blog.

A Glimpse of PyCon in Your Hometown

As in past years local Python user groups are hosting PyCon speakers to give their talks in a dress rehearsal before PyCon. It is good for the speakers, good for the local user groups, and good for PyCon (more polished talks! I did a teardown/rewrite of my talk last year based on Boston PIG feedback). Here are three dates I know of with locations and speakers:

Toronto, Feb 16, Linux Caffe

* Leigh Honeywell: Think Globally, Hack Locally - Teaching Python in Your Community
* Greg Wilson: What We've Learned From Building Basie
* Mike Fletcher: Debating 'til Dawn: Topics to keep you up all night

Boston, Jan 20, Microsoft NERD center

* Francesco Pierfederici: Python for Large Astronomical Data Reduction and Analysis Systems
* Jack Diederich: Python's Dusty Corners
* Ned Batchelder: Tests and Testability
* [added] Antonio Rodriguez: preview of his Keynote

Boston, Feb 3, Microsoft NERD center

* Peter Portante: Demystifying Non-Blocking and Asynchronous I/O
* Glyph Lefkowitz: Turtles All The Way Down: Demystifying Deferreds, Decorators, and Declarations
* Edward Abrams: DJing in Python: Audio processing fundamentals

Seattle, Jan 30, Paul Allen Center

[added from Seattle PIG in the comments]
* Speakers TBD


The Toronto announcement is here and though Boston hasn't been officially announced it was organized by Ned Batchelder this year (he organized last year's "PyCon on the Charles" too).

I'm happy to see Boston back up in the rankings with six talks this year. I intend to fully [ab]use my slot by briefly highlighting all the "dusty corners" (all the __method__ methods and object protocols) and then, based on feedback, expanding on just a subset for PyCon proper. The opportunity to see talks beforehand is a boon; you can see a talk NOW and free up a slot during PyCon and you can also see talks that you would otherwise skip altogether (I saw a great talk last year that I wouldn't have picked based on the printed program's blurb). And of course you can get ahead on your "hallway track" by catching up with people a month ahead of time.

Cthulhu and Python

Andrew Kuchling broached the topic on twitter so I think that makes it fair game.

For those of you who don't know H.P. Lovecraft wrote the short fiction horror story The Call of Cthulhu in 1926. The story would only become popular much later, and only in recent decades it has been every sci-fi writer's rite of passage to embellish and extend the genre. It is to those authors what The Aristocrats joke is to stand-up comedians - everyone tries to take the same idea and make it their own. Like The Aristocrats the Cthulhu Mythos is widely pursued because it is opened ended (inviting mutations) and nobody gets sued for riffing on it.

There are some sci-fi authors that routinely include Python in their fiction. Charlie Stross is one of those; as it happens at a PyCon I was chatting with a developer who was extolling Stross's work (his girlfriend happened to be DoD and working on a counter virus predicted by Stross and which Stross named in his book after a 1967 experimental Swedish soft-core movie). I had recently read and enjoyed Stross's short variation on the Cthulhu mythos A Colder War (in which Ollie North and Reagan damn us all to Hell using not nukes, but instead weakly godlike beings) and I've been a Stross fan ever since.

Charlie has featured python-3000 in his last couple books, the first of which was put out when py3.0 was just a joking reference. His politics are plain enough (though Scottish he's more of a Fabian Socialist) but his writing is tight so the occasional implausibilities (python dominates the world; the US economy becomes third world in ten years) are forgivable.

I would have tiwttered all that, but the medium doesn't allow it.