Archive for category Python

HOWTO: Sort a Python Dictionary/Map

I use Python all the time for quick little scripting tasks.  There’s nothing better to slice and dice a file, so I use Python for a lot of reporting tasks.  That usually involves building some kind of data structure in my script that I’m slicing and dicing from files.

In my work, I have a LOT of units of work processing in parallel on a grid.  I have GUIDs tagging each unit of work, and that GUID is the perfect key for a Map/Dictionary data structure.  There are times, though, that I want to get the values of the Map and sort by some value in the data itself.  The is important if I want to sort my results by elapsed time or some other interesting metric.

Here’s how you sort a Python Dictionary by some arbitrary value within the data structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
import time
 
work = {}
 
#
# create some sample data...
#
for i in range(10):
    key = "unit_%s" % i
    unitOfWork = {
        "id" : key,
        "data" : {
            "name" : "Turansky",
            "dob" : "03/28",
            "favoriteNumber" : int(time.time()) + i
        }
    }
    work[key] = unitOfWork
 
print "The 'work' dictionary will print the objects randomly..."
for i in work:
    print work[i]
 
print ""
print "Sprinkle some sorting magic..."
 
# but you want to sort the objects by favoriteNumber'
# get your values as a list... you want to use the list.sort() method
units = work.values()
 
# provide a lambda function that references your data structure
units.sort(key = lambda obj:obj["data"]["favoriteNumber"])
 
print ""
print "... and just like that, you have order."
for u in units:
    print u

Here is the output:

The 'work' dictionary will print the objects randomly...
{'data': {'dob': '03/28', 'favoriteNumber': 1242069926, 'name': 'Turansky'}, 'id': 'unit_5'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069925, 'name': 'Turansky'}, 'id': 'unit_4'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069928, 'name': 'Turansky'}, 'id': 'unit_7'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069927, 'name': 'Turansky'}, 'id': 'unit_6'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069922, 'name': 'Turansky'}, 'id': 'unit_1'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069921, 'name': 'Turansky'}, 'id': 'unit_0'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069924, 'name': 'Turansky'}, 'id': 'unit_3'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069923, 'name': 'Turansky'}, 'id': 'unit_2'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069930, 'name': 'Turansky'}, 'id': 'unit_9'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069929, 'name': 'Turansky'}, 'id': 'unit_8'}

Sprinkle some sorting magic...

... and just like that, you have order.
{'data': {'dob': '03/28', 'favoriteNumber': 1242069921, 'name': 'Turansky'}, 'id': 'unit_0'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069922, 'name': 'Turansky'}, 'id': 'unit_1'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069923, 'name': 'Turansky'}, 'id': 'unit_2'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069924, 'name': 'Turansky'}, 'id': 'unit_3'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069925, 'name': 'Turansky'}, 'id': 'unit_4'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069926, 'name': 'Turansky'}, 'id': 'unit_5'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069927, 'name': 'Turansky'}, 'id': 'unit_6'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069928, 'name': 'Turansky'}, 'id': 'unit_7'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069929, 'name': 'Turansky'}, 'id': 'unit_8'}
{'data': {'dob': '03/28', 'favoriteNumber': 1242069930, 'name': 'Turansky'}, 'id': 'unit_9'}

HOW TO: Download & sort pictures from your camera using Python

I’ve got tens of thousands of photos. When I last checked, the size on disk was over 25gb. And why not? Film is free!

How do I keep track of them all?

First, there’s Picasa from Google. It’s awesome. It’s the iTunes of photos.

Second, there’s a little Python script I called “DownloadPhotosFromCameraAndSort.py” (It’s .txt on the server, rename to .py if you download it).

If you couldn’t tell, I like meaningful names for my scripts. Likewise for my Java classes, components, projects, etc. Good code communicates it’s purpose clearly without too much human parsing.

The script does the following:

  1. Find my camera
  2. Download all pictures/movies to 1 or more destinations
  3. Confirm the download before deleting the picture from the camera
  4. Sort the pictures into a dated subdirectory based on the individual picture’s last modified time

The dated directories give me a running chronology of my life and the lives of my wife and daughter. It’s not a fancy archival system, but it’s simple and easy for me. It may work for you, too.

The archive looks like this:


photosdir.jpg

Horses for courses

Pick the right tool for the job.  It can save you a lot of time and effort.  This article explains how I replaced dozens of Java class files, an XSD, myriad library dependencies, a build script and a whole lot of complexity with 40 lines of Python.

A friend of mine recently bemoaned my use of the term “pick the right tool for the job.” Apparently, he thinks it’s a tired metaphor and a boring cliche. And so in deference to my friend, I will instead talk about choosing the right horse for the course.

Some horses are mudders, some are good at short-distance sprints, still others are the ones you want for the long race. If you’re eying the race’s purse, you choose the right horse for the course.

So, how do you choose the right horse? I’m guessing it has something to do with experience, breadth of knowledge, and understanding when a functional program makes more sense than an imperative one or knowing when a script is better than a fully engineered OO implementation. Here’s an example to fan the Java vs. Python flames.

Not too long ago, in my previous life working for a consulting company, I wrote a small Java application to monitor our many web applications. The requirements were simple: the server page should return “OK” (text/plain) else the contents of the entire page would be mailed to a list of interested recipients. This technique allows a developer to put whatever test they want in their server page (database connectivity, unit tests, whatever) knowing that any exception they write to the page would be mailed to them.

Not hard, right? Easy implementation? It was both times I wrote it, but one of them was a much better horse for the course.

The configuration file in Java was XML, natch, which required an XSD. The XSD required JAXB(or Castor, XMLBeans, etc.) to generate bindings for the XML. The Main program was well factored in that each class did one thing well. As a result, I had a class to poll a site, one of load the configuration, a class to send email, etc. Between Main and classes from Castor, I was up to a dozen .java files. Main, of course, required libraries (Castor, mail.jar, activation.jar, etc.), and those libraries required a script (.cmd in our case) to load them all onto the classpath. Oh, let’s not forget about building with Ant. Add a build file to the heap.

It worked, but damn, that’s a lot of files and jars and complexity for a simple monitoring program!

I rewrote it in Python in less than 40 lines of code. Two files. There are more comment lines than executable code.

Choose the right horse for the course. Occam’s Razor says the 40 line solution is the right one, but you can decide for yourself.

Here’s the python program and configuration file.

Many developers learn one or two mainstream languages and always run their favorite horse, irrespective of course. The best developers will be those that love learning new languages and techniques. I’m a better Java programmer today because of what I’ve learned from Python. I’m a better web developer today because of what I’ve learned from Ruby on Rails.

Learn more. Broaden your horizons. Try new things and new styles of development. Learn to pick the best horse for the course. Always use the right tool for the job.

Tags: , , ,

Switch to our mobile site