I wasn’t aware of generator expressions:
wwwlog = open("access-log") bytecolumn = (line.rsplit(None, 1) for line in wwwlog) bytes = (int(x) for x in bytecolumn if x != '-') print "Total", sum(bytes)
Similar to list comprehensions, but evaluated lazily. Voidspace describes it:
None of the generators are consumed until the final call to
sum. As it iterates a line at a time (not keeping the log file in memory) it can handle huge log files - and as a bonus it runs faster than a typical solution with loops!