Python: nested list restructuring

unhappy_mage

[H]ard|DCer of the Month - October 2005
Joined
Jun 29, 2004
Messages
11,455
I've got a program that is giving me back a nested list of the following form:
[[[3, 4], [5, 6]], [[7, 8], [9, 0]], [[a, b], [c d]]]
but to output the list I need to restructure it to have the following form:
[[3, 4], [5, 6], [7, 8], [9, 0], [a, b], [c, d]]
The innermost groups are what I need; I don't need things flattened all the way. What's a good way to do this?

For those interested in the application:
I've written a raytracer in Python, and I'm in the process of multithreading it using MPI. I farm out the actual ray shooting: each process makes a local array of pixels like [[3, 4], [5, 6]], and then I collect those local arrays onto one process (and obtain something like the first example list). I don't control the collection algorithm, it just takes a bunch of elements (which happen to be lists) and makes a list out of them. Then I need to write an output image file (using a list-of-lists with exactly 2 dimensions), and while I could mess with my output function to handle weirdly shaped lists, I'd rather feed it the right input than complicate it unnecessarily.

So, any ideas?
 
Well, whenever you're processing some sequence, iterating* over it is pretty much a given. What elements are you going to see along the way, and what can you do with them to build your solution?

* Or recursing, of course, which is usually the most natural solution when you're working with linked lists. I'm not sure how Python's lists work - IIRC, they've got both dynamic size and indexed access, so it's hard to tell. In any case I don't recall any clean way of decomposing a list, so iteration is probably the way to go.

Which is a shame, because problems like this can have really simple and elegant recursive solutions given the right language:
Code:
ML:
fun flat([]) = []
|   flat([a,b]::xs) = a::b::flat(xs);

Prolog:
flat([],[]).
flat([[A,B]|X],[A,B|Y]) :- flat(X,Y).
 
Figured it out (no error checking here):
def partialflatten(x):
result = []
for i in x:
for j in i:
result.append(j)
return result
I was trying to do this with a list comprehension, something like this:
>>> a=[[[3, 4], [5, 6]], [[7, 8], [9, 0]]]
>>> [(j for j in i) for i in a]
[<generator object at 0xb7d8c3ec>, <generator object at 0xb7d8c42c>]
>>> [j for j in i for i in a]
[[7, 8], [7, 8], [9, 0], [9, 0]]

The last is particularly odd. I think this is being interpreted as a single iteration through a, rather than an iteration through the elements of each i in a.

This also works, if you like compactness of notation:
>>> q = []
>>> [[q.append(j) for j in i] for i in a]
[[None, None], [None, None]]
>>> q
[[3, 4], [5, 6], [7, 8], [9, 0]]
but it'll probably use more memory to store that [[None]] array that'll just get discarded anyways. I don't know, though, the interpreter might notice that you're discarding the return value and not produce it.
 
Code:
import operator

t = [[[3, 4], [5, 6]], [[7, 8], [9, 0]], [[a, b], [c, d]]]
reduce(operator.add,t)
 
Back
Top