How to successfully do “nested” looping over a Fiona collection
I wrote some code that basically loops over all features in a shapefile with Fiona in Python, doing some things, calculating some things, etc.
I found that it would exit the for loop prematurely, depending on the data set I fed it. I found that it would work fine if I removed some code that (to me) has nothing to do with the loop we are in. This is my first attempt at GIS with Python, so I might be doing some newbie mistage with data structures I guess?
Below is a minimal example using Natural Earth data which you can grab at http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_countries.zip
import fiona from shapely.geometry import shape, Point def find_containing_area(point, collection): for record in collection: geometry = shape(record['geometry']) if point.within(geometry): return record return None with fiona.open('/tmp/ne_110m_admin_0_countries.shp', 'r') as source: for admin_record in source: properties = admin_record['properties'] print(properties['name']) # in my original code, i created a point related to the admin_record point = Point(0,0) # if the following line exists, the loop will stop after the first # admin_record in this example. In my original code, it stopped # after just few records, not the first one though. containing_area = find_containing_area(point, source)
If I run it, it prints "Afghanistan" instead of the whole list of ~177 names. If I comment out the last line, it prints them all as intended.
My only guess is that someone an iterator is passed along with the collection? The only hint in the documentation that I found is "Fiona's Collection is like a Python file, but is iterable for records rather than lines." which does help stupid-me not very much. http://toblerity.org/fiona/manual.html#data-model
edit: I have tried to copy the collection and pass the copy to the function instead, got the same bevahiour:
newsource = source[:] containing_area = find_containing_area(point, newsource)
According to the documentation:
Seeking the beginning of the file is not supported. You must reopen the collection to get back to the beginning.
Open the file again for the second loop. I changed your last line to the following and got the loop to run all the way through.
with fiona.open('ne_110m_admin_0_countries.shp', 'r') as source2: containing_area = find_containing_area(point, source2)
Reposting Kersten's comment as an answer so I can accept it as solution:
fiona.open returns an iterator so it will only return one element at a time in memory. The documentation has some examples on storing iterator results in memory, e.g. lists
My solution was to create a list from the source:
newsource = list(source)