RAII in Python - automatic destruction when leaving a scope

Question

I've been trying to find RAII in Python. Resource Allocation Is Initialization is a pattern in C++ whereby an object is initialized as it is created. If it fails, then it throws an exception. In this way, the programmer knows that the object will never be left in a half-constructed state. Python can do this much.

But RAII also works with the scoping rules of C++ to ensure the prompt destruction of the object. As soon as the variable pops off the stack it is destroyed. This may happen in Python, but only if there are no external or circular references.

More importantly, a name for an object still exists until the function it is in exits (and sometimes longer). Variables at the module level will stick around for the life of the module.

I'd like to get an error if I do something like this:

for x in some_list:
    ...

... 100 lines later ...

for i in x:
    # Oops! Forgot to define x first, but... where's my error?
    ...

I could manually delete the names after I've used it, but that would be quite ugly, and require effort on my part.

And I'd like it to Do-What-I-Mean in this case:

for x in some_list:
    surface = x.getSurface()
    new_points = []
    for x,y,z in surface.points:
        ...     # Do something with the points
        new_points.append( (x,y,z) )
    surface.points = new_points
    x.setSurface(surface)

Python does some scoping, but not at the indentation level, just at the functional level. It seems silly to require that I make a new function just to scope the variables so I can reuse a name.

Python 2.5 has the "with" statement but that requires that I explicitly put in __enter__ and __exit__ functions and generally seems more oriented towards cleaning up resources like files and mutex locks regardless of the exit vector. It doesn't help with scoping. Or am I missing something?

I've searched for "Python RAII" and "Python scope" and I wasn't able to find anything that addressed the issue directly and authoritatively. I've looked over all the PEPs. The concept doesn't seem to be addressed within Python.

Am I a bad person because I want to have scoping variables in Python? Is that just too un-Pythonic?

Am I not grokking it?

Perhaps I'm trying to take away the benefits of the dynamic aspects of the language. Is it selfish to sometimes want scope enforced?

Am I lazy for wanting the compiler/interpreter to catch my negligent variable reuse mistakes? Well, yes, of course I'm lazy, but am I lazy in a bad way?

Answer 1

tl;dr RAII is not possible, you mix it up with scoping in general and when you miss those extra scopes you're probably writing bad code.

Perhaps I don't get your question(s), or you don't get some very essential things about Python... First off, deterministic object destruction tied to scope is impossible in a garbage collected language. Variables in Python are merely references. You wouldn't want a malloc'd chunk of memory to be free'd as soon as a pointer pointing to it goes out of scope, would you? Practical exception in some circumstances if you happen to use ref counting - but no language is insane enough to set the exact implementation in stone.

And even if you have reference counting, as in CPython, it's an implementation detail. Generally, including in Python which has various implementations not using ref counting, you should code as if every object hangs around until memory runs out.

As for names existing for the rest of a function invocation: You can remove a name from the current or global scope via the del statement. However, this has nothing to do with manual memory management. It just removes the reference. That may or may not happen to trigger the referenced object to be GC'd and is not the point of the exercise.

• If your code is long enough for this to cause name clashes, you should write smaller functions. And use more descriptive, less likely-to-clash names. Same for nested loops overwriting the out loop's iteration variable: I'm yet to run into this issue, so perhaps your names are not descriptive enough or you should factor these loops apart?

You are correct, with has nothing to do with scoping, just with deterministic cleanup (so it overlaps with RAII in the ends, but not in the means).

Perhaps I'm trying to take away the benefits of the dynamic aspects of the language. Is it selfish to sometimes want scope enforced?

No. Decent lexical scoping is a merit independent of dynamic-/staticness. Admittedly, Python (2 - 3 pretty much fixed this) has weaknesses in this regard, although they're more in the realm of closures.

But to explain "why": Python must be conservative with where it starts a new scope because without declaration saying otherwise, assignment to a name makes it a local to the innermost/current scope. So e.g. if a for loop had it's own scope, you couldn't easily modify variables outside of the loop.

Am I lazy for wanting the compiler/interpreter to catch my negligent variable reuse mistakes? Well, yes, of course I'm lazy, but am I lazy in a bad way?

Again, I imagine that accidential resuse of a name (in a way that introduces errors or pitfalls) is rare and a small anyway.

Edit: To state this again as clearly as possible:

• There can't be stack-based cleanup in a language using GC. It's just not possibly, by definition: a variable is one of potentially many references to objects on the heap that neither know nor care about when variables go out of scope, and all memory management lies in the hands of the GC, which runs when it likes to, not when a stack frame is popped. Resource cleanup is solved differently, see below.
• Deterministic cleanup happens through the with statement. Yes, it doesn't introduce a new scope (see below), because that's not what it's for. It doesn't matter the name the managed object is bound to isn't removed - the cleanup happened nonetheless, what remains is a "don't touch me I'm unusable" object (e.g. a closed file stream).
• Python has a scope per function, class, and module. Period. That's how the language works, whether you like it or not. If you want/"need" more fine-grained scoping, break the code into more fine-grained functions. You might wish for more fine-grained scoping, but there isn't - and for reasons pointed out earlier in this answer (three paragraphs above the "Edit:"), there are reasons for this. Like it or not, but this is how the language works.