You can use this thread to discuss the blog post Beautiful tracebacks in Trio v0.7.0.
1 Like
Those tracebacks are beautiful! They are structured and clear and helpful, which is exactly what you want when you’re getting unexpected tracebacks. Knowing the effort that went into making them so helpful makes me feel … cared for? Thank you to everybody, and I gather especial thanks are due to John Belmonte. <3
3 Likes
Wow, I really appreciate that feedback (and I hope @belm0 sees it too!). In fact I’ve added it to our Testimonials page 
2 Likes
Aww, I’m glad this feedback reached you, and it’s an unexpected bonus that it’s a suitable testimonial. All the best to you both.
2 Likes
Those tracebacks are pretty indeed! I thought I’d come and toss in my 2c on tracebacks because IMHO the whole idea needs a re-think.
Tracebacks are an awesome thing for debugging procedural code, because the call-stack directly reflects the procedural architecture of your code.
However, procedural is a couple of paradigm’s old at this point, and tracebacks really don’t work very well for the more modern paradigms. The problem is the call stack no-longer reflects the code architecture, because we no-longer just decompose problems procedurally by execution-steps into functions that call sub-functions that call sub-sub functions, which is what the call-stack shows you. Instead we are decomposing problems by data structure with object composition, and by layers-of-implementation with inheritance.
Of course these different abstraction/decomposition methods are implemented under-the-hood using the basic procedural functions-calling-functions, so tracebacks still work, and can provide an interesting view of how all that composition/inheritance/etc maps to a sequence of function calls. But that doesn’t necessarily help you figure out where the problem is in your code, because you have to be able to re-map that call stack back to figure out where where the problem shown in the call stack corresponds to a problem in your architecture. Modern paradigms use things like virtual methods, callbacks, etc and dynamically dispatch to methods on passed in objects, which means even a relatively simple object composition structure translates into an unholy tangle of function calls that is impossible to map back to the code architecture from the data available in the traceback.
Object oriented composition architectures mean the data being operated on is at least as important as the function being called. Method calls are messages to object instances, so we really need to know the particular instance they are being called on, so we can see “Doh! I told the wrong instance to stop”.
Inheritance means the method we are calling could have been defined somewhere deep down in the base-class hierarchy, which is what the traceback shows us, but when that method calls another method on the “self” object it can bounce back up and call a method defined somewhere else in the class hierarchy. What we really need to know is the actual class of the object it’s been called on, and expose the inheritance-hierarchy call-path explicitly as “virtual calls” in the traceback instead of confusingly hiding it behind a single virtual-method dispatch.
The point of abstraction is to provide a simplified comprehensible view of the system without obscuring it with all of the low-level details. Usually the lower-levels are more mature libraries that are rarely the location of problems and there is no need to understand the details of how they work, just their public interfaces. So tracebacks interleaved with all the calls inside those libraries is just noise. We have abstraction in the composition hierarchy, with the details of small objects hidden behind large composite objects. We also have abstraction in the inheritance hierarchy, with low-level generic functionality implemented in base classes hidden behind higher level more specific subclasses. It would be nice to be able to filter out stuff in the traceback that is for lower abstraction-layers in both of those abstraction hierarchies; don’t show me all the method calls inside/between individual objects inside my “blob” objects, just show me method calls that go between “blob” objects. Or don’t show me method calls to/inside parent classes inherited by my custom classes. But then it can still be useful to be able to drill-down to the lower abstraction layers for those cases when the problem really is a bug in those lower layers.
IMHO one of the reasons for the backlash against inheritance and abstraction in general is our tooling has not kept up with the different ways we can do it (combined with abstraction-abuse and unreliable lower-levels). Things like the current tracebacks make your abstractions, which should make understanding your code easier, into a confusing mess, triggering a knee-jerk reaction “what is all this crap! Inheritance is bollocks!”.
IMHO this suggests a few simple things that could be added to tracebacks to make them more useful;
- For method calls, include the class and object reference like;
File ".../site-packages/async_generator/_impl.py", line 202, in FooClass.send on <FooSubCLass object at 0x7f4f2da2f5c0>
return self._invoke(self._it.send, value)
File ".../site-packages/async_generator/_impl.py", line 209, in FooClass._invoke on <FooSubCLass object at 0x7f4f2da2f5c0>
result = fn(*args)
- For method calls, also include the inheritance hierarchy dispatch path;
File ".../site-packages/async_generator/_impl.py", line 202, in FooClass.send on <FooSubCLass object at 0x7f4f2da2f5c0>
return self._invoke(self._it.send, value)
Method FooSubClass._invoke on <FooSubClass object at 0x7f4f2da2f5c0>
inherits FooMidClass._invoke(self, self._it.send, value)
Method FooMidClass._invoke on <FooSubClass object at 0x7f4f2da2f5c0>
inherits FooClass._invoke(self, self._it.send, value)
File ".../site-packages/async_generator/_impl.py", line 209, in FooClass._invoke on <FooSubCLass object at 0x7f4f2da2f5c0>
result = fn(*args)
- Include the composition hierarchy in the object references. This is hard, and I suspect this will never get implemented by anyone. This is complicated by composition often not being a tree, but a web, with the same object referenced as a component of multiple different composite objects, and it’s not at all clear how that could be represented. Rust’s ownership model simplifies this, as there is only one authoritative owner and the others are just references. However, even showing one of the possible composition paths would be useful, perhaps the “oldest one”, which would mostly reflect the creation sequence. It could look like this;
File ".../site-packages/async_generator/_impl.py", line 202, in FooClass.send on <ServerClass 0x7f4f2da21234>.<RequestClass 0x7f4f2da21234>.<FooSubCLass object at 0x7f4f2da2f5c0>
return self._invoke(self._it.send, value)
Method FooSubClass._invoke on ...<FooSubClass object at 0x7f4f2da2f5c0>
inherits FooMidClass._invoke(self, self._it.send, value)
Method FooMidClass._invoke on ...<FooSubClass object at 0x7f4f2da2f5c0>
inherits FooClass._invoke(self, self._it.send, value)
File ".../site-packages/async_generator/_impl.py", line 209, in FooClass._invoke on <ServerClass 0x7f4f2da21234>.<RequestClass 0x7f4f2da21234>.<FooSubCLass object at 0x7f4f2da2f5c0>
result = fn(*args)
Even nicer would be to include the attribute names on the object references like <ServerClass 0x7f4f2da21234>.requests[2]<RequestClass 0x7f4f2da21234>.dispatcher<FooSubCLass object at 0x7f4f2da2f5c0>
- Provide a mechanism to filter out traceback entries for lower abstraction layers. This is like what you’ve done with your tracebacks, and also what Python’s unittest framework does; filtering out entries for lines in your library modules. It would be nice to be able to do this based on the inheritance hierarchy, filtering out anything related to classes that are parents of MyBaseClass. Inserting the inheritance hierarchy dispatch path before filtering means you would still see dispatch-calls to the MyBaseClass level, just not the drill down below that. Also nice (but hard) would be to filter out lines based on the object composition hierarchy, so remove method call entries on objects that are sub-components of MyBaseClass objects. This needs to be adjustable in some way so you can drill down if necessary for debugging problems in the lower abstraction layers.