About Python Exception Handling

Preface

Exception handling is a topic worth paying attention to in any programming language. Good exception handling can make your program more robust, and clear error messages can better help you fix problems quickly. In Python, as in some high-level languages, try/except/finally statement blocks are used to handle exceptions. If you have experience with other programming languages, it is not difficult to put into practice. There are (at least) two kinds of errors in Python: syntax errors and exceptions (syntax errors and exceptions). Below is a brief introduction.

Syntax Errors and Exceptions

Syntax errors Syntax errors are the most obvious errors. When you encounter this kind of error, the program cannot run at all. For example

print("hello"

The code above will raise an error because a closing parenthesis is missing.

And also the following:

if a > b
    print(a)

This won't work either, because there is no colon after if. Generally speaking, once you are familiar with the syntax, this kind of error rarely occurs.

Exceptions Even if a statement or expression is syntactically correct, it may still raise an error when you try to execute it. Errors detected at runtime are called exceptions. Most exceptions are not handled by the program and need to be caught and handled by ourselves. The last line of the error message indicates what error occurred. Exceptions also have different types, and the exception type is displayed as part of the error message: the exceptions in the examples are ZeroDivisionError (ZeroDivisionError), NameError (NameError), and TypeError (TypeError). When the error message is printed, the exception type is displayed as the built-in name of the exception. This is true for all built-in exceptions, but not necessarily for user-defined exceptions (although this is a useful convention). Standard exception names are built-in identifiers (not reserved keywords).

Python Standard Exceptions

Exception Handling Statements try...excpet...finally

Sample code

def div(a, b):
    try:
        print(a / b)
    except ZeroDivisionError:
        print("Error: b should not be 0 !!")
    except Exception as e:
        print("Unexpected Error: {}".format(e))
    else:
        print('Run into else only when everything goes well')
    finally:
        print('Always run into finally block.')

# tests
div(2, 0)
div(2, 'bad type')
div(1, 2)

# Mutiple exception in one line
try:
    print(a / b)
except (ZeroDivisionError, TypeError) as e:
    print(e)

# Except block is optional when there is finally
try:
    open(database)
finally:
    close(database)

# catch all errors and log it
try:
    do_work()
except:    
    # get detail from logging module
    logging.exception('Exception caught!')
    
    # get detail from sys.exc_info() method
    error_type, error_value, trace_back = sys.exc_info()
    print(error_value)
    raise

Summary as follows

1. The except statement is not required, and the finally statement is not required either, but one of the two must be present; otherwise there is no point in using try.
2. There can be multiple except statements. Python will match the exceptions you specify in the order of the except statements. If an exception has already been handled, it will not enter subsequent except statements.
3. An except statement can specify multiple exceptions at once in tuple form; see the sample code.
4. If no exception type is specified after the except statement, it catches all exceptions by default. You can obtain the current exception via the logging or sys module.
5. If you want to re-raise an exception after catching it, use raise without any arguments or information after it.
6. It is not recommended to catch and raise the same exception; please consider refactoring your code.
7. It is not recommended to catch all exceptions when you are not clear about the logic; you might be hiding a very serious problem.
8. Try to use built-in exception-handling constructs to replace try/except statements, such as the with statement and the getattr() method.

Raising Exceptions with raise

If you need to raise an exception yourself, you can use the raise keyword, which is equivalent to throw in C# and Java. The syntax is as follows.

raise NameError("bad name!")

After the raise keyword you can specify the exception instance you want to raise. Generally speaking, the more detailed the exception you raise, the better. Python has many built-in exception types in the exceptions module. You can use the dir() function to view the exception types in exceptions, as follows:

import exceptions

print dir(exceptions)
# ['ArithmeticError', 'AssertionError'...]

Custom Exception Types

Defining your own exception types in Python is very simple; you only need to inherit from the Exception class (directly or indirectly):

class SomeCustomException(Exception):
    pass

class AnotherException(SomeCustomException):
    pass

Generally, when you define a custom exception type, the question you should consider is the scenario in which this exception applies. If a built-in exception already covers the exception you need, it is recommended to consider using the built-in exception type. For example, if you want to raise an exception when a function argument is wrong, you may not need to define an InvalidArgumentError; you can just use the built-in ValueError.

Practical Cases

Passing Exceptions re-raise Exception

If you have caught an exception but want to re-raise it (pass the exception along), you can use a raise statement without arguments:

def f1():
    print(1/0)

def f2():
    try:
        f1()
    except Exception as e:
        raise  # don't raise e !!!

f2()

In Python 2, in order to preserve the complete exception information, you must not add the exception object after raise when you catch and re-raise it; otherwise your traceback information will be truncated from this point. The above is the simplest way to re-raise an exception, and it is also the recommended way.

There are also some techniques you can consider, for example if you want to update the exception information before raising it.

def f2():
    try:
        f1()
    except Exception as e:
        e.args += ('more info',)
        raise

Python 3 has improved the re-raising of exceptions; you can try it yourself, but it's still recommended to follow the rules above.

Exception and BaseException

When we need to catch a generic exception, should we use Exception or BaseException? I suggest you take a look at the official documentation. What's the difference between these two exceptions? Let's look at their inheritance relationship.

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- StopIteration...
      +-- StandardError...
      +-- Warning...

Looking at the Exception hierarchy, BaseException is the most fundamental exception class, and Exception inherits from it. In addition to all Exceptions, BaseException also includes the three exceptions SystemExit, KeyboardInterrupt, and GeneratorExit.

From this, when your program catches all exceptions you should use Exception rather than BaseException, because the three excluded exceptions are higher-level exceptions, and the reasonable approach is to leave them to the Python interpreter.

except Exception as e and except Exception, e

Code example as follows:

try:
    do_something()
except NameError as e:  # should
    pass
except KeyError, e:  # should not
    pass

In the Python 2 era, you could use either of the two styles above. In Python 3 you can only use the first style; the second is no longer supported. The first style is more readable, and for the sake of program compatibility and the cost of later porting, you should decisively abandon the second style.

raise "Exception string"

Raising a string as an exception looks like a very concise approach, but it's actually a very bad habit.

if is_work_done():
    pass
else:
    raise "Work is not done!" # not cool

If the statement above raises an exception, it would look like this:

Traceback (most recent call last):
  File "/demo/exception_hanlding.py", line 48, in <module>
    raise "Work is not done!"
TypeError: exceptions must be old-style classes or derived from BaseException, not str

This was an acceptable practice before Python 2.4, but not specifying an exception type may prevent downstream code from correctly catching and handling the exception, making your program hard to maintain. Simply put, this style is a feudal-era bad habit and should be thrown away.

Use Built-in Syntactic Constructs Instead of try/except

Python itself provides many syntactic constructs that simplify exception handling. For example, the for statement handles the StopIteration exception, letting you write a loop smoothly.

The with statement automatically invokes finally and closes the file after opening it. When writing Python code, we should try to avoid still using a try/except/finally mindset to handle these situations.

# should not
try:
    f = open(a_file)
    do_something(f)
finally:
    f.close()

# should 
with open(a_file) as f:
    do_something(f)

For another example, when we need to access an uncertain attribute, you might write code like this:

try:
    test = Test()
    name = test.name  # not sure if we can get its name
except AttributeError:
    name = 'default'

Actually, you can use the simpler getattr() to achieve your goal.

name = getattr(test, 'name', 'default')

Best Practices

Best practices are not limited to programming languages; they are just some rules and lessons learned after filling pits.

• Only handle exceptions you know about; avoid catching all exceptions and then swallowing them.
• Raised exceptions should explain the reason; sometimes even knowing the exception type you still can't figure out why.
• Avoid doing meaningless things in catch blocks; catching exceptions also has a cost.
• Don't use exceptions to control flow; that will make your program extremely hard to understand and maintain.
• If needed, remember to use finally to release resources.
• If needed, don't forget to do cleanup or rollback after handling exceptions.

References for This Article

• Official documentation: https://docs.python.org/2.7/library/exceptions.html#bltin-exceptions
• https://segmentfault.com/a/1190000007736783
• http://www.pythondoc.com/pythontutorial3/errors.html
• http://www.runoob.com/python/python-exceptions.html
• https://betacat.online/