Python decorator with options

I have a module that has a function whose prototype is similar to that of the thread class.

def do(fn, argtuple=(), kwargdict={}, priority=0,
            block=False, timeout=0, callback=None, daemon=False)

    # do stuff

fn is a callable, and argtuple and kwargdict are positional and dictionary arguments that will be passed to the fn callable when it gets called.

I'm now trying to write a decorator for this, but I'm confused. I've never really had a very good grasp on decorators. Is there a way to make a decorator that I can set the options in the above such as timeout, but pass in argtuple and kwargdict when the function is called.

So for example:

@do(priority=2)
def decoratedTask(arg, dic=3):
    #do stuff

decoratedTask(72)

I'm开发者_运维技巧 confused how I would pass the runtime argument 72 into the decorated function. I'm thinking the decorator needs to be a class where the __call__ method returns the function call, but I'm unsure the syntax of how to pass in arguments like that.

Does this make sense?

---

What the decorator does is that it takes the function as an argument and also returns a function, typically a new function that is created in the decorator.

That new function needs to take the same parameters as the function you decorate, and it also needs to call the original function.

Now, when you have a decorator with an argument, there is an extra level going on. That decorator should take the argument, and return a decorator. The function you use is in fact not a decorator, but a decoratormaker!

Here is an example:

>>> def mydeco(count):
...     def multipass(fn):
...         def caller(*args, **kw):
...             return [fn(*args, **kw) for x in range(count)]
...         return caller
...     return multipass
... 
>>> @mydeco(5)
... def printer(text):
...     print(text)
... 
>>> printer("Yabbadabbadoo!")
Yabbadabbadoo!
Yabbadabbadoo!
Yabbadabbadoo!
Yabbadabbadoo!
Yabbadabbadoo!
[None, None, None, None, None]

You'll see examples of these decoratormakers implemented as classes. That how I prefer it too (although I usually end up not having a decorator at all anyway), but a function in a function in a function works. :)

---

That's not quite how the decorator syntax works. When you write @do(priority=2), Python will evaluate do(priority=2) and use the result of that call as the decorator. It's shorthand for

decorator=do(priority=2)
@decorator

So you actually want to make do a decorator factory: you want it to take all the positional arguments and return a decorator.

def do(args=(), kwargs={}, ...):
    def _decorator(fn):
        def newfn(*args, **kwargs):
            return fn(*args, **kwargs)
        return newfn
    return _decorator

Notice that there are actually three functions here!

• do is the function that we call do get our decorator (so e.g. do(priority=2) is an example decorator)
• _decorator is the actual decorator returned, which depends on the arguments to do
• Since a decorator takes a function as input and returns a function as output, we need to define the function that the decorator returns. newfn is that function.

---

Example:

>>> def rename(name):
...     def _decorator(fn):
...             def renamed(*args, **kwargs):
...                     return fn(*args, **kwargs)
...             renamed.__name__ = name
...             return renamed
...     return _decorator
...
>>> @rename('I like omelettes in the morning.')
... def foo():
...     return 'bar'
...
>>> foo()
'bar'
>>> foo.__name__
'I like omelettes in the morning.'

or equivalently

>>> omeletter = rename('I like omelettes in the morning.')
>>> @omeletter
... def foo():
...     return 'bar'
...
>>> foo()
'bar'
>>> foo.__name__
'I like omelettes in the morning.'

---

By the way, take care with the mutable default arguments () and {}; I'm sure you know the perils!

---

As the other answers have explained, decorators are normally passed a single implicit function argument when invoked by just using their name like this:

    @deco
    def somefunc(...): pass

Which does the same thing as:

    def somefunc(...): pass
    somefunc = deco(somefunc)

The argument in this situation is a compiled version of the function definition that immediately follows. In such cases the decorator returns a callable which is assigned to the function name instead of the compiled function body, as would normally be the case.

However if a decorator function is explicitly given one or more arguments when it's invoked, like this:

    @deco(args)
    def somefunc(...): pass

It becomes equivalent to:

    def somefunc(...): pass
    somefunc = deco(args)(somefunc)

As you can see, things in this case work somewhat differently. The decorator function still returns a callable, only this time that is expected to a "normal" single implicit-argument decorator function, which is then invoked with a function object from the following function definition like before.

Again – as others have pointed out – this makes decorators explicitly passed arguments, decorator factories in the sense that they construct and return 'regular' decorator functions.

In most if not all cases, decorators can be implemented either as a function or a class, since both are callable in Python. Personally I find functions a little easier to understand and so will use that approach in the following. On the other hand, the function approach can become tricky because it often involves one or more nested function definitions.

Here's how you could code a decorator for thedo()function in your module. In the code below I've defined what it does is print out the function's arguments before calling it.

def do(fn, args=tuple(), kwargs={}, priority=0,
       block=False, timeout=0, callback=None, daemon=False):
    # show arguments
    print ('in do(): fn={!r}, args={}, kwargs={}, priority={},\n'
           '         block={}, timeout={}, callback={}, daemon={}'
           .format(fn.__name__, args, kwargs, priority,
                   block, timeout, callback, daemon))
    # and call function 'fn' with its arguments
    print ('  calling {}({}, {})'.format(
               fn.__name__,
               ', '.join(map(str, args)) if args else '',
               ', '.join('{}={}'.format(k, v) for k,v in kwargs.items())
                    if kwargs else '')
          )
    fn(*args, **kwargs)

def do_decorator(**do_kwargs):
    def decorator(fn):
        def decorated(*args, **kwargs):
            do(fn, args, kwargs, **do_kwargs)
        return decorated
    return decorator

@do_decorator(priority=2)
def decoratedTask(arg, dic=42):
    print 'in decoratedTask(): arg={}, dic={}'.format(arg, dic)

decoratedTask(72, dic=3)

Output:

in do(): fn='decoratedTask', args=(72,), kwargs={'dic': 42}, priority=2,
         block=False, timeout=0, callback=None, daemon=False
  calling decoratedTask(72, dic=3)
in decoratedTask(): arg=72, dic=3

Here's a blow-by-blow account of how this complicated looking stuff works:

The outer decorator functiondo_decorator(), defines another inner decorator function which it returns, here creatively nameddecorator.

Whatdecoratordoes is define what happens to functions that are decorated in a simple 'no arguments' scenario—which here is defining and returning yet another – but final – nested function calleddecorated, which just calls the module'sdo()function and passes it both the arguments if any from the point of invocation, along with those intended for thedo()function's use.

This usage case is somewhat complicated by the fact that both the outer decorator and the functions being decorated have keyword arguments. Extra care is required to ensure that the names of the keywords for each are unique so they don't conflict (and that the mutablekwargsargument default value is not unintentionally changed by something in thedo()function).

---

I like @Lennart Regebro, @katrielalex and @martineau answers above, but at the risk of sounding very corny, I will risk writing a story based example.

In this two part story, we can have teachers that teach students from experience.

def self_taught_teacher(fn):
    ''' I teach students, because I had no teacher to guide me.'''
    def a_student(*real_life_issues, **details_about_my_world):
        ''' I'm a student who has been trained by a teacher 
        who has taught me about the problems I may encounter in real life.

        Thanks to my teacher for giving me extra knowledge.    
        '''
        print 'I have to remember what my teacher taught me.'
        my_answer = fn(*real_life_issues, **details_about_my_world)
        print 'Ah yes, I made the right decision.'
        #
        return my_answer
    #
    return a_student

@self_taught_teacher
def student_named_Lisa_practicing_maths(length, width, height):
    print 'Im consulting my powers of maths...'
    return length * width * height

Let's see what the student knows...

>>> answer = student_named_Lisa_practicing_maths(10, 20, height=3)
I have to remember what my teacher taught me.
Im consulting my powers of maths...
Ah yes, I made the right decision.
>>> answer
600

Very good. In the second part of the story, we introduce a professor who teaches others to become teachers. Those teachers then share what they learned with their students.

def professor_who_trains_teachers(*subjects, **to_train_teachers):
    '''I am a profeseur. I help train teachers.  ''' 
    #
    def a_teacher_who_gets_trained(fn): 
        ''' I learn subjects I should teach to my students.'''
        knowledge = [s for s in subjects]
        #
        def a_student(*real_life_issues, **details_about_my_world):
            ''' I'm a student who has been trained by a teacher 
            who has taught me about the problems I may encounter in real life.

            Thanks to my teacher for giving me extra knowledge.    
            '''
            print '(I know %s that i learned from my teacher,...)' % \
                    [information for information in knowledge]
            my_answer = fn(*real_life_issues, **details_about_my_world)
            print 'Ah yes, I made the right decision.'
            #
            return my_answer
        #
        return a_student
        #
        #
    return a_teacher_who_gets_trained 

So we can train a teacher and let them teach a student...

>>> teacher1 = professor_who_trains_teachers('math','science')
>>> teacher1
<function a_teacher_who_gets_trained at 0x104a7f500>
>>> teacher1.__name__
'a_teacher_who_gets_trained'
>>> 

@teacher1
def student_named_Lisa_practicing_maths(length, width, height):
    print 'Im consulting my powers of maths...'
    return length * width * height

That student knows their maths..

>>> answer = student_named_Lisa_practicing_maths(20, 10, 2)
(I know ['math', 'science'] that i learned from my teacher,...)
Im consulting my powers of maths...
Ah yes, I made the right decision.
>>> 
>>> answer
400

And we can create the teacher out-right as well.

@professor_who_trains_teachers('math', 'science', remember='patience')
def student_named_Lisa_practicing_maths(length, width, height):
    print 'Im consulting my powers of maths...'
    return length * width * height

Again the new student can do their maths...

>>> answer = student_named_Lisa_practicing_maths(10, 20, height=3)
(I know ['math', 'science'] that i learned from my teacher,...)
Im consulting my powers of maths...
Ah yes, I made the right decision.
>>> 
>>> answer
600