Python imports in functions

Is there a performance cost putting python imports inside functions?

I build quite complex python apps, often with Django. To simplify inter-application interfaces I sometimes use service.py modules that abstract away from the models. As these ‘aggregate functionality’, they frequently end up with circular imports which are easily eliminated by placing the import statements inside the service functions. Is there a significant performance or memory cost associated with generally moving imports as close to their point of use as possible? For example, if I only use a particular imported name in one function in a file, it seems natural to place the import in that particular function rather than at the top of the file in its conventional place. This issue is subtly different to this question because each import is in the function namespace.

@KevinBrown: I think the questions are fairly close, but not really duplicates. That one is about ‘safety’ and this one is about exploring performance penalties.

none of the answers actually answers the question! 🙁 They all are talking about import Whole module vs from XX import YYY

5 Answers 5

The point at which you import a module is not expected to cause a performance penalty, if that’s what you’re worried about. Modules are singletons and will not be import ed every single time an import statement is encountered. However, how you do the import, and subsequent attribute lookups, does have an impact.

For example, if you import math and then every time you need to use the sin(. ) function you have to do math.sin(. ) , this will generally be slower than doing from math import sin and using sin(. ) directly as the system does not have to keep looking up the function name within the module.

This lookup-penalty applies to anything that is accessed using the dot . and will be particularly noticeable in a loop. It’s therefore advisable to get a local reference to something you might need to use/invoke frequently in a performance critical loop/section.

For example, using the original import math example, right before a critical loop, you could do something like this:

# . within some function sin = math.sin for i in range(0, REALLY_BIG_NUMBER): x = sin(i) # faster than: x = math.sin(x) # . 

This is a trivial example, but note that you could do something similar with methods on other objects (e.g. lists, dictionaries, etc).

I’m probably a bit more concerned about the circular imports you mention. If your intention is to «fix» circular imports by moving the import statements into more «local» places (e.g. within a specific function, or block of code, etc) you probably have a deeper issue that you need to address.

Personally, I’d keep the imports at the top of the module as it’s normally done. Straying away from that pattern for no good reason is likely to make your code more difficult to go through because the dependencies of your module will not be immediately apparent (i.e. there’re import statements scattered throughout the code instead of in a single location).

It might also make the circular dependency issue you seem to be having more difficult to debug and easier to fall into. After all, if the module is not listed above, someone might happily think your module A has no dependency on module B and then up adding an import A in B when A already has import B hidden in some deep dark corner.

Benchmark Sample

Here’s a benchmark using the lookup notation:

>>> timeit('for i in range(0, 10000): x = math.sin(i)', setup='import math', number=50000) 89.7203312900001 

And another benchmark not using the lookup notation:

>>> timeit('for i in range(0, 10000): x = sin(i)', setup='from math import sin', number=50000) 78.27029322999988 

Here there’s a 10+ second difference.

Note that your gain depends on how much time the program spends running this code —i.e. a performance critical section instead of sporadic function calls.

Источник

Python : Import Libraries in Module or in Function [duplicate]

I have created a module with multiple small user-defined functions. Many of those functions require different library function. Now, I want to know what is a better practice — to import all the libraries in the module, or import the libraries within the functions? Is there any difference in terms of performance?

Example code

my_module.py

from math import exp from numpy import mean,random def logit(x): return exp(x)/(1+exp(x)) def sample_mean(mu,sigma,size): return mean(random.normal(mu,sigma,size)) 

def logit(x): from math import exp return exp(x)/(1+exp(x)) def sample_mean(mu,sigma,size): from numpy import mean,random return mean(random.normal(mu,sigma,size)) 

P.S.

This is just a sample code to explain my dilemma. Don’t tell me that there are ready made functions to use instead of these user-defined functions. I already know that

2 Answers 2

Depends on how related your functions are, but usually at the module level is cleaner. If your functions are not related, then having different imports for each will be completely fine and will hint at their independence. At that point, I would just make two separate python files.

It won’t be significantly beneficial. Python style guides encourage programmers to place all imports at the beginning of the module file.
Importing at the start is part of the philosophy and style of Python.
As you’ve shown in your code a way of improving efficiency is importing only what you need like : from numpy import mean,random .
If you do import inside a function it will give you a minor efficiency advantage but unless you’re working on an extremely time critical application it’s better to go with the general Pythonic style of importing everything at the start.

Module importing is quite fast, but not instant. This means that:

• Putting the imports at the top of the module is a trivial cost that’s only paid once.
• Putting the imports within a function will cause calls to that function to take longer.

So if you care about efficiency, put the imports at the top. Only move them into a function if your profiling shows that would help.

imports are cached, though, so it’s not that costly to have imports in a function, because all it does on the second time around is «check if import is in sys.modules, continue on happily if it is.»

Источник

Is it pythonic to import inside functions?

On occation, I violate PEP 8. Some times I import stuff inside functions. As a general rule, I do this if there is an import that is only used within a single function.

EDIT (the reason I feel importing in functions can be a good idea):

Main reason: It can make the code clearer.

• When looking at the code of a function I might ask myself: «What is function/class xxx?» (xxx being used inside the function). If I have all my imports at the top of the module, I have to go look there to determine what xxx is. This is more of an issue when using from m import xxx . Seeing m.xxx in the function probably tells me more. Depending on what m is: Is it a well-known top-level module/package ( import m )? Or is it a sub-module/package ( from a.b.c import m )?
• In some cases having that extra information («What is xxx?») close to where xxx is used can make the function easier to understand.

I feel it makes the code clearer in some cases. I would guess raw performance drops when importing in a function (since the import statement will execute every time the function is called).

You can answer «What is function/class xxx?» by using the import xyz syntax rather than the from xyz import abc syntax

10 Answers 10

In the long run I think you’ll appreciate having most of your imports at the top of the file, that way you can tell at a glance how complicated your module is by what it needs to import.

If I’m adding new code to an existing file I’ll usually do the import where it’s needed and then if the code stays I’ll make things more permanent by moving the import line to the top of the file.

One other point, I prefer to get an ImportError exception before any code is run — as a sanity check, so that’s another reason to import at the top.

I use pyChecker to check for unused modules.

There are two occasions where I violate PEP 8 in this regard:

• Circular imports: module A imports module B, but something in module B needs module A (though this is often a sign that I need to refactor the modules to eliminate the circular dependency)
• Inserting a pdb breakpoint: import pdb; pdb.set_trace() This is handy b/c I don’t want to put import pdb at the top of every module I might want to debug, and it easy to remember to remove the import when I remove the breakpoint.

Outside of these two cases, it’s a good idea to put everything at the top. It makes the dependencies clearer.

I agree it makes dependencies clearer with regards to the module as a whole. But I believe that it can make the code less clear at the function level to import everything at the top. When you’re looking at a the code of a function you might ask yourself: «What is function/class xxx?» (xxx is used inside the function). And you have to look at the very top of the file to see where xxx comes from. This is more of an issue when using from m import xxx. Seeing m.xxx tells you more — at least if there is no doubt as to what m is.

Here are the four import use cases that we use

1. import (and from x import y and import x as y ) at the top
2. Choices for Import. At the top.

import settings if setting.something: import this as foo else: import that as foo 

try: import this as foo except ImportError: import that as foo 

import settings module_stuff = <> module= __import__( settings.some_module, module_stuff ) x = module_stuff['x'] 

Here’s what we do to make the code clearer:

• Keep the modules short.
• If I have all my imports at the top of the module, I have to go look there to determine what a name is. If the module is short, that’s easy to do.
• In some cases having that extra information close to where a name is used can make the function easier to understand. If the module is short, that’s easy to do.

Keeping modules short is of course a very good idea. But to get the benefit of always having «import information» for functions available, the maximum module length would have to be one screen (probably 100 lines max). And that would probably be too short to be practical in most cases.

I suppose you could take this to a logical extreme. I think there might be a balance point where your module is «small enough» that you don’t need fancy import techniques to manage complexity. Our average module size is — coincidentally — about 100 lines.

One thing to bear in mind: needless imports can cause performance problems. So if this is a function that will be called frequently, you’re better off just putting the import at the top. Of course this is an optimization, so if there’s a valid case to be made that importing inside a function is more clear than importing at the top of a file, that trumps performance in most cases.

If you’re doing IronPython, I’m told that it’s better to import inside functions (since compiling code in IronPython can be slow). Thus, you may be able to get a way with importing inside functions then. But other than that, I’d argue that it’s just not worth it to fight convention.

As a general rule, I do this if there is an import that is only used within a single function.

Another point I’d like to make is that this may be a potential maintenence problem. What happens if you add a function that uses a module that was previously used by only one function? Are you going to remember to add the import to the top of the file? Or are you going to scan each and every function for imports?

FWIW, there are cases where it makes sense to import inside a function. For example, if you want to set the language in cx_Oracle, you need to set an NLS _ LANG environment variable before it is imported. Thus, you may see code like this:

import os oracle = None def InitializeOracle(lang): global oracle os.environ['NLS_LANG'] = lang import cx_Oracle oracle = cx_Oracle 

Источник