Utility To Check Imports In Python Without Running The Code (python2.7)

Is there a way to make sure all imports listed in all of my python files are in PYTHONPATH. Basically a validator for all my imports in files. My approach: Change all file path fr

Solution 1:

The best way to do this is to use the importlib (Python 3.1+) or imp (Python 2.x) module to do all the steps an import would do up to, but not including, running the code.

The key functions for each Python version are:

• 3.4-3.8: importlib.util.find_spec
• 3.3-3.3: importlib.find_loader.
• 3.1-3.2: importlib.find_module
• 3.0-3.0: imp.find_module
• 1.5-2.7: imp.find_module
• 0.9-1.4: No idea. (There were no packages yet; everything was different…)

---

Advantages of doing it this way:

• It works whether the module is a normal Python module, a .pyc-only module, a C extension module, a builtin, or some funky special type of module that you've installed a custom import hook for.
• It works even if the module is inside a .egg, or in the frozen bootstrap collection, or if the whole library is wrapped up in a .zip or even buried inside a .exe.
• It automatically takes care of the funky rules, like figuring out what is or isn't a valid namespace package extension directory (including dealing with things like site.py and old-style setuptools path-injection) that would be a huge pain to get right.

---

In 3.4+, this is literally the same code that import uses, because the entire import system is written in Python. For older versions, that's not true—but for 2.3+, it's guaranteed to get you the same thing you would get in an import hook, which is almost surely close enough.

---

Quoting the 3.7 docs:

importlib.util.find_spec(name, package=None)

Find the spec for a module, optionally relative to the specified package name. If the module is in sys.modules, then sys.modules[name].__spec__ is returned (unless the spec would be None or is not set, in which case ValueError is raised). Otherwise a search using sys.meta_path is done. None is returned if no spec is found.

If name is for a submodule (contains a dot), the parent module is automatically imported.

name and package work the same as for import_module().

(You don't really have to care what a spec is here; if Python can find the spec for a module, the module is present; if it returns None, the module is not present.)

So, this will just magically handle foo.bar.zoo.data.

If you look at the Examples, there's one that does exactly what you want, "Checking if a module can be imported".

deftest_module(name):
    ifnot importlib.util.find_spec(name):
        raise ImportError(name)

---

From the 2.7 docs:

imp.find_module(name[, path])

Try to find the module name. If path is omitted or None, the list of directory names given by sys.path is searched, but first a few special places are searched: the function tries to find a built-in module with the given name (C_BUILTIN), then a frozen module (PY_FROZEN), and on some systems some other places are looked in as well (on Windows, it looks in the registry which may point to a specific file).

Otherwise, path must be a list of directory names; each directory is searched for files with any of the suffixes returned by get_suffixes() above. Invalid names in the list are silently ignored (but all list items must be strings).

If search is successful, the return value is a 3-element tuple (file, pathname, description):

file is an open file object positioned at the beginning, pathname is the pathname of the file found, and description is a 3-element tuple as contained in the list returned by get_suffixes() describing the kind of module found.

If the module does not live in a file, the returned file is None, pathname is the empty string, and the description tuple contains empty strings for its suffix and mode; the module type is indicated as given in parentheses above. If the search is unsuccessful, ImportError is raised. Other exceptions indicate problems with the arguments or environment.

If the module is a package, file is None, pathname is the package path and the last item in the description tuple is PKG_DIRECTORY.

This function does not handle hierarchical module names (names containing dots). In order to find P.M, that is, submodule M of package P, use find_module() and load_module() to find and load package P, and then use find_module() with the path argument set to P.__path__. When P itself has a dotted name, apply this recipe recursively.

As you can see, it's a little more complicated—it doesn't magically handle foo.bar.zoo.data; you will need to find foo, verify that it's a package, load it, find bar, etc., and finally find (but not load) data.

Something like this (untested):

def test_module(name):
    parts = name.split('.')
    path = None
    for part in parts[:-1]:
        file, pathname, description = imp.find_module(part, path)
        if description[-1] != imp.PKG_DIRECTORY:
            raise ImportError(name)
        pkg = imp.load_module(part, file, pathname, description)
        path = pkg.__path__
    file, pathname, description = imp.find_module(part[-1], path)