Package & Module

Package & Module


  • Packages are modules that contain other modules.
  • Packages are generally implemented as directories containing a special __init__.py file.
  • The __init__.py file is executed when the package is imported.
  • Packages can contain sub packages which themselves are implemented with __init__.py files in directories.
  • The module objects for packages have a __path__ attribute.

sys.path

  • List of directories which Python searches for modules.

    # list directories
    >>>import sys
    >>>sys.path
    
  • Use append to attach the package directory to sys.path

    • Append the package to sys.path
    • If you append the relative path of the package to sys.path, you need to make sure the it is correct.
  • Example

    • path: path_root\package0\module0.py
    • The code of module0.py

      def test():
          print('module0 -- test !')
      
    • Test module importing

      cd root
      python
      >>>import sys
      >>>sys.append('package0')
      >>>import module0
      >>>module0.test
      module0 -- test !
      >>>exit()
      
      # It will fail if you launch python at the parent directory of root 
      
      cd ..
      python
      >>>import sys
      >>>sys.append('package0')
      >>>import module0
      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      ImportError: No module named 'module0'
      
      ### It will success if you adjust relative path as below
      >>>sys.append('path_root/package0')
      >>>import module0
      >>>module0.test
      module0 -- test !
      

PYTHONPATH

  • Environment variable adds paths to sys.path
  • Use previous module0.py to test
  • Linux

    export PYTHONPATH=package0
    python
    >>>import module0
    module0 -- test !
    
  • Windows

    set PYTHONPATH=package0
    python
    >>>import module0
    module0 -- test !
    

Package structure


  • Convert a package into a module
  • Basic structure of package0

    path_root  <--// it must be attached sys.path
    +---package0    <--// package root 
    +---__init__.py  <--// package init file
    \---module0.py
    
  • Sample code - __init__.py ( The sample code is for demo purpose)

    print('package0 --init...')
    
  • Test

    python
    >>>import package0
    package0 --init...
    >>>import package0.module0
    >>>package0.module0.test()
    module0 -- test !
    
  • Add a FileReader class into module0.py

  • Sample code of module0.py

    class FileReader:
        def __init__(self, filename):
            self.filename = filename
            self.f = open(self.filename, 'rt')
    
        def close(self):
            self.f.close()
            
        def read(self):
            return self.f.read()
    
    
  • Test

    fr=package0.module0.FileReader('package0/module0.py')
    >>>fr.read()
    >>>fr.close()
    
    
  • Update __init__.py

    from package0.module0 import FileReader
    
  • Test again

    >>>import package0
    >>>r=package0.FileReader('package0/module0.py')
    >>>r.read()
    >>>r.close()
    

Subpackage


  • Demo below shows how to add subpackages
  • Add sub-package under the package0
  • Structure

    path_root  <--// it must be attached sys.path
    +---package0    <--// package root 
    +---compress
    |   +---__init__.py
    |   +---bz.py
    |   +---gz.py
    +---__init__.py  <--// package init file
    \---module0.py
    
  • Sample code - gz.py

    import gzip
    import sys
    
    opener=gzip.open
    
    if __name__ == '__main__':
    f = gzip.open(sys.argv[1], mode='wt')
    f.write(' '.join(sys.argv[2:]))
    f.close()
    
  • Sample code - bz.py

    import bz2
    import sys
    
    opener = bz2.open
    
    if __name__ == '__main__':
    f = bz2.open(sys.argv[1], mode='wt')
    f.write(' '.join(sys.argv[2:]))
    f.close()
    
  • Test by creating two compressed files

    python3 -m package0.compress.gz test.gz data compressed with gz
    python3 -m package0.compress.bz test.bz2 data compressed with bz2
    
  • Change FileReader.py to read above files

    from package0.compress import gz, bz
    import os
    
    extension_map = {
    '.gz':gz.opener,
    '.bz2':bz.opener
    }
    
    class FileReader:
    def __init__(self, filename):
        self.filename = filename
        extension = os.path.splitext(filename)[1]
        opener = extension_map.get(extension, open)
        self.f = opener(self.filename, 'rt')
    
    def close(self):
        self.f.close()
        
    def read(self):
        return self.f.read()
    
    
  • Test

    >>> import package0
    >>> r=package0.FileReader('test.gz')
    >>> r.read()
    'data compressed with gz'
    >>> r=package0.FileReader('test.bz2')
    >>> r.read()
    'data compressed with bz2'
    
    

Import with relative path


  • Example below show how to use relative path to import packages

    path_root 
    +---package0     
    +---compress
    |   +---__init__.py
    |   +---bz.py      <--// from ..module0 import FileReader
    |   +---gz.py      <--// from .bz import bz.opener
    +---__init__.py  
    \---module0.py
    
    

Namespace package


  • Namespace packages have no __init__.py
  • Python scans all entries in sys.path.
  • If a matching directory with __init__.py is found, a normal package is loaded
  • Otherwise, all matching directories in sys.path are considered part of the namespace package
  • Example 1

    • Structure of package

      path_root0 
      +---package0
      +---module0.py
      path_root1 
      +---package0
      +---module0.py    
      
    • Test

      >>> import sys
      >>> sys.path.append('gh')
      >>> sys.path.append('path_root0')
      >>> sys.path.append('path_root1')
      >>> import package0
      >>> package0.__path__
      _NamespacePath(['gh\\package0', 'path_root0\\package0', 'path_root0\\package0'])
      
  • Example 2

    • Structure of package

      path_root0 
      +---package0
      +---module0.py
      path_root1 
      +---package0
      +---module0.py    
      path_root2 
      +---package0
      +---__init__.py  <--// Namespace should not include __init__.py
      +---module0.py            
      
    • Test

      >>> import sys
      >>> sys.path.append('path_root0')
      >>> sys.path.append('path_root1')
      >>> sys.path.append('path_root2')
      >>> import package0
      >>> package0.__path__
      _NamespacePath(['path_root2\\package0'])    
      

project_root <--// Project root directory contains everything
+---__main__.py
+---project_name
|   +---__init__.py
|   +---resource.py
|   +---package0
|   |   +---__init__.py
|   |   +---module0.py
|   +---test
|      +---__init__.py
|      +---test.py
\---setup.py      

Visual Studio Code setup


  • Install python plugin Python - donjayamanne
  • Setup launch.json

    {
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python",
            "type": "python",
            "request": "launch",
            "stopOnEntry": true,
            "pythonPath": "${config:python.pythonPath}",
            "program": "${workspaceRoot}/__main__.py",
            "cwd": "${workspaceRoot}",
            "env":{},
            "envFile": "${workspaceRoot}/.env",
            "debugOptions": [
                "WaitOnAbnormalExit",
                "WaitOnNormalExit",
                "RedirectOutput"
            ]
        }
    ]
    ...
    }
    
  • Sample code - __main__.py

    from package1 import FileReader
    
    if __name__ == "__main__":
        app = FileReader('C:/ws/python/plural/pbb/gh/test.gz')
        print(app.read())
        app.close()
    
        app = FileReader('C:/ws/python/plural/pbb/gh/test.bz2')
        print(app.read())
        app.close()
    

Function & Lambda

Function

  • statement which defines a function and binds it to a name
  • Must have a name
  • Arguments delimited by parentheses, separated by commas
  • Zero or more arguments supported -zero arguments ⇒ empty parentheses
  • Body is an indented block of statements
  • A return statement is required to return anything other than None
  • Regular functions can have docstrings‣ Easy to access for testing

Lambda

  • Expression which evaluates to a function
  • Anonymous
  • Argument list terminated by colon, separated by commas
  • Zero or more arguments supported - zero arguments ⇒ lambda:
  • Body is a single expression
  • The return value is given by the body expression. No return statement is permitted.
  • Lambdas cannot have docstrings
  • Awkward or impossible to test
  • Example

    >>>names=list(['Harry Ho', 'Harry Porter', 'Harry Charles'])
    >>>sorted(names, key=lambda name:name.split()[-1]))
    ['Harry Charles', 'Harry Ho', 'Harry Porter']
    

Callable


  • Callable instance

    • Example
    • Sample code - Resolver.py

      class Resolver:
      
      def __init__(self):
          self.cache={}
      
      def __call__(self, host):
          if host not in self.cache:
              self.cache[host]= socket.gethostbyname(host)
          return self.cache[host]
      
    • Sample code - __main__.py

      from package1 import Resolver
      
      if __name__ == "__main__":
      app = Resolver()
      print(app('harryho.github.io'))
      print(app.__call__('harryho.github.io'))
      
  • Callable class

    >>>seq_class_1 = list
    >>>sequence= seq_class_1('abc')
    >>>type(sequence)
    <class 'list'>
    >>>seq_class_1 = tuple
    >>>sequence= seq_class_1('abc')
    >>>type(sequence)
    <class 'tuple'>
    

Extended arguments and call


  • Extended arguments - syntax: def extend( *args, **kargs)

  • Example

    >>> def tag(name, **attrs):
    ...     t='<'
    ...     for k,v in attrs.items():
    ...             t+= '{key}="{val}"'.format(key=k, val=str(v))
    ...     t+='>'
    ...     return t
    ...
    >>> tag('a', href="harryho.github.io", target="_blank", id="link")
    '<href="harryho.github.io"id="link"target="_blank">'
    
  • Extended call - sample

    >>> def f1 ( a1, a2, *a3):
    ...     print(a1)
    ...     print(a2)
    ...     print(a3)
    ...
    >>> aa=(2,3)
    >>> f1(aa)
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    TypeError: f1() missing 1 required positional argument: 'a2'
    >>> f1(*aa)
    2
    3
    ()
    
    >>> def f2(a1, a2):
    ...     print(a1)
    ...     print(a2)
    ...
    
    >>> f2(*aa)
    2
    3
    >>> aa=(1,2,3,4)
    >>> f2(*aa)
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    TypeError: f2() takes 2 positional arguments but 4 were given
    
  • Transpose example

    >>> mon=[12,13,15,12,14,18,13]
    >>> tue=[11,14,16,12,11,17,14]
    >>> for d in zip(mon, tue):
    ...     print(d)
    ...
    (12, 11)
    (13, 14)
    (15, 16)
    (12, 12)
    (14, 11)
    (18, 17)
    (13, 14)
    >>> daily = [mon, tue]
    >>> from pprint import pprint as pp
    >>> pp(daily)
    [[12, 13, 15, 12, 14, 18, 13], [11, 14, 16, 12, 11, 17, 14]]
    >>> transposed = list(zip(*daily))
    >>> pp(transposed)
    [(12, 11), (13, 14), (15, 16), (12, 12), (14, 11), (18, 17), (13, 14)]