Python Note 1

Package & Module


  • Packages are modules that contain other modules.
  • Packages are generally implemented as directories containing a special __init__.py file.
  • The __init__.py file is executed when the package is imported.
  • Packages can contain sub packages which themselves are implemented with __init__.py files in directories.
  • The module objects for packages have a __path__ attribute.

sys.path

  • List of directories which Python searches for modules.
# list directories
>>>import sys
>>>sys.path
  • Use append to attach the package directory to sys.path

    • Append the package to sys.path
    • If you append the relative path of the package to sys.path, you need to make sure the it is correct.
  • Example

    • path: path_root\package0\module0.py
    • The code of module0.py
    def test():
        print('module0 -- test !')
    
    • Test module importing
    cd root
    python
    >>>import sys
    >>>sys.append('package0')
    >>>import module0
    >>>module0.test
    module0 -- test !
    >>>exit()
    
    # It will fail if you launch python at the parent directory of root 
    
    cd ..
    python
    >>>import sys
    >>>sys.append('package0')
    >>>import module0
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    ImportError: No module named 'module0'
    
    ### It will success if you adjust relative path as below
    >>>sys.append('path_root/package0')
    >>>import module0
    >>>module0.test
    module0 -- test !
    

PYTHONPATH

  • Environment variable adds paths to sys.path
  • Use previous module0.py to test
  • Linux
export PYTHONPATH=package0
python
>>>import module0
module0 -- test !
  • Windows
set PYTHONPATH=package0
python
>>>import module0
module0 -- test !

Package structure


  • Convert a package into a module
  • Basic structure of package0
path_root  <--// it must be attached sys.path
+---package0    <--// package root 
    +---__init__.py  <--// package init file
    \---module0.py

* Sample code -init.py` ( The sample code is for demo purpose)

print('package0 --init...')
  • Test

    python
    >>>import package0
    package0 --init...
    >>>import package0.module0
    >>>package0.module0.test()
    module0 -- test !
    
  • Add a FileReader class into module0.py

  • Sample code of module0.py

class FileReader:
    def __init__(self, filename):
        self.filename = filename
        self.f = open(self.filename, 'rt')

    def close(self):
        self.f.close()
    
    def read(self):
        return self.f.read()

  • Test
fr=package0.module0.FileReader('package0/module0.py')
>>>fr.read()
>>>fr.close()

  • Update __init__.py
from package0.module0 import FileReader
  • Test again
>>>import package0
>>>r=package0.FileReader('package0/module0.py')
>>>r.read()
>>>r.close()

Subpackage


  • Demo below shows how to add subpackages
  • Add sub-package under the package0
  • Structure
path_root  <--// it must be attached sys.path
+---package0    <--// package root 
    +---compress
    |   +---__init__.py
    |   +---bz.py
    |   +---gz.py
    +---__init__.py  <--// package init file
    \---module0.py
  • Sample code - gz.py
import gzip
import sys

opener=gzip.open

if __name__ == '__main__':
    f = gzip.open(sys.argv[1], mode='wt')
    f.write(' '.join(sys.argv[2:]))
    f.close()
  • Sample code - bz.py
import bz2
import sys

opener = bz2.open

if __name__ == '__main__':
    f = bz2.open(sys.argv[1], mode='wt')
    f.write(' '.join(sys.argv[2:]))
    f.close()
  • Test by creating two compressed files
python3 -m package0.compress.gz test.gz data compressed with gz
python3 -m package0.compress.bz test.bz2 data compressed with bz2
  • Change FileReader.py to read above files
from package0.compress import gz, bz
import os

extension_map = {
    '.gz':gz.opener,
    '.bz2':bz.opener
}

class FileReader:
    def __init__(self, filename):
        self.filename = filename
        extension = os.path.splitext(filename)[1]
        opener = extension_map.get(extension, open)
        self.f = opener(self.filename, 'rt')

    def close(self):
        self.f.close()
    
    def read(self):
        return self.f.read()

  • Test
>>> import package0
>>> r=package0.FileReader('test.gz')
>>> r.read()
'data compressed with gz'
>>> r=package0.FileReader('test.bz2')
>>> r.read()
'data compressed with bz2'

Import with relative path


  • Example below show how to use relative path to import packages
path_root 
+---package0     
    +---compress
    |   +---__init__.py
    |   +---bz.py      <--// from ..module0 import FileReader
    |   +---gz.py      <--// from .bz import bz.opener
    +---__init__.py  
    \---module0.py

Namespace package


  • Namespace packages have no __init__.py
  • Python scans all entries in sys.path.
  • If a matching directory with __init__.py is found, a normal package is loaded
  • Otherwise, all matching directories in sys.path are considered part of the namespace package
  • Example 1

    • Structure of package
    path_root0 
    +---package0
        +---module0.py
    path_root1 
    +---package0
        +---module0.py    
    
    • Test
    >>> import sys
    >>> sys.path.append('gh')
    >>> sys.path.append('path_root0')
    >>> sys.path.append('path_root1')
    >>> import package0
    >>> package0.__path__
    _NamespacePath(['gh\\package0', 'path_root0\\package0', 'path_root0\\package0'])
    
  • Example 2

    • Structure of package
    path_root0 
    +---package0
        +---module0.py
    path_root1 
    +---package0
        +---module0.py    
    path_root2 
    +---package0
        +---__init__.py  <--// Namespace should not include __init__.py
        +---module0.py            
    
    • Test
    >>> import sys
    >>> sys.path.append('path_root0')
    >>> sys.path.append('path_root1')
    >>> sys.path.append('path_root2')
    >>> import package0
    >>> package0.__path__
    _NamespacePath(['path_root2\\package0'])    
    

project_root <--// Project root directory contains everything
+---__main__.py
+---project_name
|   +---__init__.py
|   +---resource.py
|   +---package0
|   |   +---__init__.py
|   |   +---module0.py
|   +---test
|      +---__init__.py
|      +---test.py
\---setup.py      

Visual Studio Code setup


  • Install python plugin Python - donjayamanne
  • Setup launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python",
            "type": "python",
            "request": "launch",
            "stopOnEntry": true,
            "pythonPath": "${config:python.pythonPath}",
            "program": "${workspaceRoot}/__main__.py",
            "cwd": "${workspaceRoot}",
            "env":{},
            "envFile": "${workspaceRoot}/.env",
            "debugOptions": [
                "WaitOnAbnormalExit",
                "WaitOnNormalExit",
                "RedirectOutput"
            ]
        }
    ]
    ...
}
  • Sample code - __main__.py
from package1 import FileReader

if __name__ == "__main__":
    app = FileReader('C:/ws/python/plural/pbb/gh/test.gz')
    print(app.read())
    app.close()

    app = FileReader('C:/ws/python/plural/pbb/gh/test.bz2')
    print(app.read())
    app.close()

Function & Lambda

Function

  • statement which defines a function and binds it to a name
  • Must have a name
  • Arguments delimited by parentheses, separated by commas
  • Zero or more arguments supported -zero arguments ⇒ empty parentheses
  • Body is an indented block of statements
  • A return statement is required to return anything other than None
  • Regular functions can have docstrings‣ Easy to access for testing

Lambda

  • Expression which evaluates to a function
  • Anonymous
  • Argument list terminated by colon, separated by commas
  • Zero or more arguments supported - zero arguments ⇒ lambda:
  • Body is a single expression
  • The return value is given by the body expression. No return statement is permitted.
  • Lambdas cannot have docstrings
  • Awkward or impossible to test
  • Example
>>>names=list(['Harry Ho', 'Harry Porter', 'Harry Charles'])
>>>sorted(names, key=lambda name:name.split()[-1]))
['Harry Charles', 'Harry Ho', 'Harry Porter']

Callable


  • Callable instance

    • Example
    • Sample code - Resolver.py
    class Resolver:
    
        def __init__(self):
            self.cache={}
    
        def __call__(self, host):
            if host not in self.cache:
                self.cache[host]= socket.gethostbyname(host)
            return self.cache[host]
    
    • Sample code - __main__.py
    from package1 import Resolver
    
    if __name__ == "__main__":
        app = Resolver()
        print(app('harryho.github.io'))
        print(app.__call__('harryho.github.io'))
    
  • Callable class

>>>seq_class_1 = list
>>>sequence= seq_class_1('abc')
>>>type(sequence)
<class 'list'>
>>>seq_class_1 = tuple
>>>sequence= seq_class_1('abc')
>>>type(sequence)
<class 'tuple'>

Extended arguments and call


  • Extended arguments - syntax: def extend( *args, **kargs)

  • Example

>>> def tag(name, **attrs):
...     t='<'
...     for k,v in attrs.items():
...             t+= '{key}="{val}"'.format(key=k, val=str(v))
...     t+='>'
...     return t
...
>>> tag('a', href="harryho.github.io", target="_blank", id="link")
'<href="harryho.github.io"id="link"target="_blank">'
  • Extended call - sample
>>> def f1 ( a1, a2, *a3):
...     print(a1)
...     print(a2)
...     print(a3)
...
>>> aa=(2,3)
>>> f1(aa)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f1() missing 1 required positional argument: 'a2'
>>> f1(*aa)
2
3
()

>>> def f2(a1, a2):
...     print(a1)
...     print(a2)
...

>>> f2(*aa)
2
3
>>> aa=(1,2,3,4)
>>> f2(*aa)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f2() takes 2 positional arguments but 4 were given
  • Transpose example
>>> mon=[12,13,15,12,14,18,13]
>>> tue=[11,14,16,12,11,17,14]
>>> for d in zip(mon, tue):
...     print(d)
...
(12, 11)
(13, 14)
(15, 16)
(12, 12)
(14, 11)
(18, 17)
(13, 14)
>>> daily = [mon, tue]
>>> from pprint import pprint as pp
>>> pp(daily)
[[12, 13, 15, 12, 14, 18, 13], [11, 14, 16, 12, 11, 17, 14]]
>>> transposed = list(zip(*daily))
>>> pp(transposed)
[(12, 11), (13, 14), (15, 16), (12, 12), (14, 11), (18, 17), (13, 14)]