Ansible/タスク実行の仕組み

Ansible

Ansibleの内部の仕組みについてざっくりと調べてみる。
moduleを書いたりする時に知っていると役立つ。

タスク実行の流れ †

playbookを実行した時の動作の流れ概要は以下のとおり。

1. cli_stubからExcecutorの決定
   ansible-playbookを実行した場合は、PlaybookCLIが実行される。
2. ansible.cfgの決定
3. コマンドライン引数のパース
4. playbookのパスからロードするプラグインを探索
5. PlaybookExecutorオブジェクトを初期化しプレイブックを実行
   1. become/connection/shellのプリロード
   2. playbookのロード
   3. callbackプラグインのロード
   4. Workerの起動
      メモ
      
      • ansible/plugins/strategy/__init__.py
      • ansible/executor/process/worker.py
   5. TaskExecutorの実行
      メモ ansible/executor/task_executor.py
   6. Actionの実行
   7. moduleのコンパイル
      メモ
      
      • ansible/plugins/action/__init__.py
      • ansible/executor/module_common.py
   8. コンパイルされたファイルをリモートへ転送
      メモ ansible/plugins/action/__init__.py
   9. コンパイルされたpythonスクリプトをリモートで実行
      メモansible/plugins/connection/ssh.py
      コンパイルされたPythonスクリプト内のzipデータをtempディレクトリに展開
      例 /tmp/ansible_hello_payload_a0bPXi/ansible_hello_payload.zip

実験 単純なタスクの実行 †

以下の単純なタスクを実行して、基本的な動作の仕組みを見る。
helloモジュールは独自moduleでechoに指定された文字列を出力するだけのもの。

test.yaml

- hosts: localhost
  gather_facts: no
  tasks:
    - hello:
        echo: "hello world"

rpdbでスタックトレースを出力した結果が以下である。
スタックトレースは、リモートでhelloモジュール実行する直前のものである。

(Pdb) bt
  /home/guest/.pyenv/versions/3.8.1/bin/ansible-playbook(123)<module>()
-> exit_code = cli.run()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/cli/playbook.py(129)run()
-> results = pbex.run()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/playbook_executor.py(172)run()
-> result = self._tqm.run(play=play)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py(242)run()
-> play_return = strategy.run(iterator, play_context)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/strategy/linear.py(310)run()
-> self._queue_task(host, task, task_vars, play_context)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/strategy/__init__.py(360)_queue_task()
-> worker_prc.start()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(96)start()
-> return super(WorkerProcess, self).start()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py(121)start()
-> self._popen = self._Popen(self)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py(276)_Popen()
-> return Popen(process_obj)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py(19)__init__()
-> self._launch(process_obj)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py(75)_launch()
-> code = process_obj._bootstrap(parent_sentinel=child_r)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py(315)_bootstrap()
-> self.run()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(130)run()
-> return self._run()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(151)_run()
-> executor_result = TaskExecutor(
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_executor.py(146)run()
-> res = self._execute()
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_executor.py(646)_execute()
-> result = self._handler.run(task_vars=variables)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/normal.py(46)run()
-> result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/__init__.py(927)_execute_module()
-> res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/__init__.py(1076)_low_level_execute_command()
-> rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(1192)exec_command()
-> (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(392)wrapped()
-> return_tuple = func(self, *args, **kwargs)
  /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(1053)_run()
-> return self._bare_run(cmd, in_data, sudoable=sudoable, checkrc=checkrc)
> /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(789)_bare_run()
-> p = subprocess.Popen(cmd, stdin=slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

実行されるスクリプトは、以下のようにコンパイルされている。
使用されるテンプレートは以下に記述されている。

参考

• ansible/executor/module_common.py - on https://github.com/ansible/ansible/

#!/usr/bin/python
# -*- coding: utf-8 -*-
_ANSIBALLZ_WRAPPER = True # For test-module.py script to tell this is a ANSIBALLZ_WRAPPER
def _ansiballz_main():

    import os
    import os.path
    import sys
    import __main__
    scriptdir = None
    try:
        scriptdir = os.path.dirname(os.path.realpath(__main__.__file__))
    except (AttributeError, OSError):
        pass
    if scriptdir is not None:
        sys.path = [p for p in sys.path if p != scriptdir]
    import base64
    import runpy
    import shutil
    import tempfile
    import zipfile
    if sys.version_info < (3,):
        PY3 = False
    else:
        PY3 = True
    ZIPDATA = """ここにbase64エンコードされたzipデータが入る"""
    def invoke_module(modlib_path, temp_path, json_params):
        z = zipfile.ZipFile(modlib_path, mode='a')
        sitecustomize = u'import sys\nsys.path.insert(0,"%s")\n' %  modlib_path
        sitecustomize = sitecustomize.encode('utf-8')
        zinfo = zipfile.ZipInfo()
        zinfo.filename = 'sitecustomize.py'
        zinfo.date_time = ( 2020, 3, 7, 13, 46, 50)
        z.writestr(zinfo, sitecustomize)
        z.close()
        sys.path.insert(0, modlib_path)
        from ansible.module_utils import basic
        basic._ANSIBLE_ARGS = json_params

        runpy.run_module(mod_name='ansible.modules.hello', init_globals=None, run_name='__main__', alter_sys=True)
        print('{"msg": "New-style module did not handle its own exit", "failed": true}')
        sys.exit(1)
    def debug(command, zipped_mod, json_params):
        basedir = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'debug_dir')
        args_path = os.path.join(basedir, 'args')
        if command == 'excommunicate':
            print('The excommunicate debug command is deprecated and will be removed in 2.11.  Use execute instead.')
            command = 'execute'
        if command == 'explode':
            z = zipfile.ZipFile(zipped_mod)
            for filename in z.namelist():
                if filename.startswith('/'):
                    raise Exception('Something wrong with this module zip file: should not contain absolute paths')
                dest_filename = os.path.join(basedir, filename)
                if dest_filename.endswith(os.path.sep) and not os.path.exists(dest_filename):
                    os.makedirs(dest_filename)
                else:
                    directory = os.path.dirname(dest_filename)
                    if not os.path.exists(directory):
                        os.makedirs(directory)
                    f = open(dest_filename, 'wb')
                    f.write(z.read(filename))
                    f.close()
            f = open(args_path, 'wb')
            f.write(json_params)
            f.close()
            print('Module expanded into:')
            print('%s' % basedir)
            exitcode = 0
        elif command == 'execute':
            sys.path.insert(0, basedir)
            with open(args_path, 'rb') as f:
                json_params = f.read()
            from ansible.module_utils import basic
            basic._ANSIBLE_ARGS = json_params
            runpy.run_module(mod_name='ansible.modules.hello', init_globals=None, run_name='__main__', alter_sys=True)
            print('{"msg": "New-style module did not handle its own exit", "failed": true}')
            sys.exit(1)
        else:
            print('WARNING: Unknown debug command.  Doing nothing.')
            exitcode = 0
        return exitcode
    ANSIBALLZ_PARAMS = '{"ANSIBLE_MODULE_ARGS": {"echo": "hello world", "_ansible_check_mode": false, "_ansible_no_log": false, "_ansible_debug": false, "_ansible_diff": false, "_ansible_verbosity": 0, "_ansible_version": "2.9.5", "_ansible_module_name": "hello", "_ansible_syslog_facility": "LOG_USER", "_ansible_selinux_special_fs": ["fuse", "nfs", "vboxsf", "ramfs", "9p", "vfat"], "_ansible_string_conversion_action": "warn", "_ansible_socket": null, "_ansible_shell_executable": "/bin/sh", "_ansible_keep_remote_files": false, "_ansible_tmpdir": "/home/guest/.ansible/tmp/ansible-tmp-1583588809.357903-141465830873137/", "_ansible_remote_tmp": "~/.ansible/tmp"}}'
    if PY3:
        ANSIBALLZ_PARAMS = ANSIBALLZ_PARAMS.encode('utf-8')
    try:
        temp_path = tempfile.mkdtemp(prefix='ansible_hello_payload_')
        zipped_mod = os.path.join(temp_path, 'ansible_hello_payload.zip')
        with open(zipped_mod, 'wb') as modlib:
            modlib.write(base64.b64decode(ZIPDATA))
        if len(sys.argv) == 2:
            exitcode = debug(sys.argv[1], zipped_mod, ANSIBALLZ_PARAMS)
        else:
            invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
    finally:
        try:
            shutil.rmtree(temp_path)
        except (NameError, OSError):
            pass
    sys.exit(exitcode)
if __name__ == '__main__':
    _ansiballz_main()

moduleのパラメータは、basic._ANSIBLE_ARGSで渡されている。basic._ANSIBLE_ARGSは、global指定でAnsibleModuleから参照される。
参考 ansible/module_utils/basic.py#_load_params()

リモートで参照されるモジュール一式は、zipファイルとして作成され、base64エンコードされたデータがコンパイルスクリプトのZIPDATA変数に書かれる。このZIPDATAは、リモート実行時にtempディレクトリで展開される。

作成されるzipを展開すると以下のようになる。

$ cat /home/guest/.ansible/tmp/ansible-local-2610t6dkguao/ansiballz_cache/hello-ZIP_DEFLATED > hello-base64.zip
$ base64 -d hello-base64.zip > hello.zip
$ unzip hello.zip
Archive:  hello.zip
  inflating: ansible/__init__.py
  inflating: ansible/module_utils/__init__.py
  inflating: ansible/module_utils/basic.py
  inflating: ansible/module_utils/common/validation.py
  inflating: ansible/module_utils/parsing/__init__.py
  inflating: ansible/module_utils/common/parameters.py
  inflating: ansible/module_utils/common/_collections_compat.py
  inflating: ansible/module_utils/parsing/convert_bool.py
  inflating: ansible/module_utils/six/__init__.py
  inflating: ansible/module_utils/common/file.py
  inflating: ansible/module_utils/common/_utils.py
  inflating: ansible/module_utils/common/text/formatters.py
  inflating: ansible/module_utils/common/_json_compat.py
  inflating: ansible/module_utils/pycompat24.py
  inflating: ansible/module_utils/_text.py
  inflating: ansible/module_utils/common/__init__.py
  inflating: ansible/module_utils/common/text/__init__.py
  inflating: ansible/module_utils/common/sys_info.py
  inflating: ansible/module_utils/common/text/converters.py
  inflating: ansible/module_utils/common/process.py
  inflating: ansible/module_utils/common/collections.py
  inflating: ansible/module_utils/distro/__init__.py
  inflating: ansible/module_utils/distro/_distro.py
  inflating: ansible/modules/hello.py
  inflating: ansible/modules/__init__.py

tempに作成されるzipは以下のとおり。

$ unzip ansible_hello_payload.zip
Archive:  ansible_hello_payload.zip
  inflating: ansible/__init__.py
  inflating: ansible/module_utils/__init__.py
  inflating: ansible/module_utils/basic.py
  inflating: ansible/module_utils/parsing/__init__.py
  inflating: ansible/module_utils/common/_json_compat.py
  inflating: ansible/module_utils/common/validation.py
  inflating: ansible/module_utils/common/process.py
  inflating: ansible/module_utils/common/sys_info.py
  inflating: ansible/module_utils/common/text/formatters.py
  inflating: ansible/module_utils/common/text/converters.py
  inflating: ansible/module_utils/parsing/convert_bool.py
  inflating: ansible/module_utils/common/text/__init__.py
  inflating: ansible/module_utils/pycompat24.py
  inflating: ansible/module_utils/common/parameters.py
  inflating: ansible/module_utils/common/__init__.py
  inflating: ansible/module_utils/common/_utils.py
  inflating: ansible/module_utils/six/__init__.py
  inflating: ansible/module_utils/common/_collections_compat.py
  inflating: ansible/module_utils/common/file.py
  inflating: ansible/module_utils/_text.py
  inflating: ansible/module_utils/common/collections.py
  inflating: ansible/module_utils/distro/__init__.py
  inflating: ansible/module_utils/distro/_distro.py
  inflating: ansible/modules/hello.py
  inflating: ansible/modules/__init__.py
 extracting: sitecustomize.py

参考リンク †

• https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#ansiballz
• https://qiita.com/YumaInaura/items/67d11d1c2820644a71a2
• https://www.slideshare.net/ShingoKitayama/ansible-73396071
• https://www.school.ctc-g.co.jp/columns/super/super49.html