#author("2020-03-09T23:57:20+09:00","default:haikikyou","haikikyou")
#author("2020-03-09T23:58:09+09:00","default:haikikyou","haikikyou")
[[Ansible]]
#contents
Ansibleの内部の仕組みについてざっくりと調べてみる。~
moduleを書いたりする時に知っていると役立つ。
* タスク実行の流れ [#la3a05e4]
playbookを実行した時の動作の流れ概要は以下のとおり。
+ cli_stubからExcecutorの決定~
ansible-playbookを実行した場合は、PlaybookCLIが実行される。
+ ansible.cfgの決定
+ コマンドライン引数のパース
+ playbookのパスからロードするプラグインを探索
+ PlaybookExecutorオブジェクトを初期化しプレイブックを実行
++ become/connection/shellのプリロード
++ playbookのロード
++ callbackプラグインのロード
++ Workerの起動~
&label(memo){メモ};~
--- [[ansible/plugins/strategy/__init__.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/plugins/strategy/__init__.py#L298]]
--- [[ansible/executor/process/worker.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/executor/process/worker.py#L85]]
++ TaskExecutorの実行~
&label(memo){メモ}; [[ansible/executor/task_executor.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/executor/task_executor.py#L92]]
++ Actionの実行
++ moduleのコンパイル~
&label(memo){メモ}; ~
---[[ansible/plugins/action/__init__.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/plugins/action/__init__.py#L155]]
---[[ansible/executor/module_common.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/executor/module_common.py#L958]]
++ コンパイルされたファイルをリモートへ転送~
&label(memo){メモ}; [[ansible/plugins/action/__init__.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/plugins/action/__init__.py#L415]]
++ コンパイルされたpythonスクリプトをリモートで実行~
&label(memo){メモ};[[ansible/plugins/connection/ssh.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/plugins/connection/ssh.py#L760]]~
コンパイルされたPythonスクリプト内のzipデータをtempディレクトリに展開~
&label(sample){例}; &code(){/tmp/ansible_hello_payload_a0bPXi/ansible_hello_payload.zip};
* &label(study){実験}; 単純なタスクの実行 [#q89356e0]
以下の単純なタスクを実行して、基本的な動作の仕組みを見る。~
helloモジュールは独自moduleで&code(){echo};に指定された文字列を出力するだけのもの。
''test.yaml''
#geshi{{{
- hosts: localhost
gather_facts: no
tasks:
- hello:
echo: "hello world"
}}}
rpdbでスタックトレースを出力した結果が以下である。~
スタックトレースは、リモートでhelloモジュール実行する直前のものである。
#geshi{{{
(Pdb) bt
/home/guest/.pyenv/versions/3.8.1/bin/ansible-playbook(123)<module>()
-> exit_code = cli.run()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/cli/playbook.py(129)run()
-> results = pbex.run()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/playbook_executor.py(172)run()
-> result = self._tqm.run(play=play)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py(242)run()
-> play_return = strategy.run(iterator, play_context)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/strategy/linear.py(310)run()
-> self._queue_task(host, task, task_vars, play_context)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/strategy/__init__.py(360)_queue_task()
-> worker_prc.start()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(96)start()
-> return super(WorkerProcess, self).start()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py(121)start()
-> self._popen = self._Popen(self)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/context.py(276)_Popen()
-> return Popen(process_obj)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py(19)__init__()
-> self._launch(process_obj)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/popen_fork.py(75)_launch()
-> code = process_obj._bootstrap(parent_sentinel=child_r)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/multiprocessing/process.py(315)_bootstrap()
-> self.run()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(130)run()
-> return self._run()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/process/worker.py(151)_run()
-> executor_result = TaskExecutor(
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_executor.py(146)run()
-> res = self._execute()
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/executor/task_executor.py(646)_execute()
-> result = self._handler.run(task_vars=variables)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/normal.py(46)run()
-> result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/__init__.py(927)_execute_module()
-> res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/action/__init__.py(1076)_low_level_execute_command()
-> rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(1192)exec_command()
-> (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(392)wrapped()
-> return_tuple = func(self, *args, **kwargs)
/home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(1053)_run()
-> return self._bare_run(cmd, in_data, sudoable=sudoable, checkrc=checkrc)
> /home/guest/.pyenv/versions/3.8.1/lib/python3.8/site-packages/ansible/plugins/connection/ssh.py(789)_bare_run()
-> p = subprocess.Popen(cmd, stdin=slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
}}}
実行されるスクリプトは、以下のようにコンパイルされている。~
使用されるテンプレートは以下に記述されている。
&label(warn){参考};
- [[ansible/executor/module_common.py>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/executor/module_common.py]] - &size(11){&color(gray){on https://github.com/ansible/ansible/};};
#geshi(python){{{
#!/usr/bin/python
# -*- coding: utf-8 -*-
_ANSIBALLZ_WRAPPER = True # For test-module.py script to tell this is a ANSIBALLZ_WRAPPER
def _ansiballz_main():
import os
import os.path
import sys
import __main__
scriptdir = None
try:
scriptdir = os.path.dirname(os.path.realpath(__main__.__file__))
except (AttributeError, OSError):
pass
if scriptdir is not None:
sys.path = [p for p in sys.path if p != scriptdir]
import base64
import runpy
import shutil
import tempfile
import zipfile
if sys.version_info < (3,):
PY3 = False
else:
PY3 = True
ZIPDATA = """ここにbase64エンコードされたzipデータが入る"""
def invoke_module(modlib_path, temp_path, json_params):
z = zipfile.ZipFile(modlib_path, mode='a')
sitecustomize = u'import sys\nsys.path.insert(0,"%s")\n' % modlib_path
sitecustomize = sitecustomize.encode('utf-8')
zinfo = zipfile.ZipInfo()
zinfo.filename = 'sitecustomize.py'
zinfo.date_time = ( 2020, 3, 7, 13, 46, 50)
z.writestr(zinfo, sitecustomize)
z.close()
sys.path.insert(0, modlib_path)
from ansible.module_utils import basic
basic._ANSIBLE_ARGS = json_params
runpy.run_module(mod_name='ansible.modules.hello', init_globals=None, run_name='__main__', alter_sys=True)
print('{"msg": "New-style module did not handle its own exit", "failed": true}')
sys.exit(1)
def debug(command, zipped_mod, json_params):
basedir = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'debug_dir')
args_path = os.path.join(basedir, 'args')
if command == 'excommunicate':
print('The excommunicate debug command is deprecated and will be removed in 2.11. Use execute instead.')
command = 'execute'
if command == 'explode':
z = zipfile.ZipFile(zipped_mod)
for filename in z.namelist():
if filename.startswith('/'):
raise Exception('Something wrong with this module zip file: should not contain absolute paths')
dest_filename = os.path.join(basedir, filename)
if dest_filename.endswith(os.path.sep) and not os.path.exists(dest_filename):
os.makedirs(dest_filename)
else:
directory = os.path.dirname(dest_filename)
if not os.path.exists(directory):
os.makedirs(directory)
f = open(dest_filename, 'wb')
f.write(z.read(filename))
f.close()
f = open(args_path, 'wb')
f.write(json_params)
f.close()
print('Module expanded into:')
print('%s' % basedir)
exitcode = 0
elif command == 'execute':
sys.path.insert(0, basedir)
with open(args_path, 'rb') as f:
json_params = f.read()
from ansible.module_utils import basic
basic._ANSIBLE_ARGS = json_params
runpy.run_module(mod_name='ansible.modules.hello', init_globals=None, run_name='__main__', alter_sys=True)
print('{"msg": "New-style module did not handle its own exit", "failed": true}')
sys.exit(1)
else:
print('WARNING: Unknown debug command. Doing nothing.')
exitcode = 0
return exitcode
ANSIBALLZ_PARAMS = '{"ANSIBLE_MODULE_ARGS": {"echo": "hello world", "_ansible_check_mode": false, "_ansible_no_log": false, "_ansible_debug": false, "_ansible_diff": false, "_ansible_verbosity": 0, "_ansible_version": "2.9.5", "_ansible_module_name": "hello", "_ansible_syslog_facility": "LOG_USER", "_ansible_selinux_special_fs": ["fuse", "nfs", "vboxsf", "ramfs", "9p", "vfat"], "_ansible_string_conversion_action": "warn", "_ansible_socket": null, "_ansible_shell_executable": "/bin/sh", "_ansible_keep_remote_files": false, "_ansible_tmpdir": "/home/guest/.ansible/tmp/ansible-tmp-1583588809.357903-141465830873137/", "_ansible_remote_tmp": "~/.ansible/tmp"}}'
if PY3:
ANSIBALLZ_PARAMS = ANSIBALLZ_PARAMS.encode('utf-8')
try:
temp_path = tempfile.mkdtemp(prefix='ansible_hello_payload_')
zipped_mod = os.path.join(temp_path, 'ansible_hello_payload.zip')
with open(zipped_mod, 'wb') as modlib:
modlib.write(base64.b64decode(ZIPDATA))
if len(sys.argv) == 2:
exitcode = debug(sys.argv[1], zipped_mod, ANSIBALLZ_PARAMS)
else:
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
finally:
try:
shutil.rmtree(temp_path)
except (NameError, OSError):
pass
sys.exit(exitcode)
if __name__ == '__main__':
_ansiballz_main()
}}}
moduleのパラメータは、&code(){basic._ANSIBLE_ARGS};で渡されている。&code(){basic._ANSIBLE_ARGS};は、global指定で&code(){AnsibleModule};から参照される。~
&label(warn){参考}; [[ansible/module_utils/basic.py#_load_params()>https://github.com/ansible/ansible/blob/308723c3ca1122363e419070f1fa1d76ff5611a9/lib/ansible/module_utils/basic.py#L504]]
リモートで参照されるモジュール一式は、zipファイルとして作成され、base64エンコードされたデータがコンパイルスクリプトの&code(){ZIPDATA};変数に書かれる。この&code(){ZIPDATA};は、リモート実行時にtempディレクトリで展開される。
作成されるzipを展開すると以下のようになる。
#geshi{{{
$ cat /home/guest/.ansible/tmp/ansible-local-2610t6dkguao/ansiballz_cache/hello-ZIP_DEFLATED > hello-base64.zip
$ base64 -d hello-base64.zip > hello.zip
$ unzip hello.zip
Archive: hello.zip
inflating: ansible/__init__.py
inflating: ansible/module_utils/__init__.py
inflating: ansible/module_utils/basic.py
inflating: ansible/module_utils/common/validation.py
inflating: ansible/module_utils/parsing/__init__.py
inflating: ansible/module_utils/common/parameters.py
inflating: ansible/module_utils/common/_collections_compat.py
inflating: ansible/module_utils/parsing/convert_bool.py
inflating: ansible/module_utils/six/__init__.py
inflating: ansible/module_utils/common/file.py
inflating: ansible/module_utils/common/_utils.py
inflating: ansible/module_utils/common/text/formatters.py
inflating: ansible/module_utils/common/_json_compat.py
inflating: ansible/module_utils/pycompat24.py
inflating: ansible/module_utils/_text.py
inflating: ansible/module_utils/common/__init__.py
inflating: ansible/module_utils/common/text/__init__.py
inflating: ansible/module_utils/common/sys_info.py
inflating: ansible/module_utils/common/text/converters.py
inflating: ansible/module_utils/common/process.py
inflating: ansible/module_utils/common/collections.py
inflating: ansible/module_utils/distro/__init__.py
inflating: ansible/module_utils/distro/_distro.py
inflating: ansible/modules/hello.py
inflating: ansible/modules/__init__.py
}}}
tempに作成されるzipは以下のとおり。
#geshi{{{
$ unzip ansible_hello_payload.zip
Archive: ansible_hello_payload.zip
inflating: ansible/__init__.py
inflating: ansible/module_utils/__init__.py
inflating: ansible/module_utils/basic.py
inflating: ansible/module_utils/parsing/__init__.py
inflating: ansible/module_utils/common/_json_compat.py
inflating: ansible/module_utils/common/validation.py
inflating: ansible/module_utils/common/process.py
inflating: ansible/module_utils/common/sys_info.py
inflating: ansible/module_utils/common/text/formatters.py
inflating: ansible/module_utils/common/text/converters.py
inflating: ansible/module_utils/parsing/convert_bool.py
inflating: ansible/module_utils/common/text/__init__.py
inflating: ansible/module_utils/pycompat24.py
inflating: ansible/module_utils/common/parameters.py
inflating: ansible/module_utils/common/__init__.py
inflating: ansible/module_utils/common/_utils.py
inflating: ansible/module_utils/six/__init__.py
inflating: ansible/module_utils/common/_collections_compat.py
inflating: ansible/module_utils/common/file.py
inflating: ansible/module_utils/_text.py
inflating: ansible/module_utils/common/collections.py
inflating: ansible/module_utils/distro/__init__.py
inflating: ansible/module_utils/distro/_distro.py
inflating: ansible/modules/hello.py
inflating: ansible/modules/__init__.py
extracting: sitecustomize.py
}}}
* 参考リンク [#a1d1486a]
- https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#ansiballz
- https://qiita.com/YumaInaura/items/67d11d1c2820644a71a2
- https://www.slideshare.net/ShingoKitayama/ansible-73396071
- https://www.school.ctc-g.co.jp/columns/super/super49.html