UDN-企业互联网技术人气社区

板块导航

浏览  : 2073
回复  : 0

[干货] ansible2.0 playbook api运维应用

[复制链接]
舞操的头像 楼主
发表于 2017-1-6 16:14:05 | 显示全部楼层 |阅读模式
  写在前面:

  ansible是一个非常棒的运维工具,可以远程批量执行命令、上传文件等自动化运维操作,由于要搞配置管理,初始化等批量操作,而自己对ansible相对熟悉,因此选择了ansible playbook。不过在调用playbook API的过程中,发现原始api并不能满足我的需求,网络上多数文档还是1.0版本,因此下载了2.0源码查看,重写了部分类。因此这里总结学习,也给有相同需求的朋友参考

  ansible-playbook使用

  一、ansible-playbook简单介绍

  playbook字面意思看就是剧本,其实也很形象,ansible-playbook就是事先定义好很多task(任务)放在.yml文件中,执行的时候就像剧本一样,依次往下演(执行)

  1.   #/tmp/xx.yml

  2.   ---

  3.   - hosts: all #hosts中指定

  4.   tasks:

  5.   - name: "ls /tmp" #- name:任务名称,下面是任务过程

  6.   shell: 'ls /tmp/'
复制代码


  ansible-playbook执行:

d.png


  二、ansible-playbook api介绍

  ansible-playbook api由ansible官方提供接口,用于在python代码中更灵活的使用,但官方只提供了ansible-api的参考文档,且2.0以后变化较大,api编写更复杂,但更为灵活

  官方参考: http://docs.ansible.com/ansib...

  一个基本的ansible-playbook api调用

  1.   # -*- coding:utf-8 -*-

  2.   # 描述:

  3.   # V1 WZJ 2016-12-20

  4.   from collections import namedtuple

  5.   from ansible.parsing.dataloader import DataLoader

  6.   from ansible.vars import VariableManager

  7.   from ansible.inventory import Inventory

  8.   from ansible.executor.playbook_executor import PlaybookExecutor

  9.   # 用来加载解析yaml文件或JSON内容,并且支持vault的解密

  10.   loader = DataLoader()

  11.   # 管理变量的类,包括主机,组,扩展等变量,之前版本是在 inventory中的

  12.   variable_manager = VariableManager()

  13.   # 根据inventory加载对应变量,此处host_list参数可以有两种格式:

  14.   # 1: hosts文件(需要),

  15.   # 2: 可以是IP列表,此处使用IP列表

  16.   inventory = Inventory(loader=loader, variable_manager=variable_manager,host_list=['172.16.1.121'])

  17.   variable_manager.set_inventory(inventory)

  18.   # 设置密码,需要是dict类型

  19.   passwords=dict(conn_pass='your password')

  20.   # 初始化需要的对象

  21.   Options = namedtuple('Options',

  22.   ['connection',

  23.   'remote_user',

  24.   'ask_sudo_pass',

  25.   'verbosity',

  26.   'ack_pass',

  27.   'module_path',

  28.   'forks',

  29.   'become',

  30.   'become_method',

  31.   'become_user',

  32.   'check',

  33.   'listhosts',

  34.   'listtasks',

  35.   'listtags',

  36.   'syntax',

  37.   'sudo_user',

  38.   'sudo'])

  39.   # 初始化需要的对象

  40.   options = Options(connection='smart',

  41.   remote_user='root',

  42.   ack_pass=None,

  43.   sudo_user='root',

  44.   forks=5,

  45.   sudo='yes',

  46.   ask_sudo_pass=False,

  47.   verbosity=5,

  48.   module_path=None,

  49.   become=True,

  50.   become_method='sudo',

  51.   become_user='root',

  52.   check=None,

  53.   listhosts=None,

  54.   listtasks=None,

  55.   listtags=None,

  56.   syntax=None)

  57.   # playbooks就填写yml文件即可,可以有多个,以列表形式

  58.   playbook = PlaybookExecutor(playbooks=['/tmp/xx.yml'],inventory=inventory,

  59.   variable_manager=variable_manager,

  60.   loader=loader,options=options,passwords=passwords)

  61.   # 开始执行

  62.   playbook.run()
复制代码


  执行上面脚本:



  可以发现,这个api调用执行虽然成功了,但是并没有什么详细输出(这个跟一个名叫CallbackBase的类有关)

  重写palybook api

  1 为什么重写?

  在实际应用中,有许多地方是不太满足需求的

  1.比如inventory有host group(主机组)的概念,但是默认都放在all组里面去了,导致在编写yml的时候,无法对不同IP组进行不同的操作

  2.console输出结果有时候需要自定义

  3.调用api需要更为灵活

  当然除了以上需求外,还有一些其它意义,比如阅读源码有助于熟悉ansible的工作方式,也提升自己的脚本能力

  注意:所谓重写只是在源码的基础上继承并修改了部分代码,有可能导致ansible不稳定的情况,另外只有在有中文注释的地方才是重写内容

  2 ansible源码文件简单介绍

  源码下载地址:https://github.com/ansible/an...

c.png


  其实ansible我们所用到的源码都在lib里面,而需要关注的也主要是三个文件夹和一些类:

  用于解析和聚合host,group等环境变量: lib/ansible/inventory/__init__.py 中 Inventory 类

  用于console输出: lib/ansible/plugins/callback/default.py 中 CallbackModule 类

  用于playbook的解析工作: lib/executor/playbook_executor.py 中 PlaybookExecutor 类

  playbook底层用到的任务队列:lib/executor/task_queue_manager.py 中 TaskQueueManager 类

  几个重写的类的树状图:

b.png


  一、Inventory类重写

  1.修改可以为IP传入group name2.额外的自定义参数,如:{"test":1000}

  1.   # -*- coding:utf-8 -*-

  2.   # 描述:

  3.   # V1 WZJ 2016-12-20 Inventory重写封装api基本功能

  4.   import fnmatch

  5.   from ansible.compat.six import string_types, iteritems

  6.   from ansible import constants as C

  7.   from ansible.errors import AnsibleError

  8.   from ansible.inventory.dir import InventoryDirectory, get_file_parser

  9.   from ansible.inventory.group import Group

  10.   from ansible.inventory.host import Host

  11.   from ansible.module_utils._text import to_bytes, to_text

  12.   from ansible.parsing.utils.addresses import parse_address

  13.   from ansible.plugins import vars_loader

  14.   from ansible.utils.vars import combine_vars

  15.   from ansible.utils.path import unfrackpath

  16.   try:

  17.   from __main__ import display

  18.   except ImportError:

  19.   from ansible.utils.display import Display

  20.   display = Display()

  21.   HOSTS_PATTERNS_CACHE = {}

  22.   from ansible.inventory import Inventory

  23.   class YunweiInventory(Inventory):

  24.   '''重写Inventory'''

  25.   def __init__(self, loader, variable_manager, group_name, ext_vars=None,host_list=C.DEFAULT_HOST_LIST):

  26.   # the host file file, or script path, or list of hosts

  27.   # if a list, inventory data will NOT be loaded

  28.   # self.host_list = unfrackpath(host_list, follow=False)

  29.   # 传入的hosts

  30.   self.host_list = host_list

  31.   # 传入的项目名也是组名

  32.   self.group_name = group_name

  33.   # 传入的外部变量,格式为字典

  34.   self.ext_vars = ext_vars

  35.   # caching to avoid repeated calculations, particularly with

  36.   # external inventory scripts.

  37.   self._vars_per_host = {}

  38.   self._vars_per_group = {}

  39.   self._hosts_cache = {}

  40.   self._pattern_cache = {}

  41.   self._group_dict_cache = {}

  42.   self._vars_plugins = []

  43.   self._basedir = self.basedir()

  44.   # Contains set of filenames under group_vars directories

  45.   self._group_vars_files = self._find_group_vars_files(self._basedir)

  46.   self._host_vars_files = self._find_host_vars_files(self._basedir)

  47.   # to be set by calling set_playbook_basedir by playbook code

  48.   self._playbook_basedir = None

  49.   # the inventory object holds a list of groups

  50.   self.groups = {}

  51.   # a list of host(names) to contain current inquiries to

  52.   self._restriction = None

  53.   self._subset = None

  54.   # clear the cache here, which is only useful if more than

  55.   # one Inventory objects are created when using the API directly

  56.   self.clear_pattern_cache()

  57.   self.clear_group_dict_cache()

  58.   self.parse_inventory(host_list)

  59.   def parse_inventory(self, host_list):

  60.   if isinstance(host_list, string_types):

  61.   if "," in host_list:

  62.   host_list = host_list.split(",")

  63.   host_list = [ h for h in host_list if h and h.strip() ]

  64.   self.parser = None

  65.   # Always create the 'all' and 'ungrouped' groups, even if host_list is

  66.   # empty: in this case we will subsequently an the implicit 'localhost' to it.

  67.   ungrouped = Group('ungrouped')

  68.   all = Group('all')

  69.   all.add_child_group(ungrouped)

  70.   # 加一个本地IP

  71.   local_group = Group('local')

  72.   # 自定义组名

  73.   zdy_group_name = Group(self.group_name)

  74.   self.groups = {self.group_name:zdy_group_name,"all":all,"ungrouped":ungrouped,"local":local_group}

  75.   #self.groups = dict(all=all, ungrouped=ungrouped)

  76.   if host_list is None:

  77.   pass

  78.   elif isinstance(host_list, list):

  79.   # 默认添加一个本地IP

  80.   (lhost, lport) = parse_address('127.0.0.1', allow_ranges=False)

  81.   new_host = Host(lhost, lport)

  82.   local_group.add_host(new_host)

  83.   for h in host_list:

  84.   try:

  85.   (host, port) = parse_address(h, allow_ranges=False)

  86.   except AnsibleError as e:

  87.   display.vvv("Unable to parse address from hostname, leaving unchanged: %s" % to_text(e))

  88.   host = h

  89.   port = None

  90.   new_host = Host(host, port)

  91.   if h in C.LOCALHOST:

  92.   # set default localhost from inventory to avoid creating an implicit one. Last localhost defined 'wins'.

  93.   if self.localhost is not None:

  94.   display.warning("A duplicate localhost-like entry was found (%s). First found localhost was %s" % (h, self.localhost.name))

  95.   display.vvvv("Set default localhost to %s" % h)

  96.   self.localhost = new_host

  97.   # 为组添加host

  98.   zdy_group_name.add_host(new_host)

  99.   # 为主机组添加额外参数

  100.   # 添加外部变量

  101.   if self.ext_vars and isinstance(self.ext_vars,dict):

  102.   for k,v in self.ext_vars.items():

  103.   zdy_group_name.set_variable(k,v)

  104.   local_group.set_variable(k,v)

  105.   elif self._loader.path_exists(host_list):

  106.   # TODO: switch this to a plugin loader and a 'condition' per plugin on which it should be tried, restoring 'inventory pllugins'

  107.   if self.is_directory(host_list):

  108.   # Ensure basedir is inside the directory

  109.   host_list = os.path.join(self.host_list, "")

  110.   self.parser = InventoryDirectory(loader=self._loader, groups=self.groups, filename=host_list)

  111.   else:

  112.   self.parser = get_file_parser(host_list, self.groups, self._loader)

  113.   vars_loader.add_directory(self._basedir, with_subdir=True)

  114.   if not self.parser:

  115.   # should never happen, but JIC

  116.   raise AnsibleError("Unable to parse %s as an inventory source" % host_list)

  117.   else:

  118.   display.warning("Host file not found: %s" % to_text(host_list))

  119.   self._vars_plugins = [ x for x in vars_loader.all(self) ]

  120.   # set group vars from group_vars/ files and vars plugins

  121.   for g in self.groups:

  122.   group = self.groups[g]

  123.   group.vars = combine_vars(group.vars, self.get_group_variables(group.name))

  124.   self.get_group_vars(group)

  125.   # get host vars from host_vars/ files and vars plugins

  126.   for host in self.get_hosts(ignore_limits=True, ignore_restrictions=True):

  127.   host.vars = combine_vars(host.vars, self.get_host_variables(host.name))

  128.   self.get_host_vars(host)
复制代码


  二、collback重写,用于自定义输出

  1.以callback.default.CallbackModule为基类继承

  2.自定义了输出格式,正确输出情况,不需要的输出就忽略了,只留下几个有用的stdout

  3.最后的输出结果:

 
a.png


  1.   #-*- coding:utf-8 -*-

  2.   # 描述:

  3.   # V1 WZJ 2016-12-19 CallBack重写封装api基本功能,用于console输出

  4.   import json

  5.   from ansible import constants as C

  6.   from ansible.plugins.callback.default import CallbackModule

  7.   try:

  8.   from __main__ import display

  9.   except ImportError:

  10.   from ansible.utils.display import Display

  11.   display = Display()

  12.   class YunweiCallback(CallbackModule):

  13.   """重写console输出日志"""

  14.   # 重写2.0版本正确stdout

  15.   def v2_runner_on_ok(self, result):

  16.   if self._play.strategy == 'free' and self._last_task_banner != result._task._uuid:

  17.   self._print_task_banner(result._task)

  18.   self._clean_results(result._result, result._task.action)

  19.   #delegated_vars = result._result.get('_ansible_delegated_vars', None)

  20.   delegated_vars = self._dump_results(result._result)

  21.   #delegated_vars = result._result

  22.   #n_delegated_vars = self._dump_results(result)

  23.   #print n_delegated_vars

  24.   self._clean_results(result._result, result._task.action)

  25.   if result._task.action in ('include', 'include_role'):

  26.   return

  27.   elif result._result.get('changed', False):

  28.   if delegated_vars:

  29.   # 自定义输出

  30.   zdy_msg = self.zdy_stdout(json.loads(delegated_vars))

  31.   if zdy_msg:

  32.   msg = "changed: [%s]%s" % (result._host.get_name(), zdy_msg)

  33.   else:

  34.   msg = "changed: [%s -> %s]" % (result._host.get_name(), delegated_vars)

  35.   else:

  36.   msg = "changed: [%s]" % result._host.get_name()

  37.   color = C.COLOR_CHANGED

  38.   # 判断是否是第一步 setup

  39.   elif result._result.get('ansible_facts',False):

  40.   msg = "ok: [ %s | %s ]" % (str(result._host),str(result._host.get_groups()))

  41.   color = C.COLOR_OK

  42.   else:

  43.   if delegated_vars:

  44.   # 自定义输出

  45.   zdy_msg = self.zdy_stdout(json.loads(delegated_vars))

  46.   if zdy_msg:

  47.   msg = "ok: [%s]%s" % (result._host.get_name(), zdy_msg)

  48.   else:

  49.   msg = "ok: [%s -> %s]" % (result._host.get_name(), delegated_vars)

  50.   else:

  51.   msg = "ok: [%s]" % result._host.get_name()

  52.   color = C.COLOR_OK

  53.   if result._task.loop and 'results' in result._result:

  54.   self._process_items(result)

  55.   else:

  56.   self._display.display(msg, color=color)

  57.   self._handle_warnings(result._result)

  58.   # 自定义输出,格式清晰一些

  59.   def zdy_stdout(self,result):

  60.   msg = ''

  61.   if result.get('delta',False):

  62.   msg += u'\t执行时间:%s'%result['delta']

  63.   if result.get('cmd', False):

  64.   msg += u'\n执行命令:%s'%result['cmd']

  65.   if result.get('stderr',False):

  66.   msg += u'\n错误输出:\n%s'%result['stderr']

  67.   if result.get('stdout',False):

  68.   msg += u'\n正确输出:\n%s'%result['stdout']

  69.   if result.get('warnings',False):

  70.   msg += u'\n警告:%s'%result['warnings']

  71.   return msg
复制代码


  三、封装PlaybookExecutor

  1.封装为更容易调用的playbook api调用2.定制化一部分参数

  1.   # -*- coding:utf-8 -*-

  2.   # 描述:

  3.   # V1 WZJ 2016-12-19 PlayBook重写封装api基本功能

  4.   import json

  5.   from ansible import constants as C

  6.   from collections import namedtuple

  7.   from ansible.parsing.dataloader import DataLoader

  8.   from ansible.vars import VariableManager

  9.   from ansible.playbook.play import Play

  10.   from ansible.executor.task_queue_manager import TaskQueueManager

  11.   from ansible.executor.playbook_executor import PlaybookExecutor

  12.   from callback import YunweiCallback

  13.   from ansible.utils.ssh_functions import check_for_controlpersist

  14.   # 调用自定义Inventory

  15.   from inventory import YunweiInventory as Inventory

  16.   try:

  17.   from __main__ import display

  18.   except ImportError:

  19.   from ansible.utils.display import Display

  20.   display = Display()

  21.   class YunweiPlaybookExecutor(PlaybookExecutor):

  22.   '''重写PlayBookExecutor'''

  23.   def __init__(self, playbooks, inventory, variable_manager, loader, options, passwords, stdout_callback=None):

  24.   self._playbooks = playbooks

  25.   self._inventory = inventory

  26.   self._variable_manager = variable_manager

  27.   self._loader = loader

  28.   self._options = options

  29.   self.passwords = passwords

  30.   self._unreachable_hosts = dict()

  31.   if options.listhosts or options.listtasks or options.listtags or options.syntax:

  32.   self._tqm = None

  33.   else:

  34.   self._tqm = TaskQueueManager(inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=self.passwords, stdout_callback=stdout_callback)

  35.   # Note: We run this here to cache whether the default ansible ssh

  36.   # executable supports control persist. Sometime in the future we may

  37.   # need to enhance this to check that ansible_ssh_executable specified

  38.   # in inventory is also cached. We can't do this caching at the point

  39.   # where it is used (in task_executor) because that is post-fork and

  40.   # therefore would be discarded after every task.

  41.   check_for_controlpersist(C.ANSIBLE_SSH_EXECUTABLE)

  42.   class PlayBookJob(object):

  43.   '''封装一个playbook接口,提供给外部使用'''

  44.   def __init__(self,playbooks,host_list,ssh_user='bbs',passwords='null',project_name='all',ack_pass=False,forks=5,ext_vars=None):

  45.   self.playbooks = playbooks

  46.   self.host_list = host_list

  47.   self.ssh_user = ssh_user

  48.   self.passwords = dict(vault_pass=passwords)

  49.   self.project_name = project_name

  50.   self.ack_pass = ack_pass

  51.   self.forks = forks

  52.   self.connection='smart'

  53.   self.ext_vars = ext_vars

  54.   ## 用来加载解析yaml文件或JSON内容,并且支持vault的解密

  55.   self.loader = DataLoader()

  56.   # 管理变量的类,包括主机,组,扩展等变量,之前版本是在 inventory中的

  57.   self.variable_manager = VariableManager()

  58.   # 根据inventory加载对应变量

  59.   self.inventory = Inventory(loader=self.loader,

  60.   variable_manager=self.variable_manager,

  61.   group_name=self.project_name, # 项目名对应组名,区分当前执行的内容

  62.   ext_vars=self.ext_vars,

  63.   host_list=self.host_list)

  64.   self.variable_manager.set_inventory(self.inventory)

  65.   # 初始化需要的对象1

  66.   self.Options = namedtuple('Options',

  67.   ['connection',

  68.   'remote_user',

  69.   'ask_sudo_pass',

  70.   'verbosity',

  71.   'ack_pass',

  72.   'module_path',

  73.   'forks',

  74.   'become',

  75.   'become_method',

  76.   'become_user',

  77.   'check',

  78.   'listhosts',

  79.   'listtasks',

  80.   'listtags',

  81.   'syntax',

  82.   'sudo_user',

  83.   'sudo'

  84.   ])

  85.   # 初始化需要的对象2

  86.   self.options = self.Options(connection=self.connection,

  87.   remote_user=self.ssh_user,

  88.   ack_pass=self.ack_pass,

  89.   sudo_user=self.ssh_user,

  90.   forks=self.forks,

  91.   sudo='yes',

  92.   ask_sudo_pass=False,

  93.   verbosity=5,

  94.   module_path=None,

  95.   become=True,

  96.   become_method='sudo',

  97.   become_user='root',

  98.   check=None,

  99.   listhosts=None,

  100.   listtasks=None,

  101.   listtags=None,

  102.   syntax=None

  103.   )

  104.   # 初始化console输出

  105.   self.callback = YunweiCallback()

  106.   # 直接开始

  107.   self.run()

  108.   def run(self):

  109.   pb = None

  110.   pb = YunweiPlaybookExecutor(

  111.   playbooks = self.playbooks,

  112.   inventory = self.inventory,

  113.   variable_manager = self.variable_manager,

  114.   loader = self.loader,

  115.   options = self.options,

  116.   passwords = self.passwords,

  117.   stdout_callback = self.callback

  118.   )

  119.   result = pb.run()

  120.   # daemo

  121.   if __name__ == "__main__":

  122.   PlayBookJob(playbooks=['xx.yml'],

  123.   host_list=['10.45.176.2'],

  124.   ssh_user='root',

  125.   project_name="test",

  126.   forks=20,

  127.   ext_vars=None

  128.   )
复制代码


  参考:

  ansible-api官方文档:http://docs.ansible.com/ansib...

  一个非常好的ansible2.0 api调用文档,相见恨晚:https://serversforhackers.com...

  ansible源码:https://github.com/ansible/an...

原文作者: 青云2016 来源:开发者头条
e.png

相关帖子

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关于我们
联系我们
  • 电话:010-86393388
  • 邮件:udn@yonyou.com
  • 地址:北京市海淀区北清路68号
移动客户端下载
关注我们
  • 微信公众号:yonyouudn
  • 扫描右侧二维码关注我们
  • 专注企业互联网的技术社区
版权所有:用友网络科技股份有限公司82041 京ICP备05007539号-11 京公网网备安1101080209224 Powered by Discuz!
快速回复 返回列表 返回顶部