• 在windows中使用vagrant

    • 介绍

    vagrant封装linux开发环境,可以统一团队成员的开发环境。封装后,可以linux/windows/mac上运行.

    • 环境

    以win8安装vagrant为例子。vagrant需要virtualbox提供虚拟化支持,因此先安装virtualbox。

    • 安装
    https://www.vagrantup.com/downloads.html
    
    • vagrant镜像

    vagrant镜像下载地址

    http://www.vagrantbox.es/
    

    以centos-7.0为例子,下载地址

    https://github.com/tommy-muehle/puppet-vagrant-boxes/releases/download/1.1.0/centos-7.0-x86_64.box
    
    • 新建目录
    D:\vagrant\centos
    

    下载后的centos-7.0-x86_64.box放到此目录

    • 添加box
    D:\vagrant\centos>vagrant box add --name centos7 .\centos-7.0-x86_64.box
    ==> box: Box file was not detected as metadata. Adding it directly...
    ==> box: Adding box 'centos7' (v0) for provider:
        box: Unpacking necessary files from: file://D:/vagrant/centos/centos-7.0-x86
    _64.box
        box: Progress: 100% (Rate: 48.1M/s, Estimated time remaining: --:--:--)
    ==> box: Successfully added box 'centos7' (v0) for 'virtualbox'!
    
    • 查看box列表
    D:\vagrant\centos>vagrant box list
    centos7            (virtualbox, 0)
    
    • 初始化工作目录
    D:\vagrant\centos>vagrant init centos7
    
    • 启动
    D:\vagrant\centos>vagrant up
    Bringing machine 'default' up with 'virtualbox' provider...
    ==> default: Clearing any previously set forwarded ports...
    ==> default: Clearing any previously set network interfaces...
    ==> default: Preparing network interfaces based on configuration...
        default: Adapter 1: nat
    ==> default: Forwarding ports...
        default: 22 => 2222 (adapter 1)
    ==> default: Booting VM...
    ==> default: Waiting for machine to boot. This may take a few minutes...
        default: SSH address: 127.0.0.1:2222
        default: SSH username: vagrant
        default: SSH auth method: private key
        default: Warning: Connection timeout. Retrying...
    ==> default: Machine booted and ready!
    ==> default: Checking for guest additions in VM...
    ==> default: Mounting shared folders...
        default: /vagrant => D:/vagrant/centos
    ==> default: Machine already provisioned. Run `vagrant provision` or use the `--
    provision`
    ==> default: flag to force provisioning. Provisioners marked to run always will
    still run.
    
    • 安装ssh工具

    启动之后,登录系统需要ssh,windows没有ssh,可以安装copssh。安装后,再将ssh.exe路径配置到环境变量中。

    https://www.itefix.net/content/copssh-free-edition
    
    • 登录
    D:\vagrant\centos>vagrant ssh
    cygwin warning:
      MS-DOS style path detected: D:/vagrant/centos/.vagrant/machines/default/virtua
    lbox/private_key
      Preferred POSIX equivalent is: /cygdrive/d/vagrant/centos/.vagrant/machines/de
    fault/virtualbox/private_key
      CYGWIN environment variable option "nodosfilewarning" turns off this warning.
      Consult the user's guide for more details about POSIX paths:
        http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
    Last login: Thu Nov  5 03:10:24 2015 from 10.0.2.2
    Welcome to your Vagrant-built virtual machine.
    [vagrant@localhost ~]$ exit
    

    使用其它工具登录,初始账号是vagrant/vagrant

    • 关闭
    D:\vagrant\centos>vagrant halt
    ==> default: Attempting graceful shutdown of VM...
    
    • vagrant命令
    vagrant up  启动
    vagrant halt 普通关闭
    vagrant ssh 连接
    vagrant suspend 休眠
    vagrant reload  重启
    vagrant destroy 删除
    vagrant box add --name box_name /path/of/box/file #添加本地box
    
  • python paramiko实现ssh远程登录

    • 介绍

    paramiko是python的一个模块,遵循SSH2协议。 paramiko的功能:1、通过ssh执行命令; 2、通过ssh传输文件。

    • 示例
    import paramiko
    
    ssh = paramiko.SSHClient()
    ssh.load_system_host_keys()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy)
    ssh.connect(hostname="xxx.xxx.xxx.xxx", port=22, username="xxx", password="xxx")
    stdin, stdout, stderr = ssh.exec_command("vmstat 1 1")
    content = stdout.readlines()
    for r in content:
        print r
    ssh.close()
    
  • salt学习笔记

    salt使用主从结构配置、管理、监控远程机器。配置简单,功能强大,是很好的运维管理工具。

    • 准备环境

    使用vmware,准备两台虚拟机192.168.8.10和192.168.8.20,192.168.8.10作为master,192.168.820作为minion.

    • master
    yum install salt-master   //master
    
    • 配置master

    打开/etc/salt/master

    interface: 192.168.8.10
    
    • 启动master
    service salt-master start
    
    • minion
    yum install salt-minion   //minion
    
    • 配置minion

    打开/etc/salt/master

    master: 192.168.8.10
    
    • 启动minion
    service salt-minion start
    
    • 查看minion列表

    在master上运行

    ➜  ~  salt-key -L
    Accepted Keys:
    192.168.124.1
    Denied Keys:
    Unaccepted Keys:
    Rejected Keys:
    
    • 设置master接受所有minion
    salt-key -A
    
    • 测试minion
    salt '*' test.ping
    
    • 运行命令
    salt '192.168.8.20' cmd.run 'df -Th'
    
    • 管理master与minion关系
    salt-key -L 查看key列表
    salt-key -D 删除所有key
    salt-key -d key 删除单个key
    salt-key -A 接受所有key
    salt-key -a key 接受单个key
    
  • 去掉ssh初次登录询问yes/no

    ssh初次登录某个主机时,出现如下提示

    Are you sure you want to continue connecting (yes/no)?
    
    • 如何去掉?

    打开/etc/ssh/ssh_config,修改以下配置项

    #  StrictHostKeyChecking ask
       StrictHostKeyChecking no
    
  • gettext国际化用法示例

    • 安装gettext
    sudo yum install gettext
    
    • gettext工具

    gettext: 进行translate。

    xgettext: 从程序中抽取调用gettext进行本地化的字符串,生成一份.po结尾的配置文件。

    msgfmt: 将配置好的本地化配置文件进行转换成gettext使用的格式。

    • 准备demo.py
    #encoding=utf-8
    import gettext
    
    # demo对应mo文件名,locale为locale目录地址,zh_CN为locale目录下目录名
    zh = gettext.translation("demo", "locale", languages=["zh_CN"])
    # 激活_()
    zh.install(True)
    print _("hello world")
    
    • 生成po文件

    从程序文件中抽取,使用gettext的字符串,生成po文件

    xgettext -L python -o zh_CN.po demo.py
    
    • 编辑po文件
    ...
    #修改编码为utf-8
    "Content-Type: text/plain; charset=utf-8\n"
    ...
    
    #加上翻译
    #: demo.py:6
    msgid "hello world"
    msgstr "你好,世界"
    
    • 创建locale目录
    mkdir -p locale/zh_CN/LC_MESSAGES
    
    • 编译po文件
    msgfmt -o ~/locale/zh_CN/LC_MESSAGES/demo.mo zh_CN.po
    
    • 查看结果
    ➜  ~  python demo.py
    你好,世界
    
  • django-restful请求的访问限制

    login_required无效

    用django的restful写成的请求处理,使用auth模块中装饰器进行访问限制,出现request无user属性的错误.

    from django.views.generic import View
    from django.contrib.auth.decorators import login_required
    
    class TaskQueue(View):
        '''
        执行中的任务
        '''
        def __init__(self):
            self.manager = TaskQueueManager()
    
        def get(self, request):
            '''
            >> c.get("/task/queue/").content
            '''
            records = self.manager.list()
            return {"success": True, "msg": "", "data": records}
    
        @login_required
        def delete(self, request):
            '''
            >> c.delete("/task/queue/", json.dumps({"ids": [14]})).content
            '''
            params = json.loads(request.body)
            for id in params["ids"]:
                self.manager.delete(id)
            return {"success": True, "msg": ""}
    

    查找原因

    从django里拿到源码,调试后,发现View对象被赋给了装饰器的request.

    def user_passes_test(test_func, login_url=None, redirect_field_name=REDIRECT_FIELD_NAME):
        """
        Decorator for views that checks that the user passes the given test,
        redirecting to the log-in page if necessary. The test should be a callable
        that takes the user object and returns True if the user passes.
        """
    
        def decorator(view_func):
            @wraps(view_func, assigned=available_attrs(view_func))
            # view对象传给料request
            def _wrapped_view(request, *args, **kwargs):
                if test_func(request.user):
                    return view_func(request, *args, **kwargs)
                path = request.build_absolute_uri()
                resolved_login_url = resolve_url(login_url or settings.LOGIN_URL)
                # If the login url is the same scheme and net location then just
                # use the path as the "next" url.
                login_scheme, login_netloc = urlparse(resolved_login_url)[:2]
                current_scheme, current_netloc = urlparse(path)[:2]
                if ((not login_scheme or login_scheme == current_scheme) and
                        (not login_netloc or login_netloc == current_netloc)):
                    path = request.get_full_path()
                from django.contrib.auth.views import redirect_to_login
                return redirect_to_login(
                    path, resolved_login_url, redirect_field_name)
            return _wrapped_view
        return decorator
    
    
    def login_required(function=None, redirect_field_name=REDIRECT_FIELD_NAME, login_url=None):
        """
        Decorator for views that checks that the user is logged in, redirecting
        to the log-in page if necessary.
        """
        actual_decorator = user_passes_test(
            lambda u: u.is_authenticated(),
            login_url=login_url,
            redirect_field_name=redirect_field_name
        )
        if function:
            return actual_decorator(function)
        return actual_decorator
    

    自定义login_required

    解决的办法是进行自定义. 将user_passes_test和login_required拿出来,作如下修改

    ...
    # 加上self
    def _wrapped_view(self, request, *args, **kwargs):
        if test_func(request.user):
            # 加上self
            return view_func(self, request, *args, **kwargs)
    ...
    
    
  • 如何得到yum的rpm包

    • 安装yum-utils
    yum -y install yum-utils
    
    • 下载rpm
    yumdownloader proxychains
    

    得到proxychains-3.1-16.fc22.i686.rpm 、proxychains-3.1-16.fc22.x86_64.rpm两个文件

    • 下载源码
    yumdownloader --source proxychains
    

    得到文件proxychains-3.1-16.fc22.src.rpm

  • supervisor用法

    以什么方式运行进程?将它做成服务,再以"service xxx start/stop"方式运行。 或者以"nohup xxx &"方式运行,需要停止时,先ps获得进程id,然后kill掉. 有什么好点的办法?supervisor正好解决这个问题。

    • 安装
    sudo pip install supervisor
    
    • 创建配置文件
    echo_supervisord_conf > /etc/supervisord.conf
    
    • 取消注释

    打开/etc/supervisord.conf,取消下面两行注释,并修改files内容.

    [include]
    files = /etc/supervisor/*.ini
    
    • 新建配置目录

    新建supervisor目录及test.ini文件

    ➜  /etc  tree supervisor
    supervisor
    └── test.ini
    
    • 配置内容

    test.init内容

    [program:test]
    command=python -m SimpleHTTPServer 8000
    directory=/home/wyq/
    
    • 运行

    运行后,在浏览器访问8000端口

    supervisord 
    

    调试运行,此时需要先将test.ini中directory注释掉.

    supervisord -d
    

    supervisor也可指定配置文件

    supervisord -c xxxx
    
    • 查看日志
    tail -f supervisord.conf
    
    • 查看运行状态
    ➜  ~  supervisorctl status
    test                             RUNNING   pid 4922, uptime 0:00:07
    
    • 停止任务

    启动与重启参数为start/restart

    ➜  ~  supervisorctl stop test
    test: stopped
    ➜  ~  supervisorctl status   
    test                             STOPPED   Jul 09 04:32 PM
    
    • 停止
    supervisorctl shutdown
    
  • 利用setuptools的entry_point参数实现模块动态导入

    setuptools提供了entry_points参数,允许在安装时,动态导入模块. 下面是简单示例.

    • 目录结构
      建立如下文件
    ➜  book  tree
    .
    ├── book
    │   ├── add.py
    │   ├── __init__.py
    │   ├── remove.py
    │   └── update.py
    └── setup.py
    
    • add.py内容
      remove.py、update.py与add.py相同
    def make():
        print "add"
    
    • setup.py内容
    from setuptools import setup, find_packages
    
    setup(
        name = "book",
        version = "0.1",
        packages = find_packages(),
        entry_points={
            "book":[
                "add=book.add:make",  #add=模块:函数/类
                "update=book.update:make",
                "remove=book.remove:make",
            ]
        }
    )
    
    • 编译成egg包
    python setup.py bdist_egg
    
    • 安装到系统路径
      上面编译命令可以不用,直接用install安装
    python setup.py install
    
    • 用法
    from pkg_resources import iter_entry_points
    
    for r in iter_entry_points("book"):
        m = r.load()
        m()
    
  • apscheduler如何传递参数给job

    import tornado
    from apscheduler.schedulers.tornado import TornadoScheduler
    sched = TornadoScheduler()
    
    
    def job1(a, b, c):
        print "job1:", a,b,c
    
    
    def job2(a, b, c):
        print "job2:", a,b,c
    
    sched.add_job(job1, 'interval', seconds=1, args=["a", "b", "c"])
    sched.add_job(job2, 'interval', seconds=1, kwargs={"a": "a", "b": "b", "c": "c"})
    sched.start()
    
    tornado.ioloop.IOLoop.instance().start()